You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

(warning) Work in Progress (warning)

Overview

The sequence diagrams below are intended to be a very detailed description of the interactions that occur during the process of defining, submitting and executing a map reduce job on a secure Hadoop 2.x cluster. Different phases of the overall process are covered in each diagram.

  • Bootstrap
  • Job Definition
  • Job Submission
  • Job Initiation
  • Map Task Execution
  • Reduce Task Execution
  • Job Completion
  • Client Monitoring

Legend

The descriptions of the interactions in the sequence diagrams below take this form.

message [Protocol] ( input ) : output

The [Protocol] portion describes the protocol, authentication mechanism and identities exchanged.

Abbreviation

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="8ec8d895-ceb1-4c96-b326-8344d758ad1f"><ac:plain-text-body><![CDATA[

[KRB]

Kerberos Protocol

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9c429f6f-c5a4-4047-b09c-3a972b23c636"><ac:plain-text-body><![CDATA[

[RSK:{kerberos-service-ticket}]

RPC protocol with SASL mutual authentication using Kerberos tickets.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="8830d477-1b75-49a4-a23e-695d569fba2e"><ac:plain-text-body><![CDATA[

[RST:{access-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3768d1bd-2209-419e-afe9-0620c442e2d7"><ac:plain-text-body><![CDATA[

[RSD:{delegation-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="89077e5f-49aa-4692-9ace-5da2145e9fdc"><ac:plain-text-body><![CDATA[

[STP]

Shuffle data transfer protocol between ShuffleService and ReduceTask. HTTP protocol with TODO.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="75f010c1-ae5c-4e8e-9dfe-5bfebb73dd93"><ac:plain-text-body><![CDATA[

[DTP]

Block data transfer protocol between the DataNode and a client. HTTP protocol with block tokens plus SHA1 hash exchange.

]]></ac:plain-text-body></ac:structured-macro>

Suffixes are used in many cases to denote type.

Abbreviation

Description

tgt

Kerberos Ticket Granting Ticket

kt

Kerberos Ticket: u-jt-kt = A Kerberos Ticket for User u to access the JobTracker jt

kp

Kerberos Principal: nn-kp = The Kerberos principal for the NameNode nn

dt

Delegation Token: c-nn-dt = A delegation token for identity of the Client that can be presented to the NameNode.

tkn

Access Token: am-tkn = An access token that can be presented to the ApplicationMaster for access.

tkn-sk

Token Secret Key

Kerberos principals use the principal abbreviation and the kp suffix.

Abbreviation

Description

nn-kp

NameNode's Kerberos Principal

dn-kp

DataNode's Kerberos Principal (Unique principal for each DataNode on every node)

jt-kp

JobTracker's Kerberos Principal

tt-kp

TaskTracker's Kerberos Principal (Unique principal for each TaskTracker on every node)

Kerberos tickets use the consumer principal abbreviation, provider principal abbreviation and kt suffix.

Abbreviation

Description

u-nn-kt

Kerberos service ticket for User u to access NameNode nn

u-jt-kt

Kerberos service ticket for User u to access JobTracker jt

dn-nn-kt

Kerberos service ticket for DataNode dn to access NameNode nn

jt-nn-kt

Kerberos service ticket for JobTracker dn to access NameNode nn

tt-jt-kt

Kerberos service ticket for TaskTracker tt to access JobTracker jt

Secure MapReduce2 - Bootstrap

Secure MapReduce2 - Job Definition

Secure MapReduce2 - Job Submission

Secure MapReduce2 - Job Initiation

Secure MapReduce2 - Map Task Execution

Secure MapReduce2 - Reduce Task Execution

Secure MapReduce2 - Job Completion

Secure MapReduce2 - Client Monitoring

  • No labels