You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

(warning) Work in Progress (warning)

Overview

The sequence diagrams below are intended to be a very detailed description of the interactions that occur during the process of defining, submitting and executing a map reduce job on a secure Hadoop 2.x cluster. Different phases of the overall process are covered in each diagram.

  • Bootstrap
  • Job Definition
  • Job Submission
  • Job Initiation
  • Map Task Execution
  • Reduce Task Execution
  • Job Completion
  • Client Monitoring

Legend

The descriptions of the interactions in the sequence diagrams below take this form.

message [Protocol] ( input ) : output

The [Protocol] portion describes the protocol, authentication mechanism and identities exchanged.

Abbreviation

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9d708cbc-b383-4b82-b624-2134b854199c"><ac:plain-text-body><![CDATA[

[KRB]

Kerberos Protocol

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3ecf9aeb-871f-4b3e-948a-429cce2e2c04"><ac:plain-text-body><![CDATA[

[RSK:{ticket}]

RPC protocol with SASL mutual authentication using Kerberos tickets.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="f179516d-dbb5-4678-8cc3-7baeff2dbe03"><ac:plain-text-body><![CDATA[

[RST:{access-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="24020e05-f9f1-439e-9c14-d22d2b03c67d"><ac:plain-text-body><![CDATA[

[RSD:{delegation-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="c5cca109-81f4-4b4b-a595-d6aa4cf63104"><ac:plain-text-body><![CDATA[

[STP]

Shuffle data transfer protocol between ShuffleService and ReduceTask. HTTP protocol with TODO.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="bb59b5ce-39e8-4f22-a350-c1f38a527010"><ac:plain-text-body><![CDATA[

[DTP]

Block data transfer protocol between the DataNode and a client. HTTP protocol with block tokens plus SHA1 hash exchange.

]]></ac:plain-text-body></ac:structured-macro>

Suffixes are used in many cases to denote type.

Abbreviation

Description

tgt

Kerberos Ticket Granting Ticket

kt

Kerberos Ticket: u-jt-kt = A Kerberos Ticket for User u to access the JobTracker jt

kp

Kerberos Principal: nn-kp = The Kerberos principal for the NameNode nn

dt

Delegation Token: c-nn-dt = A delegation token for identity of the Client that can be presented to the NameNode.

tkn

Access Token: am-tkn = An access token that can be presented to the ApplicationMaster for access.

tkn-sk

Token Secret Key

Kerberos principals use the principal abbreviation and the kp suffix.

Abbreviation

Description

nn-kp

NameNode's Kerberos Principal

dn-kp

DataNode's Kerberos Principal (Unique principal for each DataNode on every node)

jt-kp

JobTracker's Kerberos Principal

tt-kp

TaskTracker's Kerberos Principal (Unique principal for each TaskTracker on every node)

Kerberos tickets use the consumer principal abbreviation, provider principal abbreviation and kt suffix.

Abbreviation

Description

u-nn-kt

Kerberos service ticket for User u to access NameNode nn

u-jt-kt

Kerberos service ticket for User u to access JobTracker jt

dn-nn-kt

Kerberos service ticket for DataNode dn to access NameNode nn

jt-nn-kt

Kerberos service ticket for JobTracker dn to access NameNode nn

tt-jt-kt

Kerberos service ticket for TaskTracker tt to access JobTracker jt

Secure MapReduce2 - Bootstrap

Secure MapReduce2 - Job Definition

Secure MapReduce2 - Job Submission

Secure MapReduce2 - Job Initiation

Secure MapReduce2 - Map Task Execution

Secure MapReduce2 - Reduce Task Execution

Secure MapReduce2 - Job Completion

Secure MapReduce2 - Client Monitoring

  • No labels