You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

(warning) Work in Progress (warning)

Overview

The sequence diagrams below are intended to be a very detailed description of the interactions that occur during the process of defining, submitting and executing a map reduce job on a secure Hadoop 2.x cluster. Different phases of the overall process are covered in each diagram.

  • Bootstrap
  • Job Definition
  • Job Submission
  • Job Initiation
  • Map Task Execution
  • Reduce Task Execution
  • Job Completion
  • Client Monitoring

Legend

The descriptions of the interactions in the sequence diagrams below take this form.

message [Protocol] ( input ) : output

The [Protocol] portion describes the protocol, authentication mechanism and identities exchanged.

Abbreviation

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="10296688-3f42-4d3f-bb45-640960afd812"><ac:plain-text-body><![CDATA[

[KRB]

Kerberos Protocol

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="54052b6a-120c-4221-bc3d-e501b411ac95"><ac:plain-text-body><![CDATA[

[RSKT:{kerberos-service-ticket}]

RPC protocol with SASL mutual authentication using Kerberos tickets.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="ba83562b-ace7-4e28-9b9f-534c6ab81ded"><ac:plain-text-body><![CDATA[

[RSAT:{access-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="eccb2d3d-f427-4c09-9ab0-6279f1eb6161"><ac:plain-text-body><![CDATA[

[RSDT:{delegation-token}]

RPC protocol with SASL mutual authentication using delegation tokens.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="84c95637-5bb8-477d-989e-4c29ec0f5b2d"><ac:plain-text-body><![CDATA[

[STP]

Shuffle data transfer protocol between ShuffleService and ReduceTask. HTTP protocol with TODO.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="10d343a0-65a6-45f5-ace9-38c3a792c559"><ac:plain-text-body><![CDATA[

[DTP]

Block data transfer protocol between the DataNode and a client. HTTP protocol with block tokens plus SHA1 hash exchange.

]]></ac:plain-text-body></ac:structured-macro>

Suffixes are used in many cases to denote type.

Abbreviation

Description

tgt

Kerberos Ticket Granting Ticket

kt

Kerberos Service Ticket: u-jt-kt = A Kerberos Ticket for User u to access the JobTracker jt

kp

Kerberos Principal: nn-kp = The Kerberos principal for the NameNode nn

dt

Delegation Token: c-nn-dt = A delegation token for identity of the Client that can be presented to the NameNode.

tkn

Access Token: am-tkn = An access token that can be presented to the ApplicationMaster for access.

tkn-sk

Token Secret Key

Kerberos principals use the principal abbreviation and the kp suffix.

Abbreviation

Description

nn-kp

NameNode's Kerberos Principal

dn-kp

DataNode's Kerberos Principal (Unique principal for each DataNode on every node)

jt-kp

JobTracker's Kerberos Principal

tt-kp

TaskTracker's Kerberos Principal (Unique principal for each TaskTracker on every node)

Kerberos tickets use the consumer principal abbreviation, provider principal abbreviation and kt suffix.

Abbreviation

Description

u-nn-kt

Kerberos service ticket for User u to access NameNode nn

u-jt-kt

Kerberos service ticket for User u to access JobTracker jt

dn-nn-kt

Kerberos service ticket for DataNode dn to access NameNode nn

jt-nn-kt

Kerberos service ticket for JobTracker dn to access NameNode nn

tt-jt-kt

Kerberos service ticket for TaskTracker tt to access JobTracker jt

Secure MapReduce2 - Bootstrap

Secure MapReduce2 - Job Definition

Secure MapReduce2 - Job Submission

Secure MapReduce2 - Job Initiation

Secure MapReduce2 - Map Task Execution

Secure MapReduce2 - Reduce Task Execution

Secure MapReduce2 - Job Completion

Secure MapReduce2 - Client Monitoring

  • No labels