Versions Compared

Old Version 22

changes.mady.by.user Kevin Minder

Saved on

compared with

New Version 23

changes.mady.by.user Kevin Minder

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Abbreviation

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="49cbaecbd70df8ae-29590737-45bd4f77-87d48fb6-3fc654fec222698f9b6ce6ce"><ac:plain-text-body><![CDATA[

[KRB]

Kerberos Protocol

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="2ee8d6db718bc225-b6def16a-4b894b2a-b26e97dc-3774e4f724acfb9f15b62f43"><ac:plain-text-body><![CDATA[

[RSKT:{kerberos-service-ticket}]

RPC protocol with SASL mutual authentication using Kerberos tickets.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="848343ab7e06b910-573a6e3a-455842db-9b05ab70-41af39288146098b266534de"><ac:plain-text-body><![CDATA[

[RSAT:{access-token}]

RPC protocol with SASL mutual authentication using access tokens (e.g. YARN Node Manager Token).

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="23ac88ba79f255ab-b49ad418-41ff4b13-a2fea161-66e681c21a89b1e3a75e32c9"><ac:plain-text-body><![CDATA[

[RSDT:{delegation-token}]

RPC protocol with SASL mutual authentication using delegation tokens (e.g. HDFS Name Node Delegation Token).

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="6c2bd8b3796dae3e-b7e78c0d-4561421e-9adf8b53-6d906009dedf4fec283c518d"><ac:plain-text-body><![CDATA[

[STP]

Shuffle data transfer protocol between ShuffleService and ReduceTask. HTTP protocol with TODO.

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="f2442571643e521c-88ed7223-4dc94ddd-b4a7bb63-9981a105a9182aef8ce5a970"><ac:plain-text-body><![CDATA[

[DTP]

Block data transfer protocol between the DataNode and a client. HTTP protocol with block tokens plus SHA1 hash exchange.

]]></ac:plain-text-body></ac:structured-macro>

...

Abbreviation

Description

u-nn-kt

Kerberos service ticket for User u to access NameNode nn

u-jt-kt

Kerberos service ticket for User u to access JobTracker jt

dn-nn-kt

Kerberos service ticket for DataNode dn to access NameNode nn

jt-nn-kt

Kerberos service ticket for JobTracker dn to access NameNode nn

tt-jt-kt

Kerberos service ticket for TaskTracker tt to access JobTracker jt

...

Bootstrap

This diagram illustrates the interactions that occur when a Hadoop system is starting up and stabilizing. It involves various master components generating secret keys and slave components registering with the masters to receive these secret keys.

...

  1. createBlockAccessTokenSecretKey -
  2. kinit/AS_REQ -
  3. TGS_REQ -
  4. register/heartbeat -
  5. createNodeManagerTokenSecretKey -
  6. createAppContainerTokenSecretKey -
  7. kinit/AS_REQ -
  8. TGS_REQ -
  9. register/heartbeat -

...

Job Definition

This diagram illustrates the steps taken by a client to define a MapReduce job that will later be submitted.

  1. TODO
  2. TODO
  3. TODO

...

Job Submission

This diagram illustrates the steps taken during the submission of a MapReduce job.

  1. TODO
  2. TODO
  3. TODO

...

Job Initiation

This diagram illustrates the steps taken when a MapReduce job is scheduled for execution.

  1. TODO
  2. TODO
  3. TODO

...

Map Task Execution

This diagram illustrates the steps taken when the Map portion of a MapReduce job is executed.

  1. TODO
  2. TODO
  3. TODO

...

Reduce Task Execution

This diagram illustrates the steps taken when the Reduce portion of a MapReduce job is executed.

  1. TODO
  2. TODO
  3. TODO

...

Job Completion

This diagram illustrates the steps taken a MapReduce job has completed.

  1. TODO
  2. TODO
  3. TODO

...

Client Monitoring

This diagram illustrates the steps taken by a Client to monitor the status of a Job throughout the Job's life-cycle. The timeframe for this diagram span several of the diagrams above starting from Job Submission all the way through Job Completion.

...