Page History

...

Pig on Tez

Cascading on Tez

TBD

Profiling in Tez

It is possible profile specific tasks in specific vertices in Tez. "tez.task-specific.launch.cmd-opts.list" and "tez.task-specific.launch.cmd-opts" options can be used for this. Examples are given below.

• tez.task-specific.launch.cmd-opts.list
  • Specifies the tasks in different vertices where additional options have to be specified
  • Examples:
    • tez.task-specific.launch.cmd-opts.list=“M5[0]" - Specifies task 0 in vertex M5
      
    • tez.task-specific.launch.cmd-opts.list=“Map10[5,20]" - Specifies task 5,20 in vertex Map 10
    • tez.task-specific.launch.cmd-opts.list=“M5[]" - Specifies all tasks in M5
    • tez.task-specific.launch.cmd-opts.list=“M5[1:3,10]" - Specifies 1,2,3,10 tasks in M5
      • However, defining partial ranges like "M5[:3]" is not yet supported.
    • tez.task-specific.launch.cmd-opts.list=“M5[0];V2[10]" - Specifies task 0 in vertex M5 and task 10 in vertex V2
      
      
• tez.task-specific.launch.cmd-opts
  • Specifies the additional task specific JVM launch options that need to be added.
  • __VERTEX_NAME__ and __TASK_INDEX__ can be specified in the option, which would be replaced at runtime.
  • Examples:
    • tez.task-specific.launch.cmd-opts="-agentpath:/opt/yourkit/bin/linux-x86-64/libyjpagent.so=disablej2ee,tracing,alloceach=1000,onexit=snapshot,tracing_settings_path=/tmp/walltime.txt, dir=/tmp/__VERTEX_NAME__/__TASK_INDEX__"