Page History

...

Application logs are produced by the component instances. The A component is free to store logs anywhere. As component Component instances are activated as the YARN user (by default it is yarn) or the user running the YARN application it is required . Therefore, you need to ensure that the component instance has write permission. The best practice is to log into a sub-folder within ${AGENT_WORK_ROOT}. This symbolic name resolves to the container specific log folder  - <yarn.nodemanager.log-dirs>/application_id/container_id.

Agent Command Logs

Agent command logs are the logs produced by an agent when commands are executed. Currently, the agent execute INSTALL, CONFIGURE, and START. These logs are available at <yarn.nodemanager.log-dirs>/application_id/container_id/app/command-log folder. You will see three types of files here:

• command-N.json - input to the command, N is a unique command id assigned by the Slider AppMaster

• output-N.txt - output of the command

• errors-N.txt - error reported by the command

Agent Logs

Agent log is the log produced by the agent itself. This log is available at <yarn.nodemanager.log-dirs>/application_id/container_id/infra/log/agent.log

AppMaster Logs

AppMaster runs within its own container. The log files are: <yarn.nodemanager.log-dirs>/application_id/container_id/slider-out.txt, and slider-err.txt. Usually, the AppMaster is the first container.  

Yarn Logs

Sometimes you may need to look into the nodemanager and resourcemanager logs. They are in the YARN log folder which is by default /var/log/hadoop-yarn/yarn.