YARN can be configured to retain logs for debugging purposes. You can set "yarn.nodemanager.delete.debug-delay-sec" to some non-zero value (for example, 3600 for an hour). The Slider dev team is working on a feature to make all logs available through YARN at one location (SLIDER-116).
Application Logs
Application logs are produced by the application component instances (e.g. HBASE_MASTER for HBASE on YARN). A component is free to store logs anywhere. Component instances are activated as the YARN user (e.g. yarn). Therefore, you need to ensure that the component instances have write permission. The best practice is to log at within ${AGENT_LOG_ROOT}. This symbolic name resolves to the container specific log folder - <yarn.nodemanager.log-dirs>/application_id/container_id. The default packages available for Slider are by default configured to log at ${AGENT_LOG_ROOT}
Why you may look into Application Logs:
- Application instance not behaving as expected
- Application component instances go down after start
Agent Command Logs
Agent command logs are the logs produced by an agent when commands are executed. Currently, the agent execute INSTALL, CONFIGURE, and START. These logs are available at <yarn.nodemanager.log-dirs>/application_id/container_id folder. You will see three types of files here:
command-N.json - input to the command, N is a unique command id assigned by the Slider AppMaster
output-N.txt - output of the command
errors-N.txt - error reported by the command
Why you may look into Agent Command Logs:
- Failures during command (INSTALL, CONFIGURE, START) execution
Agent Logs
Agent log is the log produced by the agent itself. This log is available at <yarn.nodemanager.log-dirs>/application_id/container_id/slider-agent.log
Why you may look into Agent Logs:
- AppMaster starts containers but is not able to heartbeat with the Agent (typically communication issues)
- AppMaster issues commands but commands do not get scheduled for execution
AppMaster Logs
AppMaster runs within its own container. The log files are: <yarn.nodemanager.log-dirs>/application_id/container_id/slider-out.txt, and slider-err.txt. The AppMaster is activated in the first container.
Why you may look into AppMaster Logs:
- No containers are being requested from YARN
- AppMaster fails after start
YARN logs
At times you may need to look into the nodemanager and resourcemanager logs. They are in the YARN log folder which is by default /var/log/hadoop-yarn/yarn.