...

• On the container side we start a new thread that periodically polls this endpoint described above to check if the container is valid. If its not, we shutdown the run loop and raise an error (so that the exit code is non 0 so that YARN reschedules the container)
• Calling the shutdown function on the run loop from the ContainerHeartbeat thread would causes the finally block in SamzaContainer.run() to execute, we need all the shutdown functions to execute before we can exit with a non-zero code. Therefore we need a way to indicate to The plan is to setup a monitor in the LocalContainerRunner class that the SamzaContainer class exited with an exception.This could involve maintaining state in LocalContainerRunner to denote that we we need to exit with an error code. When shutting down the LocalContainerRunner we can check for this variable and schedules a thread to check the above endpoint at regular intervals. On failure the thread modifies state on the LocalContainerRunner to denote that there was an error. This state is checked during exit in the LocalContainerRunner to exit with a non-zero exit code. 

Public Interfaces

This would introduce a few new configs:

...