Page History

...

The second behavioral change is introduced in how we log these failures. Currently, only the exception which kills the task is written with the application logs. With the additions presented in this KIP, more the following context will be logged in JSON format: 

1. The record which failed to process (if available). Any binary data will be base64 encoded. 
2. The Exception along with the stack trace.
3. Information about the stage which failed to process the record (for example: classname of the transform and its configs).
4. The various stages in the connector, along with their configurations and order of processing.
5. The number of attempts, and time taken for these attempts.

...