You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Introduction

Debugging in CloudStack using logs is cumbersome. To ease the tracking a command through logs (and upto the resource layer), the concept of contextID and sequence numbers was introduced.

The initial idea of using contextIDs was to provide create a unique ID for every thread in management server, and use the NDC (aka stack) model for pushing new contexts on the stack. It was assumed that every touch point - API, new threads, async job - will correctly push and pop the contexts and ease tracking of an API call.

After discussion with architects, it was realized this has not fully materialized. Instead of modifying existing NDC code-base, it was suggested that we use MDC (aka hash) model and pass the same logContextID, up to the resource layer.

References

Architecture and design description

Why MDC
  • Being a key, value pair object, easier to manipulate
  • More extensible than NDC stack to add more info in future if needed
Constraints Imposed
  • MDC is designed to be lightweight, hence the context hashmap will only contain String (key,value) pairs
  • Note that MDC is not a mechanism to pass information other than logContextID between methods / threads. The implementation should not be used to pass information either, and no guarantees will be made about consistency if any more data is added to the MDC
  • MDC design assumes inserts / deletes to MDC are not high frequency, and we need to respect this design
To incorporate MDC model 
  1. Create new LogContext and LogContextListener to manage semantics around MDC
  2. Modify CloudStack log4j files to provide MDC info in each log message
  3. Generate new MDC logContext on every API call
  4. The contextID should be returned in the API call (so that a user can grep through the logs using this ID)
To ensure propagation
  1. Every new thread incorporates MDC logContextID from its calling parent. If none is available, a new logContext will be generated
  2. All system threads will use a systemLogContext to be created on start-up
  3. For async jobs, save the context information when job info is serialized and retrieve when the job returns. Pass the logContextID to async job

API changes

All APIs, sync or async, to return logContextID when invoked

UI changes

None

Future enhancements

TBD

Comments

  • No labels