...

Distributed tracing, first described by Google in Dapper, a Large-Scale Distributed Systems Tracing Infrastructure paper became increasingly important topic these days. With microservices (aka SOA) gaining more and more adoption, the typical applications are built using dozens or even hundreds of small, distributed pieces. The end-to-end traceability of the requests (or any kind of work performed on user's behalf) is hard task to accomplish, particularly taking into account asyncronous or/and concurrent invocations. Apache HTrace is inspired by Dapper, a Large-Scale Distributed Systems Tracing Infrastructure paper and essentially is a full-fledged distributed tracing framework.

Distributed tracing is additional instrumentation layer on top of new or existing applications. In terms of distributed tracing, span represents a basic unit of work. For example, executing database query is a span. Spans are identified by a unique 64-bit ID for the span and another 64-bit ID for the trace the span is a part of. Spans also have other data, such as descriptions, timelines, key-value annotations, the ID of the span that caused them (parent), and process ID’s (normally IP address and process name). Spans are started and stopped, and they keep track of their timing information. Once span is created, it should be stopped at some point in the future. In turn, trace is a set of spans forming a tree-like structure. For example, if you are running a JAX-RS service, a trace might be formed by a PUT request.

Distributed Tracing in Apache CXF

Apache CXF is a very popular framework for building services and web APIs. No doubts, it is going to play even more important role in context of microservices architecture letting developers to quickly build and deploy individual JAX-RS/JAX-WS services. As it was just mentioned before, distributed tracing is an essential technique to monitor the application as whole, breaking the request to individual service traces as it goes through and crosses the boundaries of threads, processes and machines.