54. Customizations

Thanks to the SpanInjector and SpanExtractor you can customize the way spans are created and propagated.

There are currently two built-in ways to pass tracing information between processes:

via Spring Integration
via HTTP

Span ids are extracted from Zipkin-compatible (B3) headers (either Message or HTTP headers), to start or join an existing trace. Trace information is injected into any outbound requests so the next hop can extract them.

The default way of coding tracing context is done via the b3 header that contains the traceId-spanId-sampled notation (e.g. 0000000000000005-0000000000000004-1 ). For backward compatibility, if the b3 header is not present, we also check if X-B3 entries are present, and retrieve tracing context from there e.g. ( X-B3-TraceId: 0000000000000005 , X-B3-SpanId: 0000000000000004 , X-B3-Sampled: 1 ).

The key change in comparison to the previous versions of Sleuth is that Sleuth is implementing the Open Tracing’s TextMap notion. In Sleuth it’s called SpanTextMap . Basically the idea is that any means of communication (e.g. message, http request, etc.) can be abstracted via a SpanTextMap . This abstraction defines how one can insert data into the carrier and how to retrieve it from there. Thanks to this if you want to instrument a new HTTP library that uses a FooRequest as a mean of sending HTTP requests then you have to create an implementation of a SpanTextMap that delegates calls to FooRequest in terms of retrieval and insertion of HTTP headers.

54.1 Spring Integration

For Spring Integration there are 2 interfaces responsible for creation of a Span from a Message . These are:

MessagingSpanTextMapExtractor
MessagingSpanTextMapInjector

You can override them by providing your own implementation.

54.2 HTTP

For HTTP there are 2 interfaces responsible for creation of a Span from a Message . These are:

HttpSpanExtractor
HttpSpanInjector

You can override them by providing your own implementation.

54.3 Example

Let’s assume that instead of the standard Zipkin compatible tracing HTTP header names you have

for trace id - correlationId
for span id - mySpanId

This is a an example of a SpanExtractor

static class CustomHttpSpanExtractor implements HttpSpanExtractor {

	@Override public Span joinTrace(SpanTextMap carrier) {
		Map<String, String> map = TextMapUtil.asMap(carrier);
		long traceId = Span.hexToId(map.get("correlationid"));
		long spanId = Span.hexToId(map.get("myspanid"));
		// extract all necessary headers
		Span.SpanBuilder builder = Span.builder().traceId(traceId).spanId(spanId);
		// build rest of the Span
		return builder.build();
	}
}

static class CustomHttpSpanInjector implements HttpSpanInjector {

	@Override
	public void inject(Span span, SpanTextMap carrier) {
		carrier.put("correlationId", span.traceIdString());
		carrier.put("mySpanId", Span.idToHex(span.getSpanId()));
	}
}

And you could register it like this:

@Bean
HttpSpanInjector customHttpSpanInjector() {
	return new CustomHttpSpanInjector();
}

@Bean
HttpSpanExtractor customHttpSpanExtractor() {
	return new CustomHttpSpanExtractor();
}

Spring Cloud Sleuth does not add trace/span related headers to the Http Response for security reasons. If you need the headers then a custom SpanInjector that injects the headers into the Http Response and a Servlet filter which makes use of this can be added the following way:

static class CustomHttpServletResponseSpanInjector extends ZipkinHttpSpanInjector {

	@Override
	public void inject(Span span, SpanTextMap carrier) {
		super.inject(span, carrier);
		carrier.put(Span.TRACE_ID_NAME, span.traceIdString());
		carrier.put(Span.SPAN_ID_NAME, Span.idToHex(span.getSpanId()));
	}
}

static class HttpResponseInjectingTraceFilter extends GenericFilterBean {

	private final Tracer tracer;
	private final HttpSpanInjector spanInjector;

	public HttpResponseInjectingTraceFilter(Tracer tracer, HttpSpanInjector spanInjector) {
		this.tracer = tracer;
		this.spanInjector = spanInjector;
	}

	@Override
	public void doFilter(ServletRequest request, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
		HttpServletResponse response = (HttpServletResponse) servletResponse;
		Span currentSpan = this.tracer.getCurrentSpan();
		this.spanInjector.inject(currentSpan, new HttpServletResponseTextMap(response));
		filterChain.doFilter(request, response);
	}

	 class HttpServletResponseTextMap implements SpanTextMap {

		 private final HttpServletResponse delegate;

		 HttpServletResponseTextMap(HttpServletResponse delegate) {
			 this.delegate = delegate;
		 }

		 @Override
		 public Iterator<Map.Entry<String, String>> iterator() {
			 Map<String, String> map = new HashMap<>();
			 for (String header : this.delegate.getHeaderNames()) {
				map.put(header, this.delegate.getHeader(header));
			 }
			 return map.entrySet().iterator();
		 }

		 @Override
		 public void put(String key, String value) {
			this.delegate.addHeader(key, value);
		 }
	 }
}

And you could register them like this:

@Bean HttpSpanInjector customHttpServletResponseSpanInjector() {
	return new CustomHttpServletResponseSpanInjector();
}

@Bean
HttpResponseInjectingTraceFilter responseInjectingTraceFilter(Tracer tracer) {
	return new HttpResponseInjectingTraceFilter(tracer, customHttpServletResponseSpanInjector());
}

54.4 TraceFilter

You can also modify the behaviour of the TraceFilter - the component that is responsible for processing the input HTTP request and adding tags basing on the HTTP response. You can customize the tags, or modify the response headers by registering your own instance of the TraceFilter bean.

In the following example we will register the TraceFilter bean and we will add the ZIPKIN-TRACE-ID response header containing the current Span’s trace id. Also we will add to the Span a tag with key custom and a value tag .

@Bean
TraceFilter myTraceFilter(BeanFactory beanFactory, final Tracer tracer) {
	return new TraceFilter(beanFactory) {
		@Override protected void addResponseTags(HttpServletResponse response,
				Throwable e) {
			// execute the default behaviour
			super.addResponseTags(response, e);
			// for readability we're returning trace id in a hex form
			response.addHeader("ZIPKIN-TRACE-ID",
					Span.idToHex(tracer.getCurrentSpan().getTraceId()));
			// we can also add some custom tags
			tracer.addTag("custom", "tag");
		}
	};
}

To change the order of TraceFilter registration, please set the spring.sleuth.web.filter-order property.

54.5 Custom SA tag in Zipkin

Sometimes you want to create a manual Span that will wrap a call to an external service which is not instrumented. What you can do is to create a span with the peer.service tag that will contain a value of the service that you want to call. Below you can see an example of a call to Redis that is wrapped in such a span.

org.springframework.cloud.sleuth.Span newSpan = tracer.createSpan("redis");
try {
	newSpan.tag("redis.op", "get");
	newSpan.tag("lc", "redis");
	newSpan.logEvent(org.springframework.cloud.sleuth.Span.CLIENT_SEND);
	// call redis service e.g
	// return (SomeObj) redisTemplate.opsForHash().get("MYHASH", someObjKey);
} finally {
	newSpan.tag("peer.service", "redisService");
	newSpan.tag("peer.ipv4", "1.2.3.4");
	newSpan.tag("peer.port", "1234");
	newSpan.logEvent(org.springframework.cloud.sleuth.Span.CLIENT_RECV);
	tracer.close(newSpan);
}

Remember not to add both peer.service tag and the SA tag! You have to add only peer.service .

54.6 Custom service name

By default Sleuth assumes that when you send a span to Zipkin, you want the span’s service name to be equal to spring.application.name value. That’s not always the case though. There are situations in which you want to explicitly provide a different service name for all spans coming from your application. To achieve that it’s enough to just pass the following property to your application to override that value (example for foo service name):

spring.zipkin.service.name: foo

54.7 Customization of reported spans

Before reporting spans to e.g. Zipkin you can be interested in modifying that span in some way. You can achieve that by using the SpanAdjuster interface.

Example of usage:

In Sleuth we’re generating spans with a fixed name. Some users want to modify the name depending on values of tags. Implementation of the SpanAdjuster interface can be used to alter that name. Example:

@Bean
SpanAdjuster customSpanAdjuster() {
    return span -> span.toBuilder().name(scrub(span.getName())).build();
}

This will lead in changing the name of the reported span just before it gets sent to Zipkin.

Your SpanReporter should inject the SpanAdjuster and allow span manipulation before the actual reporting is done.

54.8 Host locator

In order to define the host that is corresponding to a particular span we need to resolve the host name and port. The default approach is to take it from server properties. If those for some reason are not set then we’re trying to retrieve the host name from the network interfaces.

If you have the discovery client enabled and prefer to retrieve the host address from the registered instance in a service registry then you have to set the property (it’s applicable for both HTTP and Stream based span reporting).

spring.zipkin.locator.discovery.enabled: true

Docs

Docs4dev

Title here