On this page
52. Metrics
Spring Boot Actuator includes a metrics service with ‘gauge’ and ‘counter’ support. A ‘gauge’ records a single value; and a ‘counter’ records a delta (an increment or decrement). Spring Boot Actuator also provides a PublicMetrics
interface that you can implement to expose metrics that you cannot record via one of those two mechanisms. Look at SystemPublicMetrics
for an example.
Metrics for all HTTP requests are automatically recorded, so if you hit the metrics
endpoint you should see a response similar to this:
{
"counter.status.200.root": 20,
"counter.status.200.metrics": 3,
"counter.status.200.star-star": 5,
"counter.status.401.root": 4,
"gauge.response.star-star": 6,
"gauge.response.root": 2,
"gauge.response.metrics": 3,
"classes": 5808,
"classes.loaded": 5808,
"classes.unloaded": 0,
"heap": 3728384,
"heap.committed": 986624,
"heap.init": 262144,
"heap.used": 52765,
"nonheap": 0,
"nonheap.committed": 77568,
"nonheap.init": 2496,
"nonheap.used": 75826,
"mem": 986624,
"mem.free": 933858,
"processors": 8,
"threads": 15,
"threads.daemon": 11,
"threads.peak": 15,
"threads.totalStarted": 42,
"uptime": 494836,
"instance.uptime": 489782,
"datasource.primary.active": 5,
"datasource.primary.usage": 0.25
}
Here we can see basic memory
, heap
, class loading
, processor
and thread pool
information along with some HTTP metrics. In this instance the root
(‘/’) and /metrics
URLs have returned HTTP 200
responses 20
and 3
times respectively. It also appears that the root
URL returned HTTP 401
(unauthorized) 4
times. The double asterisks (star-star
) comes from a request matched by Spring MVC as /**
(normally a static resource).
The gauge
shows the last response time for a request. So the last request to root
took 2ms
to respond and the last to /metrics
took 3ms
.
Note | |
---|---|
In this example we are actually accessing the endpoint over HTTP using the |
The following system metrics are exposed by Spring Boot:
- The total system memory in KB (
mem
) - The amount of free memory in KB (
mem.free
) - The number of processors (
processors
) - The system uptime in milliseconds (
uptime
) - The application context uptime in milliseconds (
instance.uptime
) - The average system load (
systemload.average
) - Heap information in KB (
heap
,heap.committed
,heap.init
,heap.used
) - Thread information (
threads
,thread.peak
,thread.daemon
) - Class load information (
classes
,classes.loaded
,classes.unloaded
) - Garbage collection information (
gc.xxx.count
,gc.xxx.time
)
The following metrics are exposed for each supported DataSource
defined in your application:
- The number of active connections (
datasource.xxx.active
) - The current usage of the connection pool (
datasource.xxx.usage
).
All data source metrics share the datasource.
prefix. The prefix is further qualified for each data source:
- If the data source is the primary data source (that is either the only available data source or the one flagged
@Primary
amongst the existing ones), the prefix isdatasource.primary
. - If the data source bean name ends with
DataSource
, the prefix is the name of the bean withoutDataSource
(i.e.datasource.batch
forbatchDataSource
). - In all other cases, the name of the bean is used.
It is possible to override part or all of those defaults by registering a bean with a customized version of DataSourcePublicMetrics
. By default, Spring Boot provides metadata for all supported data sources; you can add additional DataSourcePoolMetadataProvider
beans if your favorite data source isn’t supported out of the box. See DataSourcePoolMetadataProvidersConfiguration
for examples.
The following metrics are exposed for each supported cache defined in your application:
- The current size of the cache (
cache.xxx.size
) - Hit ratio (
cache.xxx.hit.ratio
) - Miss ratio (
cache.xxx.miss.ratio
)
Note | |
---|---|
Cache providers do not expose the hit/miss ratio in a consistent way. While some expose an aggregated value (i.e. the hit ratio since the last time the stats were cleared), others expose a temporal value (i.e. the hit ratio of the last second). Check your caching provider documentation for more details. |
If two different cache managers happen to define the same cache, the name of the cache is prefixed by the name of the CacheManager
bean.
It is possible to override part or all of those defaults by registering a bean with a customized version of CachePublicMetrics
. By default, Spring Boot provides cache statistics for EhCache, Hazelcast, Infinispan, JCache and Guava. You can add additional CacheStatisticsProvider
beans if your favorite caching library isn’t supported out of the box. See CacheStatisticsAutoConfiguration
for examples.
If you are using Tomcat as your embedded servlet container, session metrics will automatically be exposed. The httpsessions.active
and httpsessions.max
keys provide the number of active and maximum sessions.
To record your own metrics inject a CounterService
and/or GaugeService
into your bean. The CounterService
exposes increment
, decrement
and reset
methods; the GaugeService
provides a submit
method.
Here is a simple example that counts the number of times that a method is invoked:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.actuate.metrics.CounterService;
import org.springframework.stereotype.Service;
@Service
public class MyService {
private final CounterService counterService;
@Autowired
public MyService(CounterService counterService) {
this.counterService = counterService;
}
public void exampleMethod() {
this.counterService.increment("services.system.myservice.invoked");
}
}
Tip | |
---|---|
You can use any string as a metric name but you should follow guidelines of your chosen store/graphing technology. Some good guidelines for Graphite are available on Matt Aimonetti’s Blog . |
To add additional metrics that are computed every time the metrics endpoint is invoked, simply register additional PublicMetrics
implementation bean(s). By default, all such beans are gathered by the endpoint. You can easily change that by defining your own MetricsEndpoint
.
The default implementation of GaugeService
and CounterService
provided by Spring Boot depends on the version of Java that you are using. With Java 8 (or better) the implementation switches to a high-performance version optimized for fast writes, backed by atomic in-memory buffers, rather than by the immutable but relatively expensive Metric<?>
type (counters are approximately 5 times faster and gauges approximately twice as fast as the repository-based implementations). The Dropwizard metrics services (see below) are also very efficient even for Java 7 (they have backports of some of the Java 8 concurrency libraries), but they do not record timestamps for metric values. If performance of metric gathering is a concern then it is always advisable to use one of the high-performance options, and also to only read metrics infrequently, so that the writes are buffered locally and only read when needed.
Note | |
---|---|
The old |
Spring Boot provides a couple of implementations of a marker interface called Exporter
which can be used to copy metric readings from the in-memory buffers to a place where they can be analyzed and displayed. Indeed, if you provide a @Bean
that implements the MetricWriter
interface (or GaugeWriter
for simple use cases) and mark it @ExportMetricWriter
, then it will automatically be hooked up to an Exporter
and fed metric updates every 5 seconds (configured via spring.metrics.export.delay-millis
). In addition, any MetricReader
that you define and mark as @ExportMetricReader
will have its values exported by the default exporter.
Note | |
---|---|
This feature is enabling scheduling in your application ( |
The default exporter is a MetricCopyExporter
which tries to optimize itself by not copying values that haven’t changed since it was last called (the optimization can be switched off using a flag spring.metrics.export.send-latest
). Note also that the Dropwizard MetricRegistry
has no support for timestamps, so the optimization is not available if you are using Dropwizard metrics (all metrics will be copied on every tick).
The default values for the export trigger (delay-millis
, includes
, excludes
and send-latest
) can be set as spring.metrics.export.*
. Individual values for specific MetricWriters
can be set as spring.metrics.export.triggers.<name>.*
where <name>
is a bean name (or pattern for matching bean names).
Warning | |
---|---|
The automatic export of metrics is disabled if you switch off the default |
If you provide a @Bean
of type RedisMetricRepository
and mark it @ExportMetricWriter
the metrics are exported to a Redis cache for aggregation. The RedisMetricRepository
has two important parameters to configure it for this purpose: prefix
and key
(passed into its constructor). It is best to use a prefix that is unique to the application instance (e.g. using a random value and maybe the logical name of the application to make it possible to correlate with other instances of the same application). The “key” is used to keep a global index of all metric names, so it should be unique “globally”, whatever that means for your system (e.g. two instances of the same system could share a Redis cache if they have distinct keys).
Example:
@Bean
@ExportMetricWriter
MetricWriter metricWriter(MetricExportProperties export) {
return new RedisMetricRepository(connectionFactory,
export.getRedis().getPrefix(), export.getRedis().getKey());
}
application.properties.
spring.metrics.export.redis.prefix: metrics.mysystem.${spring.application.name:application}.${random.value:0000}
spring.metrics.export.redis.key: keys.metrics.mysystem
The prefix is constructed with the application name and id at the end, so it can easily be used to identify a group of processes with the same logical name later.
Note | |
---|---|
It’s important to set both the |
Tip | |
---|---|
The example above uses |
If you provide a @Bean
of type OpenTsdbGaugeWriter
and mark it @ExportMetricWriter
metrics are exported to Open TSDB for aggregation. The OpenTsdbGaugeWriter
has a url
property that you need to set to the Open TSDB “/put” endpoint, e.g. localhost:4242/api/put
). It also has a namingStrategy
that you can customize or configure to make the metrics match the data structure you need on the server. By default it just passes through the metric name as an Open TSDB metric name, and adds the tags “domain” (with value “org.springframework.metrics”) and “process” (with the value equal to the object hash of the naming strategy). Thus, after running the application and generating some metrics you can inspect the metrics in the TSD UI (localhost:4242 by default).
Example:
curl localhost:4242/api/query?start=1h-ago&m=max:counter.status.200.root
[
{
"metric": "counter.status.200.root",
"tags": {
"domain": "org.springframework.metrics",
"process": "b968a76"
},
"aggregateTags": [],
"dps": {
"1430492872": 2,
"1430492875": 6
}
}
]
To export metrics to Statsd, make sure first that you have added com.timgroup:java-statsd-client
as a dependency of your project (Spring Boot provides a dependency management for it). Then add a spring.metrics.export.statsd.host
value to your application.properties
file. Connections will be opened to port 8125
unless a spring.metrics.export.statsd.port
override is provided. You can use spring.metrics.export.statsd.prefix
if you want a custom prefix.
Alternatively, you can provide a @Bean
of type StatsdMetricWriter
and mark it @ExportMetricWriter
:
@Value("${spring.application.name:application}.${random.value:0000}")
private String prefix = "metrics";
@Bean
@ExportMetricWriter
MetricWriter metricWriter() {
return new StatsdMetricWriter(prefix, "localhost", 8125);
}
If you provide a @Bean
of type JmxMetricWriter
marked @ExportMetricWriter
the metrics are exported as MBeans to the local server (the MBeanExporter
is provided by Spring Boot JMX auto-configuration as long as it is switched on). Metrics can then be inspected, graphed, alerted etc. using any tool that understands JMX (e.g. JConsole or JVisualVM).
Example:
@Bean
@ExportMetricWriter
MetricWriter metricWriter(MBeanExporter exporter) {
return new JmxMetricWriter(exporter);
}
Each metric is exported as an individual MBean. The format for the ObjectNames
is given by an ObjectNamingStrategy
which can be injected into the JmxMetricWriter
(the default breaks up the metric name and tags the first two period-separated sections in a way that should make the metrics group nicely in JVisualVM or JConsole).
There is an AggregateMetricReader
that you can use to consolidate metrics from different physical sources. Sources for the same logical metric just need to publish them with a period-separated prefix, and the reader will aggregate (by truncating the metric names, and dropping the prefix). Counters are summed and everything else (i.e. gauges) take their most recent value.
This is very useful if multiple application instances are feeding to a central (e.g. Redis) repository and you want to display the results. Particularly recommended in conjunction with a MetricReaderPublicMetrics
for hooking up to the results to the “/metrics” endpoint.
Example:
@Autowired
private MetricExportProperties export;
@Bean
public PublicMetrics metricsAggregate() {
return new MetricReaderPublicMetrics(aggregatesMetricReader());
}
private MetricReader globalMetricsForAggregation() {
return new RedisMetricRepository(this.connectionFactory,
this.export.getRedis().getAggregatePrefix(), this.export.getRedis().getKey());
}
private MetricReader aggregatesMetricReader() {
AggregateMetricReader repository = new AggregateMetricReader(
globalMetricsForAggregation());
return repository;
}
Note | |
---|---|
The example above uses |
Note | |
---|---|
The |
A default MetricRegistry
Spring bean will be created when you declare a dependency to the io.dropwizard.metrics:metrics-core
library; you can also register you own @Bean
instance if you need customizations. Users of the Dropwizard ‘Metrics’ library will find that Spring Boot metrics are automatically published to com.codahale.metrics.MetricRegistry
. Metrics from the MetricRegistry
are also automatically exposed via the /metrics
endpoint
When Dropwizard metrics are in use, the default CounterService
and GaugeService
are replaced with a DropwizardMetricServices
, which is a wrapper around the MetricRegistry
(so you can @Autowired
one of those services and use it as normal). You can also create “special” Dropwizard metrics by prefixing your metric names with the appropriate type (i.e. timer.*
, histogram.*
for gauges, and meter.*
for counters).
If a MessageChannel
bean called metricsChannel
exists, then a MetricWriter
will be created that writes metrics to that channel. Each message sent to the channel will contain a Delta
or Metric
payload and have a metricName
header. The writer is automatically hooked up to an exporter (as for all writers), so all metric values will appear on the channel, and additional analysis or actions can be taken by subscribers (it’s up to you to provide the channel and any subscribers you need).