Logbook

Dependencies

Java 8 (for Spring 6 / Spring Boot 3 and JAX-RS 3.x, Java 17 is required)
Any build tool using Maven Central, or direct download
Servlet Container (optional)
Apache HTTP Client 4.x or 5.x (optional)
JAX-RS 3.x (aka Jakarta RESTful Web Services) Client and Server (optional)
JAX-RS 2.x Client and Server (optional)
Netty 4.x (optional)
OkHttp 2.x or 3.x (optional)
Spring 6.x or Spring 5.x (optional, see instructions below)
Spring Boot 3.x or 2.x (optional)
Ktor (optional)
logstash-logback-encoder 5.x (optional)

Strategy

Logbook used to have a very rigid strategy how to do request/response logging:

Requests/responses are logged separately
Requests/responses are logged soon as possible
Requests/responses are logged as a pair or not logged at all
(i.e. no partial logging of traffic)

Some of those restrictions could be mitigated with custom HttpLogWriter implementations, but they were never ideal.

Starting with version 2.0 Logbook now comes with a Strategy pattern at its core. Make sure you read the documentation of the Strategy interface to understand the implications.

Logbook comes with some built-in strategies:

Attribute Extractor

Starting with version 3.4.0, Logbook is equipped with a feature called Attribute Extractor. Attributes are basically a list of key/value pairs that can be extracted from request and/or response, and logged with them. The idea was sprouted from issue 381, where a feature was requested to extract the subject claim from JWT tokens in the authorization header.

The AttributeExtractor interface has two extract methods: One that can extract attributes from the request only, and one that has both request and response at its avail. The both return an instance of the HttpAttributes class, which is basically a fancy Map<String, Object>. Notice that since the map values are of type Object, they should have a proper toString() method in order for them to appear in the logs in a meaningful way. Alternatively, log formatters can work around this by implementing their own serialization logic. For instance, the built-in log formatter JsonHttpLogFormatter uses ObjectMapper to serialize the values.

Here is an example:

final class OriginExtractor implements AttributeExtractor {

  @Override
  public HttpAttributes extract(final HttpRequest request) {
    return HttpAttributes.of("origin", request.getOrigin());
  }
    
}

Logbook must then be created by registering this attribute extractor:

final Logbook logbook = Logbook.builder()
        .attributeExtractor(new OriginExtractor())
        .build();

This will result in request logs to include something like:

"attributes":{"origin":"LOCAL"}

For more advanced examples, look at the JwtFirstMatchingClaimExtractor and JwtAllMatchingClaimsExtractor classes. The former extracts the first claim matching a list of claim names from the request JWT token. The latter extracts all claims matching a list of claim names from the request JWT token.

If you require to incorporate multiple AttributeExtractors, you can use the class CompositeAttributeExtractor:

final List<AttributeExtractor> extractors = List.of(
    extractor1,
    extractor2,
    extractor3
);

final Logbook logbook = Logbook.builder()
        .attributeExtractor(new CompositeAttributeExtractor(extractors))
        .build();

Phases

Logbook works in several different phases:

Conditional,
Filtering,
Formatting and
Writing

Each phase is represented by one or more interfaces that can be used for customization. Every phase has a sensible default.

Conditional

Logging HTTP messages and including their bodies is a rather expensive task, so it makes a lot of sense to disable logging for certain requests. A common use case would be to ignore health check requests from a load balancer, or any request to management endpoints typically issued by developers.

Defining a condition is as easy as writing a special Predicate that decides whether a request (and its corresponding response) should be logged or not. Alternatively you can use and combine predefined predicates:

Logbook logbook = Logbook.builder()
    .condition(exclude(
        requestTo("/health"),
        requestTo("/admin/**"),
        contentType("application/octet-stream"),
        header("X-Secret", newHashSet("1", "true")::contains)))
    .build();

Exclusion patterns, e.g. /admin/**, are loosely following Ant’s style of path patterns without taking the the query string of the URL into consideration.

JsonPath body filtering (experimental)

You can apply JSON Path filtering to JSON bodies. Here are some examples:

import static org.zalando.logbook.json.JsonPathBodyFilters.jsonPath;
import static java.util.regex.Pattern.compile;

Logbook logbook = Logbook.builder()
        .bodyFilter(jsonPath("$.password").delete())
        .bodyFilter(jsonPath("$.active").replace("unknown"))
        .bodyFilter(jsonPath("$.address").replace("X"))
        .bodyFilter(jsonPath("$.name").replace(compile("^(\\w).+"), "$1."))
        .bodyFilter(jsonPath("$.friends.*.name").replace(compile("^(\\w).+"), "$1."))
        .bodyFilter(jsonPath("$.grades.*").replace(1.0))
        .build();

Take a look at the following example, before and after filtering was applied:

Before

```json { "id": 1, "name": "Alice", "password": "s3cr3t", "active": true, "address": "Anhalter Straße 17 13, 67278 Bockenheim an der Weinstraße", "friends": [ { "id": 2, "name": "Bob" }, { "id": 3, "name": "Charlie" } ], "grades": { "Math": 1.0, "English": 2.2, "Science": 1.9, "PE": 4.0 } } ```

After

```json { "id": 1, "name": "Alice", "active": "unknown", "address": "XXX", "friends": [ { "id": 2, "name": "B." }, { "id": 3, "name": "C." } ], "grades": { "Math": 1.0, "English": 1.0, "Science": 1.0, "PE": 1.0 } } ```

Correlation

Logbook uses a correlation id to correlate requests and responses. This allows match-related requests and responses that would usually be located in different places in the log file.

If the default implementation of the correlation id is insufficient for your use case, you may provide a custom implementation:

Logbook logbook = Logbook.builder()
    .correlationId(new CustomCorrelationId())
    .build();

Formatting

Formatting defines how requests and responses will be transformed to strings basically. Formatters do not specify where requests and responses are logged to — writers do that work.

Logbook comes with two different default formatters: HTTP and JSON.

HTTP

HTTP is the default formatting style, provided by the DefaultHttpLogFormatter. It is primarily designed to be used for local development and debugging, not for production use. This is because it’s not as readily machine-readable as JSON.

Request

Incoming Request: 2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b
GET http://example.org/test HTTP/1.1
Accept: application/json
Host: localhost
Content-Type: text/plain

Hello world!

Response

Outgoing Response: 2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b
Duration: 25 ms
HTTP/1.1 200
Content-Type: application/json

{"value":"Hello world!"}

JSON

JSON is an alternative formatting style, provided by the JsonHttpLogFormatter. Unlike HTTP, it is primarily designed for production use — parsers and log consumers can easily consume it.

Requires the following dependency:

<dependency>
  <groupId>org.zalando</groupId>
  <artifactId>logbook-json</artifactId>
</dependency>

Request

{
  "origin": "remote",
  "type": "request",
  "correlation": "2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b",
  "protocol": "HTTP/1.1",
  "sender": "127.0.0.1",
  "method": "GET",
  "uri": "http://example.org/test",
  "host": "example.org",
  "path": "/test",
  "scheme": "http",
  "port": null,
  "headers": {
    "Accept": ["application/json"],
    "Content-Type": ["text/plain"]
  },
  "body": "Hello world!"
}

Response

{
  "origin": "local",
  "type": "response",
  "correlation": "2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b",
  "duration": 25,
  "protocol": "HTTP/1.1",
  "status": 200,
  "headers": {
    "Content-Type": ["text/plain"]
  },
  "body": "Hello world!"
}

Note: Bodies of type application/json (and application/*+json) will be inlined into the resulting JSON tree. I.e., a JSON response body will not be escaped and represented as a string:

{
  "origin": "local",
  "type": "response",
  "correlation": "2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b",
  "duration": 25,
  "protocol": "HTTP/1.1",
  "status": 200,
  "headers": {
    "Content-Type": ["application/json"]
  },
  "body": {
    "greeting": "Hello, world!"
  }
}

cURL

cURL is an alternative formatting style, provided by the CurlHttpLogFormatter which will render requests as executable cURL commands. Unlike JSON, it is primarily designed for humans.

Request

curl -v -X GET 'http://localhost/test' -H 'Accept: application/json'

Response

See HTTP or provide own fallback for responses:

new CurlHttpLogFormatter(new JsonHttpLogFormatter());

Splunk

Splunk is an alternative formatting style, provided by the SplunkHttpLogFormatter which will render requests and response as key-value pairs.

Request

origin=remote type=request correlation=2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b protocol=HTTP/1.1 sender=127.0.0.1 method=POST uri=http://example.org/test host=example.org scheme=http port=null path=/test headers={Accept=[application/json], Content-Type=[text/plain]} body=Hello world!

Response

origin=local type=response correlation=2d66e4bc-9a0d-11e5-a84c-1f39510f0d6b duration=25 protocol=HTTP/1.1 status=200 headers={Content-Type=[text/plain]} body=Hello world!

Writing

Writing defines where formatted requests and responses are written to. Logbook comes with three implementations: Logger, Stream and Chunking.

Logger

By default, requests and responses are logged with an slf4j logger that uses the org.zalando.logbook.Logbook category and the log level trace. This can be customized:

Logbook logbook = Logbook.builder()
    .sink(new DefaultSink(
            new DefaultHttpLogFormatter(),
            new DefaultHttpLogWriter()
    ))
    .build();

Stream

An alternative implementation is to log requests and responses to a PrintStream, e.g. System.out or System.err. This is usually a bad choice for running in production, but can sometimes be useful for short-term local development and/or investigation.

Logbook logbook = Logbook.builder()
    .sink(new DefaultSink(
            new DefaultHttpLogFormatter(),
            new StreamHttpLogWriter(System.err)
    ))
    .build();

Chunking

The ChunkingSink will split long messages into smaller chunks and will write them individually while delegating to another sink:

Logbook logbook = Logbook.builder()
    .sink(new ChunkingSink(sink, 1000))
    .build();

Sink

The combination of HttpLogFormatter and HttpLogWriter suits most use cases well, but it has limitations. Implementing the Sink interface directly allows for more sophisticated use cases, e.g. writing requests/responses to a structured persistent storage like a database.

Multiple sinks can be combined into one using the CompositeSink.

Form Requests

As of Logbook 1.5.0, you can now specify one of three strategies that define how Logbook deals with this situation by using the logbook.servlet.form-request system property:

Value	Pros	Cons
`body` (default)	Body is logged	Downstream code can *not use `getParameter()`**
`parameter`	Body is logged (but it’s reconstructed from parameters)	Downstream code can not use `getInputStream()`
`off`	Downstream code can decide whether to use `getInputStream()` or `getParameter*()`	Body is not logged

Security

Secure applications usually need a slightly different setup. You should generally avoid logging unauthorized requests, especially the body, because it quickly allows attackers to flood your logfile — and, consequently, your precious disk space. Assuming that your application handles authorization inside another filter, you have two choices:

Don’t log unauthorized requests
Log unauthorized requests without the request body

You can easily achieve the former setup by placing the LogbookFilter after your security filter. The latter is a little bit more sophisticated. You’ll need two LogbookFilter instances — one before your security filter, and one after it:

context.addFilter("SecureLogbookFilter", new SecureLogbookFilter(logbook))
    .addMappingForUrlPatterns(EnumSet.of(REQUEST, ASYNC), true, "/*");
context.addFilter("securityFilter", new SecurityFilter())
    .addMappingForUrlPatterns(EnumSet.of(REQUEST), true, "/*");
context.addFilter("LogbookFilter", new LogbookFilter(logbook))
    .addMappingForUrlPatterns(EnumSet.of(REQUEST, ASYNC), true, "/*");

The first logbook filter will log unauthorized requests only. The second filter will log authorized requests, as always.

HTTP Client

The logbook-httpclient module contains both an HttpRequestInterceptor and an HttpResponseInterceptor to use with the HttpClient:

CloseableHttpClient client = HttpClientBuilder.create()
        .addInterceptorFirst(new LogbookHttpRequestInterceptor(logbook))
        .addInterceptorFirst(new LogbookHttpResponseInterceptor())
        .build();

Since the LogbookHttpResponseInterceptor is incompatible with the HttpAsyncClient there is another way to log responses:

CloseableHttpAsyncClient client = HttpAsyncClientBuilder.create()
        .addInterceptorFirst(new LogbookHttpRequestInterceptor(logbook))
        .build();
        
// and then wrap your response consumer
client.execute(producer, new LogbookHttpAsyncResponseConsumer<>(consumer), callback)

HTTP Client 5

The logbook-httpclient5 module contains an ExecHandler to use with the HttpClient:

CloseableHttpClient client = HttpClientBuilder.create()
        .addExecInterceptorFirst("Logbook", new LogbookHttpExecHandler(logbook))
        .build();

The Handler should be added first, such that a compression is performed after logging and decompression is performed before logging.

To avoid a breaking change, there is also an HttpRequestInterceptor and an HttpResponseInterceptor to use with the HttpClient, which works fine as long as compression (or other ExecHandlers) is not used:

CloseableHttpClient client = HttpClientBuilder.create()
        .addRequestInterceptorFirst(new LogbookHttpRequestInterceptor(logbook))
        .addResponseInterceptorFirst(new LogbookHttpResponseInterceptor())
        .build();

Since the LogbookHttpResponseInterceptor is incompatible with the HttpAsyncClient there is another way to log responses:

CloseableHttpAsyncClient client = HttpAsyncClientBuilder.create()
        .addRequestInterceptorFirst(new LogbookHttpRequestInterceptor(logbook))
        .build();
        
// and then wrap your response consumer
client.execute(producer, new LogbookHttpAsyncResponseConsumer<>(consumer), callback)

Netty

The logbook-netty module contains:

A LogbookClientHandler to be used with an HttpClient:

HttpClient httpClient =
        HttpClient.create()
                .doOnConnected(
                        (connection -> connection.addHandlerLast(new LogbookClientHandler(logbook)))
                );

A LogbookServerHandler for use used with an HttpServer:

HttpServer httpServer =
        HttpServer.create()
                .doOnConnection(
                        connection -> connection.addHandlerLast(new LogbookServerHandler(logbook))
                );

Spring WebFlux

Users of Spring WebFlux can pick any of the following options:

Programmatically create a NettyWebServer (passing an HttpServer)
Register a custom NettyServerCustomizer
Programmatically create a ReactorClientHttpConnector (passing an HttpClient)
Register a custom WebClientCustomizer
Use separate connector-independent module logbook-spring-webflux

Micronaut

Users of Micronaut can follow the official docs on how to integrate Logbook with Micronaut.

:warning: Even though Quarkus and Vert.x use Netty under the hood, unfortunately neither of them allows accessing or customizing it (yet).

Ktor

The logbook-ktor-client module contains:

A LogbookClient to be used with an HttpClient:

private val client = HttpClient(CIO) {
    install(LogbookClient) {
        logbook = logbook
    }
}

The logbook-ktor-server module contains:

A LogbookServer to be used with an Application:

private val server = embeddedServer(CIO) {
    install(LogbookServer) {
        logbook = logbook
    }
}

Alternatively, you can use logbook-ktor, which ships both logbook-ktor-client and logbook-ktor-server modules.

Spring

The logbook-spring module contains a ClientHttpRequestInterceptor to use with RestTemplate:

    LogbookClientHttpRequestInterceptor interceptor = new LogbookClientHttpRequestInterceptor(logbook);
    RestTemplate restTemplate = new RestTemplate();
    restTemplate.getInterceptors().add(interceptor);

Spring Boot Starter

Logbook comes with a convenient auto configuration for Spring Boot users. It sets up all of the following parts automatically with sensible defaults:

Servlet filter
Second Servlet filter for unauthorized requests (if Spring Security is detected)
Header-/Parameter-/Body-Filters
HTTP-/JSON-style formatter
Logging writer

Instead of declaring a dependency to logbook-core declare one to the Spring Boot Starter:

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-spring-boot-starter</artifactId>
    <version>${logbook.version}</version>
</dependency>

Every bean can be overridden and customized if needed, e.g. like this:

@Bean
public BodyFilter bodyFilter() {
    return merge(
            defaultValue(), 
            replaceJsonStringProperty(singleton("secret"), "XXX"));
}

Please refer to LogbookAutoConfiguration or the following table to see a list of possible integration points:

Type	Name	Default
`FilterRegistrationBean`	`secureLogbookFilter`	Based on `LogbookFilter`
`FilterRegistrationBean`	`logbookFilter`	Based on `LogbookFilter`
`Logbook`		Based on condition, filters, formatter and writer
`Predicate<HttpRequest>`	`requestCondition`	No filter; is later combined with `logbook.exclude` and `logbook.exclude`
`HeaderFilter`		Based on `logbook.obfuscate.headers`
`PathFilter`		Based on `logbook.obfuscate.paths`
`QueryFilter`		Based on `logbook.obfuscate.parameters`
`BodyFilter`		`BodyFilters.defaultValue()`, see filtering
`RequestFilter`		`RequestFilters.defaultValue()`, see filtering
`ResponseFilter`		`ResponseFilters.defaultValue()`, see filtering
`Strategy`		`DefaultStrategy`
`AttributeExtractor`		`NoOpAttributeExtractor`
`Sink`		`DefaultSink`
`HttpLogFormatter`		`JsonHttpLogFormatter`
`HttpLogWriter`		`DefaultHttpLogWriter`

Multiple filters are merged into one.

Autoconfigured beans from `logbook-spring`

Some classes from logbook-spring are included in the auto configuration.

You can autowire LogbookClientHttpRequestInterceptor with code like:

private final RestTemplate restTemplate;
MyClient(RestTemplateBuilder builder, LogbookClientHttpRequestInterceptor interceptor){
  this.restTemplate = builder
    .additionalInterceptors(interceptor)
    .build();
}

Configuration

The following tables show the available configuration (sorted alphabetically):

Configuration	Description	Default
`logbook.attribute-extractors`	List of AttributeExtractors, including configurations such as `type` (currently `JwtFirstMatchingClaimExtractor` or `JwtAllMatchingClaimsExtractor`), `claim-names` and `claim-key`.	`[]`
`logbook.filter.enabled`	Enable the `LogbookFilter`	`true`
`logbook.filter.form-request-mode`	Determines how form requests are handled	`body`
`logbook.filters.body.default-enabled`	Enables/disables default body filters that are collected by java.util.ServiceLoader	`true`
`logbook.format.style`	Formatting style (`http`, `json`, `curl` or `splunk`)	`json`
`logbook.httpclient.decompress-response`	Enables/disables additional decompression process for HttpClient with gzip encoded body (to logging purposes only). This means extra decompression and possible performance impact.	`false` (disabled)
`logbook.minimum-status`	Minimum status to enable logging (`status-at-least` and `body-only-if-status-at-least`)	`400`
`logbook.obfuscate.headers`	List of header names that need obfuscation	`[Authorization]`
`logbook.obfuscate.json-body-fields`	List of JSON body fields to be obfuscated	`[]`
`logbook.obfuscate.parameters`	List of parameter names that need obfuscation	`[access_token]`
`logbook.obfuscate.paths`	List of paths that need obfuscation. Check Filtering for syntax.	`[]`
`logbook.obfuscate.replacement`	A value to be used instead of an obfuscated one	`XXX`
`logbook.predicate.include`	Include only certain paths and methods (if defined)	`[]`
`logbook.predicate.exclude`	Exclude certain paths and methods (overrides `logbook.predicate.include`)	`[]`
`logbook.secure-filter.enabled`	Enable the `SecureLogbookFilter`	`true`
`logbook.strategy`	Strategy (`default`, `status-at-least`, `body-only-if-status-at-least`, `without-body`)	`default`
`logbook.write.chunk-size`	Splits log lines into smaller chunks of size up-to `chunk-size`.	`0` (disabled)
`logbook.write.max-body-size`	Truncates the body up to `max-body-size` and appends `...`. :warning: Logbook will still buffer the full body, if the request is eligible for logging, regardless of the `logbook.write.max-body-size` value	`-1` (disabled)

Example configuration

logbook:
  predicate:
    include:
      - path: /api/**
        methods: 
         - GET
         - POST
      - path: /actuator/**
    exclude:
      - path: /actuator/health
      - path: /api/admin/**
        methods: 
         - POST
  filter.enabled: true
  secure-filter.enabled: true
  format.style: http
  strategy: body-only-if-status-at-least
  minimum-status: 400
  obfuscate:
    headers:
      - Authorization
      - X-Secret
    parameters:
      - access_token
      - password
  write:
    chunk-size: 1000
  attribute-extractors:
    - type: JwtFirstMatchingClaimExtractor
      claim-names: [ "sub", "subject" ]
      claim-key: Principal
    - type: JwtAllMatchingClaimsExtractor
      claim-names: [ "sub", "iat" ]

logstash-logback-encoder

For basic Logback configuraton

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

configure Logbook with a LogstashLogbackSink

HttpLogFormatter formatter = new JsonHttpLogFormatter();
LogstashLogbackSink sink = new LogstashLogbackSink(formatter);

for outputs like

{
  "@timestamp" : "2019-03-08T09:37:46.239+01:00",
  "@version" : "1",
  "message" : "GET http://localhost/test?limit=1",
  "logger_name" : "org.zalando.logbook.Logbook",
  "thread_name" : "main",
  "level" : "TRACE",
  "level_value" : 5000,
  "http" : {
     // logbook request/response contents
  }
}

Customizing default Logging Level

You have the flexibility to customize the default logging level by initializing LogstashLogbackSink with a specific level. For instance:

LogstashLogbackSink sink = new LogstashLogbackSink(formatter, Level.INFO);

Getting Help with Logbook

If you have questions, concerns, bug reports, etc., please file an issue in this repository’s Issue Tracker.

Dependencies

Strategy

Attribute Extractor

Phases

Conditional

JsonPath body filtering (experimental)

Correlation

Formatting

HTTP

Request

Response

JSON

Request

Response

cURL

Request

Response

Splunk

Request

Response

Writing

Logger

Stream

Chunking

Sink

Form Requests

Security

HTTP Client

HTTP Client 5

Netty

Spring WebFlux

Micronaut

Ktor

Spring

Spring Boot Starter

Autoconfigured beans from logbook-spring

Configuration

Example configuration

logstash-logback-encoder

Customizing default Logging Level

Getting Help with Logbook

Articles

Autoconfigured beans from `logbook-spring`