Details
-
New Feature
-
Resolution: Fixed
-
Major
-
None
Description
Modern centralized logging assumes you work with structured data and even beyond logger/message/exception you enhance data with MDC (like httpUrl / httpMethod or customerEmail, etc) and with keyValue of SLF4J fluent API.
Some tracing solution (OpenTelemetry/OpenTracing) might add additional dimension to data, via MDC, for example: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/logger-mdc-instrumentation.md
So:
- logging data gets lots of independent dimensions (via MDC or keyValue)
- plain text file ambiguity is no longer acceptable - who wants to deal with parsing multiline stack traces anymore?
One of the popular structured format is NDJSON.
http://ndjson.org/ tells that NDJSON is a file with valid JSON on each line.
Popular file log collectors supports NDJSON parsers:
- FluentBit https://docs.fluentbit.io/manual/pipeline/parsers/json
- FileBeat (from Elastic) https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html
Today to log in NDJSON format you need to relay on 3rd party layout providers, like https://github.com/logfellow/logstash-logback-encoder
I wish Logback has support for NDJSON out of the box.
https://github.com/qos-ch/logback-contrib/blob/master/json/classic/src/main/java/ch/qos/logback/contrib/json/classic/JsonLayout.java is not a solution:
- it lacks customization: possibility to rename attributes
- it lacks customization: possibility to flatten MDC
- it lacks support for Markers
- it lacks support for keyValue of fluent API
- it lacks support for nanoseconds
- it lacks auto-incremented sequence number - centralized logging solutions store events in random order, if a query returns several events within a millisecond (or nanosecond) - they will be presented in random order, not the original order!
Attachments
Issue Links
- relates to (out)
-
LOGBACK-212 Add a layout that produces JSON
- Open
- links to