GitHub

Benchmark

Logging should have no significant impact on performance. This is why multiple benchmarks are part of the tinylog project. These are helpful for comparing different logging frameworks. Additionally, benchmarks are used during development for testing performance optimizations and detecting performance bugs at an early stage.

The benchmarks are based on JMH and hosted on GitHub. The first benchmark measures how many log entries per second can be output to a log file. Usually, there are also trace and debug log statements that should be ignored in production. How many log statements can be discarded per second is measured in the second benchmark.

To ensure statistically reliable data, each benchmark for each logging framework was executed sequentially in 10 forks, with 10 warm-up iterations and 10 relevant iterations. In the benchmark diagrams, you see the average value of the total 100 relevant iterations (10 forks × 10 iterations). The test machine was an Intel Core i7-4790 with 16 GB RAM and a 512 GB SSD from SanDisk (model X300s). Java Runtime Environment 9.0.0 was used to execute the benchmarks.

Writing Log Entries to a File

In this benchmark, all logging frameworks have to write info log entries to a log file, and {date:yyyy-MM-dd HH:mm:ss} [{thread}] {class}.{method}(): {message} is used as the format pattern.

First, the log entries are written synchronously and unbuffered (yellow bars). This supports every logging framework, and is the default for all of them. To prevent an application from being blocked by slow I/O operations, all logging frameworks (except for java.util.logging) can also write log entries asynchronously and buffered. The recommended configuration for asynchronous output for each logging framework has been benchmarked in the second step (blue bars).

Framework Processed Log Entries per Second
tinylog 2.0-SNAPSHOT
 
75,273
with writing thread
 
282,601
tinylog 1.3.5
 
77,399
with writing thread
 
247,334
Log4j 2.11.0
 
33,595
with async logger
 
107,188
Log4j 1.2.17
 
29,801
with async appender
 
49,573
Logback 1.2.3
 
26,776
with async appender
 
44,207
java.util.logging
 
51,434

Discarding Log Entries

Debug and trace log entries are helpful for developers in their development environment, but are not usually output in the production environment. The second benchmark measures how many log entries (which are to be discarded) can be processed per second by the logging frameworks.

Framework Processed Log Entries per Second
tinylog 2.0-SNAPSHOT
 
3,934,674,600
with writing thread
 
3,924,682,099
tinylog 1.3.5
 
1,114,731,827
with writing thread
 
1,112,835,347
Log4j 2.11.0
 
582,503,870
with async logger
 
581,429,395
Log4j 1.2.17
 
669,507,524
with async appender
 
669,585,952
Logback 1.2.3
 
661,902,703
with async appender
 
661,317,894
java.util.logging
 
1,104,430,545

tinylog 2 loads the visibility of the severity levels while class loading, and stores them as final boolean values. Each logging method first checks against this boolean value whether a severity level can be visible or not. This allows the JVM to easily identify which logging methods will never output anything at runtime and remove the calls. For this reason, the performance of tinylog 2 for disabled severity levels is identical to an invocation of an empty method that does nothing.

No-Op Benchmark Invocations per Second
Empty Method
 
3,931,872,534