GitHub

Benchmark

Logging should have no significant impact on performance. This is why multiple benchmarks are part of the tinylog project. These are helpful for comparing different logging frameworks. Additionally, benchmarks are used during development for testing performance optimizations and detecting performance bugs at an early stage.

The benchmarks are based on JMH and hosted on GitHub. The first benchmark measures how many log entries per second can be output to a log file. Usually, there are also trace and debug log statements that should be ignored in production. How many log statements can be discarded per second is measured in the second benchmark.

To ensure statistically reliable data, each benchmark for each logging framework was executed sequentially in 10 forks, with 10 warm-up iterations and 10 relevant iterations. In the benchmark diagrams, you see the average value of the total 100 relevant iterations (10 forks × 10 iterations). The test machine was an Intel Core i7-4790 with 16 GB RAM and a 512 GB SSD from SanDisk (model X300s). Java Runtime Environment 11.0.6 was used to execute the benchmarks.

Writing Log Entries to a File

In this benchmark, all logging frameworks have to write info log entries to a log file, and {date:yyyy-MM-dd HH:mm:ss} [{thread}] {class}.{method}(): {message} is used as the format pattern.

First, the log entries are written synchronously and unbuffered (yellow bars). This supports every logging framework, and is the default for all of them. To prevent an application from being blocked by slow I/O operations, all logging frameworks (except for java.util.logging) can also write log entries asynchronously and buffered. The recommended configuration for asynchronous output for each logging framework has been benchmarked in the second step (blue bars).

Framework Processed Log Entries per Second
tinylog 2.1.0
 
85,115
with writing thread
 
427,774
tinylog 1.3.6
 
89,945
with writing thread
 
380,230
Log4j 2.12.1
 
46,490
with async logger
 
94,692
Log4j 1.2.17
 
43,703
with async appender
 
80,111
Logback 1.2.3
 
42,894
with async appender
 
72,881
java.util.logging
 
55,892

Discarding Log Entries

Debug and trace log entries are helpful for developers in their development environment, but are not usually output in the production environment. The second benchmark measures how many log entries (which are to be discarded) can be processed per second by the logging frameworks.

Framework Processed Log Entries per Second
tinylog 2.1.0
 
2,443,854,755
with writing thread
 
2,344,761,131
tinylog 1.3.6
 
935,020,822
with writing thread
 
933,573,859
Log4j 2.12.1
 
512,255,316
with async logger
 
516,118,831
Log4j 1.2.17
 
554,779,540
with async appender
 
554,718,392
Logback 1.2.3
 
571,848,193
with async appender
 
571,816,430
java.util.logging
 
926,042,812

tinylog 2 loads the visibility of the severity levels while class loading, and stores them as final boolean values. Each logging method first checks against this boolean value whether a severity level can be visible or not. This allows the JVM to easily identify which logging methods will never output anything at runtime and remove the calls. For this reason, the performance of tinylog 2 for disabled severity levels is identical to an invocation of an empty method that does nothing.

No-Op Benchmark Invocations per Second
Empty Method
 
2,381,223,621