Setup ELK Stack to capture Spring Boot logs

ELK stack is one of the good stack available to organize and visualize Spring Boot application logs. So, what’s ELK Stack?

“ELK” is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline and Kibana lets users visualize data with charts and graphs in Elasticsearch. Please refer ELK official documentation for more details.

In this article, we are setting up ELK stack to capture Spring Boot application logs.

Technologies Used

1. Java 8/11
2. Spring Boot 2.x
3. Elastic Search 6.6.1
4. LogStash 6.6.1
5. Kibana 6.6.1

Setup Instructions

1. Download Elastic Search and unzip it
2. Start Elastic search with the following command and go to URL http://localhost:9200

$ bin/elasticsearch

3. Download LogStash and unzip it

4. Create logstash-elk.conf file on a logstash home directory with the following content and change the log file location and index name based on your settings

input {
    file {
        path => "/Users/<username>/ELK-Stack/logback/*.log"
        codec => "json"
        type => "logback"
    }
}
 
output {
    if [type]=="logback" {
         elasticsearch {
             hosts => [ "localhost:9200" ]
             index => "logstash"
        }
    }
}

5. Start the logstash with the command

$ bin/logstash -f logstash-elk.conf

6. Download Kibana and unzip it, run it with the following command

$ bin/kibana

7. Go to http://localhost:5601 to see Kibana UI

8. Go to Spring Boot application.yml file and add logging file location properties

# Logging
logging:
file: /Users/<username>/ELK-Stack/logback/springsecuritydata.log

9. Restart the Spring Boot application to capture logs in the newly created file springsecuritydata.log

10. Go to Kibana UI -> Management -> Create Index pattern -> Type index name (You should see the index name below)

11. Go to Kibana UI -> Dashboard to see Spring Boot log data, modify the filter to see data from past