log ingestor system that can efficiently handle vast volumes of log data, and offer a simple interface for querying this data using full-text search or specific field filters.
java 21 docker desktop Intellij (Recommended) maven
- In the parent directory, run the below commands
- docker-compose up setup (Take several minutes)
- docker-compose up -d (wait till both kibana and elastic search will start)
- ./mvnw spring-boot:run
Go to http://localhost:5601 and enter user credentials.
- username=elastic
- password=pass1234
Create index pattern in kibana by navigating to http://localhost:5601/app/management/kibana/dataViews to create the view for the index by passing parameters. name=ingest-log pattern=ingest-logs*
The service will run on 3000 port. Call the post API by sending log data in request body.
Example request:
curl --location 'http://localhost:3000'
--header 'Content-Type: application/json'
--data '{
"level": "error",
"message": "Failed to connect to DB",
"resourceId": "server-1234",
"timestamp": "2023-02-18T12:00:00Z",
"traceId": "abc123",
"spanId": "span-xyz123",
"commit": "342ff",
"metadata": {
"parentResourceId": "server-5678"
}
}'
Go to kibana to discover http://localhost:5601/app/discover# and change the index patterns or data view to ingest-log
You can query the data using filters in the discover page.
For further reference, please consider the following sections:
- Official Apache Maven documentation
- Spring Boot Maven Plugin Reference Guide
- Create an OCI image
- Spring Web
- Spring Data Elasticsearch (Access+Driver)
- Kibana Discover
The following guides illustrate how to use some features concretely: