Logstash vs Beats

Both Logstash and Beats are part of ELK Stack so you’ll need Elasticsearch and Kibana to store the logs (Elasticsearch) and to view them (Kibana).

Running ELK Stack Locally

We use deviantony/docker-elk for development. Note: Docker images backing this stack include X-Pack with paid features enabled by default (see How to disable paid features to disable them). The trial license is valid for 30 days. After this license expires, you can continue using the free features seamlessly, without losing any data.

Clone deviantony/docker-elk then docker-compose up.

cd ~/project
git clone https://github.com/deviantony/docker-elk.git
cd docker-elk
docker-compose up

The stack is pre-configured with the following privileged bootstrap user:

• user: elastic
• password: changeme

Endpoints

Service	Endpoint URL
Kibana – plain HTTP	http://localhost:5601/
Elasticsearch REST API – plain HTTP	http://localhost:9200/
Logstash REST API – plain HTTP	http://localhost:9600/

Web Logs Dashboard in Kibana

When to use Beats over Logstash?

As described in https://www.elastic.co/guide/en/beats/filebeat/current/diff-logstash-beats.html:

• Beats are lightweight data shippers that you install as agents on your servers to send specific types of operational data to Elasticsearch. Beats have a small footprint and use fewer system resources than Logstash.
• Logstash has a larger footprint, but provides a broad array of input, filter, and output plugins for collecting, enriching, and transforming data from a variety of sources.

In short, if you just want to send simple stuff and does not need any transformation, use Beats. If Beats doesn’t fit the job, or you need to transform the data before sending to Elasticsearch, “upgrade” to Logstash.