The course is designed for anyone who wants to learn how to store large amounts of data using Elastic Stack. We learn how to store, search, and visualize logs using Kibana.
Gradually we will go through the whole process from installing individual components (Beats, Logstash, Elasticsearch, Kibana) through their use to cluster management.
On real-life examples, we try different storage architectures that we collect from different sources, enriching them with additional information and storing them into Elasticsearch. The participant gets acquainted with the Elasticsearch repository so that it can efficiently manage and scalable a large amount of data. In Kibana, we learn how to visualize logs, create dashboards, and understand the data more deeply.
Audience:
Application developers
System Administrators
IT Professionals
Prerequisites:
Basic knowledge of Elasticsearch, HTTP protocol, JSON format, general knowledge of database systems.
Course goals:
Participants will learn:
how to store different logs using Elastic Stack
how to design logging architecture for different uses
how to install and configure individual data processing components (Beats, Logstash, Elasticsearch, Kibana)
Elasticsearch technology more in depth, learn how to use storage tools, how to manage, scale and monitor
create dashboards and work with the Kibana tool
Course content:
Logging and Elasticsearch
Event log management
Visualization of logs
Examples of use of Elastic Stack in practice
Elasticsearch
Individual components of Elasticsearch
Basic work with cluster, nodes, indexes
Lab
Installation and Configuration
Install Elasticsearch
Configuration for logging
Sample configurations from practice
Cluster, nodes, indexes
Architecture
Tools for working with a cluster
Understanding and Configuring Different Types of Nodes
Working with indexes, setting indexes for storing logs
Lab
Data Collection
Beats
Collecting application logs from files
Collect metrics from the server
Possible architectures for data collection
Ingest Node
Lab
Logstash
Data collection from different sources
Configuration, deployment examples on real-world applications
Input, Filter, Output
Grok filter
Save to Elasticsearch
Filebeat
Lab
Kibana
Configuration, index patterns
Discovery of interface
Aggregation using Kibana
Creation of visualizations
Dashboards
Data search
Timelion - Time series work
Sample dashboards, examples from practice
Lab
Log monitoring
Track Log Changes
Detection of anomalies, notification
Elastic Stack
ElastAlert
Lab
Distributed search in logs
Search in Elasticsearch
Inverted index, relevance and more
Data analysis, mapping, dynamic templates
Lab
Index management
Capacity planning and configuration
Managing indexes, compression
Cache
Rollover, Shrink API
Lab
Cluster management
Restart (rolling, full-cluster)
Manage snapshots, repositories
Cluster upgrade (minor, major version)
Lab
Cluster monitoring
Elastic cluster monitoring setup
What ever, how to monitor
Monitoring tools
Lab