ELK Monitoring – Part 2 – Setup Elasticsearch
In this blog article, we will set up and run an elasticsearch node in our local machine. We will also see some important configurations.
For a better understanding, it is recommended to go through the blog articles on ELK topic in order from 1-6
https://myknowtech.com/tag/elk
What is elasticsearch?
– Open source search and analytics engine
– Build on Apache lucene and uses simple REST APIs for the communication
– Mainly used for searching, analytics and monitoring
What is elasticsearch index?
It is a collection of documents that are related to each other.
Note: It is somewhat related to indexing in data tables.
All the incoming data gets indexed and stored as documents inside the index. If you think of a log file, each entry line in a log file is stored as a document in elasticsearch. Each document contains a list of fields that provide more details
How to set up elasticsearch in Windows?
Step 1: Download the elasticsearch binaries
https://www.elastic.co/downloads/elasticsearch
Step 2: Unzip and install the binaries on the local machine
Step 3: Setup some important configurations
All the configuration files will be found in the config folder.
The main three configuration files are elasticsearch.yml, jvm.options and log4j2.properties
a) Cluster name and node name
Here you can specify the name for the elastic cluster and the current node.
The configuration stays in the elasticsearch.yml
By default, the cluster name is elasticsearch. You can always specify your own cluster name
Note: Do not use the same cluster name across environments
Node name – It identifies the elasticsearch instance. By default, it uses the hostname, you can always specify your own node name as a good practice.
b) Network settings
Here you can specify the hostname and port name
By default, elasticsearch binds to localhost.
For this tutorial, since I hosted the elastic server on my local machine, I am not going to update any of the configurations. If needed you can update the configurations here.
c) Paths to logs and data
By default, the log and data directories get created in your elastic home folder.
Note: data folder will be created once you start the server.
It is good practice to change the data and log directories. The reason is if you upgrade the instance then there is a high chance of these files getting overridden.
You can change it here. For now, I created a folder – elasticdata and included the path.
d) VM heap size
The configuration for JVM heap size resides in a separate file – jvm.options
By default., the minimum and maximum heap size if 1 GB. You can update if you want.
We are done with important configurations
Step 4: Start the elasticsearch
The Windows batch file to start the elastic server stays in the bin folder.
Open the Windows PowerShell and switch to elastic home directory.
Now execute the below command – .binelasticsearch.bat
In the logs, you will see it loads all the plugins.
You should see the node started message.
But seems the node name is still the host machine name!
Step 5: Verify the elasticsearch server status
Click on http://localhost:9200
It seems the configurations are not updated. Because I didn’t save the file.
Now save the configuration file and start the server again.
You see the node name and cluster name are updated accordingly.
Final verification, check if the data directory points to new folder.
All looks good.
We have successfully set up Elastic server in our local machine.
In the next articles, we will configure Kibana and use REST API to create index and documents in the elastic server.
