Elasticsearch and Kibana are essential tools for data search, analysis, and visualization, forming a core part of the Elastic Stack. Elasticsearch is a search and analytics engine, while Kibana is a visualization tool that lets you view and explore data stored in Elasticsearch. If you're using Ubuntu and want to set up Elasticsearch and Kibana, this guide will walk you through the installation and initial configuration of both tools
Prerequisites
Before diving in, ensure that you have the following:
- Ubuntu 18.04 or later: This guide focuses on Ubuntu, but the process is similar for Debian-based systems.
- Root or Sudo Access: Administrative privileges are required to install packages.
- Java: Elasticsearch requires Java to run. This guide will cover the installation if it’s not already present on your system.
Step 1: Update and Install Java
First, update your package lists to ensure you’re installing the latest versions:
sudo apt update
sudo apt upgrade
Check if Java is already installed by running:
java -version
If Java is not installed, install the OpenJDK package, which is compatible with Elasticsearch
sudo apt install openjdk-11-jdk -y
To confirm Java installation, verify the version again
java -version
Step 2: Install Elasticsearch
Elasticsearch packages need to be verified using a GPG key. Download and add the GPG key to your system:
Add the Elasticsearch GPG Key
Elasticsearch packages need to be verified using a GPG key. Download and add the GPG key to your system:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Add the Elasticsearch Repository
To make Elasticsearch available as a package, add the Elastic repository
sudo sh -c 'echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" > /etc/apt/sources.list.d/elastic-8.x.list'
Install Elasticsearch
Update the package list again to include Elasticsearch and then install it:
sudo apt update
sudo apt install elasticsearch -y
Configure Elasticsearch
Before starting Elasticsearch, it’s a good idea to set some configuration options. Open the Elasticsearch configuration file:
sudo nano /etc/elasticsearch/elasticsearch.yml
Modify the following lines:
-
Set the network host to
localhostto restrict external access for security:network.host: localhost - Optional: Adjust the memory settings in
/etc/elasticsearch/jvm.optionsto suit your system’s RAM.
Start and Enable Elasticsearch
Now that Elasticsearch is configured, start and enable it to run on boot:
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch
Verify Elasticsearch Installation
To confirm that Elasticsearch is running, use the curl command to make a request to localhost on port 9200:
curl -X GET "localhost:9200/"
You should see a JSON response with details about your Elasticsearch instance, confirming the successful installation.
Step 3: Install Kibana
With Elasticsearch up and running, you can now install Kibana, the visualization layer for exploring your Elasticsearch data.
Install Kibana
Since the Elastic repository is already added, install Kibana by running:
sudo apt install kibana -y
Configure Kibana
To configure Kibana, open its configuration file:
sudo nano /etc/kibana/kibana.yml
Modify the following lines:
-
Set the server host to
localhost:server.host: "localhost" - Optional: Configure the Elasticsearch URL that Kibana should connect to
elasticsearch.hosts: ["http://localhost:9200"]
Start and Enable Kibana
Start Kibana and set it to launch on boot:
sudo systemctl start kibana
sudo systemctl enable kibana
Access Kibana
By default, Kibana runs on port 5601. Open a web browser and go to:
http://localhost:5601
If you’re installing on a remote server, make sure to open port 5601 on the firewall and replace localhost with the server's IP address in the URL. You’ll see the Kibana web interface, indicating that Kibana is successfully installed and running.
Step 4: Testing the Installation
With both Elasticsearch and Kibana installed, it’s time to test the setup to make sure everything is functioning properly.
Check Elasticsearch Health
In your terminal, check the health of your Elasticsearch cluster:
curl -X GET "localhost:9200/_cluster/health?pretty"
You should see a JSON response with cluster health status, indicating that Elasticsearch is up and running.
Troubleshooting Common Issues
If you encounter any issues, here are some common troubleshooting tips:
-
Elasticsearch or Kibana Won’t Start: Check their respective logs for error messages:
- Elasticsearch:
/var/log/elasticsearch/elasticsearch.log - Kibana:
/var/log/kibana/kibana.log
- Elasticsearch:
-
Firewall Issues: If you’re accessing Kibana remotely, ensure ports
9200(for Elasticsearch) and5601(for Kibana) are open. -
Insufficient Memory: Elasticsearch requires a significant amount of memory. Consider adjusting the
jvm.optionsfile or increasing the memory of your server if you encounter memory-related errors.
Conclusion
Setting up Elasticsearch and Kibana on Ubuntu can seem complex at first, but following these steps will ensure a smooth installation and configuration. With Elasticsearch handling search and analytics, and Kibana providing powerful visualizations, you have a robust data processing and exploration setup. From monitoring application logs to analyzing business data, Elasticsearch and Kibana can offer deep insights that drive smarter decision-making.
Explore the Elastic Stack further to leverage its full potential. Once you’re comfortable with the basics, you can expand your setup with other components like Logstash and Beats for complete data pipeline management!