What is Elasticsearch and where is it used?
If you're looking for a powerful search engine and data visualization tool,
then you need to install Elasticsearch and Kibana. Elasticsearch is a
distributed, RESTful search and analytics engine that helps you find the right
information quickly.
What is Kibana and where is it used?
Kibana is a data visualization tool that lets you see your data in creative
and exciting ways. Having both of these tools installed can be extremely
beneficial for anyone who wants to be able to quickly find and visualize their
data.
Step-by-step guide on how to install Elasticsearch and Kibana
Elasticsearch and Kibana are two powerful tools that can be used to store and
analyze data. Having them installed can be beneficial because they offer a
quick and easy way to search and visualize data.
To install Elasticsearch and Kibana, follow these steps:
1. Download the latest version of Elasticsearch from the official
website
ElasticSearch download
2. Unzip the downloaded file and move it to the desired location on your
computer.
3. Open the terminal in the same folder where you have placed your unzip ES
file and enter the below command to start the ES server.
bin/elasticsearch
4 The ES server runs on port 9200. If everything is fine then you must
get status like below.
{
"name": "INSML-W4FQHY5",
"cluster_name": "elasticsearch",
"cluster_uuid": "PCSKqJ4nQsq8mTadAH92ZQ",
"version": {
"number": "8.4.3",
"build_flavor": "default",
"build_type": "tar",
"build_hash": "42f05b9372a9a4a470db3b52817899b99a76ee73",
"build_date": "2022-10-04T07:17:24.662462378Z",
"build_snapshot": false,
"lucene_version": "9.3.0",
"minimum_wire_compatibility_version": "7.17.0",
"minimum_index_compatibility_version": "7.0.0"
},
"tagline": "You Know, for Search"
}
5. Install Kibana by running the following command in a terminal window:
sudo apt-get install Kibana
By downloading the file from
Kibana download
6. Start Kibana by running the following command in a terminal window:
bin/kibana 7. Open your browser and type localhost:5601
Connect Kibana with ElasticSearch
When you start Elasticsearch for the first time, the server generates an
enrollment token for Kibana which is used to help Kibana connect with our
Elasticsearch server.
Enrollment tokens are valid for 30 minutes.
Now open Kibana in your browser, paste the enrollment token that was generated
by Elasticsearch, and click the connect button.
Kibana configuration file
You can also configure Kibana manually using a configuration file. Kibana
loads all configuration from its /config/kibana.yml file by
default.
How to use Elasticsearch and Kibana examples
Let's get our hands on elastic search by simply creating an index and
inserting some documents into it.
If you have already worked with NoSQL databases, index and document may sound
familiar to you.
Node (an instance running psychical or virtually) stores the data that we add
to Elasticsearch.
The term cluster refers to a collection of nodes.
- An index is like a table which contains all data.
- Data is stored as document which are JSON objects mostly.
- JSON fields act as column names, like in a table, for filtering and searching documents in Elasticsearch.
Go to Kibana and under dev Tools -> console. The console is where we
interact with elastic search by writing an elastic query and seeing the result
of the query. It also helps to write queries easily by auto-suggesting key
words.
Kibana uses the REST API under the hook. All operations to Elasticsearch are
performed using REST API calls.
Create an index in elasticsearch
To create an index in elastic search we have to call an HTTP PUT endpoint with
indexname as a parameter.
PUT /indexname to create an index
We can also set sharding and replication of the index during index creation
using the request body with settings like below.
{
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 2
}
}
}
Delete index in elasticsearch
To delete an index, we have to call the HTTP delete endpoint and pass the
index name that we want to delete.
Delete /indexname to delete an index
Let's add some documents to our index.
Add document to our index
To add a document to the index we need to use _doc endpoint (HTTP POST) which
take index name and document (JSON structure) we want to add.
In the below, we have added employee documents to our index.
POST /indexname/_doc
{
"empid": "SJ011MS",
"personal": {
"name": "Smith Jones",
"gender": "Male",
"age": 28,
"address": {
"streetaddress": "724th Street",
"city": "New York",
"state": "NY",
"postalcode": "10038"
}
},
"profile": {
"designation": "Deputy General",
"department": "Finance"
}
}
The API response looks like this.
_id acts as the primary identifier of the document.
_shards tell whether the request is successful or failed.
{
"_index": "employeeindex",
"_id": "sWh4y4MBviYtB2C8KnaN", -> auto generated
"_version": 1,
"result": "created", -> flag tell the document status
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1
}
If you want to pass or create your own ID for each document when creating
POST /indexname/_doc/id
Retrieve all documents from the index
In order to retrieve a document from an index, we need to use the _search
endpoint, which searches the index and returns the requested document when
found.
GET /index name/_search
{
"query": {
"match_all": {} -> match all document in the index
}
}
Retrieving document using ID from index
To fetch a particular document we can pass the document ID.
GET /indexname/_doc/doc ID if we have one.
The response will include a found flag which is a Boolean if doc is found for
given ID then it sets to true otherwise false. The response contains a source
part that contains the actual document value.
{
"_index": "employeeindex",
"_id": "sWh4y4MBviYtB2C8KnaN",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"found": true,
"_source": {
"empid": "SJ011MS",
"personal": {
"name": "Smith Jones",
"gender": "Male",
"age": 28,
"address": {
"streetaddress": "724th Street",
"city": "New York",
"state": "NY",
"postalcode": "10038"
}
},
"profile": {
"designation": "Deputy General",
"department": "Finance"
}
}
}
Benefits of having Elasticsearch and Kibana
Elasticsearch and Kibana are a powerful combination for anyone who wants quick
answers from large datasets. With Elasticsearch, you can build your data in a
way that allows for fast searches, and with Kibana's built-in features, you
can see information more quickly than ever before. This enables businesses to
quickly access actionable intelligence and make informed decisions without
spending hours sifting through raw data. Not only will this save time, but it
will also improve accuracy when interpreting complex sets of info - making it
easier than ever before to gain insight into customer behavior or market
trends.
Tags:
ElasticSearch