Elasticsearch Guide

What it is and when to use it

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. It stores JSON documents in an inverted index, which makes full-text search, relevance ranking, filtering and aggregations fast across large datasets. It also supports vector search for semantic retrieval.

Use it for product and site search, log and event analytics, autocomplete, and faceted navigation. It is not a transactional database: it offers near-real-time search rather than immediate consistency, and it lacks multi-document transactions. Keep your primary data in a relational store and index a searchable copy.

Quick start

docker run -d --name es -p 9200:9200 \
  -e "discovery.type=single-node" \
  -e "xpack.security.enabled=false" \
  -e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
  docker.elastic.co/elasticsearch/elasticsearch:8.14.0

curl -s localhost:9200/_cluster/health?pretty

Disabling security is acceptable only for local experiments.

Core concepts

Index, document, shard. An index is a collection of documents. It is split into primary shards, each with replicas for availability and read throughput. The primary shard count is fixed at creation, while replicas can change at any time.

Mappings. Define field types explicitly rather than relying on dynamic mapping. Use text for analyzed full-text search and keyword for exact matching, sorting and aggregations.

curl -X PUT localhost:9200/products -H 'Content-Type: application/json' -d '
{
  "settings": { "number_of_shards": 1, "number_of_replicas": 1 },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "name":     { "type": "text", "fields": { "raw": { "type": "keyword" } } },
      "category": { "type": "keyword" },
      "price":    { "type": "scaled_float", "scaling_factor": 100 },
      "created":  { "type": "date" }
    }
  }
}'

Analyzers break text into tokens: lowercase, remove stop words, stem. The same analysis must apply at index and query time for matches to work.

Indexing and searching

curl -X POST localhost:9200/products/_doc/1 -H 'Content-Type: application/json' \
  -d '{"name":"Trail running shoes","category":"footwear","price":89.99,"created":"2026-05-01"}'

curl -X POST 'localhost:9200/products/_search?pretty' -H 'Content-Type: application/json' -d '
{
  "query": {
    "bool": {
      "must":   [ { "match": { "name": "running shoes" } } ],
      "filter": [ { "term": { "category": "footwear" } },
                  { "range": { "price": { "lte": 120 } } } ]
    }
  },
  "aggs": { "by_category": { "terms": { "field": "category" } } }
}'

Clauses under filter skip scoring and are cached, so put exact conditions there and keep must for relevance.

For bulk loads use the _bulk API with batches of several megabytes, and set refresh_interval to 30s or -1 temporarily during large imports.

Pitfalls

• Oversharding. Many tiny shards waste heap. Aim for shards in the tens of gigabytes and avoid hundreds per node.
• Dynamic mapping surprises. A stray field can create a wrong type or a mapping explosion. Use strict mappings or templates.
• Deep pagination. from plus size beyond 10000 is rejected and costly. Use search_after with a point in time.
• Heap sizing. Set the heap to at most half of RAM and under roughly 31 GB so compressed object pointers stay enabled.
• Treating it as a primary store. Keep a source of truth elsewhere and be able to reindex.
• Changing a mapping in place. Field types cannot change. Create a new index and use the _reindex API behind an alias.
• Split brain and unsafe topologies. Run at least three master-eligible nodes in production.
• Wildcard and leading-wildcard queries on large fields are slow. Use n-gram analyzers instead.

How it fits an architecture

A common pattern keeps a relational database as truth and streams changes into Elasticsearch through change data capture or an indexing worker fed by a queue. Applications query through an alias, so reindexing is a zero-downtime swap. For logs and metrics, use data streams with index lifecycle management to roll over and delete old data, with hot, warm and cold tiers. Monitor cluster status, heap use, search latency and rejected thread-pool requests.

Key takeaways

• Elasticsearch excels at relevance, filtering and aggregation, not transactions.
• Define explicit mappings and distinguish text from keyword.
• Put exact conditions in filter context for speed and caching.
• Size shards deliberately and avoid oversharding.
• Always query through aliases so reindexing is painless.
• Use lifecycle policies for time-based data.