Blog

Elasticsearch in 5 Minutes

Key Takeaways

  • Elasticsearch is a distributed, Lucene-based search and analytics engine that indexes JSON documents.
  • Use analyzed text fields for full-text search and keyword fields for exact filters, sorting, and aggregations.
  • Search is near-real-time: a successful write isn't necessarily searchable until a refresh.
  • The walkthrough below pins Elasticsearch 9.5.5 and uses authenticated HTTPS requests with certificate validation.
  • Choose Elasticsearch when its search capabilities justify a separate index, synchronization pipeline, and operational cost.

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. It organizes documents into searchable indexes and exposes an HTTP REST API for indexing, querying, and aggregation. For application search, a useful starting architecture is to keep your transactional database as the source of truth and build a derived Elasticsearch index from it.

Why use Elasticsearch?

Consider an e-commerce catalog. A shopper searches for “waterproof hiking boots,” filters by brand and size, and expects relevant products plus category counts. Elasticsearch supports full-text relevance, structured filters, and aggregations—the summaries behind features such as faceted navigation.

That makes product search a good use case, not a performance guarantee. Mapping, query design, hardware, document size, and concurrency all affect results. Before adding another system, establish whether your database's existing search and indexes already meet your requirements.

How does Elasticsearch work?

The Elasticsearch indexing model has a few essential terms:

  • Document: a JSON object representing something searchable, such as a product or user.
  • Index: a named collection of documents with mappings and settings.
  • Mapping: the definition of field types and how those fields are indexed.
  • Shard: a partition of an index; each shard is a self-contained Lucene index.
  • Replica: a copy of a primary shard that provides redundancy and can serve reads.

A node is a running Elasticsearch instance. Distributing shards across nodes lets a cluster spread data and work across machines, but each additional shard also introduces overhead.

What is Lucene, and how do inverted indexes work?

Apache Lucene is a Java search library, not a standalone search service. Elasticsearch builds the distributed service and REST interface around it.

Think of an inverted index as the index at the back of a book: instead of reading every page to find a word, you look up the word and follow its references. Lucene's inverted index maps terms to the documents containing them.

Illustrative product titles and analyzed terms:

Document A: Blue running shoes
Document B: Red running shoes

blue    -> A
red     -> B
running -> A, B
shoes   -> A, B

For a text field, an analyzer processes the input into searchable terms. A keyword field keeps a value as a single term, making it appropriate for identifiers, categories, and exact filters.

Explicit mappings versus dynamic mapping

Dynamic mapping infers types when new fields arrive. Under the usual dynamic rules, ordinary strings become text fields with a .keyword subfield, while date-like strings can become dates. That convenience can produce an unintended schema.

Define important fields explicitly before loading application data. The dynamic mapping rules explain inference; the walkthrough uses dynamic: strict to reject documents containing undeclared fields.

Getting started with the Elasticsearch REST API

This local walkthrough follows Elastic's secured single-node Docker procedure, pinned to Elasticsearch 9.5.5. You'll need Docker, Bash, and curl with --fail-with-body support. Docker Desktop users should allocate at least 4 GB to Docker.

Elasticsearch is implemented in Java, but deployment support depends on the package, runtime, and host—not simply the language. Use the installation requirements for your environment, including any host configuration required by Docker.

Start Elasticsearch with security enabled

Run these commands in your first terminal, using unused network and container names. Leave the container running:

docker network create elastic-demo

docker run --name es01 --net elastic-demo \
  -p 127.0.0.1:9200:9200 -it -m 1GB \
  docker.elastic.co/elasticsearch/elasticsearch:9.5.5

The fresh container generates TLS certificates and prints the elastic user's password. Copy that password. The port mapping binds the published endpoint to IPv4 loopback rather than all host interfaces.

In a second Bash terminal, copy the certificate authority certificate, read the password without echoing it, and define an authenticated request helper:

docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .

export ES_URL='https://localhost:9200'
export ES_CA_CERT="$PWD/http_ca.crt"
read -r -s -p 'Paste the generated elastic password: ' ELASTIC_PASSWORD
printf '\n'
export ELASTIC_PASSWORD

es() {
  curl --fail-with-body --silent --show-error \
    --cacert "$ES_CA_CERT" \
    --user "elastic:${ELASTIC_PASSWORD}" "$@"
}

es "$ES_URL"

The final request should return cluster and version information. If you lost the password, run docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic, then read the replacement into ELASTIC_PASSWORD again.

Create an index with explicit mappings

PUT /users creates the index. Here, name is searchable text, age is an integer, and status supports exact matching:

es -X PUT "$ES_URL/users" \
  -H 'Content-Type: application/json' \
  --data-binary @- <<'JSON'
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "name": { "type": "text" },
      "age": { "type": "integer" },
      "status": { "type": "keyword" }
    }
  }
}
JSON

Zero replicas is intentional for this single-node exercise. Elasticsearch won't allocate a replica on the same node as its primary, so replicas require additional nodes to provide redundancy.

Populate documents with the Bulk API

The Bulk API accepts newline-delimited JSON, or NDJSON. Each indexing action occupies one line and its document occupies the next; the body must end with a newline.

es -X POST "$ES_URL/users/_bulk?refresh=wait_for" \
  -H 'Content-Type: application/x-ndjson' \
  --data-binary @- <<'NDJSON'
{"index":{"_id":"1"}}
{"name":"Sam","age":36,"status":"active"}
{"index":{"_id":"2"}}
{"name":"Sara","age":32,"status":"inactive"}
NDJSON

The heredoc supplies the final newline, and --data-binary preserves the NDJSON formatting. Ages are JSON numbers rather than strings.

Check that the response contains "errors": false before continuing. A successful HTTP response can still contain failed bulk items; inspect their individual error details if errors is true.

Search one index with Query DSL

Query DSL is Elasticsearch's JSON query language. This request searches only users, combining a full-text query with an exact filter:

es -X POST "$ES_URL/users/_search" \
  -H 'Content-Type: application/json' \
  --data-binary @- <<'JSON'
{
  "query": {
    "bool": {
      "must": [
        { "match": { "name": "Sam" } }
      ],
      "filter": [
        { "term": { "status": "active" } }
      ]
    }
  }
}
JSON

The match query analyzes “Sam” for the text field. The term filter requires the exact keyword value active and doesn't contribute to relevance scoring. The boolean query combines both requirements, returning Sam from the sample data.

Keep application searches scoped to the intended index or alias. A bare /_search request can search across indices rather than just users.

Understand refreshes and document updates

Elasticsearch search is near-real-time. A refresh makes newly indexed changes searchable; acknowledging a write doesn't by itself mean a subsequent search will find it.

The bulk example uses refresh=wait_for so its changes are visible before the response returns. Use that when your workflow requires search visibility, not automatically on every production write.

To change Sam's age, send a partial document to the Update API:

es -X POST "$ES_URL/users/_update/1?refresh=wait_for" \
  -H 'Content-Type: application/json' \
  --data-binary '{"doc":{"age":37}}'

An update that changes a document generally indexes a new document version and marks the previous one deleted. Lucene segment merges reclaim the old version's space later. Heavy update rates therefore add indexing and merge work; benchmark that workload rather than assuming updates are either free or inherently unsuitable.

Keeping Elasticsearch synchronized with your database

Elasticsearch doesn't automatically track changes in MongoDB, a relational database, or an event stream. You must configure an ingestion mechanism, such as a connector, scheduled export, or change data capture (CDC)—a pipeline that captures database changes.

Source-of-truth database
          |
          | connector / CDC / export
          v
Elasticsearch index <--- application search requests

Search freshness = ingestion lag + refresh visibility

For a product catalog, one possible design is to publish database changes through CDC and have a consumer index product documents using stable product IDs. Treat synchronization as eventually consistent: search can temporarily show older data.

Plan retries, duplicate delivery, out-of-order changes, deletions, and rebuilding the index from the source. Read our change data capture introduction for the architecture, and check Elastic's connector documentation for supported integrations. Neither Hadoop nor MongoDB is a prerequisite.

Elasticsearch vs MongoDB, Solr, and OpenSearch

How is Elasticsearch different from MongoDB?

MongoDB is a general-purpose document database; Elasticsearch specializes in search and analytics. Both can persist documents, but that doesn't make them interchangeable.

QuestionMongoDBElasticsearch
Primary roleStore and query application documentsIndex documents for search and analytics
Search optionsDatabase indexes and MongoDB SearchSearch-oriented mappings, queries, and aggregations
Used togetherCan remain the system of recordCan hold a derived, independently refreshed search index

Don't assume MongoDB lacks capable full-text search: evaluate its current search features before introducing another platform. Our MongoDB index performance guide also helps establish a database-side baseline.

How does Elasticsearch compare to Apache Solr?

Both are Lucene-based search platforms. Solr supports full-text search, faceting, relevance tuning, and distributed indexing and querying; it isn't merely an analytics alternative.

Compare your actual queries, schema configuration, ingestion tooling, deployment model, ecosystem, and team experience. SolrCloud uses ZooKeeper for cluster coordination, which is an operational consideration. Start with our Apache Solr introduction and the official Solr overview, then test representative workloads.

Licensing and OpenSearch are separate decisions

Lucene uses Apache License 2.0. Elastic's default Elasticsearch distribution uses Elastic License 2.0; free portions of its source code also offer AGPLv3 and SSPL licensing options. The library and the packaged product don't have identical licensing—consult Elastic's licensing FAQ.

OpenSearch is a separate Apache-2.0-licensed search project that originated as a fork of Elasticsearch and Kibana. Treat it as a distinct platform to evaluate, not another name for current Elasticsearch.

Where vector and hybrid search fit

Elasticsearch also supports dense-vector retrieval: embeddings represent content as numeric vectors, and k-nearest-neighbor search finds nearby vectors. Hybrid search combines lexical retrieval with semantic retrieval, helping balance exact terminology with similarity in meaning.

That can serve the retrieval stage of retrieval-augmented generation (RAG), where an application retrieves relevant passages before asking a model to generate an answer. Keep permission filters in the retrieval path, and measure retrieval quality, latency, and resource use. Vector search is an additional retrieval option, not a reason to discard lexical search.

Should you use Elasticsearch in your project?

  • Start with database search when its relevance, filtering, and latency already satisfy your application.
  • Consider Elasticsearch when you need dedicated relevance tuning, complex search-plus-aggregation workloads, or independent search scaling.
  • Consider managed search when operating a cluster isn't your team's priority. You still own data synchronization, access control, and application-level search quality.

Small projects can justify Elasticsearch when search is central to the product. Dataset size alone isn't a good selection rule.

Production Checklist

  • Budget resources: allow memory beyond the JVM heap, including filesystem cache, and disk headroom for replicas, indexing, and merges.
  • Size shards from measurements: more shards aren't automatically faster. Follow the shard-sizing guidance and benchmark representative queries and updates.
  • Monitor allocation and lag: yellow health means replicas are unassigned; red means at least one primary is unassigned. Also monitor ingestion failures and search freshness.
  • Back up and restore: replicas aren't backups. Configure snapshots and test restoration and index rebuilding.
  • Keep security enabled: validate TLS certificates, restrict network access, and grant application credentials only the permissions they need.

FAQ

What is Elasticsearch used for?

Elasticsearch is used for full-text application search, product catalogs, log search, and analytics over indexed documents. It also supports vector and hybrid retrieval for semantic search applications.

Can Elasticsearch replace a primary database?

Elasticsearch persists documents and can be authoritative for some search-centric workloads. It isn't a drop-in replacement for a transactional database: evaluate durability, transaction requirements, and search visibility before making it your system of record.

How is Elasticsearch different from MongoDB?

MongoDB is a general-purpose document database, while Elasticsearch specializes in search and analytics. You can use either platform's search capabilities or synchronize MongoDB data into a separate Elasticsearch index when that architecture earns its cost.

Why are documents not immediately searchable after indexing?

Search visibility follows a refresh, so a successful indexing response doesn't necessarily mean the document is searchable yet. Use refresh=wait_for on a write when that workflow needs the change visible to search before the response returns.

Does Elasticsearch require authentication by default?

A fresh installation using Elasticsearch's automatic security setup enables authentication and configures TLS for its HTTP endpoint. Existing configurations and explicitly changed security settings can alter that behavior; the Docker walkthrough uses HTTPS, the generated password, and the generated CA certificate.

Further Reading

Written by Sam Erickson

Software engineer. I write every article on StackChief myself. More about me

Related reading