Elasticsearch

Updated Nov 16, 2026 · 4 min read
On this page

    A shopper types "red jacket" into your search box. The first version runs SELECT * FROM products WHERE title LIKE '%red%jacket%'. It misses "Jacket, Red", misses "red rain jackets", cannot put the best match first, and on 5 million products it scans the whole table for every keystroke. Search takes two seconds and returns the wrong things.

    A relational index cannot help, because LIKE '%…%' cannot use a B-tree. Text search needs a different data structure, an inverted index, and a system built around it. Elasticsearch (and its fork OpenSearch) is the common choice.

    How it works

    The inverted index

    Inverted index: three product titles are tokenized into words, and each word maps to the documents that contain it; the query "red jacket" intersects {1,3} and {2,3} to find document 3.
    Inverted index: three product titles are tokenized into words, and each word maps to the documents that contain it; the query "red jacket" intersects {1,3} and {2,3} to find document 3.

    When a document is indexed, its text goes through an analyzer: split into words (tokenize), lowercase, remove very common words, and reduce words to a root ("jackets" → "jacket"). Each resulting term points to the list of documents that contain it, the posting list.

    A query runs through the same analyzer. "Red Jackets" becomes red, jacket. Elasticsearch fetches both posting lists, combines them, and scores each document, by default with BM25: a match on a rare word counts more than a match on a common one, and a match in a short title counts more than one buried in a long description. The top results come back in a few milliseconds.

    More than text

    • Filters: exact conditions such as category = shoes, price < 100, in_stock = true. These are cached and cheap.
    • Aggregations: counts by brand, price buckets, used for the facet sidebar ("Nike (120), Adidas (98)").
    • Fuzzy matching and synonyms: "jakcet" still finds "jacket"; "sneakers" matches "trainers".
    • Geo queries: "within 5 km of this point" (see location-based search).

    Shards and replicas

    An index is split into primary shards, each a self-contained index on one node, so a large index spreads across machines. Each shard has replicas on other nodes for failover and extra query capacity. A query goes to one copy of every shard in parallel, and a coordinator merges the results.

    Keeping it in sync with the database

    Sync from the database: the Product Service writes to the Products DB, changes flow through CDC or an outbox into an indexer that updates the search index about a second later; the Search Service quer
    Sync from the database: the Product Service writes to the Products DB, changes flow through CDC or an outbox into an indexer that updates the search index about a second later; the Search Service queries the index for the top 20 IDs and fetches details from the database.
    1. The Product Service writes to the database, the source of truth.
    2. Changes are captured (change data capture from the database log, or an outbox table) and published as events.
    3. An indexer consumes the events and updates the search index. New documents become searchable after the next refresh, by default about once a second.
    4. The Search Service queries the index for matching IDs, then (if needed) reads current price and stock from the database or cache.

    If the index is ever corrupted or the mapping changes, you rebuild it from the database into a new index and switch an alias over.

    Trade-offs and costs

    • Near-real-time, not real-time: a product added now is searchable in about a second. Fine for search; not fine for "did my payment go through".
    • Not a primary database: no multi-document transactions, and mapping changes often need a full reindex. Keep the truth in the database.
    • Resource-hungry: it needs plenty of memory and fast disks; a cluster is a real operational cost.
    • Shard count is hard to change. Too few shards limits growth; too many small shards wastes memory. A common target is shards of roughly 10–50 GB.
    • Deep pagination is expensive. Page 500 requires every shard to sort 10,000 results. Use cursor-style search_after instead.

    When to reach for it

    • Full-text search over titles, descriptions, posts, messages, documents.
    • Search with filters and facets (e-commerce, job boards, listings).
    • Typo tolerance, synonyms, relevance ranking.
    • Log and event search and dashboards (a very common use outside product search).

    Do not use it for simple lookups by ID or exact key (the database or cache does that), or as the only copy of data you cannot lose.

    Example

    Product search for a shop. Assumptions: 5 million products, about 2 KB of searchable text each, 2,000 searches per second at peak, 50,000 product updates per hour.

    • Index size: 5 M × 2 KB ≈ 10 GB of source, maybe 20 GB indexed. Two primary shards of about 10 GB each, with one replica each, on three nodes.
    • Query capacity: each copy of a shard can serve hundreds of simple queries per second; with replicas and a result cache, 2,000 per second fits. Add replicas if load grows.
    • Mapping: title and description as analyzed text (English analyzer, synonyms file), brand and category as keyword for exact filters and facets, price as a number, in_stock as a boolean.
    • Sync: updates go through CDC into a topic; the indexer batches them (bulk requests of 500), so 50,000 updates per hour is light work.
    • Read path: the Search Service asks for the top 20 IDs with facets, then reads live price and stock for those 20 from the cache, so a price change never waits for reindexing.

    Common mistakes

    • Using LIKE '%term%' on the database at scale.
    • Making Elasticsearch the source of truth and having nothing to rebuild from.
    • Dual writes from the service (write DB, then write index) without an outbox, so the two drift apart after a crash.
    • Ignoring refresh delay, then being surprised that a just-created item is not found.
    • Oversharding (hundreds of tiny shards) or undersharding a fast-growing index.
    • Not saying what is analyzed and what is a keyword. Filters on analyzed fields give odd results.

    Say this in the interview

    "Search needs ranking and typo tolerance, so I'll index products in Elasticsearch. Postgres stays the source of truth, and changes stream in through CDC to an indexer, so search is about a second behind. Title and description are analyzed text; brand and category are keywords for filters and facets. The Search Service gets the top 20 IDs from the index and reads live price and stock from the cache. If we change the mapping, we reindex into a new index and swap the alias."

    Used in