Search is hard and you should probably not build it

Every product eventually adds a search box. The distance between "a search box that works" and "a search box users trust" is much larger than it looks, and most teams discover this after committing.

the levels#

Level 0: LIKE '%query%'.

Works for tiny datasets. No ranking, no stemming, no typo tolerance, and a full table scan on every query.

Fine for an admin tool with a thousand rows. Not fine for anything a customer touches.

Level 1: your database's full-text search.

Postgres tsvector, MySQL full-text, SQLite FTS5. You get stemming, stop words, ranking, and index support.

ALTER TABLE articles ADD COLUMN search tsvector
  GENERATED ALWAYS AS (
    setweight(to_tsvector('english', coalesce(title,'')), 'A') ||
    setweight(to_tsvector('english', coalesce(body,'')), 'B')
  ) STORED;

CREATE INDEX ON articles USING GIN (search);

SELECT id, title, ts_rank(search, q) AS rank
FROM articles, websearch_to_tsquery('english', $1) q
WHERE search @@ q
ORDER BY rank DESC LIMIT 20;

The setweight calls are the part people miss: a match in the title should outrank a match in the body, and without weighting it does not.

This handles most applications. If you have under a few million documents and your users search for terms that appear in them, stop here. One system, no synchronization problem, joins to your relational data.

Level 2: a dedicated search engine.

Elasticsearch, OpenSearch, Typesense, Meilisearch. You get: typo tolerance, faceting, synonyms, custom analyzers, distributed scaling, and much better relevance tuning.

The cost is a second system with a synchronization problem. Your search index is now eventually consistent with your database, and every write path must update both. That inconsistency will produce bugs — a deleted item still appearing in results is the classic — and handling it correctly is real work.

Level 3: hybrid semantic search.

Vector embeddings alongside keyword search, combined with reciprocal rank fusion or a learned reranker.

This handles the case where the user's words are not the document's words. "How do I cancel" should find "Terminating your subscription."

Important: hybrid, not pure vector. Pure semantic search is bad at exact matches — product codes, error numbers, names, function names — precisely the queries where users are most certain about what they want and least tolerant of a wrong answer.

what makes search actually good#

The engine is the easy part. Relevance is the hard part, and it is mostly not about the algorithm.

Weight your fields. Title beats body. Exact phrase beats individual terms. Recent beats old, for content where recency matters.

Use behavioral signals. What users clicked on for similar queries is the strongest relevance signal available, and it requires logging queries and clicks from day one. Retrofitting this means starting your data collection from zero.

Handle the empty result. "No results for X" is a failure. Show something: did-you-mean, related content, popular items, a way to browse. An empty page is where users leave.

Handle the head queries manually. A small number of queries make up a large share of volume. Look at your top hundred, check what they return, and pin the correct answer where it is wrong. This is unglamorous, takes an afternoon, and improves perceived quality more than any algorithmic change.

Log everything. Query, result count, position clicked, whether anything was clicked. The searches that return nothing and the searches where nobody clicks are your improvement backlog, delivered for free.

the instrumentation that matters#

Three metrics:

1. Zero-result rate. Should be low. Every zero-result query is a user who did not find what they wanted.
2. Click-through rate, and the position clicked. If people consistently click the fifth result, your ranking is wrong.
3. Query refinement rate. Users who search, then immediately search again with different words, did not find it the first time.

Most teams have none of these and are tuning relevance by intuition.

the recommendation#

Start at level 1. Postgres full-text search with weighted fields covers more applications than people expect, and it does not introduce a synchronization problem.

Move to level 2 when you have a specific complaint you cannot fix — typo tolerance, faceting at scale, or performance. Move to level 3 when you have evidence that users search for concepts rather than terms.

And whatever level you are at: log the queries. That data is the input to every future improvement and you cannot get it retroactively.