Once you represent items as vectors (embeddings), recommendation and semantic search reduce to a single operation: given a query vector, find the nearest vectors among millions or billions of stored ones. Doing this exactly is too slow at scale, so the field relies on approximate nearest neighbor (ANN) search. In a 2016 paper, Yury Malkov and Dmitry Yashunin introduced Hierarchical Navigable Small World graphs, or HNSW, which became the leading ANN method.
HNSW builds a multi-layer graph. Each stored vector is a node connected to its near neighbors, and nodes are assigned to layers with exponentially decreasing probability, so the top layers are sparse and the bottom layer holds everything. A search starts at the sparse top, greedily hops toward the query, then descends layer by layer, refining the neighborhood at each level. This structure gives roughly logarithmic search time while delivering high recall, and it handles clustered, high-dimensional data well, outperforming earlier tree- and graph-based methods.
HNSW is now the default index in many vector databases and similarity-search libraries, including FAISS, and underpins the retrieval step in semantic search and retrieval-augmented generation systems.
For a business reader, HNSW is the unglamorous infrastructure that makes “find me the most similar items” feel instant, whether the items are products, documents, faces, or passages an AI assistant needs to ground its answer.