Search Semantic Search Guide

When to use this capability

SKILL.md routes here when the user is asking about search retrieval setup or design. Concrete triggers:

• Phrases: “semantic search”, “hybrid search”, “vector index”, “k-NN”, “build a RAG app”, “Bedrock embeddings”, “sparse vectors”, “dense vectors”, “ELSER”, “neural search”, “FAISS or Lucene”
• Tasks: pick an embedding model, set up a Bedrock connector, configure a vector index, design hybrid scoring, evaluate retrieval quality, troubleshoot relevance

All search files (capability index)

After loading this entry, you can discover every search-capability file from this list.

Cross-cutting refs you may also load: vector-knn.md (vector sizing math, k-NN engines), sizing.md, security.md.

Vector / k-NN target shape

• Serverless NextGen Vector Search collections use a simplified API — no engine/mode selection (system auto-picks); supports custom document IDs and 32x compression by default.
• Serverless Classic Vector Search collections require explicit engine: faiss; Lucene/IVF/PQ are NOT supported on Classic Serverless.
• Managed Domain supports all engines: Lucene, FAISS HNSW, FAISS IVF, FAISS PQ.
• NMSLIB is removed in OS 3.x. For the engine-by-engine breakdown, see vector-knn.md.

Sizing-related universal rules (apply when this capability sizes a vector index)

• Current-generation instances. Default to Graviton (r7g/r8g for memory-optimized; m7g/m8g for cluster managers). r6g/r6gd only with explicit justification.
• Input honesty. When sizing on UNKNOWN inputs, lead with [BLOCKER — need input] OR present 2–3 tiered bands. Never present a single point estimate built on invented numbers.

Cross-capability handoff

• For provisioning the underlying domain or collection: see provisioning-reference.md.
• For migrating an existing search workload into AOS: see assessment-workflow.md.
• For post-deploy log analytics on the same domain: see log-analytics-guide.md.
• For embedding model selection beyond Bedrock: see search-dense-vector-models.md and search-sparse-vector-models.md.

1. BM25 (Lexical Search)

1.1 Overview

BM25 is the default ranking algorithm in OpenSearch. It calculates relevance based on term frequency (TF), inverse document frequency (IDF), and document length normalization.

1.2 Accuracy Characteristics

Strengths:

• Perfect for exact keyword matching
• Handles rare/domain-specific terminology well
• No vocabulary mismatch between query and index

Weaknesses:

• Cannot understand semantic meaning
• Fails on synonyms (e.g., “car” vs “automobile”)
• Language-dependent (requires language-specific analyzers)

1.3 Cost Profile

Storage Estimation:

Scaling Behavior:

• Cost&Latency grows sub-linearly with data size
• Horizontal scaling is straightforward
• Query complexity significantly affects latency

1.5 Unique Features & Query Types

BM25 supports several special query types that vector search cannot:

1.6 Language Support

1.7 When to Use BM25

Recommended:

• Exact keyword/phrase search requirements
• Autocomplete and typeahead features
• Domain-specific terminology search
• Regex or wildcard pattern matching
• Maximum cost efficiency required
• Low-latency requirements at any scale

Not Recommended:

• Semantic similarity search
• Cross-lingual search
• Synonym handling without manual configuration
• User queries differ significantly from document terminology

2. Dense Vector Search

2.1 Overview

Dense vector search uses neural network embeddings to represent text as dense floating-point vectors (typically 384-1536 dimensions). Similarity is computed using cosine similarity, dot product, or L2 distance.

2.2 Accuracy Characteristics

Strengths:

• Understands semantic meaning
• Handles paraphrases and synonyms naturally
• Supports cross-lingual search with multilingual models
• Zero-shot transfer to new domains

Weaknesses:

• May miss exact keyword matches
• Requires embedding model
• Higher computational cost
• Quality depends heavily on embedding model choice

2.3 Index Algorithms (Core Structure)

2.3.1 HNSW (Hierarchical Navigable Small World)

Overview: Graph-based approximate nearest neighbor (ANN) algorithm. Default and most popular choice.

Memory Estimation (Raw):

Best For:

• Small to medium datasets that fit in memory
• Low-latency requirements
• High accuracy requirements

2.3.2 IVF (Inverted File Index)

Overview: Clustering-based approach that partitions vectors into clusters (buckets).

Best For:

• Larger datasets where memory is constrained
• Can tolerate slightly lower accuracy
• Batch search workloads

2.3.3 Disk-based Vector Search (mode: on_disk)

Overview: OpenSearch’s solution for billion-scale vector search with limited memory (requires OpenSearch 2.17+). Uses Binary Quantization (BQ) to keep a compressed index in memory while storing full-precision vectors on disk.

Memory Estimation:

Best For:

• Billion-scale datasets
• Cost-efficiency (trading RAM for SSD)
• High-throughput scenarios where RAM is the bottleneck

2.4 Compression & Quantization

2.4.1 Product Quantization (PQ)

Compression technique that breaks vectors into sub-vectors and encodes them.

2.4.2 Binary Quantization (BQ)

Extreme compression using 1-bit representations.

2.5 Total Latency Composition

1. Embedding Inference: 5-200ms depending on deployment (API vs GPU vs CPU)
2. Vector Search (KNN): 1-100ms depending on algorithm

Critical Note: Often, inference time dominates the total latency.

2.6 Language Support

2.7 When to Use Dense Vector

Recommended:

• Semantic similarity search
• Cross-lingual search requirements
• Synonym and paraphrase handling needed
• Natural language queries from users
• Question-answering systems
• RAG (Retrieval Augmented Generation) applications

Not Recommended:

• Exact keyword matching is critical
• Highly specialized domain vocabulary not covered by model
• Extremely cost-sensitive deployments
• Real-time autocomplete/typeahead
• Sub-millisecond latency requirements

3. Sparse Vector Search

3.1 Overview

Sparse vector search uses learned sparse representations where most dimensions are zero. Unlike dense vectors with 384-1536 dimensions all populated, sparse vectors may have 30,000+ dimensions but only 100-500 non-zero values.

3.2 How Neural Sparse Works

Uses neural networks to learn sparse representations with semantic meaning:

1. Documents and queries are encoded into sparse vectors
2. Each dimension corresponds to a vocabulary token
3. Weights indicate semantic importance (not just term frequency)

Advantages over BM25:

• Learns semantic term expansion (e.g., “dog” activates “puppy”, “canine”)
• Trained on relevance signals
• Better zero-shot domain transfer

3.3 Search Modes: Doc-only (Recommended) vs Bi-encoder

3.3.1 Doc-only Mode (Recommended)

• Ingestion: Documents encoded using a specialized doc-only model
• Search: Query processed using a simple tokenizer (not full model inference)

Why recommended: Zero query inference, low latency (10x+ faster), lower cost.

3.3.2 Bi-encoder Mode

• Both documents and queries processed by the same deep neural network
• Higher relevance but higher latency

3.4 Index Backends

3.4.1 rank_features Field (Inverted Index Based)

• Exact search (no approximation)
• Best for smaller datasets (< 50M documents)

3.4.2 SEISMIC (ANN-based Sparse Search)

• Approximate nearest neighbor for sparse vectors
• Best for large-scale datasets (> 10M documents) with latency sensitivity

3.5 Accuracy Characteristics

3.6 When to Use Sparse Vector

Recommended:

• Balance between lexical and semantic search
• Users want semantic search without query-time model inference
• Extreme fast semantic search (doc-only + SEISMIC)
• Interpretability is important
• Lower memory budget than dense vectors

Not Recommended:

• Cross-lingual search
• Maximum semantic understanding needed

4. Hybrid Search

4.1 Overview

Hybrid search combines multiple retrieval methods (BM25, dense vector, sparse vector) to leverage the strengths of each. OpenSearch supports hybrid search through the hybrid query type and score normalization.

4.2 Score Normalization

OpenSearch provides several normalization techniques (Min-Max, L2, Harmonic Mean, etc.) to ensure scores are comparable before combination.

4.3 Combination Strategy for Relevance

• Hybrid Scope Rule: Use at most two retrieval methods per hybrid query.

• Recommended Combinations:

  • Dense + Sparse: Best search relevance. Two layers of semantic understanding.
  • Dense + BM25: Robust baseline combining semantic understanding with exact keyword precision.

• Not Recommended:

  • Sparse + BM25: Generally redundant. Sparse vectors already capture keyword information.

4.4 When to Use Hybrid Search

Recommended:

• Maximum Relevance: When accuracy and recall are the top priorities.
• Mixed query types (some exact, some semantic).
• Unknown query distribution.
• Can afford additional infrastructure cost.

Not Recommended:

• Strict cost constraints
• Simple use cases where one method suffices
• Sub-10ms latency requirements
• Development/prototype phase (start simple)