Search Dense Vector Models

• OpenSearch node (CPU) pretrained models tend to be older baselines: convenient for quick starts, but not SOTA for retrieval quality.
• Default recommendation for most users: Amazon Titan Embeddings (via Amazon Bedrock) for strong quality + managed ops.
• External Embedding API Services: OpenSearch can work with any embedding service via ML Commons Connectors; the list below is just common examples.

1. OpenSearch Node Deployment (CPU)

Deploy models directly on OpenSearch nodes using CPU inference.

When to use

• Dev / POC / low QPS workloads
• Environments where you cannot run GPU endpoints
• You prioritize simplicity over best retrieval quality

Caveat

• The pretrained models available on OpenSearch nodes are generally older and may not match the quality of newer retrieval-optimized models (e.g., E5/BGE or vendor-managed models like Titan).

1.1 Supported Pre-trained Models (examples)

OpenSearch provides a repository of pre-trained models that can be registered directly.

1.2 Custom Models

Not Supported. Custom or fine-tuned dense embedding models cannot be deployed on OpenSearch Nodes. You must use a SageMaker GPU Endpoint.

2. SageMaker GPU Endpoint (Recommended for Custom / High-QPS)

Deploy models on AWS SageMaker with GPU acceleration for high throughput and low latency. This is the recommended approach for:

• High QPS / large batch ingestion
• Larger or retrieval-optimized models (E5/BGE family, etc.)
• Custom/fine-tuned models and custom inference logic

2.1 Recommended Models (examples)

Any model compatible with Hugging Face Text Embeddings Inference (TEI) or a custom SageMaker inference script can be used.

2.2 Custom Models

If you have a custom or fine-tuned dense embedding model, deploy it using a SageMaker GPU Endpoint. This mode supports custom model weights and custom inference logic that you control.

3. External Embedding API Services (Managed Providers)

Use managed API services to generate embeddings. OpenSearch connects via the ML Commons Connector.

Important: OpenSearch can integrate with any embedding provider/service as long as:

• You can call an HTTP endpoint from OpenSearch (or from the connector runtime),
• The service returns a numeric embedding vector,
• You can configure authentication and request/response transformation.

So the providers below are examples of common choices, not an exhaustive list.

3.1 Common Providers (Examples)

Why default recommend Amazon Titan

• Strong general-purpose embedding quality
• Fully managed + straightforward operations on AWS
• IAM-based auth and Bedrock integration reduces operational overhead

Summary of Trade-offs

Practical Tips (Common Gotchas)

• Dimensions must match your index mapping (knn_vector dimension).
• If your model recommends normalization (common for cosine similarity), apply it consistently at ingestion and query time.
• For E5/BGE-style retrieval models, follow their recommended query/document formatting (e.g., prefixes) for best results.