Setting the file. One moment.
Chapter 61 · Amazon Elasticache
Subchapter 61.8
references/genai/embedding-providers.mdMarkdown9 KBView on GitHub
Load this reference when the user needs to choose or configure an embedding provider for vector search with ElastiCache (Valkey).
Important: Vector search (FT.CREATE / FT.SEARCH) requires node-based ElastiCache Valkey 8.2 or later (recommend 9.0). It is not available on ElastiCache Serverless. If using serverless, see the application-side comparison approach in semantic-cache.md.
First, check .elasticache/requirements.json. If infrastructure.embedding_provider, infrastructure.embedding_model, and infrastructure.embedding_dim are already set, use those values. Do not re-ask.
If not set, ask the user: “Do you have a preferred embedding model or provider?”
After selection:
{
"infrastructure": {
"embedding_provider": "bedrock",
"embedding_model": "amazon.titan-embed-text-v2:0",
"embedding_dim": 1024,
"embedding_module": "utils/embeddings.py"
}
}Create a file (default: utils/embeddings.py, or wherever fits the user’s project structure) that exports:
generate_embedding(text: str) -> list[float]embedding_to_bytes(embedding: list[float]) -> bytesVECTOR_DIM: intUse the provider-specific code from the Standard Functions section below. This file is generated ONCE. Every subsequent file the model generates imports from it:
from utils.embeddings import generate_embedding, embedding_to_bytes, VECTOR_DIMSave the file path in requirements.json as infrastructure.embedding_module so the model never regenerates it.
If infrastructure.embedding_module is set in requirements.json, read that file to confirm it exists. If it exists, import from it. Never regenerate. If the file was deleted, regenerate it from the stored provider/model/dim values.
| Provider | Model | Dimensions | Requires API | Best for |
|---|---|---|---|---|
| Bedrock Titan | amazon.titan-embed-text-v2:0 | 256/512/1024 | Yes (Bedrock) | Production |
| Bedrock Cohere | cohere.embed-english-v3 | 1024 | Yes (Bedrock) | English-only (use cohere.embed-multilingual-v3 for multilingual) |
| fastembed | BAAI/bge-small-en-v1.5 | 384 | No | Prototyping |
| sentence-transformers | all-MiniLM-L6-v2 | 384 | No | Prototyping with more model choice |
amazon.titan-embed-text-v2:0bedrock:InvokeModel on the Titan Embed model ARNcohere.embed-english-v3 or cohere.embed-multilingual-v3bedrock:InvokeModel on the Cohere Embed model ARNtexts (list) instead of inputText (string), and requires input_type. Use "search_document" when storing and "search_query" when querying.pip install fastembedBAAI/bge-small-en-v1.5 (384 dims) or BAAI/bge-base-en-v1.5 (768 dims)cohere.embed-v4:0inputs fieldbedrock:InvokeModel on the Cohere Embed v4 model ARNinputs field schema.pip install sentence-transformersall-MiniLM-L6-v2 (384 dims)Use these in the embedder block of mem0’s config dict. See agent-memory.md for the full mem0 configuration.
Bedrock Titan:
{
"embedder": {
"provider": "aws_bedrock",
"config": {
"model": "amazon.titan-embed-text-v2:0",
"aws_region": "us-east-1"
}
}
}Bedrock Cohere:
{
"embedder": {
"provider": "aws_bedrock",
"config": {
"model": "cohere.embed-english-v3",
"aws_region": "us-east-1"
}
}
}fastembed / sentence-transformers:
{
"embedder": {
"provider": "huggingface",
"config": {
"model": "BAAI/bge-small-en-v1.5"
}
}
}These are the canonical implementations for the reusable embedding utility file. Use the one matching the user’s chosen provider.
Bedrock Titan:
import boto3, json, struct
_bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
VECTOR_DIM = 1024
def generate_embedding(text: str) -> list[float]:
response = _bedrock.invoke_model(
modelId="amazon.titan-embed-text-v2:0",
# "embeddingTypes" is optional; float is the default. Include only if you need
# a specific type (e.g., "binary"). Omitting it returns float embeddings.
body=json.dumps({"inputText": text, "dimensions": VECTOR_DIM}),
)
return json.loads(response["body"].read())["embedding"]
def embedding_to_bytes(embedding: list[float]) -> bytes:
return struct.pack(f"{VECTOR_DIM}f", *embedding)Bedrock Cohere:
import boto3, json, struct
_bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
VECTOR_DIM = 1024
def generate_embedding(text: str, query: bool = False) -> list[float]:
response = _bedrock.invoke_model(
modelId="cohere.embed-english-v3",
body=json.dumps({
"texts": [text],
"input_type": "search_query" if query else "search_document",
"truncate": "END",
}),
)
return json.loads(response["body"].read())["embeddings"][0]
def embedding_to_bytes(embedding: list[float]) -> bytes:
return struct.pack(f"{VECTOR_DIM}f", *embedding)Cohere note: Pass query=True when embedding a search query (retrieval), query=False (default) when embedding documents for storage. Other providers ignore this parameter.
fastembed:
import struct
from fastembed import TextEmbedding
_model = TextEmbedding("BAAI/bge-small-en-v1.5")
VECTOR_DIM = 384
def generate_embedding(text: str) -> list[float]:
return list(_model.embed([text]))[0].tolist()
def embedding_to_bytes(embedding: list[float]) -> bytes:
return struct.pack(f"{VECTOR_DIM}f", *embedding)sentence-transformers:
import struct
from sentence_transformers import SentenceTransformer
_model = SentenceTransformer("all-MiniLM-L6-v2")
VECTOR_DIM = 384
def generate_embedding(text: str) -> list[float]:
return _model.encode(text).tolist()
def embedding_to_bytes(embedding: list[float]) -> bytes:
return struct.pack(f"{VECTOR_DIM}f", *embedding)For bulk ingestion (>1K documents), batch embedding calls to avoid per-request overhead. Titan accepts sequential calls (add exponential backoff for throttling). Cohere natively supports batching via texts: [list] with up to 96 texts per call.
The FT.CREATE index DIM must match your embedding model’s output dimension exactly. If you change embedding providers, you must:
FT.DROPINDEX <index_name>SCAN + DEL by prefixDIMThis is destructive. Choose your embedding model before ingesting production data.
Backfill warning: After recreating an index with FT.CREATE, queries (FT.SEARCH) are not allowed while the index is backfilling and will return an error. Use FT.INFO <index_name> and check the state field – wait until it reports ready before issuing queries.