Standard Operating Procedure (SOP): Integrating Elasticsearch Open Source with Vector Databases

Purpose

Dense vector search enables semantic retrieval by mapping similar content to nearby points in vector spaceelastic.co. This SOP outlines how to configure Elasticsearch (Open Source) with vector fields and optionally integrate external vector DBs to support efficient vector similarity and hybrid (lexical + semantic) search. The resulting setup powers Retrieval-Augmented Generation (RAG) pipelines, where external embeddings augment LLM knowledgezilliz.com elastic.co. By following these steps, teams can build scalable AI search applications that combine keyword queries and semantic matching in a repeatable, production-ready workflow.

Scope

Applies to engineering and ML teams working on AI-powered search solutions using Elasticsearch and vector databases. Typical use cases include:

Semantic search: Retrieving content by meaning or intent (e.g. question-answering, recommendation) using text embeddings.
Embeddings storage: Managing and indexing high-dimensional vectors for documents or images.
Hybrid search/RAG: Combining BM25 keyword search with dense-vector similarity for more relevant resultsmilvus.io elastic.co.
Multi-modal search: Integrating different data types (text, image) via their vector representations.

Pre-Requisites

Elasticsearch (Open Source) installed (latest version from GitHub recommended). Ensure a running cluster or single node.
Docker or Kubernetes for local cluster setup (optional but helpful for testing).
At least one vector database (e.g. Pinecone, Weaviate, Milvus, Qdrant, or pgVector/Postgres) if using external vector store.

Python 3.9+ environment with required libraries:

pip install elasticsearch sentence-transformers numpy

API keys/credentials for any external vector DB (Pinecone, Weaviate, etc.) if used.

Roles & Responsibilities

DevOps Engineer: Set up and manage the Elasticsearch cluster (nodes, scaling, backups, and monitoring).
Data/ML Engineer: Generate and validate embeddings from data using models (e.g. SentenceTransformers, OpenAI, Hugging Face)elastic.co.
Backend Engineer: Implement services/APIs to index data into Elasticsearch and sync embeddings with the vector DB; construct query logic that merges ES and vector DB resultsmilvus.io milvus.io.
QA Engineer: Test the accuracy and performance of vector similarity search and hybrid queries; verify relevance of search results in both ES-only and ES+vector-DB setups.

Procedure

Configure Elasticsearch for Vector Fields: Elasticsearch’s dense_vector field type allows storing fixed-length embeddings for kNN searchelastic.co. Create an index with a mapping that includes a vector field. For example:
```
PUT my-vector-index
{
  "mappings": {
    "properties": {
      "title":   { "type": "text" },
      "content": { "type": "text" },
      "embedding": {
        "type": "dense_vector",
        "dims": 768         // set to your embedding size (e.g. 768 for BERT)
      }
    }
  }
}
```
This defines embedding as a 768-dimensional dense vector field. The dense_vector type is designed for kNN search and cannot be used for sorting or aggregationselastic.co. Ensure the dims matches your model’s output dimension (e.g. 384 for MiniLM-L6-v2elastic.co).
Generate Embeddings: Use a pre-trained model to convert text into vector embeddings. Frameworks like SentenceTransformers are recommended for this taskelastic.co. For example, in Python:
```
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')  # 384-dimensional vectors:contentReference[oaicite:12]{index=12}
text = "Elasticsearch is a powerful search engine"
embedding = model.encode(text).tolist()  # produces a fixed-length vector
```
The model outputs a floating-point vector (e.g. 384 dims for all-MiniLM-L6-v2). Verify the dimensionality so it matches the index mapping. Pre-compute embeddings during data ingestion to avoid query-time costs. Keep embeddings up-to-date if the model or data changes.
Index Documents with Vectors: Use the Elasticsearch client to index each document along with its embedding. Include the vector under the dense_vector field:
```
from elasticsearch import Elasticsearch
es = Elasticsearch("http://localhost:9200")

doc = {
    "title":   "Elasticsearch Overview",
    "content": "Elasticsearch is a powerful open-source search engine.",
    "embedding": embedding
}
es.index(index="my-vector-index", document=doc)
```
This stores the document text and its embedding in ES. As noted in the documentation, you should store embeddings via the dense_vector fieldelastic.co to enable kNN. If embedding storage is disabled (index: false), ES won’t index the vectors for search; ensure indexing is enabled (default) or explicitly set for ANN search.
Perform Vector Search in Elasticsearch: Use the k-NN query to find nearest neighbors by vector similarityelastic.co. For example, to retrieve the top 5 nearest documents:
```
POST my-vector-index/_search
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.12, -0.08, ...], 
    "k": 5,
    "num_candidates": 100
  }
}
```
This returns the 5 documents whose embeddings are closest (e.g. by cosine similarity or dot product) to the query vector. Elasticsearch automatically uses the HNSW algorithm for fast ANN search when indexing is enabledelastic.co. The knn query is specifically designed for dense vectorselastic.co. Adjust num_candidates for search precision vs. speed trade-off.
Integrate with an External Vector Database (Optional): For very large vector collections or specialized ANN requirements, use a dedicated vector database alongside Elasticsearch. There are two common patterns:
- Dual Indexing: Store metadata (title, content, etc.) in Elasticsearch and store the embeddings in the vector DB (e.g. Pinecone, Weaviate, or Milvus). Keep both in sync (via pipelines or CDC). At query time, search both: ES for keyword matches and the vector DB for semantic matches, then merge resultsmilvus.io milvus.io. For example, product descriptions reside in ES; their embeddings in Pinecone. A user query goes to both systems, and results (exact keyword hits and semantically similar items) are fused, often by score fusion or ranking modelsmilvus.io.
- Meta+Vector Split: Use Elasticsearch for filtering/metadata and the vector DB for nearest-neighbor search. In this RAG-style flow: embed the user query, query the vector DB to get top-N document IDs by similarity, then retrieve full documents from ES by those IDsmilvus.io. For instance:
  1. Query embedding: Convert user input to a vector.
  2. Vector DB search: Find top-K similar vectors (returns doc IDs).
  3. Fetch documents: Retrieve those documents’ metadata (title/content) from ES.
  4. Augment LLM: Send the aggregated content to the LLM for answer generation.
Both approaches leverage each system’s strengths: vector DBs scale efficiently for high-dimensional search, and Elasticsearch excels at full-text queries and filtering. Synchronize data (e.g. with change-data-capture or update hooks) so both stores remain consistentmilvus.io milvus.io.
Implement Hybrid Search (Text + Vectors): To combine lexical and semantic relevance in a single query, use a boolean query that includes both match and k-NN clauseselastic.co. For example:
```
POST my-vector-index/_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "content": "distributed search engine" }},
        {
          "knn": {
            "field": "embedding",
            "query_vector": [0.12, -0.08, ...],
            "k": 10,
            "num_candidates": 100
          }
        }
      ]
    }
  }
}
```
This hybrid query retrieves documents matching the text query and/or having embeddings similar to the query vector. Elastic’s search supports mixing a lexical search clause with a KNN clause in the same requestelastic.co. The results will include exact keyword hits as well as semantically related documents. (More advanced hybrid features like the retriever API and rank fusion are available in Elastic Enterprise editions, but the above bool + knn approach works in open-source as well.)
Monitor & Maintain: Use Elasticsearch’s operational tools to ensure reliability and performance:
- Index Lifecycle Management (ILM): Define ILM policies to rollover, shrink, or delete indices as they age or growelastic.co. For example, create daily indices and delete those older than a retention period. ILM helps manage storage and performance for time-series or large-scale dataelastic.co.
- Performance Monitoring: Use Kibana (or other dashboards) to monitor search latency, query throughput, and resource usage. Track vector search metrics (e.g. KNN query times) as data volume grows.
- Model Updates: Periodically retrain or update your embedding models if the domain language evolves. Re-generate embeddings for new or changed documents. Use evaluation metrics (e.g. recall@K, precision) on test queries to ensure search quality remains high.

Validation

Relevance Testing: Manually and automatically verify that vector queries return semantically relevant results. Compare ES-only vs. ES+vector results to ensure hybrid search improves outcomesmilvus.io elastic.co.
Accuracy Benchmarks: For known query sets with expected results, measure recall/precision for KNN search and keyword search. Ensure combined results meet business requirements (e.g. user relevance).
Latency Benchmark: Measure query latency as index size scales, comparing pure ES vs. external vector DB setups. Adjust architecture if performance degrades (e.g. shard counts, ANN parameters).
End-to-End Testing: In RAG applications, validate that LLM answers improve with the retrieved context. Ensure no sensitive data leaks through embeddings (tokenize or anonymize data as needed prior to embedding).

Security & Compliance

Secure Communications: Enable TLS/SSL on Elasticsearch HTTP and transport layers. Use API keys or basic authentication to restrict access to the ES cluster. Similarly secure connections to any vector DB service.
Access Control: Implement role-based access so that only authorized services/users can query vector fields. For external vector DBs, use scoped API keys.
Data Privacy: Avoid embedding sensitive PII or confidential content. If needed, tokenize or hash sensitive fields before embedding. Ensure compliance with data regulations (GDPR, HIPAA) when storing user data.
Auditing: Log search queries and indexing operations for auditing. Monitor for unusual activity (e.g. excessive vector queries) that may indicate misuse.

References

Elasticsearch Reference: Dense vector field typeelastic.co (mapping and usage).
Elasticsearch Documentation: Dense vector search and kNN querieselastic.co elastic.co.
Milvus (Vector DB) Guide: Integrating a vector database with Elasticsearchmilvus.io milvus.io.
Elastic Blog: Optimize your RAG workflows with Elasticsearch and Vectorizeelastic.co elastic.co.
Zilliz Blog: Weaviate vs Elasticsearch (vector DB comparison)zilliz.com zilliz.com.
Elasticsearch Labs Tutorial: Generate Embeddings using SentenceTransformerselastic.co elastic.co.
Elasticsearch Labs Blog: Hybrid search support in Elasticsearchelastic.co.
Elasticsearch Docs: Index Lifecycle Management (ILM)elastic.co.

SOP, Elastic Search, Vector DatabaseFrancesca Tabor13 September 2025