The OpenAI Embeddings API: A Developer’s Guide to Vector Search and Semantic Retrieval
Disclosure: This post contains one or more links to providers we have a real, registered affiliate/referral relationship with. We may earn a commission at no extra cost to you if you sign up through them.
The OpenAI embeddings API converts text into numerical vectors that capture semantic meaning, enabling search, clustering, and recommendation systems that understand context rather than just matching keywords. This guide covers how the API works, how to integrate it into a production pipeline, and how to run the supporting infrastructure reliably.
What the OpenAI Embeddings API Actually Returns
When you call the OpenAI embeddings API, you send a string (or a batch of strings) and receive back an array of floating-point numbers — the embedding vector. Each dimension in that vector doesn’t correspond to anything a human can read directly; instead, the vector’s position in high-dimensional space encodes semantic relationships. Text with similar meaning produces vectors that sit close together when measured with cosine similarity or dot product.
This is fundamentally different from traditional keyword search. A query like “how to restart a crashed container” and a document titled “recovering a failed Docker service” share almost no overlapping words, but their embeddings will be close in vector space because the underlying meaning overlaps. That property is what makes the openai embeddings api useful for retrieval-augmented generation (RAG), semantic search, deduplication, and classification tasks.
The current generation of models (text-embedding-3-small and text-embedding-3-large) also supports a dimensions parameter, letting you request a shorter vector than the model’s native output size. Shorter vectors cost less to store and search, at some tradeoff in retrieval quality — a decision worth testing against your own dataset rather than assuming a default is correct.
Model Selection Tradeoffs
text-embedding-3-small is cheaper and faster, and for many internal tools (log clustering, ticket deduplication, FAQ matching) it performs well enough that the larger model isn’t worth the extra cost. text-embedding-3-large produces higher-dimensional vectors with generally stronger retrieval accuracy, which matters more for customer-facing search where relevance directly affects user experience.
A practical approach: prototype with the small model, measure retrieval quality against a labeled test set of query/document pairs, and only upgrade if the small model’s failure rate is actually a problem. Guessing at which model you need without measurement usually leads to over-provisioning.
Setting Up a Basic Integration
A minimal integration is just an HTTP call. Here’s a shell example using curl to sanity-check your API key and see the raw response shape before writing any application code:
curl https://api.openai.com/v1/embeddings
-H "Authorization: Bearer $OPENAI_API_KEY"
-H "Content-Type: application/json"
-d '{
"model": "text-embedding-3-small",
"input": "Docker Compose lets you define multi-container applications in a single YAML file."
}'The response contains a data array with one object per input string, each holding an embedding field (the vector) and an index. For production use, you’ll want to batch multiple strings into a single request — the input field accepts an array — since batching reduces per-request overhead and is generally cheaper than issuing one call per document.
Storing Vectors for Retrieval
Once you have embeddings, you need somewhere to store and query them. Options range from a dedicated vector database (Pinecone, Weaviate, Qdrant) to using pgvector inside an existing PostgreSQL instance, which is often the simplest choice if you’re already running Postgres for other application data.
A pgvector-backed table might look like this:
# docker-compose.yml snippet for a Postgres instance with pgvector
services:
postgres:
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: changeme
POSTGRES_DB: embeddings_db
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:If you’re already comfortable running Postgres in containers, this guide on Postgres Docker Compose setup covers the broader configuration details, and the Docker Compose secrets guide is worth reading before you put a real API key or database password into a compose file — plaintext environment variables in version control are a common and avoidable mistake.
Batching and Rate Limits
The openai embeddings api enforces rate limits based on requests-per-minute and tokens-per-minute, which vary by account tier. When embedding a large corpus (migrating an existing document set, for instance), you should:
429 responses rather than a fixed retry delayBuilding a Semantic Search Pipeline
A typical pipeline has three stages: ingest and embed source documents, store the vectors alongside metadata, and query with a user’s input embedded the same way, then rank results by similarity.
The critical detail engineers miss is that the query and the documents must use the same model and same dimensionality. Mixing text-embedding-3-small vectors with text-embedding-3-large vectors in the same similarity comparison produces meaningless results — the vector spaces aren’t compatible across models.
Chunking Strategy
Long documents need to be split into chunks before embedding, since a single embedding for a 10,000-word article averages out too much meaning to be useful for precise retrieval. A common approach is fixed-size chunking (500-1000 tokens per chunk) with some overlap between consecutive chunks to avoid cutting a relevant sentence in half at a boundary.
There’s no universally correct chunk size — it depends on your content structure and how precise you need retrieval to be. Documentation with clear section headers benefits from chunking along those natural boundaries rather than a fixed token count.
Similarity Search at Query Time
Once you have stored vectors, a query-time lookup with pgvector uses standard SQL with a vector distance operator:
SELECT content, embedding <=> '[0.012, -0.045, ...]' AS distance
FROM document_chunks
ORDER BY distance
LIMIT 5;For workloads beyond a few hundred thousand vectors, you’ll want an approximate nearest-neighbor index (HNSW or IVFFlat in pgvector) rather than an exact scan, since exact comparison against every stored vector doesn’t scale well as the table grows.
Automating Embedding Pipelines With Workflow Tools
If you’re generating embeddings as part of a larger content pipeline — for example, embedding every newly published article for internal search, or re-embedding a knowledge base whenever documents change — running that logic through a workflow orchestrator instead of a standalone cron script gives you retries, logging, and visibility for free.
Tools like n8n can call the openai embeddings api directly via an HTTP Request node, store results in Postgres, and trigger downstream steps like updating a search index. If you’re self-hosting this kind of automation, the n8n self-hosted installation guide and n8n automation guide cover getting an instance running on a VPS, and the n8n API guide is useful if you want to trigger embedding jobs programmatically rather than only on a schedule.
For teams comparing tools before committing to one, the n8n vs Make comparison is a reasonable starting point — the tradeoffs matter more once you’re running production embedding jobs at scale, not just prototyping.
Cost Management and Monitoring
Embedding costs scale with the number of tokens processed, not the number of API calls, so batching reduces overhead but not the underlying token cost. Before running a large-scale embedding job, estimate token count first — most tokenizer libraries let you count tokens locally without making an API call, which is worth doing before committing to embedding a multi-gigabyte corpus.
Ongoing considerations for cost control:
If you’re already tracking OpenAI spend for other parts of your stack, the OpenAI API pricing guide and OpenAI API cost breakdown are useful references for understanding how embedding costs fit into your overall API bill, and the OpenAI API reference documents the full set of parameters available on the embeddings endpoint beyond what’s covered here.
Deployment and Infrastructure Considerations
If your embedding pipeline runs as a background service — polling for new content, generating vectors, and writing to a database — it needs somewhere reliable to run. A small VPS is generally sufficient for moderate embedding volume, since the actual compute work (the API call itself) happens on OpenAI’s infrastructure, not locally. Your server’s job is orchestration: fetching content, calling the API, and writing results.
For teams self-hosting this kind of service, providers like DigitalOcean or Hetzner offer VPS instances suitable for running a lightweight embedding pipeline alongside a Postgres/pgvector instance, particularly if you’re already running other Docker-based services on the same box. Keep the pipeline containerized so it’s easy to redeploy — the Dockerfile vs Docker Compose guide is a good primer if you’re deciding how to structure the deployment.
Regardless of where it runs, treat your OpenAI API key the same way you’d treat a database credential: never commit it to source control, and inject it via environment variables or a secrets manager rather than hardcoding it into application code.
Recommended: Ready to put this into practice? DigitalOcean is a tool we use for exactly this, and we have a real, disclosed affiliate relationship with them.
FAQ
What’s the difference between text-embedding-3-small and text-embedding-3-large?
The small model produces lower-dimensional vectors and costs less per token, while the large model generally retrieves more accurately at higher cost and storage overhead. Which one you need depends on your accuracy requirements — test both against a sample of your actual queries rather than assuming the larger model is always worth it.
Can I compare embeddings from different OpenAI models?
No. Vectors from different models occupy different, incompatible spaces, even if they have the same number of dimensions. Always embed both your documents and your queries with the same model.
How do I reduce the cost of embedding a large document set?
Deduplicate identical or near-identical text before embedding, cache results so you never re-embed unchanged content, and batch multiple inputs into single API calls rather than issuing one request per string.
Do I need a dedicated vector database, or can I use Postgres?
For most small-to-medium workloads, pgvector on an existing Postgres instance is sufficient and avoids adding a new piece of infrastructure to operate. Dedicated vector databases become more attractive at very large scale or when you need specialized indexing features they offer that pgvector doesn’t.
Conclusion
The openai embeddings api is a straightforward HTTP interface, but building a reliable system around it — chunking strategy, storage, rate-limit handling, cost tracking, and deployment — is where the real engineering work happens. Start with a small model and a simple pgvector setup, measure retrieval quality against real queries, and only add complexity (larger models, dedicated vector databases, more sophisticated chunking) once you’ve confirmed the simpler approach isn’t meeting your needs. For further detail on request parameters and response formats, the official OpenAI API documentation and OpenAI API reference are the authoritative source, and general vector-search concepts are also well covered in the PostgreSQL documentation if you’re building on pgvector.
Leave a Reply