What Is a Vector Database and How Do You Choose One?

Understand vector databases without writing code: how they work, how they differ from regular databases, and what to ask before you pick one.

At the first meeting of an AI project, someone from IT often asks, "Do we need to buy a vector database?" and the room goes quiet, because nobody is sure what it is, how necessary it is, or how it differs from the database the company already uses. This article will help you discuss it with your team confidently.

What is a vector database?

A vector database is a system designed to store and search vectors, or embeddings: lists of numbers that represent the meaning of text, images or other data. Its main job is answering "which items are closest in meaning to what was asked?", which is the heart of the retrieval step in a RAG system.

Ordinary databases are good at exact lookups, such as "find employee ID 1024". A vector database is good at "similar" lookups. For example, the question "Can I take leave for my brother's ordination?" should find the document on ordination leave even if the wording is completely different.

How does it search so fast?

Exact search: compare everything

The straightforward way is to compare the question with every vector in the store. It is the most accurate, but it gets slower and slower as you reach millions of items.

Approximate search: trade a little accuracy for speed

Most systems therefore use an approximate index (Approximate Nearest Neighbor). A very popular one is HNSW, from research by Malkov and Yashunin (2016). It is like finding a house in a big city: start with a national map, zoom into the province, the district and then the street, instead of knocking on every door.

The documentation for pgvector, a PostgreSQL extension, explains that approximate search "trades some recall for speed", so results can differ slightly from exact search. It also notes that HNSW has a better speed-recall trade-off than IVFFlat indexes, but builds more slowly and uses more memory.

The main options available

• An extension to your existing database: for example pgvector on PostgreSQL. A good fit if your team already uses Postgres; business data and vectors live together and are easy to maintain.
• A dedicated vector database: for example Pinecone, Weaviate, Milvus, Qdrant or Chroma, built specifically for this job, available as cloud services or self-hosted.
• A search engine with vector capabilities: many search systems support both keyword and vector search in one place, which suits work that needs hybrid search.

Questions to ask before you choose

1. How much data? A few thousand pages of documents and millions of products need very different systems.
2. What does your team know? A system your team can already run is often worth more than a stronger one nobody understands.
3. Do you need filters? For example, only search your own department's documents, or only the latest year's versions.
4. Where must the data live? Some organisations must keep data in the country or on their own systems, per policy and Thailand's PDPA.
5. Long-term cost: service fees, hardware, and the team time needed to maintain it.

An example decision

A mid-sized insurance company wants a chatbot that answers sales agents' questions from about a hundred product manuals, and it already runs PostgreSQL. The team enables pgvector in the existing system rather than adding a new one. An online store with millions of products that needs meaning-based search every time a customer types, on the other hand, may find a dedicated system built for high volume worth the investment.

Watch out for

• Don't choose based on hype or vendor benchmarks alone. Test with your own data and questions.
• Approximate search can miss some relevant documents. Periodically measure recall against exact search.
• A vector database does not fix poor-quality documents. If the content is wrong or badly chunked, fast search still returns wrong answers.
• Set access permissions to match the source documents, rather than letting everyone find everything.

The best database is the one your team can actually run.

How to get started

1. Check which databases your organisation already uses and whether they support vectors.
2. Roughly estimate your document volume and questions per day.
3. Build a small prototype with the simplest option first.
4. Measure with a test question set, for both accuracy and speed.
5. Move to a dedicated system only when the numbers show you need it.

To understand the earlier step that creates the vectors, read Chunking and Embedding next. And if you are interested in data and AI roles, browse jobs on Plants.work.