by francoismassot on 1/23/24, 10:34 AM with 167 comments
by francoismassot on 1/23/24, 1:19 PM
There are a lot of OSS vector search databases out there, we could probably list the main ones:
- Qdrant: https://github.com/qdrant/qdrant
- Weaviate: https://github.com/weaviate/weaviate
- Milvus: https://github.com/milvus-io/milvus
What else?
by lettergram on 1/23/24, 1:03 PM
I’ve been working with vectors for over a decade; particularly with embeddings used in AI. We’re talking projects from 100k to 100B+ records, used for AI applications
Postgres, particularly with pgvector and derivatives, can handle to millions of records very rapidly no problem. It’s very cheap, scales great, and is accurate.
I’m sure some of these open source solutions are improvements. That said, weigh vendor lock in, cost, risk and in the end it usually makes very little sense.
by tw1984 on 1/23/24, 1:22 PM
by clbrmbr on 1/23/24, 11:30 AM
by weinzierl on 1/23/24, 11:33 AM
They are also a Rust shop.
Who says Germany has no cool startups.
EDIT: Yes, it was Grok.
by mindvirus on 1/23/24, 11:32 AM
What have your experiences with vector databases been? I've been using https://weaviate.io/ which works great, but just for little tech demos, so I'm not really sure how to compare one versus another or even what to look for really.
by tw1984 on 1/23/24, 2:25 PM
for now is the keyword here.
When the company grow to 10k or 30k people, there will be teams competing for visibility, someone is going to build their inhouse "vector database" to get his/her slice of the pie. Do you still believe that any AI major player is going to reply on some external vector databases?
by hartator on 1/23/24, 3:54 PM
Small revolution indeed.
by anonzzzies on 1/23/24, 11:49 AM
by anonzzzies on 1/23/24, 11:47 AM
by softwaredoug on 1/23/24, 3:02 PM
I genuinely ask - there are a lot of other problems in the RAG, fine tuning, AI/LLM, retireval space, to solve. And more and more vector retrieval is, while not 100% solved, at least is something the community has a grasp on the tradeoffs. Solved to the point that squeezing a bit more recall out of vector retrieval isn't the problem anymore.
by shanghaikid on 1/23/24, 11:38 AM
What do you think you milvus? https://milvus.io/. The difference seems significant from the architecture perspective.
by infecto on 1/23/24, 12:26 PM
by redwood on 1/23/24, 11:29 AM
by yujian on 1/23/24, 3:01 PM
(now I'm gonna plug what I work on)
If you're interested in a more scalable vector database written in Go, check out Milvus (https://github.com/milvus-io/milvus)
by rvz on 1/23/24, 11:59 AM
Let’s see what they can do in a year or more with that new capital.
by braza on 1/23/24, 6:23 PM
by ancorevard on 1/23/24, 5:30 PM
by wahnfrieden on 1/23/24, 5:32 PM
by yding on 1/23/24, 6:55 PM
by spullara on 1/23/24, 6:15 PM
by _mh56 on 1/23/24, 11:46 AM
"We are getting many applications for this position. Usually, a test task would help preselect suitable candidates. However, since we develop open-source software, we rely on contribution.
You can build an open-source Qdrant connector to another framework or library. The simplest one would be, for example, a Streamlit data connector. But other ideas are more than welcome!
No limitations and no deadline. As long as this job position is online, we accept submissions. After you are done, send us an email to career@qdrant.com with the link to the repo. We will review it and get back to you asap."
No interviews, conversation before this email. Hope they see and fix this.
Edit : No Pay.