Skip to content

Conversation

@jba
Copy link
Contributor

@jba jba commented Jun 8, 2024

Vector DBs only register two actions: an indexer and a retriever.
Continue to allow users to look up these actions by name.
But also return them from Init.

Vector DBs only register two actions: an indexer and a retriever.
Continue to allow users to look up these actions by name.
But also return them from Init.
@jba jba requested review from ianlancetaylor and pavelgj June 8, 2024 11:03
@pavelgj
Copy link
Collaborator

pavelgj commented Jun 8, 2024

LGTM

One use case I'd like to point out, and I think it works fine with this proposal -- initializing multiple vector stores from the plugin:

	cfi, cfr, err := localvec.Init(context.Background(), localvec.Config{
		Name:     "cat-facts",
		Embedder: googleai.Embedder("embedding-001"),
	})
	dfi, dfr, err := localvec.Init(context.Background(), localvec.Config{
		Name:     "dog-facts",
		Embedder: googleai.Embedder("embedding-001"),
	})

Init conveys the semantics well -- you are only allowed to initialize it once per given Name.

@jba
Copy link
Contributor Author

jba commented Jun 10, 2024

Actually, I think you've provided a counterexample. I didn't realize you might want to instantiate a vector DB with two indexes in the same program. That makes the vector DB case look more like the generative model case, where there could be multiple named entities of the same kind. Now that I took a better look at your rag test app, I see that you actually use that ability.

So I think for consistency, a single Init should completely initialize the plugin.

Once we have multiple retrievers and indexers, I think pinecone.Retriever("my-index") is the best we can do.

@jba jba closed this Jun 10, 2024
@jba jba reopened this Jun 10, 2024
@jba
Copy link
Contributor Author

jba commented Jun 10, 2024

...or let's have Init return slices of actions. Best of both worlds.
You can just write actions[0] or actions[1] if there aren't too many, or you can call Name on them.
This also solves the problem of how to know what models were registered when you init a generator plugin with no explicit models, so that it calls ListModels.

@jba jba merged commit 0eab45f into main Jun 10, 2024
@jba jba deleted the jba-docstore-return-values branch June 10, 2024 11:43
randall77 pushed a commit that referenced this pull request Jun 10, 2024
Continue to allow users to look up actions by name.
But also return them from Init.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants