vectorstores: Removed the custom ID option for embeddings when using PGVector

[pieterclaerhout]

PR Checklist

• Read the Contributing documentation.
• Read the Code of conduct documentation.
• Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
• Describes the source of new concepts.
• References existing implementations as appropriate.
• Contains test coverage for new functions.
• Passes all golangci-lint checks.

---

See this discussion for more details:

#561 (comment)

[eliben]

It would be good for other folks who worked on pgvector to chime in

@Abirdcfly

@tmc WDYT?

[tmc]

I agree that with the metdata field this can go there, we can revisit if necessary.

[Abirdcfly]

It would be good for other folks who worked on pgvector to chime in

@Abirdcfly

@tmc WDYT?

The pgvector stores the data in postgresql, IMO, the code is changeable, the data is a bit more precious.

When creating the table structure, I basically copied the langchain-python structure, and I wanted the table created by langchain-python could be reused by the langchain-go code.

https://github.com/langchain-ai/langchain/pull/1630/files#diff-dc91245f981bb33717e95ed1c7347e5dfbd214a1a243a3f516a0895481a655a5R92 langchain/vectorstores/pgvector.py:92

class EmbeddingStore(BaseModel):
    __tablename__ = "langchain_pg_embedding"

    collection_id: Mapped[UUID] = sqlalchemy.Column(
        UUID(as_uuid=True),
        sqlalchemy.ForeignKey(
            f"{CollectionStore.__tablename__}.uuid",
            ondelete="CASCADE",
        ),
    )
    collection = relationship(CollectionStore, back_populates="embeddings")

    embedding: Vector = sqlalchemy.Column(Vector(ADA_TOKEN_COUNT))
    document = sqlalchemy.Column(sqlalchemy.String, nullable=True)
    cmetadata = sqlalchemy.Column(sqlalchemy.JSON, nullable=True)

    # custom_id : any user defined id
    custom_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)

custom_id is a field that has existed since the very beginning of the pull request.

langchain-python uses this field by passing in optional parameters:

https://github.com/langchain-ai/langchain/blob/c37ca45825b37eed90d773691772a6be4da4322b/libs/community/langchain_community/vectorstores/pgvector.py#L405

    def add_texts(
        self,
        texts: Iterable[str],
        metadatas: Optional[List[dict]] = None,
        ids: Optional[List[str]] = None,
        **kwargs: Any,
    ) -> List[str]:
        """Run more texts through the embeddings and add to the vectorstore.

        Args:
            texts: Iterable of strings to add to the vectorstore.
            metadatas: Optional list of metadatas associated with the texts.
            kwargs: vectorstore specific parameters

        Returns:
            List of ids from adding the texts into the vectorstore.
        """
        embeddings = self.embedding_function.embed_documents(list(texts))
        return self.add_embeddings(
            texts=texts, embeddings=embeddings, metadatas=metadatas, ids=ids, **kwargs
        )

I now think it's possible to expose the filter and use this field using vectorstores.Option, and it's fine to create it using another pull request if anyone needs it.

[tmc]

Hmm I should have referenced the python implementation. With it as a separate field there I'm more inclined to preserve it.