Support bedrock rerank API

[wuodar]

As AWS Bedrock supports now rerank API, I wanted to ask whether there are any plans to integrate it into langchain-aws, either into bedrock retriever or as a standalone module, like Document Compressor? Here is boto3 documentation

[3coins]

@wuodar
Thanks for sharing info about the rerank API. It seems like it would be flexible to create this as a separate runnable, so it could be used with other retrievers, and not just Bedrock KnowlegeBases. On the other hand, there are some benefits in abstracting it inside the KnowledgeBases retriever, that reduces onus on users to plug this into their application pipeline.

Do you have a specific use case that can help with this decision? Are there other APIs like this that are available from other providers, and how have they implemented this?

[jpfcabral]

I supose this is a good feature to implement, since I need to compare the Bedrock KnowlegeBases with other retrievers. It would be easier to switch between retrievers and choose a better option.

[tomaszdudek7]

I suppose the possibility of using it both in Bedrock KB and as a standalone are needed for some groups of people.

In my case - I'm using both approaches in several projects and I would love if LC supported them. :)

[jpfcabral]

@wuodar Thanks for sharing info about the rerank API. It seems like it would be flexible to create this as a separate runnable, so it could be used with other retrievers, and not just Bedrock KnowlegeBases. On the other hand, there are some benefits in abstracting it inside the KnowledgeBases retriever, that reduces onus on users to plug this into their application pipeline.

Do you have a specific use case that can help with this decision? Are there other APIs like this that are available from other providers, and how have they implemented this?

The Cohere rerank is used as contextual retrievers on large applications.

Based on the documentation, we can see the use case:

from langchain.retrievers.contextual_compression import ContextualCompressionRetriever
from langchain_cohere import CohereRerank
from langchain_community.llms import Cohere

llm = Cohere(temperature=0)
compressor = CohereRerank(model="rerank-english-v3.0")
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
pretty_print_docs(compressed_docs)

This kind of implementation is very relevant when you have to use other tools, such as Langgraph. Usually we need to create a single compressed retriever with rerank capability to create nodes.

See: https://medium.com/@myscale/enhancing-advanced-rag-systems-using-reranking-with-langchain-523a0b840311

Since rerank APIs are still emerging (no many companies offer the service) I think it is very valid that there is already a set of tools that facilitate its use.

I already created a PR #331 with the code I used on internal projects. Its based on the Cohere Rerank implementation, so its replaceable with the Langchain patterns (i.e. Document Compression)

The PR comments show some snippets and some comparisons with the documentation provided by Cohere