Update samples for the In-Memory VectorStore Driver

[joehu21]

Updated samples for the In-Memory VectorStore Driver, and added the missing PDF test file. Manually tested with MemoryDB 7.1.1 cluster.

[3coins]

Can simplify the import.

from langchain_aws.llms import BedrockLLM

Also, why are we not using the ChatBedrock class?

[joehu21]

Tried changing it to ChatBedrock, but the RetrievalQA code below failed. I think the code below also needs to change. Will investigate when I get some free time. It is a learning opportunity for me. We can leave it as is for now.

qa_prompt = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vds.as_retriever(),
    return_source_documents=True,
    chain_type_kwargs={"prompt": PROMPT},
    #verbose="true"
)
query = "How do i create a MemoryDB cluster"
result = qa_prompt({"query": query})

[3coins]

RetrievalQA chain has been deprecated. Use this instead.

from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
from langchain_aws import ChatBedrock


retriever = ...  # Your retriever
llm = ChatBedrock(...)

system_prompt = (
    "Use the given context to answer the question. "
    "If you don't know the answer, say you don't know. "
    "Use three sentence maximum and keep the answer concise. "
    "Context: {context}"
)
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
chain = create_retrieval_chain(retriever, question_answer_chain)

chain.invoke({"input": query})

[joehu21]

Still getting the same error:

ValueError: Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: subject must not be valid against schema {"required":["messages"]}#: extraneous key [max_tokens_to_sample] is not permitted, please reformat your input and try again.

code:

system_prompt = (
    "Use the given context to answer the question. "
    "If you don't know the answer, say you don't know. "
    "Use three sentence maximum and keep the answer concise. "
    "Context: {context}"
)
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
chain = create_retrieval_chain(retriever, question_answer_chain)
query = "How do i create a MemoryDB cluster?"
chain.invoke({"input": query})

[joehu21]

Thanks for the offline help. Pushed the 3rd commit to change to ChatBedrock and ChatPromptTemplate.

[3coins]

See my comment above. Even if you want to use the dotenv, the correct syntax would be to use the magics in a new cell. You don't need the import above for this.

%load_ext dotenv
%dotenv

[joehu21]

Good to know.

[3coins]

Users are most likely to use claude-3 models.

[joehu21]

Tried to change it to claude-3, but it fails:

ValueError: Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: The provided model identifier is invalid.

Here's my test:

from langchain_aws import BedrockLLM

# initialize the Bedrock LLM
llm = BedrockLLM(
    model_id="anthropic.claude-v3"
)

prompt = "What is the largest city in Vermont?"

# return a response to the prompt
response = llm.invoke(prompt)
print(response)

Any idea why the above fails?

BTW, langchain-aws README still uses claude-2.

[3coins]

That model id seems wrong. You can use one of these.

anthropic.claude-3-haiku-20240307-v1:0
anthropic.claude-3-opus-20240229-v1:0
anthropic.claude-3-sonnet-20240229-v1:0

Also, can you use ChatBedrock.

[joehu21]

updated