Add stuff-summarisation using runnable

[andy-symonds]

Context

First pass as adding stuff summarisation functionality using Langchain Expression Language (LCEL) runnables.

Changes proposed in this pull request

1. I pulled in changs from REDBOX 337 chat file selection #556 and from Turning RAG into a runnable function #554
2. Keep a single build chain method in chat.py which uses the routing layer to decide with chain to build.
3. Pulled out specific build_'x'_chains into a separate file, called build_chains.py. In build_chains.py there is the original build_vanilla_chain and build_retrieval_chain (both of which we will want to change to LCEL runnables as soon as possible, but that is not the focus of this PR)
4. The ROUTING_RESPONSES dict has been updated, so that if the summarisation route is chosen, the build_chain selected is build_stuff_chain. The idea behind this is it would be extensive to many different types of chains mapped to different routes in future.
5. This is a draft! So build_stuff_chain is not yet complete. It uses the make_stuff_document_runnable. @wpfl-dbt could you add retrieval of the documents for summarisation?

Guidance to review

1. Not ready for PR review yet - first pass. Please could you sense check all of this!
2. I was trying to figure this out and seeing what would work, so have not got to tests yet. Sharing this early, so you can pick it up in the morning :)

Relevant links

Things to check

• I have added any new ENV vars in all deployed environments
• I have tested any code added or changed
• I have run integration tests

[wpfl-dbt]

I initially wasn't sold on the direction, but the more I think about it, the more I like it.

Overall, I think:

• We should have a principle of one build_* function per route (even the mundane ones), and these functions should use common keyword arguments and datatypes, and identical output shapes
• This enables build_chain to focus on what it's really about -- routing logic -- because it makes chain building trivial
• The summary chain needs work
• If Turning RAG into a runnable function #554 gets in before this PR, why not bring LCEL on the original runnables into this PR? As I see it, this is the "everything's in, plumb it up" PR

[wpfl-dbt]

See my previous comment about **kwargs. Here you could then safely do...

build_chain(
    **{
        "chat_request": chat_request,
        "user_uuid": user_uuid,
        "llm": llm,
        "vector_store": vector_store,
        "storage_handler": storage_handler
    }
)

...content in the knowledge it'll work for every single build_* function. No more treating the chat function differently, no more worrying about how to handle future use cases!

[wpfl-dbt]

-> tuple[Runnable, dict[str, Any]] on all these to help understand what they're doing

[wpfl-dbt]

1. Add **kwargs to all these function signatures
2. Ensure (as you currently have) that when they use a common keyword argument, it takes the same data
3. Now you can call them all with unpacked dictionaries of keyword arguments

This means that build_vanilla_chain can be called in just the same way as all the other build_* functions even though it only needs a fraction of the arguments.

[wpfl-dbt]

Suggest generalising to build_summary_chain -- I think these build_* functions should match the routes even if they wrangle multiple runnables to do so.

See summarise() in the summarise notebook for my take on this function. You're going to need a storage_handler: ElasticsearchStorageHandler arg so you can retrieve the documents to summarise using core_api.src.format.get_file_chunked_to_tokens().

[wpfl-dbt]

See the unit test for what this should be (and actually a good example of what you need in this function overall)

[wpfl-dbt]

This overwrites its own function name -- change it

[wpfl-dbt]

Get an output signature on this.

I also think a name like route_input_to_chain() would be more helpful to understand what's going on here.

As I suggest above, I think this function's main role is to deal with routing logic. Getting from route -> chain can be made trivial, especially in a-build_-function-per-route world.

[wpfl-dbt]

⚠️ I am 50/50 on the quality of this suggestion

Imo make this a nice consistent dict[str, Callable] -- make all values functions. Ways you might tackle this:

1. build_precanned_chain(response: ChatPromptTemplate, ...), and functools.partial a version with the response filled into each route
2. Start the pattern of "one build_* function per route" on the principle that it makes them all extensible, and most of them will one day need to be

I like this because it means in the function that deals with routing, once you've got the route name, you can just

    ...
    return ROUTE_RESPONSES.get(route.name)(
        **{
            "chat_request": chat_request,
            "user_uuid": user_uuid,
            "llm": llm,
            "vector_store": vector_store,
            "storage_handler": storage_handler
        }
    )

This means that function can focus on the routing logic, which will get more complex as user override is added, and the "what chain do I need" stuff is moved out of the way.

Tbh, if you got to the point where the above was possible, consider changing build_chain to pure routing logic, returning a string, because that little snippet would be all you needed to connect routes with the configured runnables you need. Just put it in the endpoint directly.

As I say, 50/50 on this one...

[wpfl-dbt]

Although strap in for mypy's take on that code...

[wpfl-dbt]

Merged as part of #570