RAG, Langchain and Chromadb example #31
Conversation
MichaelClifford
force-pushed
the
rag-langchain
branch
from
e3fce64 to
ad0243d
Compare
MichaelClifford
force-pushed
the
rag-langchain
branch
from
bbcb65f to
371502d
Compare
MichaelClifford
changed the title
| arch: | ||
| - arm64 | ||
| - amd64 | ||
| - name: chromadb-server |
There was a problem hiding this comment.
We need a way to differentiate it from the sample app
There was a problem hiding this comment.
Then idk how fast it is to start but if it's slow and the sample app fails to connect we need a way to listen to its port so when it's ready we can restart the sample app container. I need to test it
There was a problem hiding this comment.
We talked with @jeffmaury earlier. Now we find the port to listen to by using the containerfile (we look at the EXPOSE). If we don't have it as we use an image, maybe it's better to specify the port inside the ai-studio.yaml. if you decompile the image you could see multiple ports (also some belonging to the parent image)
There was a problem hiding this comment.
Isen't it differentiated by the model-service=True parameter?
There was a problem hiding this comment.
I agree - any information you need to build the application will be best if included in the ai.yaml - so you only have to depend on that single file
There was a problem hiding this comment.
wouldn't there be instances where you'd want the user to define the ports to use just before runtime?
There was a problem hiding this comment.
so atm what we do is, if model_service=true we mount the volume/set he model_path else we set the model_endpoint env.
When running the chromadb service the model_endpoint is not really necessary, so it would be nice to know if the item we are handling is the sample app or the vector db.
Regarding the port, by reading the readme i see that we should run podman run -it -p 8000:8000 chroma but where do i take the port value i have to open if we do not add it in the ai-studio?
There was a problem hiding this comment.
should we add a flag vectordb=true?
There was a problem hiding this comment.
we should keep in mind to be as much generic as we can. I don't see why we need to differentiate client app and vector db. If we need to pass the port to another container then we should do like what is done in Kubernetes ie have env var named CONTAINER_NAME_PORT
There was a problem hiding this comment.
@jeffmaury i don't get your point. We need to know if we are working with a sample app or a vectorDB. When dealing with the sample app container we need to open its port into the pod so that it can be reached, nothing for the vectorDB. For the sample app we also need to fill the env variable so that it can connect to the model service, nothing for the vectorDB apparently. So there is a difference when adding a sample app or a vectorDB to the pod.
MichaelClifford
force-pushed
the
rag-langchain
branch
from
2c3e7f9 to
2759190
Compare
This PR adds a new recipe
rag-langchainto the repo. In this example we build only the ai-application code and rely on the existing playground image for our model service and the external chormaDB image for our vectorDB service.