This repository contains code to load CrewAI configuration using Gradio UI from Excel xls and generate the Python code for a crew-job combination. A job is a sequence of tasks. The source of the crews is defined in the xls. It allows for quick prototyping and iterations without having to edit code, There is a set of basic agents which have tools defined. This is only a subset of what currently is available through crewai tools.
CrewAI is a framework for managing and executing tasks using AI agents. It allows for quick prototyping and iterations without having to edit code. A new clean requirements.txt is generated; command is in .sh file
Default pip install of the package
Default install of the crewai tools
Langchain is a library for building applications with language models. It provides tools for managing and orchestrating language model interactions. Other langchain packages that are used are listed in requirements.txt
Excel format is xlsx, list of sheets are read and loaded. The actual preparation of the crews is done beforehand as the crew details do not change much once developed.
API keys need to be provided as part of the environment, both running straight from source and when using docker container eg when hosting on render.com
The application is containerized using Docker. Below are the key configurations and steps to build and run the Docker container:
- Base Image: The application uses a slim version of Python 3.10.13 as the base image.
- Environment Variables:
PYTHONDONTWRITEBYTECODE=1: Prevents Python from writing.pycfiles.PYTHONUNBUFFERED=1: Ensures that Python output is not buffered, which is useful for logging.
- Working Directory: The application code is located in the
/appdirectory within the container. - Dependencies: Dependencies are installed from
requirements.txtusingpip. - Port: The application listens on port
8000. - Command: The application is run using Uvicorn with the command:
CMD ["python", "-m", "uvicorn", "main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "8000", "--reload"]
First, get the project dependencies installed
pip install -r requirements.txt
just run locally as
python3 main.py
and follow the gradio instructions on screen to open your browser and use the UI.
The Makefile provides several commands to manage Docker containers and the development environment:
- build: Builds the Docker image with the default image name
crew-ai-xls. - run: Runs the Docker container, stopping and removing any existing container with the same name.
- stop: Stops the running Docker container.
- rm: Removes the Docker container.
- logs: Follows the logs from the running Docker container.
- rerun: Stops, removes, rebuilds, and runs the Docker container.
- clean: Cleans up all unused Docker images and containers.
- conda: Activates the specified Conda environment. This is useful for managing dependencies in a consistent environment.
- pyrun: Runs the application using Uvicorn for local development. This command is useful for testing the application locally before deploying.
- reqs: Generates a
requirements.txtfile usingpipreqs. To allow easy deployment a docker image is provided;
When you're ready, start your application by running:
docker compose up --build.
Your application will be available at http://localhost:8000.
First, build your image, e.g.: docker build -t myapp ..
If your cloud uses a different CPU architecture than your development
machine (e.g., you are on a Mac M1 and your cloud provider is amd64),
you'll want to build the image for that platform, e.g.:
docker build --platform=linux/amd64 -t myapp ..
Then, push it to your registry, e.g. docker push myregistry.com/myapp.
Consult Docker's getting started docs for more detail on building and pushing.
The source code is organized in the src directory as follows:
src/
├── __init__.py
├── config.py
├── gradio_interface.py
├── generate_crew.py
├── main.py
├── complex_logger.py
├── init.py
├── document_crew.py
├── crew_operations.py
├── excel_operations.py
├── templates/
│ ├── agent_template.py
│ ├── agents_class_template.py
│ ├── crew_agent_list_template.py
│ ├── crew_class_template.py
│ ├── crew_task_list_template.py
│ ├── llm_class_template.py
│ ├── llm_list_template.py
│ ├── task_template.py
│ └── tasks_class_template.py
└── tools/
├── __init__.py
├── book_publishing_tools.py
├── browser_tools.py
├── calculator_tools.py
└── search_tools.py
The excel file contains the complete setup to run a crew. Please note , not all parameters are implemented yet, but most are and up-to-date with version 0.41 from crewAI.
Supporting additonal parameters entails adding the column, matching the name to the template {variable} taking care of quoteds strings when needed. If something more fancy is needed, you have to either add to the generate_crew code, or within the class constructor.
Defining the providers and available models. The templates/llm_class_template.py has the import statement for the provider. To support additional providers, you need to add these to the class template, additional models for already configured providders, you only need to add a row in the xls sheet with the details.
To stick to crewAI terminology, a job is a collection of tasks
The xls contains the trip planner example, other jobs and tasks are development test cases, not necessarily working well.
Crews are defined to tackle specific jobs. Agens have the tools and are assebled into crews.
