Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Minor fix to docker bake github actions #529

Merged
merged 8 commits into from
Sep 28, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/dev-push-to-hub.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Build and push images to docker hub on merge to dev
run-name: ${{ github.actor }} is building and pushing images to docker hub

on:
push:
branches:
- "dev"

jobs:
Bake-Push-Images:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: julepai
password: "${{ secrets.DOCKER_HUB_PASSWORD }}"

- name: Build and push agent images
uses: docker/bake-action@v5
with:
files: |
./docker-bake.hcl
targets: agents-api
push: true
set: |
*.cache-from=type=gha
*.cache-to=type=gha,mode=max
*.args.TAG=dev
*.args.GIT_SHA=${{ github.sha }}

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/dev' }}
38 changes: 38 additions & 0 deletions .github/workflows/docker-bake-on-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Bake images on PR
run-name: ${{ github.actor }} is baking images

on:
pull_request:

jobs:
Bake-Images:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: julepai
password: "${{ secrets.DOCKER_HUB_PASSWORD }}"

- name: Bake images
uses: docker/bake-action@v5
with:
files: |
./docker-bake.hcl
targets: agents-api
push: false
set: |
*.cache-from=type=gha
*.cache-to=type=gha,mode=max
*.args.TAG=pr-${{ github.ref_name }}
*.args.GIT_SHA=${{ github.sha }}

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/dev' }}
40 changes: 40 additions & 0 deletions .github/workflows/main-push-to-hub.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Build and push images to docker hub on merge to main
run-name: ${{ github.actor }} is building and pushing images to docker hub

on:
push:
branches:
- "main"

jobs:
Bake-Push-Images:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: julepai
password: "${{ secrets.DOCKER_HUB_PASSWORD }}"

- name: Build and push agent images
uses: docker/bake-action@v5
with:
files: |
./docker-bake.hcl
targets: agents-api
push: true
set: |
*.cache-from=type=gha
*.cache-to=type=gha,mode=max
*.args.TAG=latest
*.args.GIT_SHA=${{ github.sha }}

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
163 changes: 0 additions & 163 deletions .github/workflows/push-to-hub.yml

This file was deleted.

40 changes: 40 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,44 @@ Improvements to documentation are always appreciated! If you see areas that coul

We'd love to hear your feedback and ideas for the project! Feel free to submit an issue or contact the maintainers directly to share your thoughts. Your input is very valuable in shaping the future direction of the project.

## Building Docker Images with Buildx Bake

We use Docker Buildx Bake to build our Docker images. This allows us to build multiple images concurrently and efficiently. Follow these steps to build the Docker images:

1. Ensure you have Docker and Docker Buildx installed on your system.

2. Navigate to the root directory of the project where the `docker-bake.hcl` file is located.

3. To build all services, run:
```
docker buildx bake --file docker-bake.hcl
```

4. To build a specific service, use:
```
docker buildx bake --file docker-bake.hcl <service-name>
```
Replace `<service-name>` with one of the following:
- agents-api
- agents-api-worker
- cozo-migrate
- memory-store
- integrations
- gateway
- embedding-service-cpu
- embedding-service-gpu

5. To set a custom tag for the images, use:
```
docker buildx bake --file docker-bake.hcl --set *.tags=myorg/myimage:v1.0
```
Replace `myorg/myimage:v1.0` with your desired image name and tag.

6. By default, the images are built with the "latest" tag. To specify a different tag, you can set the TAG variable:
```
docker buildx bake --file docker-bake.hcl --set TAG=v1.2.3
```

Note: The `docker-bake.hcl` file defines the build contexts, Dockerfiles, and tags for each service. If you need to modify the build process for a specific service, update the corresponding target in the `docker-bake.hcl` file.

Thank you for your interest in contributing to this project!
9 changes: 5 additions & 4 deletions agents-api/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# syntax=docker/dockerfile:1
# check=error=true

FROM python:3.12-slim

ENV PYTHONUNBUFFERED True
ENV PYTHONUNBUFFERED=1
ENV POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /app

RUN pip install 'poetry<=1.9.0' \
RUN pip install 'poetry>=1.8.0,<1.9.0' \
&& poetry config virtualenvs.create false

COPY pyproject.toml poetry.lock ./
Expand All @@ -14,6 +17,4 @@ RUN poetry install --no-dev --no-root

COPY . ./

RUN poetry install --no-dev

ENTRYPOINT ["python", "-m", "agents_api.web", "--host", "0.0.0.0", "--port", "8080"]
10 changes: 8 additions & 2 deletions agents-api/Dockerfile.migration
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# syntax=docker/dockerfile:1
# check=error=true

FROM python:3.11-slim

ENV PYTHONUNBUFFERED True
ENV PYTHONUNBUFFERED=1
ENV POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /app
Expand All @@ -9,8 +12,11 @@ RUN pip install --no-cache-dir --upgrade cozo-migrate

COPY . ./
ENV COZO_HOST="http://cozo:9070"
ENV COZO_AUTH_TOKEN="myauthkey"

# Expected environment variables:
# COZO_AUTH_TOKEN="myauthkey"

SHELL ["/bin/bash", "-c"]
ENTRYPOINT \
cozo-migrate -e http -h $COZO_HOST --auth $COZO_AUTH_TOKEN init \
; cozo-migrate -e http -h $COZO_HOST --auth $COZO_AUTH_TOKEN -d ./migrations apply -ay
7 changes: 5 additions & 2 deletions agents-api/Dockerfile.worker
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# syntax=docker/dockerfile:1
# check=error=true

FROM python:3.12-slim

ENV PYTHONUNBUFFERED True
ENV PYTHONUNBUFFERED=1
ENV POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /app

RUN pip install 'poetry<=1.9.0' \
RUN pip install 'poetry>=1.8.0,<1.9.0' \
&& poetry config virtualenvs.create false

COPY pyproject.toml poetry.lock ./
Expand Down
Loading
Loading