Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11631. Add Run with Docker Compose guide #101

Draft
wants to merge 4 commits into
base: HDDS-9225-website-v2
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cspell.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ words:
- ratis
- OM
- SCM
- DN
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike OM and SCM we don't usually use DN as an abbreviation for datanode when talking about Ozone in an official context, so it was deliberately omitted here. It might be better to use an inline ignore to skip it in the mermaid code block.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

- datanode
- datanodes
- ofs
Expand Down
7 changes: 7 additions & 0 deletions docs/06-troubleshooting/15-docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
sidebar_label: Docker
---

# Troubleshooting Docker
ptlrs marked this conversation as resolved.
Show resolved Hide resolved

**TODO:** File a subtask under [HDDS-9860](https://issues.apache.org/jira/browse/HDDS-9860) and complete this page or section.
140 changes: 140 additions & 0 deletions docs/08-developer-guide/02-run/02-docker-compose.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,143 @@ sidebar_label: Docker Compose
- Define the Ozone runner image at `ozone-docker-runner` and its purpose to wrap Ozone binaries.
- How to define which image gets used in the cluster (runner or other pre-built image).
- Changing configurations in Docker Compose (`docker-config` file, transformation.py, etc).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can delete the TODO above this section since we are filling in the page in this PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

This guide explains how to run Apache Ozone using Docker Compose, either with locally built sources or pre-built images.

## Prerequisites

- Docker Engine 20.10.0 or higher
- Docker Compose V2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's link to the relevant pages on the docker website for installing these components.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

- Built Ozone distribution (if running from local build)

## Running Ozone

You can run Ozone either using your locally built version or using official pre-built Docker images.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should organize this a little differently. We can have one section on running docker-compose from a pre-build Ozone tarball, regardless of where the tarball came from (built from source or Ozone release). Then we can mention the pre-built docker images for each release separately if people want to use those for other things.

I don't see why someone would pull just the compose file from the tarball and manually pair it with a docker image built from the same version.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


### Option 1: Running from Local Build

If you've built Ozone from source, follow these steps:

1. Navigate to the compose directory in your build output:

```bash
cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone
```

2. Start the cluster:

```bash
docker-compose up -d
```

The local build uses the `ozone-docker-runner` image, which is automatically created during the build process to wrap your compiled Ozone binaries.

### Option 2: Running from Pre-built Images

If you haven't built Ozone locally, you can quickly start a cluster using official pre-built images from Docker Hub:

1. Create a directory for your Ozone deployment:

```bash
mkdir ozone && cd ozone
```

2. Download the example compose file:

```bash
curl \
https://raw.githubusercontent.com/apache/ozone/master/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml \
-o docker-compose.yaml
```

3. Start the cluster:

```bash
docker-compose up -d
```

This will pull the official Apache Ozone images from Docker Hub.

## Architecture Diagram
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid duplicate content I think we should leave this information for the page dedicated to Ozone architecture. Default ports will be covered in docs/administrator-guide/configuration/basic/network/. This may mean we don't need to add mermaid in this PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a new user, it is not immediately apparent how the Ozone services will be divvied across containers. It could be the case that all Ozone services run within one container. There are several possible configurations.

I added the mermaid diagram to visualize how many containers will be available and how each Ozone service maps to those containers.

It just so happens that each Ozone service is a separate container and the container diagram looks similar to what could easily have been an Ozone architecture diagram. Perhaps the title and wording add to the confusion.

If it is still a cause of confusion, I can remove the diagram. Not a problem.


This image shows the containers that will be created when running the `docker-compose up -d` command.

```mermaid
graph TB
subgraph "Apache Ozone Architecture"
scm["SCM<br/>Storage Container Manager<br/>(Port: 9876, 9860)"]
om["OM<br/>Ozone Manager<br/>(Port: 9874, 9862)"]
s3g["S3G<br/>S3 Gateway<br/>(Port: 9878)"]
recon["Recon<br/>Monitoring Service<br/>(Port: 9888)"]
httpfs["HttpFS<br/>HTTP FileSystem<br/>(Port: 14000)"]

subgraph "Data Nodes"
dn1["DataNode 1<br/>(Port: 19864, 9882)"]
dnN["DataNode N<br/>(Port: 19864, 9882)"]
end

%% Connections
scm --> dn1
scm --> dnN
om --> dn1
om --> dnN
s3g --> om
s3g --> scm
httpfs --> om
httpfs --> scm
recon --> scm
recon --> om
end

classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px;
classDef datanode fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
classDef manager fill:#e8f5e9,stroke:#388e3c,stroke-width:2px;

class dn1,dnN datanode;
class scm,om manager;
```

## Cluster Configuration

### Default Services

The default Docker Compose configuration includes:

- Storage Container Manager (SCM)
- Ozone Manager (OM)
- S3 Gateway
- Recon (Monitoring Service)
- Datanodes
- HttpFS

### Port Mappings

The default setup exposes the following ports:
Copy link
Contributor

@errose28 errose28 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure the RPC connections will work with clients outside the compose network. The configuration and hostnames would need to be configured manually at least. HTTP endpoints should work out of the box though.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default ports will be covered in docs/administrator-guide/configuration/basic/network/

Removed the port mappings section here


- SCM: 9876 (RPC), 9860 (Client)
- OM: 9874 (RPC), 9862 (Client)
- S3 Gateway: 9878
- Recon: 9888
- HttpFS: 14000
- Datanodes: 19864 (Container), 9882 (Client)

## Cluster Management

Common Docker Compose commands:

```bash
# Start the cluster
docker-compose up -d
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use docker compose since v2 is the standard going forward.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


# Stop the cluster
docker-compose down

# View service logs
docker-compose logs -f [service_name]

# Scale data nodes
docker-compose up -d --scale datanode=3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works on clusters that provide a service called datanode. Some of the compose clusters like topology and upgrade that need to use individual datanodes differently give each datanode a different service.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


# Check service status
docker-compose ps
```
2 changes: 2 additions & 0 deletions docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ const config = {
],

markdown: {
mermaid: true,
/*
Validate markdown frontmatter against a more restrictive schema than what Docusaurus allows.
This ensures all pages are using a minimal set of consistent keys.
Expand Down Expand Up @@ -170,6 +171,7 @@ const config = {
]
],

themes: ['@docusaurus/theme-mermaid'],
themeConfig:
/** @type {import('@docusaurus/preset-classic').ThemeConfig} */
({
Expand Down
13 changes: 7 additions & 6 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"packageManager": "pnpm@8.7.1",
"packageManager": "pnpm@9.12.3",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify why we need the pnpm version bump? Is mermaid incompatible with older versions?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With [email protected], running docker-compose fails with the following error:

0.280 ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-8.7.1.tgz
1.109  ERR_PNPM_LOCKFILE_BREAKING_CHANGE  Lockfile /ozone-site/pnpm-lock.yaml not compatible with current pnpm
1.109 
1.109 Run with the --force parameter to recreate the lockfile.
------
failed to solve: process "/bin/sh -c pnpm install --prod --frozen-lockfile" did not complete successfully: exit code: 1
`docker-compose` process finished with exit code 17

I added the docusaurus/theme-mermaid package via pnpm add @docusaurus/[email protected] on my mac. This updated the pnpm-lock.yaml and the package.json files.

"name": "doc-site",
"version": "0.1.0",
"private": true,
Expand All @@ -16,22 +16,23 @@
},
"dependencies": {
"@docusaurus/core": "3.3.2",
"@docusaurus/preset-classic": "3.3.2",
"@docusaurus/plugin-pwa": "3.3.2",
"@docusaurus/preset-classic": "3.3.2",
"@docusaurus/theme-mermaid": "3.3.2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked out this change and ran docker compse up --build to build an image with the new dependencies but got this error. Not sure exactly what the cause is:

ozone-site-site-1  | 
ozone-site-site-1  | > [email protected] start /ozone-site
ozone-site-site-1  | > docusaurus start --port 3001 "--host" "0.0.0.0"
ozone-site-site-1  | 
ozone-site-site-1  | [INFO] Starting the development server...
ozone-site-site-1  | 
ozone-site-site-1  | [ERROR] Error: Docusaurus was unable to resolve the "@docusaurus/theme-mermaid" theme. Make sure one of the following packages are installed:
ozone-site-site-1  | - @docusaurus/theme-mermaid
ozone-site-site-1  | - @docusaurus/docusaurus-theme-theme-mermaid
ozone-site-site-1  |     at resolveModuleName (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/moduleShorthand.js:41:15)
ozone-site-site-1  |     at normalizeShorthand (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/configs.js:76:60)
ozone-site-site-1  |     at /ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:63
ozone-site-site-1  |     at Array.map (<anonymous>)
ozone-site-site-1  |     at loadPluginConfigs (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:48)
ozone-site-site-1  |     at async initPlugins (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/init.js:42:27)
ozone-site-site-1  |     at async /ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/plugins/plugins.js:136:36
ozone-site-site-1  |     at async loadSite (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/server/site.js:127:45)
ozone-site-site-1  |     at async createReloadableSite (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/commands/start/utils.js:46:16)
ozone-site-site-1  |     at async Command.start (/ozone-site/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/@docusaurus/core/lib/commands/start/start.js:22:28)
ozone-site-site-1  | [INFO] Docusaurus version: 3.3.2
ozone-site-site-1  | Node version: v20.18.0
ozone-site-site-1  |  ELIFECYCLE  Command failed with exit code 1.
ozone-site-site-1 exited with code 1

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw that I faced the same problem after uninstalling docusaurus/theme-mermaid.
I was able to resolve it by reinstalling the package again on my mac.
Next, I ran docker compse up --build and then ran it with no issues.

"@mdx-js/react": "^3.0.0",
"ajv": "^8.12.0",
"clsx": "^2.1.0",
"prism-react-renderer": "^2.3.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"ajv": "^8.12.0"
"react-dom": "^18.2.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "3.3.2",
"cspell": "^8.2.1",
"@cspell/dict-docker": "^1.1.7",
"@cspell/dict-java": "^5.0.6",
"@cspell/dict-markdown": "^2.0.1",
"@cspell/dict-shell": "^1.0.6",
"@docusaurus/module-type-aliases": "3.3.2",
"cspell": "^8.2.1",
"markdownlint-cli": "^0.39.0"
},
"browserslist": {
Expand Down
Loading