mvnio is a horizontal scalable Maven Repository Manager with first class support for S3 buckets.
The underlying webserver is implemented with reactive streams, or more specific by using Spring webflux which uses project-reactor under the hood. For the S3 bucket interaction, the Amazon Async S3 client library is used, which is also a non-blocking API, and it has support for other S3 compatible storage providers as well (e.g. MinIO).
There are plenty of Maven Repository Managers and they are great and have a lot of features, but most of them don't support S3 buckets at all or not very well or require a special license.
As alternative, you could use a so called maven-wagon which basically adds S3 bucket support to the maven client. There are also plenty of them (1, 2, 3, ...), but they require an extra configuration, which in fact often is not supported by third-party products. For example Jenkins picks not up the extra configuration (see https://issues.jenkins-ci.org/browse/JENKINS-30058). They also cannot take into account that Maven artifacts are immutable. You could lock the objects in S3 but then the metadata cannot be updated anymore.
- Standard Repository Layout
- upload and download artifacts
- browse artifacts and retrieve metadata easily
- stateless and horizontal scalable
- support for any S3 compatible storage provider
- multiple repositories
- immutability protection
- quick startup time and less memory consumption
The diagram below shows how mvnio handles maven client requests and how they are mapped to the S3 storage provider.
In general, a maven client interacts with a maven repository when it tries to install an artifact with a GET request and with a PUT request when it wants to deploy an artifact. For example, when a client requests GET /maven/releases/foo/bar/1.0.1/bar-1.0.1.pom from mvnio then mvnio tries to get an object with key foo/bar/1.0.1/bar-1.0.1.pom in bucket releases from S3.
The AppTests and the corresponding requests and responses could also be helpful if you want more details.
AppProperties contains a list of all available configuration parameters, and their default values.
mvnio expects a Basic Authorization Header sent along with any client request. The header contains the S3 Credentials of the corresponding S3 Bucket. It also requires the following permissions:
s3:GetObject- to download artifactss3:HeadObjectands3:PutObject- to upload artifactss3:ListBucket- to request metadata or to list versions (optional)
Note: Clients can bypass the repository and modify artifacts in S3 directly!
This example shows how mvnio can be set up with MinIO as storage provider. It requires docker-compose and mvn installed properly and, for sake of simplicity, the MinIO admin user will be used to connect to S3/MinIO. For production environments you should use an extra user with strong credentials, and the minimum required permissions (see Security)!
Let's get started:
- run
docker-compose upto start upmvnioandMinIOas well (ports: 8080 and 9000) - run
docker-compose run --rm mc config host add minio http://minio:9000 admin long-passwordonce to register MinIO - run
docker-compose run --rm mc mb minio/releasesto create a bucket namedreleases - adjust your
~/.m2/settings.xml:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>maven-releases</id>
<username>admin</username>
<password>long-password</password>
</server>
</servers>
</settings>
- create a file named
pom.xmlin an empty directory with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>foo</groupId>
<artifactId>bar</artifactId>
<version>1.0.1</version>
<distributionManagement>
<repository>
<id>maven-releases</id>
<url>http://localhost:8080/maven/releases</url>
</repository>
</distributionManagement>
</project>
- run
mvn deploy
...
[INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ bar ---
Uploading to maven-releases: http://localhost:8080/maven/releases/foo/bar/1.0.1/bar-1.0.1.jar
Uploaded to maven-releases: http://localhost:8080/maven/releases/foo/bar/1.0.1/bar-1.0.1.jar (1.4 kB at 1.1 kB/s)
Uploading to maven-releases: http://localhost:8080/maven/releases/foo/bar/1.0.1/bar-1.0.1.pom
Uploaded to maven-releases: http://localhost:8080/maven/releases/foo/bar/1.0.1/bar-1.0.1.pom (601 B at 5.1 kB/s)
Downloading from maven-releases: http://localhost:8080/maven/releases/foo/bar/maven-metadata.xml
Uploading to maven-releases: http://localhost:8080/maven/releases/foo/bar/maven-metadata.xml
Uploaded to maven-releases: http://localhost:8080/maven/releases/foo/bar/maven-metadata.xml (286 B at 2.1 kB/s)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.917 s
[INFO] Finished at: 2020-10-23T13:22:45+02:00
[INFO] Final Memory: 12M/209M
[INFO] ------------------------------------------------------------------------
The artifacts should now be available in MinIO.
- open http://localhost:9000 in your browser (accessKey:
admin, secretKey:long-password)
- mirror for central (light version, just proxy the request)
- alternative user/account management
- independent of S3
- central configuration (via
vaultfor example) - storage provider is configurable per repository
- proxy/mirror for repositories (central and external)
- users accessKey/secretKey in S3, are used as username/password in Maven (see
~/.m2/settings.xml) - users can bypass the repository and access objects directly in S3