Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(catalog): Dynamically build Camel Catalog #1169

Merged
merged 1 commit into from
Jun 19, 2024
Merged

feat(catalog): Dynamically build Camel Catalog #1169

merged 1 commit into from
Jun 19, 2024

Conversation

lordrip
Copy link
Member

@lordrip lordrip commented Jun 17, 2024 •
edited
Loading

Context

This commit adds a Camel Catalog generator CLI to build the following structure:

dist/
├─ camel-main/
│  ├─ 4.4.0/
│  ├─ 4.6.0/
├─ camel-quarkus/
│  ├─ 3.8.0/
├─ camel-springboot/
│  ├─ 4.4.0/
│  ├─ 4.6.0/
├─ index.json

How to run?

# Install dependencies
./mvnw clean install;

# Package and run
./mvnw package; java -jar ./target/catalog-generator-0.0.1-SNAPSHOT.jar -o ./dist/camel-catalog -k 4.6.0 -m 4.6.0 -m 4.4.0 -q 3.8.0 -s 4.6.0

Alternatively, yarn can be used as well:

yarn workspace @kaoto/catalog-generator run build

This will generate:

  • Camel Main 4.4.0
  • Camel Main 4.6.0
  • Camel Quarkus 3.8.0
  • Camel SpringBoot 4.6.0

Usage

usage: catalog-generator
 -k,--kamelets <kameletsVersion>   Kamelets catalog version
 -m,--main <version>               Camel Main version
 -o,--output <outputDir>           Output directory. It will be cleaned
                                   before generating the catalogs
 -q,--quarkus <version>            Camel Extensions for Quarkus version
 -s,--springboot <version>         Camel SpringBoot version
 -v,--verbose                      Be more verbose

fix: #1109

@lordrip lordrip requested a review from igarashitm June 17, 2024 15:46
@lordrip lordrip mentioned this pull request Jun 17, 2024
Closed
9 tasks
@lordrip lordrip force-pushed the feat/generate-catalogs branch from 7e388ad to 2c6c1bb Compare June 17, 2024 15:53
@lordrip lordrip marked this pull request as draft June 18, 2024 01:56
@lordrip lordrip force-pushed the feat/generate-catalogs branch 2 times, most recently from d41206c to b600156 Compare June 18, 2024 02:14
apupier
apupier reviewed Jun 18, 2024
View reviewed changes
...s/catalog-generator/src/main/java/io/kaoto/camelcatalog/maven/CamelCatalogVersionLoader.java

public class CamelCatalogVersionLoader {
private static final Logger LOGGER = Logger.getLogger(CamelCatalogVersionLoader.class.getName());
private final KaotoMavenVersionManager KAOTO_VERSION_MANAGER = new KaotoMavenVersionManager();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either decalre it as static or rename following the regular expression '^[a-z][a-zA-Z0-9]*$'.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that it cannot be static for the use case, the field should be renamed to follow Java guidelines on naming with the regular expression '^[a-z][a-zA-Z0-9]*$'.

@lhein
Copy link
Contributor

lhein commented Jun 18, 2024

addressed the mentioned problems in lordrip#30

@apupier
Copy link
Member

apupier commented Jun 18, 2024

it doesn't compile

[@kaoto/catalog-generator]: [ERROR] /__w/kaoto/kaoto/packages/catalog-generator/src/main/java/io/kaoto/camelcatalog/Build.java:[19,20] cannot find symbol
[@kaoto/catalog-generator]: [ERROR]   symbol:   variable Level
[@kaoto/catalog-generator]: [ERROR]   location: class io.kaoto.camelcatalog.Build

the import package is missing.

@lhein
Copy link
Contributor

lhein commented Jun 18, 2024

My PR should have the fix

apupier
apupier reviewed Jun 18, 2024
View reviewed changes
Copy link
Member

@apupier apupier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be worthy to rework commit history, there is currently several commits with same message

@lhein
Copy link
Contributor

lhein commented Jun 18, 2024

yep, that was caused by accepting your PR suggestions.

@lhein
Copy link
Contributor

lhein commented Jun 18, 2024 •
edited
Loading

cleaned up the history on the PR for this PR :D So you only need to squash rename after u merged

@lordrip lordrip force-pushed the feat/generate-catalogs branch 2 times, most recently from 5b9682b to 24f0e92 Compare June 19, 2024 15:10
@lordrip
feat(catalog): Dynamically build Camel Catalog
3845547 
This commit adds a Camel Catalog generator CLI to build the following
structure:

```
dist/
├─ camel-main/
│  ├─ 4.4.0/
│  ├─ 4.6.0/
├─ camel-quarkus/
│  ├─ 3.8.0/
├─ camel-springboot/
│  ├─ 4.4.0/
│  ├─ 4.6.0/
├─ index.json
```

* How to run?
```bash
mvn package; java -jar ./target/catalog-generator-0.0.1-SNAPSHOT.jar -o ./dist -k 4.6.0 -m 4.6.0 -m 4.4.0 -q 3.8.0 -s 4.6.0
```

This will generate:
* Camel Main 4.4.0
* Camel Main 4.6.0
* Camel Quarkus 3.8.0
* Camel SpringBoot 4.6.0

* Usage
```
usage: catalog-generator
 -k,--kamelets <kameletsVersion>   Kamelets catalog version
 -m,--main <version>               Camel Main version
 -o,--output <outputDir>           Output directory. It will be cleaned
                                   before generating the catalogs
 -q,--quarkus <version>            Camel Extensions for Quarkus version
 -s,--springboot <version>         Camel SpringBoot version
 -v,--verbose                      Be more verbose
```

relates: KaotoIO#1109
@lordrip lordrip force-pushed the feat/generate-catalogs branch from 24f0e92 to 3845547 Compare June 19, 2024 15:10
@lordrip lordrip marked this pull request as ready for review June 19, 2024 15:11
@lordrip lordrip merged commit 7fcea34 into KaotoIO:main Jun 19, 2024
10 checks passed
@lordrip lordrip deleted the feat/generate-catalogs branch June 19, 2024 15:30
apupier
apupier reviewed Jun 20, 2024
View reviewed changes
...s/catalog-generator/src/main/java/io/kaoto/camelcatalog/commands/GenerateCommandOptions.java
ConfigBean configBean = new ConfigBean();
CommandLineParser parser = new DefaultParser();

CommandLine cmd = null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be initialized directly in side the try

...s/catalog-generator/src/main/java/io/kaoto/camelcatalog/maven/CamelCatalogVersionLoader.java

public class CamelCatalogVersionLoader {
private static final Logger LOGGER = Logger.getLogger(CamelCatalogVersionLoader.class.getName());
private final KaotoMavenVersionManager KAOTO_VERSION_MANAGER = new KaotoMavenVersionManager();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that it cannot be static for the use case, the field should be renamed to follow Java guidelines on naming with the regular expression '^[a-z][a-zA-Z0-9]*$'.

...es/catalog-generator/src/main/java/io/kaoto/camelcatalog/maven/KaotoMavenVersionManager.java
Comment on lines +125 to +126
LOGGER.log(Level.WARNING,
String.format("Cannot resolve artifact {} due to {}", gav, e.getMessage()), e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here it is assuming that it is an error resolving the artifact, it can be another kind of error

...es/catalog-generator/src/main/java/io/kaoto/camelcatalog/maven/KaotoMavenVersionManager.java
Comment on lines +155 to +166
URL found = null;
Enumeration<URL> urls = getClassLoader().getResources(name);
while (urls.hasMoreElements()) {
URL url = urls.nextElement();
if (url.getPath().contains(version)) {
found = url;
break;
}
}
if (found != null) {
return found.openStream();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can directly return found.openStream instead of using a found and a break

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you be more precise here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    private InputStream doGetResourceAsStream(String name, String version) {
        if (version != null) {
            try {
                Enumeration<URL> urls = getClassLoader().getResources(name);
                while (urls.hasMoreElements()) {
                    URL url = urls.nextElement();
                    if (url.getPath().contains(version)) {
                        return url.openStream();
                    }
                }
                
            } catch (IOException e) {
                if (getLog()) {
                    LOGGER.log(Level.WARNING, String.format("Cannot open resource {} and version {} due {}", name, version,
                            e.getMessage(), e));

                }
            }
        }
        return null;

?

packages/catalog-generator/src/main/resources/schemas/PipeErrorHandler.json
"type" : "object",
"additionalProperties" : false,
"title": "Pipe Error Handler",
"description": "Camel K Pipe ErrorHandler. See https://camel.apache.org/camel-k/latest/pipe-step.html#_error_handler for more details.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Page not found for the link

packages/catalog-generator/src/test/java/io/kaoto/camelcatalog/MainTest.java

@Test
public void testGenerate() throws IOException {
// Main.main(new String[] {});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we uncomment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lordrip ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to bring and adapt the tests from the @kaoto/camel-catalog project, we can uncomment, but nevertheless, the entire file will be replaced.

@lhein lhein mentioned this pull request Jun 20, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@apupier apupier apupier left review comments

@lhein lhein lhein approved these changes

@igarashitm igarashitm igarashitm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support different Camel catalog versions
4 participants
@lordrip @lhein @apupier @igarashitm