circleci config validate fails for project on GitHub Enterprise and CircleCI server

[jmrieger]

Meta:

CircleCI CLI Version:
0.1.24783+93e3e61 (homebrew)

Operating System:
macOS Ventura 13.2.1

CircleCI CLI Diagnostic:

---
CircleCI CLI Diagnostics
---
Debugger mode: false
Config found: --redacted--
API host: --redacted, but correctly points to my GitHub Enterprise server --
API endpoint: graphql-unstable
OK, got a token.
Trying an introspection query on API... 
Ok.
Hello, Rieger, Joe.

---

Current behavior:

When running the CLI locally for CircleCI server and GitHub Enterprise as the location for the project, I receive the following:

circleci config validate produces the following output:

Validating config with following values
        id:     00000000-0000-0000-0000-000000000001    number: 1       project.git_url:        https://github.com/CircleCI-Public/circleci-cli project.type:   github  git.tag:                git.branch:     main    git.revision:  --redacted-- git.base_revision:      --redacted-- Error: Not Found

Expected behavior:

The CLI should respect and acknowledge that projects can live places other than public GitHub

When did this begin / Was this previously working?:

Began when I ran a brew update today for CircleCI CLI

Additional Information:

[elliotforbes]

Hi @jmrieger - are you able to try adding --host https://server.com to the commands you run to validate. I've got a few examples of the commands I've used to validate this in this PR - #880

[jmrieger]

#880

Same results.

circleci config --host --redacted-- validate --org-slug --redacted

Validating config with following values
        git.revision:   (redacted)        git.base_revision:      (redacted)        id:     00000000-0000-0000-0000-000000000001    number: 1       project.git_url:       https://github.com/CircleCI-Public/circleci-cli  project.type:   github  git.tag:                git.branch:     (redacted)Error: Not Found

Edit: It's worth noting that I've used circleci setup to set my host and PAT locally.

[jmrieger]

If I manually download previous versions, the last version that works is v0.1.23845. 0.1.24435 starts to throw issues with private (server) orbs not being able to be found in the registry.

[elliotforbes]

@jmrieger - can I ask you to pin to v0.1.23845 for now.

The reason this is failing appears to be far more nuanced and will require some deeper investigation into the internal services that serve these commands.

[jmrieger]

@jmrieger - can I ask you to pin to v0.1.23845 for now.

The reason this is failing appears to be far more nuanced and will require some deeper investigation into the internal services that serve these commands.

Of course! If there's any other local debug or diagnosis that you'd like me to test, reach out directly - I'd be happy to help.

[elliotforbes]

I'll likely have to take you up on that offer! Leave it with me for now and I'll dig into our o11y tools tomorrow during my day and see if I can sniff out the underlying reason this appears to have regressed.

[elliotforbes]

I've been unable to replicate this issue in any of my tests unfortunately. Instead, I've taken a look at cleaning up the architecture of the code base and improving the error handling to be more explicit here - #884

It still needs to go through review process, but if you'd like to try make build and testing this out, it could really help with the investigations!

[jmrieger]

I've been unable to replicate this issue in any of my tests unfortunately. Instead, I've taken a look at cleaning up the architecture of the code base and improving the error handling to be more explicit here - #884

It still needs to go through review process, but if you'd like to try make build and testing this out, it could really help with the investigations!

Trying now; will update with any findings.

[jmrieger]

circleci config --host (redacted) validate --org-slug (org/repo from GitHub Enterprise)

Could not fetch a valid org-id from collaborators endpoint.
Check if you have access to this org by hitting https://circleci.com/api/v2/me/collaborations
Continuing on - private orb resolution will not work as intended
Error: config compilation request returned an error: Not Found

[elliotforbes]

Interesting, ok that helps move the needle a little.

If you hit https://redacted-host/api/v2/me/collaborations, can you see the org-slug in the array of collaborations returned to you?

This gives me more to investigate though, I'll try and see if there is another internal source that is queried for the org-id in these situations.

[jmrieger]

Interesting, ok that helps move the needle a little.

If you hit https://redacted-host/api/v2/me/collaborations, can you see the org-slug in the array of collaborations returned to you?

This gives me more to investigate though, I'll try and see if there is another internal source that is queried for the org-id in these situations.

I can, but see the org slug was in a different format than what I was expecting - gh/org, and not orgname/reponame format.

Modifying that to circleci config --host https://circleci.foc.zone validate --org-slug gh/my-org results in Error: config compilation request returned an error: Not Found

[jmrieger]

I made a change in config.go in the `ConfigQuery function as follows, to get some additional debugging:

requestURL := &url.URL{
		Path: "compile-config-with-defaults",
	}
	req, err := c.compileRestClient.NewRequest(
		"POST",
		requestURL,
		compileRequest,
	)
	if err != nil {
		return nil, fmt.Errorf("an error occurred creating the request: %w", err)
	}

	configCompilationResp := &ConfigResponse{}
	statusCode, err := c.compileRestClient.DoRequest(req, configCompilationResp)
	if err != nil {
		fmt.Printf("%+v\n", requestURL)
                 fmt.Printf("%s\n", req.URL)
		return nil, fmt.Errorf("config compilation request returned an error: %w (%w)", err, statusCode)
	}

Output:

circleci config --host (redacted) validate --org-slug gh/(redacted)
compile-config-with-defaults
https://(redacted)/api/v2/compile-config-with-defaults
Error: config compilation request returned an error: Not Found (%!w(int=404))

I'm less fluent in Go, but what I think this is leading to is that the path that's supposed to be set on that URL is intended to be pulled from / compiled from configuration, but isn't being fully transpiled.

[elliotforbes]

@jmrieger I appreciate this is asking a lot with you helping me investigate - I've just pushed a new commit to that branch though, if you could test again with that I'm hoping it might help!

[jmrieger]

Similar results:

circleci config --host (redacted) validate --org-slug gh/(redacted) --verbose
Validating config with following values
        git.base_revision:      6b7b0aa378920e4db24f93bfeaa82e05cd52126d        id:     00000000-0000-0000-0000-000000000001    number: 1       project.git_url:        https://github.com/CircleCI-Public/circleci-cli project.type:  github   git.tag:                git.branch:     (redacted)   git.revision:   (redacted)dcompile-config-with-defaults
(redacted)/api/v2/compile-config-with-defaults
Error: config compilation request returned an error: Not Found (%!w(int=404))

[jmrieger]

With most recent commit (ede5ced):

circleci config --host (redacted) validate --org-slug gh/(redacted) --verbose
Validating config with following values
        git.base_revision:      (redacted)        id:     00000000-0000-0000-0000-000000000001    number: 1       project.git_url:        https://github.com/CircleCI-Public/circleci-cli project.type:  github   git.tag:                git.branch:     (redacted)   git.revision:   (redacted git sha)(redacted base URL that I promise is our CircleCI server instance)/api/v2/me/collaborations
Error: config compilation request returned an error: Not Found

[elliotforbes]

I've figured this out - this will be due to the server version you are running on not having the underlying service endpoint exposed.

The only course of action here will be to pin your version until the underlying server instance is updated.

[jmrieger]

I've figured this out - this will be due to the server version you are running on not having the underlying service endpoint exposed.

The only course of action here will be to pin your version until the underlying server instance is updated.

That make sense now, and understood.

I realize that the CLI is not in version 1 yet, and thus all of this would technically not be subject to BC breaks, BUT, would it be a consideration to support the older version of Server? Or, provide some way via the package managers (Brew, Chocolatey) to actually support the older version of the CLI? At least with brew, they don't have an easy way to pin without having some (major) version specifier.

[elliotforbes]

I'll have to check with the team if they can make it easier to support breaking versions via proper versioning.

Unfortunately, we wont be able to continue to support older versions as the intention is to remove certain internal services altogether. We are in the process of attempting to improve stability and simplify our architecture with these changes.

[elliotforbes]

@jmrieger - the latest release should make it explicitly obvious that this is the underlying reason for config compilation and validation issues now.

I'm going to take a look into improving the documentation to show how folks can install older versions (not specific to brew) which should hopefully help other folks in your situation.

Are you happy for me to close this issue for now though?

[jmrieger]

@jmrieger - the latest release should make it explicitly obvious that this is the underlying reason for config compilation and validation issues now.

I'm going to take a look into improving the documentation to show how folks can install older versions (not specific to brew) which should hopefully help other folks in your situation.

Are you happy for me to close this issue for now though?

@elliotforbes Absolutely. I appreciate the extra inches in the CLI being more verbose about the underlying errors.