Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support private repository configuration #265

Conversation

ricardojdsilva87
Copy link
Contributor

@ricardojdsilva87 ricardojdsilva87 commented Nov 8, 2024

Pull Request

Proposed Changes

This PR intends to allow the configuration of private registries on the dependabot.yml file based on a file created on the repository.

Should fix #199

The following changes were done to the code:

  • Modified the way that the yaml file is managed across all the python files
    • With the usage of the ruamel.yaml library it should be easier to add more configurations to the yaml if needed in the future. Differences to PyYaml
    • Dependabot has multiple other settings that could be retrieved from a configuration file, like it's done with the private registries (for example if we need different schedules for different types of dependencies)
  • Fixed existing tests to be compliant with the new changes
  • The existing configuration is updated if new packages are found
  • Added tests
  • Updated README file with the usage and some examples

Example: (Testers needed if possible)

Setting the environment variable as follow:

DEPENDABOT_CONFIG_FILE = "dependabot-config.yaml"

It expects a file with the name dependabot-config.yaml to exist on the repository and it should have the following structure to add the needed private repositories:

npm:
  type: 'npm'
  url: 'https://yourprivateregistry/npm/'
  username: '${{secrets.username}}'
  password: '${{secrets.password}}'
maven:
  type: 'maven'
  url: 'https://yourprivateregistry/maven/'
  username: '${{secrets.username}}'
  password: '${{secrets.password}}'

The action code will check if there is any key on the file that matches the ones that dependabot is looking for and will add the following to the configuration of each package-ecosystem configuration based on the found package:

updates:
  - package-ecosystem: 'npm'
    directory: '/'
    registries:  --> added configuration
      - 'npm'    --> added configuration
    schedule:
      interval: 'weekly'
    labels:
      - 'test'
      - 'dependabot'
      - 'new'

Current code coverage --> Still missing some tests regarding the existence of a configuration file (The existing tests were adapted to the current configuration)

---------- coverage: platform darwin, python 3.13.0-final-0 ----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
auth.py                 28      0   100%
dependabot_file.py      81      0   100%
env.py                 132      4    97%   46-47, 167-168
evergreen.py           143      3    98%   323-324, 377
--------------------------------------------------
TOTAL                  384      7    98%

Required test coverage of 80% reached. Total coverage: 98.18%

Readiness Checklist

Author/Contributor

  • If documentation is needed for this change, has that been included in this pull request
  • run make lint and fix any issues that you have introduced
  • run make test and ensure you have test coverage for the lines you are introducing
  • If publishing new data to the public (scorecards, security scan results, code quality results, live dashboards, etc.), please request review from @jeffrey-luszcz

Reviewer

  • Label as either fix, documentation, enhancement, infrastructure, maintenance or breaking

@ricardojdsilva87 ricardojdsilva87 marked this pull request as ready for review November 11, 2024 16:04
Makefile Outdated Show resolved Hide resolved
Copy link
Member

@jmeridth jmeridth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of nits and still looking at rest of PR. Still determining if switch to ruamel.yaml from PyYAML is the best choice. It looks like it but confirming.

Small suggestion, when making a change like that, maybe link to something like this in your PR description to show comparison. wdyt?

.gitignore Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@ricardojdsilva87
Copy link
Contributor Author

@jmeridth I've added the needed

Couple of nits and still looking at rest of PR. Still determining if switch to ruamel.yaml from PyYAML is the best choice. It looks like it but confirming.

Small suggestion, when making a change like that, maybe link to something like this in your PR description to show comparison. wdyt?

@jmeridth I've added the needed changes and also mentioned the differences between PyYaml and ruamel.
As I understood this library is an extension to the PyYaml that allows for example:

  • Automatic indentation of the generated Yaml
  • Possibility to quote the entries
    These were some that I've used here.
    Feel free to revise the library usage, during the change I found it easier to manage the generated yaml using a library instead of the prints with indentation
    Thanks

@jmeridth
Copy link
Member

@jmeridth I've added the needed

Couple of nits and still looking at rest of PR. Still determining if switch to ruamel.yaml from PyYAML is the best choice. It looks like it but confirming.
Small suggestion, when making a change like that, maybe link to something like this in your PR description to show comparison. wdyt?

@jmeridth I've added the needed changes and also mentioned the differences between PyYaml and ruamel. As I understood this library is an extension to the PyYaml that allows for example:

  • Automatic indentation of the generated Yaml
  • Possibility to quote the entries
    These were some that I've used here.
    Feel free to revise the library usage, during the change I found it easier to manage the generated yaml using a library instead of the prints with indentation
    Thanks

Thank you. Yeah, the library is built on PyYAML and has many improvements. I'm currently leaning towards a merge but double checking a few things. I do agree this new library simplifies multiple things we were handling manually (i.e., indention).

password: "${{secrets.password}}"
```

The principal key of each configuration need to match the package managers that the [script is looking for](https://github.com/github/evergreen/blob/main/dependabot_file.py#L78).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this link to a specific SHA(permalink)? If we add any changes to that file line 78 won't be the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree @jmeridth , for example now it's already moved to another place. Should we add this page instead from the oficial documentation?
https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem
Even though currently the script does not support all the available packages I believe the missing ones shouldn't be an issue to add.
Checking the list the supported package-ecosystems still not added to the script are:

  • devcontainers
  • elm
  • gitsubmodule
  • gradle
  • pub
  • swift

Following the same logic that is now implemented it shouldn't be too much dificult to add new supported packages.
Thanks

@jmeridth jmeridth merged commit d5af164 into github:main Nov 16, 2024
8 checks passed
if dependabot_file is None:
print("\tNo (new) compatible package manager found")
continue
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmeridth found this issue that I've mistakenly introduced by removing the continue if a repository does not need any dependabot file the code breaks.
This continue should be here to break the for loop and continue to other repositories. Could this be added to the existing release or if necessary I can create a small fix PR and also add the changes mentioned here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Does Evergreen support Dependabot configuration for private registries?
2 participants