-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Package JSON manifest should support generic attributes #385
Comments
We can do whatever we want. Since there is, so far, 1 method that writes the manifest, and one that decodes an entry, and they are both private interfaces, changing the format is fair game as we need it. Another reason for moving to that now might be that we may have to add more user/owner types for #370 . Rather than add more fields at the top level, add them to the embedded struct. |
One more requirement: We need operators around this
|
Been thinking about this on and off. Initial thoughts:
So, something like this: [
{
"label": "//:foo-config",
"type": "FILE",
"source": "bazel-out/k8-linux-fastbuild/foo", // or whatever bazel provides us
"destination": "{PREFIX}/etc/foo", // Templated destination
"mode": "rw-r--r--", // or 0644
"user": "root",
"group": "root",
"rpm_filetag": "%config" // Custom attribute for pkg_rpm
"zip_compression": "STORE" // Theoretical custom attribute for pkg_zip
"unix_extended_permissions": "+i" // Theoretical custom attribute for UNIX-style installers (set immutable bit)
},
...
] This is simple enough to meet all the above requirements and is also relatively futureproof. It also has the unfortunate consequences of requiring more code and redoing a bit of the logic that we already have for the manifest system as used in |
The current way that rules_pkg communicates with must packagers is using a manifest file, which is currently a JSON data structure based on a an array of arrays. While generally readable, it looks strange, as it was to reduce Bazel resource usage (JSON strings in memory). Further, our Python code is directly bound to this data structure format. However, if we want to add more or change this, it becomes cumbersome on both the Starlark and Python sides. This change alleviates concerns generally by: - Converting all manifests to a JSON "object" style, improving readability. Numerous "golden" tests were updated to support this. - Replace the `collections.namedtuple`-based `ManifestEntry` object with one that is a little more flexible and type-safe. - Providing a function (`read_entries_from`) that converts a file-like object into a list of `ManifestEntry`s, and replacing all JSON reading in packagers (`tar`, `zip`, `install`) and their tests with this function. Other convenience factors or things addressed: - `ManifestEntry.entry_type` is now just `ManifestEntry.type` - Bazel 6 now stringifies repository-local labels with a preceding `@`, unlike prior versions. Adapt to this in the manifest writer. Future changes will extend this interface to allow for custom attributes to be passed from `pkg_files` and friends, which, among other things, will be necessary to more generically support `pkg_rpm`. Provides groundwork for (but doesn't resolve) bazelbuild#385.
Genericize package manifest system and interface The current way that rules_pkg communicates with must packagers is using a manifest file, which is currently a JSON data structure based on a an array of arrays. While generally readable, it looks strange, as it was to reduce Bazel resource usage (JSON strings in memory). Further, our Python code is directly bound to this data structure format. However, if we want to add more or change this, it becomes cumbersome on both the Starlark and Python sides. This change alleviates concerns generally by: - Converting all manifests to a JSON "object" style, improving readability. Numerous golden tests were updated to support this. - Replace the `collections.namedtuple`-based `ManifestEntry` object with one that is a little more flexible and type-safe. - Providing a function (`read_entries_from`) that converts a file-like object into a list of `ManifestEntry`s, and replacing all JSON reading in packagers (`tar`, `zip`, `install`) and their tests with this function. Other convenience factors or things addressed: - `ManifestEntry.entry_type` is now just `ManifestEntry.type` - Bazel 6 now stringifies repository-local labels with a preceding `@`, unlike prior versions. Adapt to this in the manifest writer. Future changes will extend this interface to allow for custom attributes to be passed from `pkg_files` and friends, which, among other things, will be necessary to more generically support `pkg_rpm`. Provides groundwork for (but doesn't resolve) #385.
I like the idea of the JSON manifest format -- it allows us to create backends for a generic packaging frontends.
I would like to adopt this for
pkg_rpm
at some point, but in order to do this, the manifest would need to support a generic "attributes" format to support file tags (like%config
). This will also be needed for supporting packages in Windows.So, instead of separate
mode
,user
, andgroup
fields, perhaps we should have anattributes
object that can contain arbitrary information, mirroringpkg_attributes
in thepkg_filegroup
framework?The text was updated successfully, but these errors were encountered: