Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add and refactor Debianization #1741

Merged
merged 2 commits into from
Jan 18, 2024
Merged

Add and refactor Debianization #1741

merged 2 commits into from
Jan 18, 2024

Conversation

legoktm
Copy link
Member

@legoktm legoktm commented Dec 14, 2023

Status

Ready to review

Description

This copies in all the packaging files from securedrop-builder plus the keyring and workstation-config components and refactors them to use a single source package for the build, like securedrop server.

Building all workstation Debian packages now has one entrypoint, ./scripts/build-debs.sh (a make target could be added in the future); all packages use the same version.

As part of the migration, dh-virtualenv was replaced with ./debian/setup-venv.sh that invokes virtualenv and sed directly to implement the same functionality.

Package building is now done by GitHub Actions in a singular job, having this build and push nightlies will be done in a future commit.

Test plan

  • Run ./scripts/build-debs.sh, see bullseye debs are in the build/ directory
  • Download the latest bullseye nightlies from https://apt-test.freedom.press/pool/nightlies/s/ , run debdiff and then diffoscope against the nightly vs the newly built deb. Versions will be different, but all the expected files should be there. (Some packages lost their prerm/postrm that did nothing, and many no longer have a triggers file because we moved away from dh-virtualenv)
  • Run DEBIAN_VERSION=bookworm ./scripts/build-debs.sh and see that it builds bookworm debs.

@legoktm legoktm force-pushed the better-debian branch 9 times, most recently from 302a37a to 00e114a Compare January 3, 2024 01:01
This copies in all the packaging files from securedrop-builder plus the
keyring and workstation-config components and refactors them to use a
single source package for the build, like securedrop server.

Building all workstation Debian packages now has one entrypoint,
`./scripts/build-debs.sh` (a make target could be added in the future);
all packages use the same version.

As part of the migration, dh-virtualenv was replaced with
`./debian/setup-venv.sh` that invokes virtualenv and sed directly to
implement the same functionality.

Package building is now done by GitHub Actions in a singular job, having
this build and push nightlies will be done in a future commit.
@zenmonkeykstop
Copy link
Contributor

securedrop-client:

securedrop-client debdiff
debdiff securedrop-client_0.9.0+bullseye_all.deb securedrop-client_0.9.0.dev20240118060412+bullseye_all.deb 
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .deb but not in first
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dev20240118060412.dist-info/top_level.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client/sdk/__init__.py
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client/sdk/sdlocalobjects.py
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client/sdk/timestamps.py
-rw-r--r--  root/root   DEBIAN/triggers
-rwxr-xr-x  root/root   DEBIAN/prerm

Files in first .deb but not in second
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/sdclientapi/__init__.py
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/sdclientapi/sdlocalobjects.py
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/sdclientapi/timestamps.py
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_client-0.9.0.dist-info/top_level.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-client/lib/python3.9/site-packages/securedrop_sdk-0.4.0.dist-info/top_level.txt

Control files: lines which differ (wdiff format)
------------------------------------------------
Installed-Size: [-19194-] {+19154+}
Version: [-0.9.0+bullseye-] {+0.9.0.dev20240118060412+bullseye+}

Only major difference is due to the merge of the SDK into the client as per #1746.

securedrop-export

debdiff
debdiff securedrop-export_0.9.0*
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .deb but not in first
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.9.0.dev20240118060403.dist-info/top_level.txt
-rw-r--r--  root/root   DEBIAN/triggers
-rwxr-xr-x  root/root   DEBIAN/prerm

Files in first .deb but not in second
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-export/lib/python3.9/site-packages/securedrop_export-0.3.0.dist-info/top_level.txt

Control files: lines which differ (wdiff format)
------------------------------------------------
Homepage: [-https://github.com/freedomofpress/securedrop-client-] {+https://github.com/freedomofpress/securedrop-export+}
Installed-Size: [-3143-] {+3148+}
Section: [-unknown-]
[-Source: securedrop-client-] {+utils+}
Version: [-0.9.0+bullseye-] {+0.9.0.dev20240118060403+bullseye+}
No significant differences.

securedrop-keyring (0.2.1 vs new 0.9.0)

debdiff and diffoscope all good, no meaningful difference.

securedrop-log

debdiff and diffoscope all good, no meaningful difference.

securedrop-proxy

debdiff results
 debdiff securedrop-proxy_0.9.0*
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .deb but not in first
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.9.0.dev20240118060418.dist-info/top_level.txt
-rw-r--r--  root/root   /usr/share/securedrop-proxy/config-example.yaml
-rw-r--r--  root/root   DEBIAN/triggers
-rwxr-xr-x  root/root   DEBIAN/postinst
-rwxr-xr-x  root/root   DEBIAN/prerm

Files in first .deb but not in second
-------------------------------------
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/INSTALLER
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/LICENSE
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/METADATA
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/REQUESTED
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/WHEEL
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/entry_points.txt
-rw-r--r--  root/root   /opt/venvs/securedrop-proxy/lib/python3.9/site-packages/securedrop_proxy-0.4.1.dist-info/top_level.txt

Control files: lines which differ (wdiff format)
------------------------------------------------
Homepage: [-https://github.com/freedomofpress/securedrop-client-] {+https://github.com/freedomofpress/securedrop-proxy+}
Installed-Size: [-6240-] {+6246+}
[-Source: securedrop-client-]
Version: [-0.9.0+bullseye-] {+0.9.0.dev20240118060418+bullseye+}
No significant difference (dropped config-example.yaml but not required).

securedrop-workstation-config

No significant difference.

securedrop-workstation-viewer

No significant difference.

zenmonkeykstop
zenmonkeykstop previously approved these changes Jan 18, 2024
Copy link
Contributor

@zenmonkeykstop zenmonkeykstop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

packages building successfully for both bullseye and bookworm, no surprises or problems in the debdiff and diffoscope results. LGTM!

Copy link
Contributor

@zenmonkeykstop zenmonkeykstop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once again now the internationalisation check is passing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants