Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyinstaller builds for macOS and Linux in the CI #1348

Merged
merged 97 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
fa84772
Build for macOS
C0rn3j Dec 17, 2024
e87cbbb
Add sdl2
C0rn3j Dec 17, 2024
c2b3c98
add sdl2_image
C0rn3j Dec 17, 2024
555100c
Add assets back
C0rn3j Dec 17, 2024
48e1c1c
Try pysdl2-dll
C0rn3j Dec 17, 2024
df5f6d7
Pack theme
C0rn3j Dec 17, 2024
6598d8c
Add templates
C0rn3j Dec 17, 2024
2f71f25
add librsvg
C0rn3j Dec 17, 2024
6c4a902
List all files in .app
C0rn3j Dec 17, 2024
7191357
Try build with --wheel
C0rn3j Dec 17, 2024
fd79d67
Attempt to bundle phazor
C0rn3j Dec 17, 2024
608a99a
Try adding pysdl2-dll and Rsvg as hidden imports
C0rn3j Dec 17, 2024
d8fd9c5
Attempt to throw DYLD_LIBRARY_PATH back into spec
C0rn3j Dec 17, 2024
b06a3b3
Try adding librsvg-2.2.dylib
C0rn3j Dec 17, 2024
ada4974
Try fixing localization
C0rn3j Dec 17, 2024
827aa1f
rsvg hidden import try #2
C0rn3j Dec 17, 2024
59988eb
Stop duping CustomLoggingFormatter
C0rn3j Dec 17, 2024
a9fbbe2
compile_translations.py: Throw a proper exception on error
C0rn3j Dec 17, 2024
a5e8417
Fix up command order
C0rn3j Dec 17, 2024
83d383e
Install gettext
C0rn3j Dec 17, 2024
ed208e6
Use partial path for gettext
C0rn3j Dec 17, 2024
fe5e704
Move logging classes to a separate module
C0rn3j Dec 17, 2024
73ff9e1
Cleanup and brew upgrade
C0rn3j Dec 17, 2024
774a175
Add locale dir
C0rn3j Dec 17, 2024
ec9f37d
Fix brew commands
C0rn3j Dec 17, 2024
9b95b88
Which genius decided to litter the FS with this garbage
C0rn3j Dec 17, 2024
d443fdc
Remove extra space
C0rn3j Dec 17, 2024
ba7b009
cleanup glib and gettext
C0rn3j Dec 17, 2024
4045313
Attempt to bundle SDL2 prebuilt frameworks
C0rn3j Dec 17, 2024
97f2f3f
Try importing hidden chromecast
C0rn3j Dec 17, 2024
048dbdc
try lower case pychromecast
C0rn3j Dec 17, 2024
99103f9
Fix error message for setproctitle
C0rn3j Dec 17, 2024
c22a8d2
Add support for PEP508 deps definitions
C0rn3j Dec 17, 2024
609ca7e
Remove duped dep
C0rn3j Dec 17, 2024
e4b8155
Add initial optdepends
C0rn3j Dec 17, 2024
1fd6b57
Fix up deps
C0rn3j Dec 17, 2024
ef76d6f
Move out import and add test exception
C0rn3j Dec 18, 2024
9592672
pyinstaller debug build
C0rn3j Dec 18, 2024
e821df3
Try reverting deps changes
C0rn3j Dec 18, 2024
847eb3a
Kill optdepends in pyproject
C0rn3j Dec 18, 2024
7df2496
Fix ext-modules
C0rn3j Dec 18, 2024
58b498e
Fix optional
C0rn3j Dec 18, 2024
51afa35
install pyinstaller in unified deps
C0rn3j Dec 18, 2024
083a802
Fix opus include
C0rn3j Dec 18, 2024
dfe5094
Init linux.spec
C0rn3j Dec 18, 2024
20c283f
Add Linux CI
C0rn3j Dec 18, 2024
25c3ce0
Swap macOS CI to unified reqs
C0rn3j Dec 18, 2024
2e93950
Try fixing up hiddenimports
C0rn3j Dec 18, 2024
5d8dd95
Linux CI: Add gobject-introspection
C0rn3j Dec 18, 2024
c8050b6
Linux CI: Add python3-gi-cairo
C0rn3j Dec 18, 2024
265bc9a
try adding libgirepository1.0-dev
C0rn3j Dec 18, 2024
df2d5de
Add libcairo2-dev
C0rn3j Dec 18, 2024
33aca20
Linux CI: moar deps
C0rn3j Dec 18, 2024
e488046
Log error for SDL renderer errors too
C0rn3j Dec 18, 2024
a866e6f
Fix up CI
C0rn3j Dec 18, 2024
edaea45
Editor is rendering pipes curved here...
C0rn3j Dec 18, 2024
ba8e459
Linux CI: Fix syntax
C0rn3j Dec 18, 2024
bfba2cf
Jpeg XL dep
C0rn3j Dec 18, 2024
3b27972
Try 24.10
C0rn3j Dec 18, 2024
86c01de
Make requirements.txt be the cross-platform full one
C0rn3j Dec 19, 2024
a0a583a
yeet JXLPY to temp pass CI
C0rn3j Dec 19, 2024
ce04ae1
Fix backslash
C0rn3j Dec 19, 2024
5355d91
test if sdl framework still gets copied
C0rn3j Dec 19, 2024
e192762
Linux CI: Clone submodules too
C0rn3j Dec 19, 2024
e37f533
Linux CI: Add devel packages for audio
C0rn3j Dec 19, 2024
e6e8082
Linux CI: Add libsamplerate0-dev
C0rn3j Dec 19, 2024
cede69d
Linux CI: Fix backslash
C0rn3j Dec 19, 2024
ebb743a
Try copying the entire sdl2dll dir
C0rn3j Dec 19, 2024
d8a67ba
Rename compile translations job
C0rn3j Dec 19, 2024
ecfac75
Try adding zeroconf to reqs
C0rn3j Dec 19, 2024
9b6e4ac
Fix pychromecast on pyinstaller
C0rn3j Dec 19, 2024
3054bde
Fix pyproject.toml indents
C0rn3j Dec 19, 2024
a697f3d
Clean up requirements a bit
C0rn3j Dec 19, 2024
c60ec29
Stop always printing pychromecast exception, now just to debug
C0rn3j Dec 19, 2024
f743516
Move Chrome() creation back to try block
C0rn3j Dec 19, 2024
7b3be40
pysdl fix attempt
C0rn3j Dec 19, 2024
a43a39e
try to switch over to dev pyinstaller better
C0rn3j Dec 19, 2024
19bcf49
Fix up reqs
C0rn3j Dec 19, 2024
761fa69
Use pysdl2-dll on all OSs
C0rn3j Dec 19, 2024
338cd94
Fix up Linux spec file
C0rn3j Dec 19, 2024
4262078
Try bumping ffmpeg from v5 to v7
C0rn3j Dec 20, 2024
3b97287
Ffmpeg v2 attempt numero duo
C0rn3j Dec 20, 2024
f83e1ad
Attempt to add a Rsvg hook
C0rn3j Dec 20, 2024
387f72b
Indent user files location
C0rn3j Dec 20, 2024
1feb331
Add toggle console button to Misc
C0rn3j Dec 20, 2024
97b31f1
Log PATH when looking for ffmpeg
C0rn3j Dec 20, 2024
b5458d5
Add the second part necessary for Rsvg
C0rn3j Dec 20, 2024
0212496
Fix up DConsole self reference
C0rn3j Dec 20, 2024
9df8cdb
Fix up optdeps
C0rn3j Dec 20, 2024
1e60cd8
try hacking libjxl on Linux
C0rn3j Dec 20, 2024
40669f9
Cleanup and document workarounds
C0rn3j Dec 20, 2024
bd71c36
Fix dpkg -i
C0rn3j Dec 20, 2024
055488e
Add libgif dep for jxl
C0rn3j Dec 20, 2024
79f6f96
Cleanup TODO notes
C0rn3j Dec 20, 2024
21a0ea8
Try packaging ffmpeg better
C0rn3j Dec 20, 2024
963ede8
Cleanup docs
C0rn3j Dec 20, 2024
fd32d56
Fix up action names
C0rn3j Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/build_Linux.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Build Linux app

on:
push:
pull_request:

jobs:
build:
runs-on: ubuntu-24.04
steps:
- name: Checkout source code
uses: actions/checkout@v4
with:
submodules: true

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.13'

- name: Install deps
run: |
sudo apt-get update
sudo apt-get install -y \
gettext \
gobject-introspection \
libgirepository1.0-dev \
python3-gi-cairo \
libcairo2-dev \
libpipewire-0.3-dev \
libdbus-1-dev \
libjxl-dev \
libflac-dev \
libgme-dev \
libmpg123-dev \
libopenmpt-dev \
libopusfile-dev \
libsamplerate0-dev \
libvorbis-dev \
libwavpack-dev
# JPEG-XL hack since 24.04 is too old
sudo apt-get install -y \
libgif7 \
wget
wget http://mirrors.kernel.org/ubuntu/pool/universe/j/jpeg-xl/libjxl-dev_0.10.3-4ubuntu1_amd64.deb
wget http://mirrors.kernel.org/ubuntu/pool/universe/j/jpeg-xl/libjxl0.10_0.10.3-4ubuntu1_amd64.deb
wget http://mirrors.kernel.org/ubuntu/pool/universe/h/highway/libhwy-dev_1.2.0-3ubuntu2_amd64.deb
wget http://mirrors.kernel.org/ubuntu/pool/universe/h/highway/libhwy1t64_1.2.0-3ubuntu2_amd64.deb
wget http://mirrors.kernel.org/ubuntu/pool/main/l/lcms2/liblcms2-dev_2.14-2build1_amd64.deb
sudo dpkg -i *.deb

- name: Install Python dependencies and setup venv
run: |
python -m pip install --upgrade pip
python -m venv .venv
source .venv/bin/activate
pip install \
-r requirements.txt \
build \
pyinstaller

- name: Build the project using python-build
run: |
source .venv/bin/activate
python -m compile_translations
python -m build --wheel

- name: Install the project into a venv
run: |
source .venv/bin/activate
pip install --prefix ".venv" dist/*.whl

- name: "[DEBUG] List all files"
run: find .

- name: Build Linux App with PyInstaller
run: |
source .venv/bin/activate
pyinstaller --log-level=DEBUG linux.spec

- name: Create ZIP
run: |
mkdir -p dist/zip
APP_NAME="TauonMusicBox"
APP_PATH="dist/${APP_NAME}"
ZIP_PATH="dist/zip/${APP_NAME}.zip"

zip -r "${ZIP_PATH}" "${APP_PATH}"

- name: Upload ZIP artifact
uses: actions/upload-artifact@v4
with:
name: TauonMusicBox-linux
path: dist/zip/TauonMusicBox.zip
95 changes: 95 additions & 0 deletions .github/workflows/build_macOS.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: Build macOS app

on:
push:
pull_request:

jobs:
build:
runs-on: macos-latest

steps:
- name: Checkout source code
uses: actions/checkout@v4
with:
submodules: true

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.13'

- name: brew update and upgrade
run: brew update && brew upgrade

- name: Install brew dependencies
run: |
brew install \
gobject-introspection \
gtk+3 \
pango \
sdl2 \
sdl2_image \
jpeg-xl \
ffmpeg \
librsvg \
opusfile \
libopenmpt \
wavpack \
game-music-emu

- name: Install Python dependencies and setup venv
run: |
python -m pip install --upgrade pip
python -m venv .venv
source .venv/bin/activate
pip install \
-r requirements.txt \
build
# Hack until https://github.com/pyinstaller/pyinstaller/issues/8936 is resolved
pip install https://github.com/rokm/pyinstaller/archive/refs/heads/macos-nested-framework-bundles.zip
# \
# pyinstaller
# pip uninstall pyinstaller
# CFLAGS: "-I/opt/homebrew/include"
# LDFLAGS: "-L/opt/homebrew/lib"

- name: Build the project using python-build
run: |
source .venv/bin/activate
python -m compile_translations
python -m build --wheel

- name: Install the project into a venv
run: |
source .venv/bin/activate
pip install --prefix ".venv" dist/*.whl

- name: "[DEBUG] List all files"
run: find .

- name: Build macOS app with PyInstaller
run: |
source .venv/bin/activate
pyinstaller --log-level=DEBUG mac.spec
env:
DYLD_LIBRARY_PATH: "/opt/homebrew/lib"

- name: "[DEBUG] List all files in .app"
run: find "dist/TauonMusicBox.app"

- name: Create DMG
run: |
mkdir -p dist/dmg
APP_NAME="TauonMusicBox"
APP_PATH="dist/${APP_NAME}.app"
DMG_PATH="dist/dmg/${APP_NAME}.dmg"

# Create a .dmg package
hdiutil create -volname "$APP_NAME" -srcfolder "$APP_PATH" -ov -format UDZO "$DMG_PATH"

- name: Upload DMG artifact
uses: actions/upload-artifact@v4
with:
name: TauonMusicBox-dmg
path: dist/dmg/TauonMusicBox.dmg
2 changes: 1 addition & 1 deletion .github/workflows/compile_translations.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Tauon Linux CI
name: Compile translations on Linux

on:
push:
Expand Down
2 changes: 2 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
recursive-include src/phazor/kissfft *.h
recursive-include src/phazor/miniaudio *.h
40 changes: 4 additions & 36 deletions compile_translations.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,38 +3,7 @@
import subprocess
from pathlib import Path


# TODO(Martin): import this class from tauon.py instead
class CustomLoggingFormatter(logging.Formatter):
"""Nicely format logging.loglevel logs"""

grey = "\x1b[38;20m"
grey_bold = "\x1b[38;1m"
yellow = "\x1b[33;20m"
yellow_bold = "\x1b[33;1m"
red = "\x1b[31;20m"
bold_red = "\x1b[31;1m"
reset = "\x1b[0m"
format = "%(asctime)s [%(levelname)s] [%(module)s] %(message)s"
format_verbose = "%(asctime)s [%(levelname)s] [%(module)s] %(message)s (%(filename)s:%(lineno)d)"

FORMATS = {
logging.DEBUG: grey_bold + format_verbose + reset,
logging.INFO: yellow + format + reset,
logging.WARNING: yellow_bold + format + reset,
logging.ERROR: red + format + reset,
logging.CRITICAL: bold_red + format_verbose + reset,
}

def format(self, record: dict) -> str:
log_fmt = self.FORMATS.get(record.levelno)
# Remove the miliseconds(%f) from the default string
date_fmt = "%Y-%m-%d %H:%M:%S"
formatter = logging.Formatter(log_fmt, date_fmt)
# Center align + min length things to prevent logs jumping around when switching between different values
record.levelname = f"{record.levelname:^7}"
record.module = f"{record.module:^10}"
return formatter.format(record)
from src.tauon.t_modules.logging import CustomLoggingFormatter

# DEBUG+ to file and std_err
logging.basicConfig(
Expand All @@ -56,7 +25,7 @@ def main() -> None:

for lang_file in languages:

if lang_file.name == "messages.pot":
if lang_file.name in ("messages.pot", ".DS_Store"):
continue

po_path = locale_folder / lang_file.name / "LC_MESSAGES" / "tauon.po"
Expand All @@ -69,10 +38,9 @@ def main() -> None:

if po_path.exists():
try:
subprocess.run(["/usr/bin/msgfmt", "-o", mo_path, po_path], check=True)
subprocess.run(["msgfmt", "-o", mo_path, po_path], check=True)
except Exception:
# Don't log the exception to make the build log clear
logging.error(f"Failed to compile translations for {lang_file.name}")
logging.exception(f"Failed to compile translations for {lang_file.name}")
compile_failure = True
else:
logging.info(f"Compiled: {lang_file.name}")
Expand Down
5 changes: 5 additions & 0 deletions extra/pyinstaller-hooks/hook-gi.repository.Rsvg.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from PyInstaller.utils.hooks.gi import GiModuleInfo

module_info = GiModuleInfo("Rsvg", "2.0")
if module_info.available:
binaries, datas, hiddenimports = module_info.collect_typelib_data()
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#-----------------------------------------------------------------------------
# Copyright (c) 2005-2023, PyInstaller Development Team.
#
# Distributed under the terms of the GNU General Public License (version 2
# or later) with exception for distributing the bootloader.
#
# The full license is in the file COPYING.txt, distributed with this software.
#
# SPDX-License-Identifier: (GPL-2.0-or-later WITH Bootloader-exception)
#-----------------------------------------------------------------------------


def pre_safe_import_module(api):
# PyGObject modules loaded through the gi repository are marked as MissingModules by modulegraph, so we convert them
# to RuntimeModules in order for their hooks to be loaded and executed.
api.add_runtime_module(api.module_name)
58 changes: 58 additions & 0 deletions linux.spec
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@


a = Analysis(
["src/tauon/__main__.py"],
pathex=[],
binaries=[],
datas=[
("src/tauon/assets", "assets"),
("src/tauon/locale", "locale"),
("src/tauon/theme", "theme"),
("src/tauon/templates", "templates"),
# This could only have SDL2.framework and SDL2_image.framework to save space...
(".venv/lib/python3.13/site-packages/sdl2dll/dll", "sdl2dll/dll"),
# (".venv/lib/python3.13/site-packages/sdl2dll/dll/SDL2.framework", "sdl2dll/dll/SDL2.framework"),
# (".venv/lib/python3.13/site-packages/sdl2dll/dll/SDL2_image.framework", "sdl2dll/dll/SDL2_image.framework"),
],
hiddenimports=[
"pylast",
"phazor",
# Zeroconf is hacked until this issue is resolved: https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/840
"zeroconf._utils.ipaddress",
"zeroconf._handlers.answers",
],
hookspath=["extra/pyinstaller-hooks"],
hooksconfig={},
runtime_hooks=[],
excludes=[],
noarchive=False,
optimize=0,
)
pyz = PYZ(a.pure)

exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name="Tauon Music Box",
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name="TauonMusicBox",
)
Loading
Loading