Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v3.6.0 #205

Merged
merged 51 commits into from
Oct 4, 2023
Merged

Release v3.6.0 #205

merged 51 commits into from
Oct 4, 2023

Conversation

P403n1x87
Copy link
Owner

This is the v3.6.0 release PR.

@P403n1x87 P403n1x87 added the release New release label Oct 3, 2023
@codecov
Copy link

codecov bot commented Oct 3, 2023

Codecov Report

Attention: 165 lines in your changes are missing coverage. Please review.

Comparison is base (b7eb5e2) 69.75% compared to head (90005bf) 68.63%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #205      +/-   ##
==========================================
- Coverage   69.75%   68.63%   -1.12%     
==========================================
  Files          25       27       +2     
  Lines        2473     2519      +46     
  Branches      730      774      +44     
==========================================
+ Hits         1725     1729       +4     
- Misses        416      458      +42     
  Partials      332      332              
Files Coverage Δ
src/argparse.c 60.27% <ø> (-4.80%) ⬇️
src/error.c 100.00% <ø> (ø)
src/logging.c 73.46% <100.00%> (-8.54%) ⬇️
src/mem.h 85.71% <ø> (ø)
src/mojo.h 100.00% <ø> (ø)
src/msg.h 100.00% <ø> (ø)
src/python/symbols.h 100.00% <ø> (ø)
src/timer.h 100.00% <100.00%> (ø)
src/timing.h 100.00% <ø> (ø)
src/version.h 60.00% <50.00%> (+2.42%) ⬆️
... and 6 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link

github-actions bot commented Oct 3, 2023

Austin Benchmarks

Running Austin benchmarks with Python 3.10.13

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
Wall time [sampling interval: 1] 🟢 🟡 🟡 🟡
Wall time [sampling interval: 1000] 🟡 🟡 🟡 🟢
CPU time [sampling interval: 1000] 🔴 🟡 🟡 🔴
RSA keygen [sampling interval: 10] 🟢 🟡 🟡 🟡
Multiprocess wall time [sampling interval: 10] 🟡 🟡 🟡 🟢
Multiprocess wall time [sampling interval: 1000] 🟡 🟡 🟡 🟢

Benchmark Results

Wall time [sampling interval: 1]

Wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 116000 ± 3000 1 ± 0 8e-06 ± 4e-06 13.4 ± 0.5
dev 119000 ± 3000 1 ± 0 8e-06 ± 5e-06 13.1 ± 0.6
Wall time [sampling interval: 10]

Wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 115000 ± 3000 0.545 ± 0.002 1e-05 ± 4e-06 13.4 ± 0.5
dev 116000 ± 4000 0.545 ± 0.003 8e-06 ± 4e-06 13.4 ± 0.5
Wall time [sampling interval: 100]

Wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 10600 ± 300 0.00081 ± 0.0001 4e-05 ± 3e-05 15.1 ± 0.6
dev 10400 ± 200 0.001 ± 0.0004 3e-05 ± 3e-05 14.5 ± 0.7
Wall time [sampling interval: 1000]

Wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 1840 ± 30 0.0001 ± 0.0002 4e-05 ± 9e-05 22 ± 3
dev 1860 ± 20 0.0002 ± 0.0003 4e-05 ± 9e-05 19 ± 1
CPU time [sampling interval: 1]

CPU time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 51000 ± 5000 1 ± 0 1.8e-05 ± 9e-06 23.2 ± 0.9
dev 51000 ± 5000 1 ± 0 2e-05 ± 1e-05 23 ± 1
CPU time [sampling interval: 10]

CPU time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 51000 ± 4000 0.9979 ± 0.0002 2e-05 ± 1e-05 22.8 ± 0.9
dev 53000 ± 1000 0.9977 ± 0.0007 1.3e-05 ± 7e-06 22.5 ± 0.7
CPU time [sampling interval: 100]

CPU time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 5200 ± 100 0.0014 ± 0.0006 8e-05 ± 7e-05 24 ± 1
dev 5300 ± 100 0.0016 ± 0.0005 7e-05 ± 7e-05 24 ± 1
CPU time [sampling interval: 1000]

CPU time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 945 ± 4 0.0003 ± 0.0006 0.0001 ± 0.0002 33 ± 6
dev 941 ± 2 0.0004 ± 0.0006 0.0002 ± 0.0004 40 ± 2
RSA keygen [sampling interval: 1]

RSA keygen [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 38800 ± 500 1 ± 0 0.0002 ± 0.0001 25.0 ± 0.5
dev 39000 ± 1000 1 ± 0 0.0002 ± 0.0002 24.9 ± 0.7
RSA keygen [sampling interval: 10]

RSA keygen [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 38600 ± 400 0.985 ± 0.009 0.0003 ± 0.0002 25 ± 0
dev 39200 ± 400 0.986 ± 0.006 0.00014 ± 9e-05 24.9 ± 0.3
RSA keygen [sampling interval: 100]

RSA keygen [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 6232 ± 9 0.003 ± 0.002 0.0008 ± 0.0009 31.0 ± 0.9
dev 6200 ± 80 0.003 ± 0.003 0.0008 ± 0.0009 30 ± 2
RSA keygen [sampling interval: 1000]

RSA keygen [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 941 ± 1 0.0002 ± 0.0005 0.0004 ± 0.0007 35 ± 2
dev 940 ± 2 0 ± 0 0.0002 ± 0.0005 34 ± 4
Full metrics [sampling interval: 1]

Full metrics [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 50000 ± 3000 1 ± 0 2.3e-05 ± 8e-06 31 ± 2
dev 50000 ± 3000 1 ± 0 2e-05 ± 9e-06 31 ± 2
Full metrics [sampling interval: 10]

Full metrics [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 50000 ± 3000 1 ± 0 2e-05 ± 1e-05 31 ± 2
dev 50000 ± 3000 1 ± 0 2e-05 ± 1e-05 31 ± 2
Full metrics [sampling interval: 100]

Full metrics [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 9700 ± 400 0.0028 ± 0.0009 5e-05 ± 2e-05 34.7 ± 0.9
dev 9800 ± 300 0.0027 ± 0.001 2e-05 ± 4e-05 34 ± 1
Full metrics [sampling interval: 1000]

Full metrics [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 1850 ± 20 0.0003 ± 0.0003 0.0001 ± 0.0001 41 ± 2
dev 1850 ± 20 0.0002 ± 0.0002 0 ± 0 40 ± 3
Multiprocess wall time [sampling interval: 1]

Multiprocess wall time [sampling interval: 1]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 3300 ± 200 1 ± 0 0.00014 ± 6e-05 290 ± 20
dev 3300 ± 100 1 ± 0 0.00015 ± 4e-05 290 ± 10
Multiprocess wall time [sampling interval: 10]

Multiprocess wall time [sampling interval: 10]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 3100 ± 100 1 ± 0 0.00015 ± 3e-05 310 ± 10
dev 3300 ± 200 1 ± 0 0.00012 ± 5e-05 290 ± 20
Multiprocess wall time [sampling interval: 100]

Multiprocess wall time [sampling interval: 100]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 3100 ± 100 0.05 ± 0.01 9e-05 ± 3e-05 300 ± 10
dev 3300 ± 100 0.05 ± 0.01 6e-05 ± 4e-05 290 ± 10
Multiprocess wall time [sampling interval: 1000]

Multiprocess wall time [sampling interval: 1000]

Sample Rate Saturation Error Rate Sampling Speed
3.5.0 2900 ± 100 0.009 ± 0.001 2e-05 ± 2e-05 77 ± 5
dev 3100 ± 200 0.007 ± 0.003 2e-05 ± 2e-05 60 ± 10

This change improves the report from benchmark runs by adding t-tests
for each scenario. This allows the benchmarks to provide a summary at
the end with the scenarios that are likely to show perfomance difference
between the latest released version and the one that comes with the PR
the benchmarks run for.

This also updates the versions that are being benchmarked to include the
latest releases.
This change adds a data validation workflow. Data is validated by
performing a Hotelling T2 test on the collected data.
This change adds extra steps to the CI workflows to build Python
wheels so that the Austin binary can be installed with pip from
PyPI.
Use the more correct expression for computing the lasti index with
Python 3.11.
The version of cppcheck available from the package manager tends to be
outdated. In this change we build from sources instead. We cache the
build result for faster execution.
This change ensures that Austin reports the expected error code and
message when running without the superuser privileges.
The wrong symbol name was being used on OSX, preventing the inference of
the Python version from symbols.
Python < 3.8 is no longer maintained, so we drop support for it.
Previous versions of Austin can still be used if needed.
Since we no longer support CPython < 3.8, we drop the tracking of
symbols that are no longer exported by the CPython binaries in later
versions. We also rationalise the error propagation for a more robust
error handling and propagation logic.
We run the uwsgi tests in separated virtual environment to ensure that
we install it using the interpreter that Austin is being tested against.
Currently, we install the dependency in the environment used to run
pytest, which means that we test uwsgi with the same Python version in
every job.
We bump the checkout action version from 2 to 3, and setup-python
from 2 to 4 (the latest at the time this change is being made) to
resolve the node12 deprecation warnings.
We add path filtering to some GitHub workflows so that they are
triggered when necessary.
We group tests in categories to allow for better fine-graining when
running the suite.
We try to collect core dump on Linux once more
We also improve the wording of MacOS compatibility to explicitly
mention limitations with code-signing.
We add more details to the logs generated by Austin on Windows to
include a timestamp and process information. This allows us to collect
the data in tests for helping with investigations.
To ensure that we get the expected exit code and error messages, we set
the correct error in the internal _py_proc__run method on the fail paths
With the removal of the heap scanning logic, we no longer need to keep
track of VM bounds so we remove the relevant code.
We make sure that threads are resumed regardless of potential errors
during the sampling of the threads.
We make sure that we handle signals and propagate the signal number as
exit code. We negate the number to distinguish it from Austin error
codes.
When Austin is given a command to launch, we make sure that any
termination signal that Austin receives is propagated to the spawned
process to prevent it from running potentially indefinitely in the
background.
A segmentation fault was caught in CI exposing a missing NULL check on
a local variable in recently-added code. We add a check to prevent this
from happening.
We add support for sub-interpreters.
We speed up the benchmarks CI run by only running the latest release
and dev. Any issues caught in this way can be analysed locally if
needed.
The documentation sources are updated to include mention of the CPython
3.12 support and other features that are shipped with the upcoming 3.6
release of Austin.
@P403n1x87 P403n1x87 marked this pull request as ready for review October 4, 2023 10:17
@P403n1x87 P403n1x87 merged commit 7991c9d into master Oct 4, 2023
68 of 81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release New release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant