Merge opaque closure modules with the rest of the workqueue #50724
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pchintalapudi
added
compiler:codegen
vtjnash
reviewed
Jul 31, 2023
|
Maybe codegen_params can hang onto a reference to (workqueue, compiled_functions, external_functions) and have that be provided by JIT/AOT? That way we should be able to deduplicate both across the JIT and across whatever code is being AOT compiled. |
|
I think each of those are largely an implementation artifact of not being properly isolated from the enclosing scope, rather than an actual proper feature. In a future implementation, those should all be removable since each compilation unit would be self-contained + a list of other units that are required. It would then be up to the system, post-linking, to ensure all of those dependencies are satisfied. Right now all of that happens before compilation, which limits threading scalability of compilation. |
pchintalapudi
added
the
backport 1.10
|
Marking for backport to fix the issues noted by Keno earlier. |
36 tasks
|
@pchintalapudi, the open PRs from you with backport label tend to have a lot of conflicts that I cannot resolve myself. You could take on the effort of cherry picking these on top of #50708? You could just push them there when its done. |
pchintalapudi
added a commit
that referenced
this pull request
Aug 10, 2023
This sticks the compiled opaque closure module into the `compiled_functions` list of modules that we have compiled for the particular `jl_codegen_params_t`. We probably should manage that vector in codegen_params, since it lets us see if a particular codeinst has already been compiled but not yet emitted. (cherry picked from commit 441fcb1)
KristofferC
added a commit
that referenced
this pull request
Aug 16, 2023
Backported PRs: - [x] #50637 <!-- Remove SparseArrays legacy code --> - [x] #50665 <!-- print `@time` msg into print buffer --> - [x] #50523 <!-- Avoid generic call in most cases for getproperty --> - [x] #50635 <!-- `versioninfo()`: include build info and unofficial warning --> - [x] #50670 <!-- Make reinterpret specialize fully. --> - [x] #50666 <!-- include `--pkgimage=no` caches for stdlibs --> - [x] #50765 - [x] #50764 - [x] #50768 - [x] #50767 - [x] #50618 <!-- inference: continue const-prop' when concrete-eval returns non-inlineable --> - [x] #50689 <!-- Attach `tanpi` docstring to method --> - [x] #50671 <!-- Fix rdiv of complex lhs by real factorizations --> - [x] #50598 <!-- only limit types in stack traces in the REPL --> - [x] #50766 <!-- Don't partition alwaysinline functions --> - [x] #50771 <!-- re-allow non-string values in ENV `get!` --> - [x] #50682 <!-- Add fallback if we have make a weird GC decision. --> - [x] #50781 <!-- fix `bit_map!` with aliasing --> - [x] #50172 <!-- print feature flags used for matching pkgimage --> - [x] #50844 <!-- Bump OpenBLAS binaries to use the new GEMM multithreading threshold --> - [x] #50826 <!-- Update dependency builds --> - [x] #50845 <!-- fix #50438, use default pool for at-threads --> - [x] #50568 <!-- `Array(::AbstractRange)` should return an `Array` --> - [x] #50655 <!-- fix hashing regression. --> - [x] #50779 <!-- Minor refactor to image generation --> - [x] #50791 <!-- Make symbols internal in jl_create_native, and only externalize them when partitioning --> - [x] #50724 <!-- Merge opaque closure modules with the rest of the workqueue --> - [x] #50738 <!-- Add alignment to constant globals --> - [x] #50871 <!-- macOS: Don't inspect dead threadtls during exception handling. --> Need manual backport: Contains multiple commits, manual intervention needed: Non-merged PRs with backport label: - [ ] #50850 <!-- Remove weird Rational dispatch and add pi functions to list --> - [ ] #50823 <!-- Make ranges more robust with unsigned indexes. --> - [ ] #50809 <!-- Limit type-printing in MethodError --> - [ ] #50663 <!-- Fix Expr(:loopinfo) codegen --> - [ ] #50594 <!-- Disallow non-index Integer types in isassigned --> - [ ] #50385 <!-- Precompile pidlocks: add to NEWS and docs --> - [ ] #49805 <!-- Limit TimeType subtraction to AbstractDateTime -->
KristofferC
removed
the
backport 1.10
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This sticks the compiled opaque closure module into the
compiled_functionslist of modules that we have compiled for the particularjl_codegen_params_t. We probably should manage that vector in codegen_params, since it lets us see if a particular codeinst has already been compiled but not yet emitted.