Huge Make-Over, Version 0.0.2 #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a pretty massiv commit, sorry. There is now a near-standalone backend providing `RBFSurrogate`, similar to Flux/Lux models. We use the same methods for `RBFModel`. It still needs some interface definitions, but evaluation and construction appears to work well. Actually did some profiling.
Enabling `T<:Number` in `RBFParamaters` and `RBFTrainingBuffers` allows for automatic differentiation and testing with `ForwardDiff`.
This commit introduces `@ignoraise` and `@ignorebreak` to propagate objects of type `AbstractStoppingCriterion`. The need arose in handling stopping based on maximum number of function evaluations. The tests were broken and log messages wrong. I have also made the `check_num_calls` function a bit simpler, but the interface does not seem ideal yet.
Now intersecting polyhedron before backtracking. Additionally: Simplication(?) of some types.
This required new wrappers.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RBF Surrogate Models
The internals of how RBF Surrogates are constructed has been redone.
As before, the construction is based off “Derivative-Free Optimization Algorithms For Computationally Expensive Functions,” (Wild, 2009).
In the old version, I did not really care for the matrix factorizations.
Finding a poised set of points for fully-linear interpolation needs repeated QR factorizations
of the point matrix.
Afterwards, additional points are found by updating the Cholesky factorization of
some symmetric matrix product involving the RBF Gram matrix.
those in
FastLapackInterface.Once I manage to make a pull request
to avoid even more allocations, we can also make
FastLapackInterfacea dependency.and the factors are used to compute the surrogate coefficients.
points, and we work with views mainly.
Such temporary buffers are stored in
RBFTrainingBuffers.RBFModelnow only needsRBFParametersfor successful training andreproducible evaluation.
Most importantly, evaluation is decoupled from the
RBFDatabase!!In older versions, we would view into the database to query interpolation points.
These are now copied instead, so that changes to the database don't invalidate a model.
we can thus share a database in multiple optimization runs.
For most of the RBF related changes, commit ab5cba8
is most relevant.
Other changes
In old versions, I tried to avoid the computation of an initial steplength by making it part
of the descent direction sub-problem, but accounting for the change in criticality measure
did not work out well.
Commit f1386c2
makes things look a bit more elegant.
AbstractNonlinearOperatorinterface.A new
AbstractNonlinearOperatorWrapper <: AbstractNonlinearOperator.AlgorithmOptions. Stopping based mainly on minimum radius.ReturnObject).