Intermittent assertion error in HagerZhang

[baggepinnen]

I'm getting some occasional assertion errors that are not really telling me what's wrong with my problem, example:

  Got exception outside of a @test
  AssertionError: B > A
  Stacktrace:
   [1] (::LineSearches.HagerZhang{Float64,Base.RefValue{Bool}})(::Function, ::LineSearches.var"#ϕdϕ#6"{Optim.ManifoldObjective{OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}},Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Float64, ::Float64, ::Float64) at /home/travis/.julia/packages/LineSearches/WrsMD/src/hagerzhang.jl:277
   [2] HagerZhang at /home/travis/.julia/packages/LineSearches/WrsMD/src/hagerzhang.jl:101 [inlined]
   [3] perform_linesearch!(::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::LBFGS{Nothing,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}},Optim.var"#19#21"}, ::Optim.ManifoldObjective{OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}}) at /home/travis/.julia/packages/Optim/UkDyx/src/utilities/perform_linesearch.jl:53
   [4] update_state!(::OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::LBFGS{Nothing,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}},Optim.var"#19#21"}) at /home/travis/.julia/packages/Optim/UkDyx/src/multivariate/solvers/first_order/l_bfgs.jl:198
   [5] optimize(::OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::LBFGS{Nothing,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}},Optim.var"#19#21"}, ::Optim.Options{Float64,Nothing}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}) at /home/travis/.julia/packages/Optim/UkDyx/src/multivariate/optimize/optimize.jl:57
   [6] optimize(::OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::LBFGS{Nothing,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64,Base.RefValue{Bool}},Optim.var"#19#21"}, ::Optim.Options{Float64,Nothing}) at /home/travis/.julia/packages/Optim/UkDyx/src/multivariate/optimize/optimize.jl:33

https://travis-ci.org/github/baggepinnen/SpectralDistances.jl/jobs/672377449#L934

The call to optimize occurs here

[pkofod]

Something went wrong with the bracketing. Did you try MoreThuente instead? They're sort of "related". You can also try BackTracking.

[baggepinnen]

Do you have any feeling for when this error occurs? Is it my problem that has numerical problems? Too non-smooth etc?

[pkofod]

Is your objective not smooth?

[baggepinnen]

In theory it's smooth, but it solves an inner optimization problem the is subject to some numerical tolerance etc. so in practice it might not be perfectly smooth.

[pkofod]

Okay, if that means that the objective is noisy then this can certainly be a problem in terms of line search. I know because we've had that same issue in Pumas.jl where some fit methods have separate fits on the inside. We had only finite difference gradients in some places and were able to replace these with AD, but it also helped to solve the interior models to an ever so slightly more accurate tolerance...