Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization algorithm to determine HDIs does not converge well #13

Open
michaelosthege opened this issue Nov 15, 2021 · 0 comments
Open
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@michaelosthege
Copy link
Member

michaelosthege commented Nov 15, 2021

While working on #12 I added tests and came up with one to check if calibr8.core._get_hdi is doing a good job.

It does not.

I believe the scipy.optimize.fmin doesn't cope well with the switching inside the hdi_objective, or at least not in combination with the step-function like area calculation by _interval_prob.

Some starting points to fix this:

  • Make a detailed heatmap visualization of the objective function that's being optimized, based on a coarse x_cdf and cdf input. (Hypothesis: It's not smooth, but has step functions in one dimension.)
  • Adapt test_interval_prob to an interpolation that should give much more accurate results compared to the step-function. (Also nicer conceptually!)
  • Use the triangular example from test_get_hdi as a starting point, because it's HDI is easy to determine analytically (centerline of the triangle to its right flank).
@michaelosthege michaelosthege added bug Something isn't working help wanted Extra attention is needed labels Nov 15, 2021
michaelosthege added a commit that referenced this issue Nov 16, 2021
lhelleckes pushed a commit that referenced this issue Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant