Skip to content
This repository has been archived by the owner on Apr 11, 2024. It is now read-only.

Remove Stan fit files from tests #6

Open
seantalts opened this issue Jan 4, 2020 · 2 comments
Open

Remove Stan fit files from tests #6

seantalts opened this issue Jan 4, 2020 · 2 comments

Comments

@seantalts
Copy link
Member

Copying the goals from discussion on stan-dev/stanc3#409,

Hmm. I think we could formulate these [tests in test/integration/tfp] as cram tests (aka gold tests, or expect tests in OCaml). Then we can have a way to automatically generate the correct values from Stan and an automated test of the TFP backend against those values. How does that sound?

I don't want to be prescriptive here - I think some goals in maybe descending order of preference from my perspective would be:

  1. Easy to understand what is being tested and what is going wrong when a test failure occurs
  2. Easy to create new tests or update the tested values when something changes (though these are testing end-to-end stuff that isn't supposed to change except for bug fixes and major version number bumps)
  3. Has a small footprint
  4. Separates code requiring the Stan toolchain from code requiring the TFP toolchain (not a huge priority but could be nice to have)

So it's possible that the best solutions to 1 and 2 have a larger footprint and that's okay, we can just organize around it.

@bob-carpenter
Copy link
Contributor

bob-carpenter commented Jan 6, 2020 via email

@adamhaber
Copy link
Contributor

Resurfacing this. The current TFP integration tests are not ideal (see the discussions on stan-dev/stanc3#409 and stan-dev/stanc3#388 ), and I'd like to improve them. Specifically, removing the Stan fit files and automating the log_prob comparison would be great. Would be happy to hear your thoughts on this.

A question in this regard - given a .stan file that has K parameters, can we compute the lp__ of N predefined vectors in R^K (unconstrained) during testing?

@WardBrian WardBrian transferred this issue from stan-dev/stanc3 Nov 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants