Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add warning about using ppc_stat stat="mean" #321

Closed
avehtari opened this issue Mar 15, 2024 · 0 comments · Fixed by #329
Closed

Add warning about using ppc_stat stat="mean" #321

avehtari opened this issue Mar 15, 2024 · 0 comments · Fixed by #329

Comments

@avehtari
Copy link
Contributor

Most of the models do include parameters that do make the posterior predictive mean to match the observed mean, and the test statistic "mean" is perfectly dependent and not ancillary as required so that ppc would make some sense. stat="mean" might make sense given grouping if the groups don't have their own intercept parameters. I have recently seen too many times people posting ppc_stat plots with stat="mean" and the plots have zero information content. I guess we can't change the default to not break existing code.

  • When calling ppc_stat() with the default stat or with explicit stat="mean", create a warning saying "Test statistic "mean" is unlikely to be ancillary and thus likely to not provide any useful information. We recommend using some other test statistic." or if we want to avoid the word "ancillary" then warn: "Test statistic "mean" is very weak to detect anything interesting in most cases. We recommend using some other test statistic."
  • Change the ppc_stat doc to describe the problem with "mean" test statistic and use some other test statistic in the example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant