You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I believe there might be a logical error is selecting the best model based on 'error_scores' in refresh.R.
Based on my understanding, we want to choose the model with the highest error score. However, the code snippet below shows that the sorting is done on 'solID' first and then 'error_score', and given the solIDs are unique, the selection is done on the string of solIDs not on error score. I have seen this in practice too, seeing the next model for refresh is selected based on the solIDs strings.
In Refresh.R:
## Select winner model for current refresh
OutputCollectRF$resultHypParam <- OutputCollectRF$resultHypParam %>%
arrange(.data$solID, desc(.data$error_score)) %>%
select(.data$solID, everything()) %>%
ungroup()
bestMod <- OutputCollectRF$resultHypParam$solID[1]
Environment & Robyn version
Robyn version 3.7 and above
The text was updated successfully, but these errors were encountered:
sahbakn
changed the title
Selecting the best model for Refresh
Logical error in selecting the best model for Robyn_refresh
Apr 4, 2023
I've just deployed a fix directly to main branch @sahbakn
Thanks again for double checking for us. Please update Robyn and re-run the refresh function to automatically get the max(errors_score) model by default (and not the first one based on IDs!).
Also, after validating it runs as expected, feel free to close this ticket.
Describe issue
I believe there might be a logical error is selecting the best model based on 'error_scores' in refresh.R.
Based on my understanding, we want to choose the model with the highest error score. However, the code snippet below shows that the sorting is done on 'solID' first and then 'error_score', and given the solIDs are unique, the selection is done on the string of solIDs not on error score. I have seen this in practice too, seeing the next model for refresh is selected based on the solIDs strings.
In Refresh.R:
Environment & Robyn version
Robyn version 3.7 and above
The text was updated successfully, but these errors were encountered: