A Shiny app and module to monitor runtime progress.

[wlandau]

Prework

• I understand and agree to the code of conduct and the contributing guidelines.
• I have already submitted an issue to the issue tracker to discuss my idea with the maintainer.
• This pull request is not a draft.

Related GitHub issues and pull requests

• Ref: #

Summary

This PR implements tar_watch(), a new function to deploy a built-in Shiny app. The app repeatedly calls tar_visnetwork() at regular intervals. Call it in a separate R session to monitor the progress of the pipeline. tar_watch_ui() and tar_watch_server() are Shiny modules to help you build this functionality into your own apps that run targets pipelines internally.

Example

# Define an example _targets.R file with a slow pipeline.
library(targets)
tar_script({
  sleep_run <- function(...) {
    Sys.sleep(10)
  }
  tar_pipeline(
    tar_target(settings, sleep_run()),
    tar_target(data1, sleep_run(settings)),
    tar_target(data2, sleep_run(settings)),
    tar_target(data3, sleep_run(settings)),
    tar_target(model1, sleep_run(data1)),
    tar_target(model2, sleep_run(data2)),
    tar_target(model3, sleep_run(data3)),
    tar_target(figure1, sleep_run(model1)),
    tar_target(figure2, sleep_run(model2)),
    tar_target(figure3, sleep_run(model3)),
    tar_target(conclusions, sleep_run(c(figure1, figure2, figure3)))
  )
})

# Run the pipeline in a separate R process.
px <- tar_make(callr_function = callr::r_bg)

# Right away, run the app in the main process.
# Watch it refresh every 10 seconds.
# New targets should appear built each time.
tar_watch(seconds = 10, outdated = FALSE, targets_only = TRUE)

[wlandau]

cc @rpodcast. You might be interested in this module for your own apps.

[codecov-io]

Codecov Report

Merging #248 (870cfe3) into main (deec0c4) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #248   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          136       136           
  Lines         4233      4233           
=========================================
  Hits          4233      4233           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update deec0c4...870cfe3. Read the comment docs.

[wlandau]

Update: as of tonight, tar_watch() runs in the background by default (can be disabled). So now you can run tar_watch() and then tar_make() in the same R session.

[ginolhac]

Hello,
I recently presented targets and the custom Build in RStudio (I think from Noam Ross initially) here:
https://ulhpc-tutorials.readthedocs.io/en/latest/maths/R/PS10_targets.html#19
this is very convenient and also allows to run tar_watch() in the main process then (tested yesterday).
Maybe this custom build for runnning tar_make() could be in the manual as most people were interested by it.
Thanks again for this package!

[wlandau]

Awesome, thanks for sharing!

tar_make() actually has a callr_function argument to control how the pipeline gets deployed. If you run tar_make(callr_function = callr::r_bg()), the pipeline will run in the background and not block your R session. You don't have the console output anymore, but you do have tar_progress(), tar_visnetwork(), and of course tar_watch().

Incidentally, I just posted what I learned about non-blocking Shiny apps: https://wlandau.github.io/posts/2020-12-18-non-blocking-app/