Merge remote-tracking branch 'rstudio_origin/master' into master

* rstudio_origin/master: fix rstudio#1905: a chunk with the class "fold-hide" should only hide itself (rstudio#1906) bump version after merging rstudio#1899 lua filter -> Lua filter Add an argument `lua_filters` to pandoc_options() to store them in the output format object (rstudio#1899) recursve into the parent dir at the end also suppor dir input in proj_root() list.files() with full.names = TRUE (so the files could be read with corret paths), and pass down the `file`/`pattern` arguments in the recursion use our own proj_root() instead of introducing a dependency rename package_root() to proj_root() generalize the package_root() function to work with more types of projects Discover site generators in index.Rmd files in parent directories (rstudio#1898) only install pkgdown for website build (rstudio#1900) Generalize number sections (rstudio#1879) enable figure cropping in pdf_document() only when both pdfcrop and ghostscript are found (yihui/knitr#954), and do not limit this feature to non-Windows platforms add the citation entry for the R Markdown Cookbook add lua filter vignette to pkgdown website (rstudio#1894) close rstudio#1889: evaluate the output argument before changing working directory in pandoc_convert() (rstudio#1890)
jonathan-g · Sep 24, 2020 · b92fcdb · b92fcdb
2 parents f8a77bb + ad2c8fd
commit b92fcdb
Show file tree

Hide file tree

Showing 39 changed files with 419 additions and 112 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -29,6 +29,8 @@ jobs:
     - r: release
       name: R release / RStudio Pandoc
       env: NETLIFY_SITE_ID=5d77c13c-e2ee-4a31-87ca-2fe657196160
+      before_cache:
+        - Rscript -e 'xfun::pkg_load2("pkgdown")'
       after_success:
         - "[[ ${TRAVIS_BRANCH}  = master  ]] || exit 0"
         - "[[ -z ${NETLIFY_AUTH_TOKEN} ]] && exit 0"

diff --git a/DESCRIPTION b/DESCRIPTION
@@ -1,7 +1,7 @@
 Package: rmarkdown
 Type: Package
 Title: Dynamic Documents for R
-Version: 2.3.3
+Version: 2.3.9
 Authors@R: c(
   person("JJ", "Allaire", role = "aut", email = "[email protected]"),
   person("Yihui", "Xie", role = c("aut", "cre"), email = "[email protected]", comment = c(ORCID = "0000-0003-0645-5666")),
@@ -16,7 +16,9 @@ Authors@R: c(
   #
   # Contributors, ordered alphabetically by first name
   person("Andrew", "Dunning", role = "ctb", comment = c(ORCID = "0000-0003-0464-5036")),
-  person("Atsushi", "Yasumoto", role = "ctb", comment = c(ORCID = "0000-0002-8335-495X")),
+  person("Atsushi", "Yasumoto", role = c("ctb", "cph"),
+         comment = c(ORCID = "0000-0002-8335-495X",
+                     cph = "Number sections Lua filter")),
   person("Barret", "Schloerke", role = "ctb"),
   person("Christophe", "Dervieux", role = "ctb"),
   person("Frederik", "Aust", role = "ctb", email = "[email protected]", comment = c(ORCID = "0000-0003-4900-788X")),
@@ -70,9 +72,9 @@ Authors@R: c(
   person("Aidan", "Lister", role = c("ctb", "cph"), 
          comment = "jQuery StickyTabs"),
   person("Benct Philip", "Jonsson", role = c("ctb", "cph"), 
-         comment = "pagebreak lua filter"),
+         comment = "pagebreak Lua filter"),
   person("Albert", "Krewinkel", role = c("ctb", "cph"), 
-         comment = "pagebreak lua filter")
+         comment = "pagebreak Lua filter")
   )
 Maintainer: Yihui Xie <[email protected]>
 Description: Convert R Markdown documents into a variety of formats.
@@ -99,7 +101,6 @@ Suggests:
     dygraphs,
     tibble,
     fs,
-    pkgdown,
     rsconnect
 SystemRequirements: pandoc (>= 1.14) - http://pandoc.org
 URL: https://github.com/rstudio/rmarkdown

diff --git a/NAMESPACE b/NAMESPACE
@@ -62,6 +62,7 @@ export(pandoc_exec)
 export(pandoc_highlight_args)
 export(pandoc_include_args)
 export(pandoc_latex_engine_args)
+export(pandoc_lua_filter_args)
 export(pandoc_metadata_arg)
 export(pandoc_options)
 export(pandoc_path_arg)

diff --git a/NEWS.md b/NEWS.md
@@ -1,13 +1,22 @@
 rmarkdown 2.4
 ================================================================================
 
+- Lua filters handling has been improved internally with some user-facing changes (#1899):
+    - New exported function `pandoc_lua_filter_args()` to return the Pandoc command-line argument to add a Lua filter.
+    - New argument `lua_filters` in `pandoc_options()` to pass the Lua filter paths to use with a format. This allow output format authors to add filters for a custom format using the `pandoc` argument of `output_format()` and to get filters from a format using `fmt$pandoc$lua_filters`.
+    - The Lua filters of an output format are now passed to Pandoc in `render()`. By default, they are passed to Pandoc before any other format-defined or user-defined Pandoc arguments (usually via the `pandoc_args` option of an output format). This ensures that filters of an output format are executed first. To change the default, you need to deal with it in a custom format (i.e., modify the elements in `fmt$pandoc$lua_filters`, such as reordering them).
+
 - Since **rmarkdown** 1.16, Pandoc's fenced `Div`'s are converted to LaTeX environments when the output format is LaTeX, e.g., `::: {.center data-latex=""}` is converted to `\begin{center}`. The attribute `data-latex` of the `Div` was mandatory, even if it is empty. In **rmarkdown** 2.2, we silently drop this requirement, which means `::: {.center}` is converted to `\begin{center}`. This turns out to be a bad idea, because users have no control over which Div's to be converted to LaTeX environments. Previously, they could opt-in by the `data-latex` attribute, but with **rmarkdown** 2.3, all Div's are converted to LaTeX environments unconditionally. What's more, this change led to bugs like https://stackoverflow.com/q/62340425/559676 and https://github.com/rstudio/bookdown/issues/883. Therefore the `data-latex` attribute became mandatory again in this version. If the LaTeX environment does not need arguments, you may use `data-latex=""`.
 
 - The two Lua fitlers `pagebreak.lua` and `latex-div.lua` (introduced in **rmarkdown** 1.16) are also applied to the output format `beamer_presentation` now (thanks, @XiangyunHuang, #1815).
 
 - When customizing formats with the `output_format` function, `pre_knit`, `opts_hooks`, and `knit_hooks` can now refer to `rmarkdown::metadata`. Previously, `rmarkdown::metadata` returned `list()` in these functions (thanks, @atusy, #1855).
 
-- Added the `number_sections` argument to the `word_document()` output format. This requires Pandoc >= v2.10.1, and is set to `FALSE` by default (thanks, @jooyoungseo, #1869).
+- Added the `number_sections` argument to following formats: `github_document`, `ioslides_presentation`, `md_document`, `odt_document`, `powerpoint_presentation`, `rtf_document`, `slidy_presentation`, `word_document`. These are powered by a Lua filter and requires Pandoc > 2.0. It will silently have no effect has before with previous pandoc version (thanks @atusy 1893).  Pandoc >= 2.10.1 adds `--number-sections` for docx format, and thus `word_document` prefers the native feature to the Lua filter (thanks, @jooyoungseo, #1869).
+
+- For the output format `pdf_document`, the option `fig_crop` will not be enabled unless both the programs `pdfcrop` and `ghostscript` are found (thanks, @dalupus, yihui/knitr#954).
+
+- Fixed a bug that a chunk with a class `fold-hide` hides the rest of the chunks even the output format setting `html_document(code_folding = "show")` (thanks, @atusy, #1906).
 
 - Updated documentation for `render` to make it clearer how options are set for the `output_format` parameter (See <https://github.com/rstudio/bookdown/issues/930>) (thanks, @jonathan-g, #1880)
 
@@ -149,13 +158,13 @@ rmarkdown 1.16
 
 - Added `self_contained` argument to `html_vignette` to keep intermediate directory if `self_contained = FALSE` (thanks, @cderv, #1641).
 
-- It is now possible to add pagebreak in HTML, Word, LaTeX, and ODT documents using the `\newpage` or `\pagebreak` command in an Rmd file. This is possible thanks to the [Pandoc's pagebreak lua filter](https://github.com/pandoc/lua-filters/tree/master/pagebreak). See `vignette("lua-filters", package = "rmarkdown")` (thanks, @cderv, #1626).
+- It is now possible to add pagebreak in HTML, Word, LaTeX, and ODT documents using the `\newpage` or `\pagebreak` command in an Rmd file. This is possible thanks to the [Pandoc's pagebreak Lua filter](https://github.com/pandoc/lua-filters/tree/master/pagebreak). See `vignette("lua-filters", package = "rmarkdown")` (thanks, @cderv, #1626).
 
 - The Pandoc extension `ascii_identifiers` is no longer enabled by default. If you still need it, you may use the argument `md_extensions = "+ascii_identifiers"` in the output format function. However, please note that this will trigger an error in a future version of Pandoc.
 
 - Output formats can be configured by arbitrary YAML files, which used to be restricted to `_output.yml` or `_output.yaml`. They can be specified via the `output_yaml` argument of `render()` or the `output_yaml` top-level parameter of YAML front matter, and the first existing one will be used. If `output_yaml` is specified both for `render()` and YAML front matter, then `render()` has the priority. If none are found, then `_output.yml` or `_output.yaml` will be used if they exist (thanks, @atusy, #1634).
 
-- Added a Pandoc lua filter to convert fenced Divs to LaTeX environments when the output format is `latex` or `beamer`. Basically a fenced Div `::: {.NAME data-latex="[OPTIONS]"}` is converted to `\begin{NAME}[OPTIONS] \end{NAME}` in LaTeX. The attribute `data-latex` must be provided, even if it is an empty string (meaning that the LaTeX environment does not have any optional arguments). For example, `::: {.verbatim data-latex=""}` generates a `verbatim` environment, and `::: {.minipage data-latex="{.5\textwidth}"}` generates `\begin{minipage}{.5\textwidth}`. This lua filter was originally written by @RLesur at https://github.com/yihui/bookdown-crc/issues/1. It will allow users to create custom blocks that work for both HTML and LaTeX output (e.g., info boxes or warning boxes).
+- Added a Pandoc Lua filter to convert fenced Divs to LaTeX environments when the output format is `latex` or `beamer`. Basically a fenced Div `::: {.NAME data-latex="[OPTIONS]"}` is converted to `\begin{NAME}[OPTIONS] \end{NAME}` in LaTeX. The attribute `data-latex` must be provided, even if it is an empty string (meaning that the LaTeX environment does not have any optional arguments). For example, `::: {.verbatim data-latex=""}` generates a `verbatim` environment, and `::: {.minipage data-latex="{.5\textwidth}"}` generates `\begin{minipage}{.5\textwidth}`. This Lua filter was originally written by @RLesur at https://github.com/yihui/bookdown-crc/issues/1. It will allow users to create custom blocks that work for both HTML and LaTeX output (e.g., info boxes or warning boxes).
 
 - Added `keep_html` argument to `github_document` so to save a preview HTML file in a working directory (thanks, @atusy, #1650).
 

diff --git a/R/beamer_presentation.R b/R/beamer_presentation.R
@@ -126,8 +126,6 @@ beamer_presentation <- function(toc = FALSE,
   # make sure the graphics package is always loaded
   if (identical(template, "default")) args <- c(args, "--variable", "graphics=yes")
 
-  args <- c(args, pandoc_lua_filters(c("pagebreak.lua", "latex-div.lua")))
-
   # custom args
   args <- c(args, pandoc_args)
 
@@ -151,11 +149,14 @@ beamer_presentation <- function(toc = FALSE,
   # return format
   output_format(
     knitr = knitr_options_pdf(fig_width, fig_height, fig_crop, dev),
-    pandoc = pandoc_options(to = "beamer",
-                            from = from_rmarkdown(fig_caption, md_extensions),
-                            args = args,
-                            latex_engine = latex_engine,
-                            keep_tex = keep_tex),
+    pandoc = pandoc_options(
+      to = "beamer",
+      from = from_rmarkdown(fig_caption, md_extensions),
+      args = args,
+      latex_engine = latex_engine,
+      keep_tex = keep_tex,
+      lua_filters = pkg_file_lua(c("pagebreak.lua", "latex-div.lua"))
+    ),
     pre_processor = pre_processor,
     intermediates_generator = intermediates_generator,
     clean_supporting = !keep_tex,

diff --git a/R/context_document.R b/R/context_document.R
@@ -90,9 +90,6 @@ context_document <- function(toc = FALSE,
   # content includes
   args <- c(args, includes_to_pandoc_args(includes))
 
-  # lua filters (added if pandoc > 2)
-  args <- c(args, pandoc_lua_filters("pagebreak.lua"))
-
   # args args
   args <- c(args, pandoc_args)
 
@@ -129,11 +126,13 @@ context_document <- function(toc = FALSE,
   # return format
   output_format(
     knitr = knitr_options_pdf(fig_width, fig_height, fig_crop, dev),
-    pandoc = pandoc_options(to = paste(c("context", output_extensions), collapse = ""),
-                            from = from_rmarkdown(fig_caption, md_extensions),
-                            args = args,
-                            keep_tex = FALSE,
-                            ext = ext),
+    pandoc = pandoc_options(
+      to = paste(c("context", output_extensions), collapse = ""),
+      from = from_rmarkdown(fig_caption, md_extensions),
+      args = args,
+      keep_tex = FALSE,
+      ext = ext,
+      lua_filters = pkg_file_lua("pagebreak.lua")),
     clean_supporting = !isTRUE(keep_tex),
     keep_md = keep_md,
     df_print = df_print,

diff --git a/R/github_document.R b/R/github_document.R
@@ -18,6 +18,7 @@
 #' @export
 github_document <- function(toc = FALSE,
                             toc_depth = 3,
+                            number_sections = FALSE,
                             fig_width = 7,
                             fig_height = 5,
                             dev = 'png',
@@ -30,20 +31,23 @@ github_document <- function(toc = FALSE,
                             keep_html = FALSE) {
 
   # add special markdown rendering template to ensure we include the title fields
+  # and add an optional feature to number sections
   pandoc_args <- c(
-    pandoc_args, "--template", pkg_file_arg(
-      "rmarkdown/templates/github_document/resources/default.md")
+    pandoc_args, "--template",
+    pkg_file_arg("rmarkdown/templates/github_document/resources/default.md")
   )
 
+
   pandoc2 <- pandoc2.0()
   # use md_document as base
   variant <- if (pandoc2) "gfm" else "markdown_github"
   if (!hard_line_breaks) variant <- paste0(variant, "-hard_line_breaks")
 
   format <- md_document(
     variant = variant, toc = toc, toc_depth = toc_depth,
-    fig_width = fig_width, fig_height = fig_height, dev = dev,
-    df_print = df_print, includes = includes, md_extensions = md_extensions,
+    number_sections = number_sections, fig_width = fig_width,
+    fig_height = fig_height, dev = dev, df_print = df_print,
+    includes = includes, md_extensions = md_extensions,
     pandoc_args = pandoc_args
   )
 

diff --git a/R/html_document_base.R b/R/html_document_base.R
@@ -106,9 +106,6 @@ html_document_base <- function(theme = NULL,
 
     preserved_chunks <<- extract_preserve_chunks(input_file)
 
-    # a lua filters added if pandoc2.0
-    args <- c(args, pandoc_lua_filters(c("pagebreak.lua", "latex-div.lua")))
-
     args
   }
 
@@ -170,7 +167,9 @@ html_document_base <- function(theme = NULL,
 
   output_format(
     knitr = NULL,
-    pandoc = pandoc_options(to = "html", from = NULL, args = args),
+    pandoc = pandoc_options(
+      to = "html", from = NULL, args = args,
+      lua_filters = pkg_file_lua(c("pagebreak.lua", "latex-div.lua"))),
     keep_md = FALSE,
     clean_supporting = FALSE,
     pre_knit = pre_knit,

diff --git a/R/ioslides_presentation.R b/R/ioslides_presentation.R
@@ -223,7 +223,8 @@
 #'   To create a PDF version of a presentation you can use Print to PDF
 #'   from Google Chrome.
 #' @export
-ioslides_presentation <- function(logo = NULL,
+ioslides_presentation <- function(number_sections = FALSE,
+                                  logo = NULL,
                                   slide_level = 2,
                                   incremental = FALSE,
                                   fig_width = 7.5,
@@ -343,6 +344,10 @@ ioslides_presentation <- function(logo = NULL,
     # add any custom pandoc args
     args <- c(args, pandoc_args)
 
+    # number sections
+    if (number_sections)
+      args <- c(args, pandoc_lua_filter_args(pkg_file_lua("number-sections.lua")))
+
     lua_writer <- file.path(dirname(input_file), "ioslides_presentation.lua")
     # The input directory may not be writable (on e.g. Shiny Server), so write
     # to the output directory in this case. We don't always do this since

diff --git a/R/md_document.R b/R/md_document.R
@@ -37,6 +37,7 @@ md_document <- function(variant = "markdown_strict",
                         preserve_yaml = FALSE,
                         toc = FALSE,
                         toc_depth = 3,
+                        number_sections = FALSE,
                         fig_width = 7,
                         fig_height = 5,
                         fig_retina = NULL,
@@ -75,10 +76,12 @@ md_document <- function(variant = "markdown_strict",
   # return format
   output_format(
     knitr = knitr_options_html(fig_width, fig_height, fig_retina, FALSE, dev),
-    pandoc = pandoc_options(to = variant,
-                            from = from_rmarkdown(extensions = md_extensions),
-                            args = args,
-                            ext = ext),
+    pandoc = pandoc_options(
+      to = variant,
+      from = from_rmarkdown(extensions = md_extensions),
+      args = args,
+      ext = ext,
+      lua_filters = if (number_sections) pkg_file_lua("number-sections.lua")),
     clean_supporting = FALSE,
     df_print = df_print,
     post_processor = post_processor

diff --git a/R/odt_document.R b/R/odt_document.R
@@ -31,7 +31,8 @@
 #' render("input.Rmd", odt_document(highlight = "zenburn"))
 #' }
 #' @export
-odt_document <- function(fig_width = 5,
+odt_document <- function(number_sections = FALSE,
+                         fig_width = 5,
                          fig_height = 4,
                          fig_caption = TRUE,
                          template = "default",
@@ -62,9 +63,6 @@ odt_document <- function(fig_width = 5,
   # reference odt
   args <- c(args, reference_doc_args("odt", reference_odt))
 
-  # lua filters (added if pandoc > 2)
-  args <- c(args, pandoc_lua_filters("pagebreak.lua"))
-
   # pandoc args
   args <- c(args, pandoc_args)
 
@@ -81,9 +79,12 @@ odt_document <- function(fig_width = 5,
   # return output format
   output_format(
     knitr = knitr,
-    pandoc = pandoc_options(to = "odt",
-                            from = from_rmarkdown(fig_caption, md_extensions),
-                            args = args),
+    pandoc = pandoc_options(
+      to = "odt",
+      from = from_rmarkdown(fig_caption, md_extensions),
+      args = args,
+      lua_filters = pkg_file_lua(c(
+        "pagebreak.lua", if (number_sections) "number-sections.lua"))),
     keep_md = keep_md,
     pre_processor = pre_processor,
     intermediates_generator = intermediates_generator

diff --git a/R/output_format.R b/R/output_format.R
@@ -198,6 +198,7 @@ merge_pandoc_options <- function(base,
 
   res <- merge_lists(base, overlay, recursive = FALSE)
   res$args <- c(base$args, overlay$args)
+  res$lua_filters <- c(base$lua_filters, overlay$lua_filters)
   res
 }
 
@@ -258,8 +259,8 @@ knitr_options_pdf <- function(fig_width,
 
   knit_hooks <- NULL
 
-  # apply cropping if requested and we have pdfcrop
-  crop <- fig_crop && !is_windows() && nzchar(find_program("pdfcrop"))
+  # apply cropping if requested and we have pdfcrop and ghostscript
+  crop <- fig_crop && find_program("pdfcrop") != '' && tools::find_gs_cmd() != ''
   if (crop) {
     knit_hooks = list(crop = knitr::hook_pdfcrop)
     opts_chunk$crop = TRUE
@@ -290,6 +291,10 @@ knitr_options_pdf <- function(fig_width,
 #'   chooses default based on \code{to}). This is typically used to force
 #'   the final output of a latex or beamer conversion to be \code{.tex}
 #'   rather than \code{.pdf}.
+#' @param lua_filters  Character vector of file paths to Lua filters to use with
+#'   this format. They will be added to pandoc command line call using
+#'   \code{--lua-filter} argument. See \code{vignette("lua-filters", package =
+#'   "rmarkdown")} to know more about Lua filters.
 #' @return An list that can be passed as the \code{pandoc} argument of the
 #'   \code{\link{output_format}} function.
 #' @seealso \link{output_format}, \link{rmarkdown_format}
@@ -299,13 +304,15 @@ pandoc_options <- function(to,
                            args = NULL,
                            keep_tex = FALSE,
                            latex_engine = c("pdflatex", "lualatex", "xelatex"),
-                           ext = NULL) {
+                           ext = NULL,
+                           lua_filters = NULL) {
   list(to = to,
        from = from,
        args = args,
        keep_tex = keep_tex,
        latex_engine = match.arg(latex_engine),
-       ext = ext)
+       ext = ext,
+       lua_filters = lua_filters)
 }
 
 #' R Markdown input format definition

diff --git a/R/pandoc.R b/R/pandoc.R
@@ -49,6 +49,9 @@ pandoc_convert <- function(input,
   # ensure we've scanned for pandoc
   find_pandoc()
 
+  # evaluate path arguments before changing working directory
+  force(output)
+
   # execute in specified working directory
   if (is.null(wd)) {
     wd <- base_dir(input)
@@ -714,9 +717,13 @@ pandoc_citeproc <- function() {
   if (file.exists(p)) p else bin
 }
 
-pandoc_lua_filters <- function(...) {
-  # lua filters was introduced in pandoc 2.0
-  if (pandoc2.0()) c(rbind("--lua-filter", pkg_file("rmd", "lua", ...)))
+#' @rdname pandoc_args
+#' @param lua_files Character vector of file paths to Lua filter files. Paths
+#'   will be transformed by \code{\link{pandoc_path_arg}}.
+#' @export
+pandoc_lua_filter_args <- function(lua_files) {
+  # Lua filters was introduced in pandoc 2.0
+  if (pandoc2.0()) c(rbind("--lua-filter", pandoc_path_arg(lua_files)))
 }