plot() not finding graphviz on Windows if graphviz is installed system wide.

[t-kalinowski]

First reported in Manning book discussion forum by George Fox.

[ggeeoorrgg]

Is this problem resolved?
I am triyng to reproduce some examples from the book "Deep learning with R". Calling plot(model) after Listing 7.11 falls with error: "See ?keras::plot.keras.engine.training.Model for instructions on how to install graphviz and pydot Error in plot(model) : ImportError: You must install pydot (pip install pydot) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) for plot_model to work."
But I have installed pydot, pydotplus, graphviz.
Moreover the same code in Python from its sibling "Deep Learning with Python" (with plot_model(model)) works fine.
Any thoughts?
P.S. On Ubuntu in R it works fine... What could went wrong on Windows?

[t-kalinowski]

I can only reproduce this in the RStudio IDE R console, I can't reproduce this in a terminal (in or out of the IDE). I don't think this is a reticulate or keras issue (at least, not directly).

To reproduce locally:

library(reticulate)
# virtualenv_create("rstudio-find-graphviz", python = install_python(), packages = "pydot")
use_virtualenv("rstudio-find-graphviz")
py_run_string("import pydot; d = pydot.Dot(); d.create()")

Gives output in the IDE:

  OSError: [WinError 6] The handle is invalid

but both these succeed in terminal:

C:/Users/kalin/Documents/.virtualenvs/rstudio-find-graphviz/Scripts/python.exe -c "import pydot; d = pydot.Dot(); print(d.create())"

C:/PROGRA~1/R/R-42~1.2/bin/Rscript.exe -e "library(reticulate); use_virtualenv('rstudio-find-graphviz'); py_run_string('import pydot; d = pydot.Dot(); print(d.create())')

expected output:

b"%!PS-Adobe-3.0\r\n%%Creator: graphviz version 7.1.0 (20230121.1956)\r\n%%Title: G\r\n%%Pages: (atend)\r\n%%BoundingBox: (atend)\r\n%%EndComments\r\nsave\r\n%%BeginProlog\r\n/DotDict 200 dict def\r\nDotDict begin\r\n\r\n/setupLatin1 {\r\nmark\r\n/EncodingVector 256 array def\r\n EncodingVector 0\r\n\r\nISOLatin1Encoding 0 255 getinterval putinterval\r\nEncodingVector 45 /hyphen put\r\n\r\n% Set up ISO Latin 1 character encoding\r\n/starnetISO {\r\n        dup dup findfont dup length dict begin\r\n        { 1 index /FID ne { def }{ pop pop } ifelse\r\n        } forall\r\n        /Encoding EncodingVector def\r\n        currentdict end definefont\r\n} def\r\n/Times-Roman starnetISO def\r\n/Times-Italic starnetISO def\r\n/Times-Bold starnetISO def\r\n/Times-BoldItalic starnetISO def\r\n/Helvetica starnetISO def\r\n/Helvetica-Oblique starnetISO def\r\n/Helvetica-Bold starnetISO def\r\n/Helvetica-BoldOblique starnetISO def\r\n/Courier starnetISO def\r\n/Courier-Oblique starnetISO def\r\n/Courier-Bold starnetISO def\r\n/Courier-BoldOblique starnetISO def\r\ncleartomark\r\n} bind def\r\n\r\n%%BeginResource: procset graphviz 0 0\r\n/coord-font-family /Times-Roman def\r\n/default-font-family /Times-Roman def\r\n/coordfont coord-font-family findfont 8 scalefont def\r\n\r\n/InvScaleFactor 1.0 def\r\n/set_scale {\r\n       dup 1 exch div /InvScaleFactor exch def\r\n       scale\r\n} bind def\r\n\r\n% styles\r\n/solid { [] 0 setdash } bind def\r\n/dashed { [9 InvScaleFactor mul dup ] 0 setdash } bind def\r\n/dotted { [1 InvScaleFactor mul 6 InvScaleFactor mul] 0 setdash } bind def\r\n/invis {/fill {newpath} def /stroke {newpath} def /show {pop newpath} def} bind def\r\n/bold { 2 setlinewidth } bind def\r\n/filled { } bind def\r\n/unfilled { } bind def\r\n/rounded { } bind def\r\n/diagonals { } bind def\r\n/tapered { } bind def\r\n\r\n% hooks for setting color \r\n/nodecolor { sethsbcolor } bind def\r\n/edgecolor { sethsbcolor } bind def\r\n/graphcolor { sethsbcolor } bind def\r\n/nopcolor {pop pop pop} bind def\r\n\r\n/beginpage {\t% i j npages\r\n\t/npages exch def\r\n\t/j exch def\r\n\t/i exch def\r\n\t/str 10 string def\r\n\tnpages 1 gt {\r\n\t\tgsave\r\n\t\t\tcoordfont setfont\r\n\t\t\t0 0 moveto\r\n\t\t\t(\\() show i str cvs show (,) show j str cvs show (\\)) show\r\n\t\tgrestore\r\n\t} if\r\n} bind def\r\n\r\n/set_font {\r\n\tfindfont exch\r\n\tscalefont setfont\r\n} def\r\n\r\n% draw text fitted to its expected width\r\n/alignedtext {\t\t\t% width text\r\n\t/text exch def\r\n\t/width exch def\r\n\tgsave\r\n\t\twidth 0 gt {\r\n\t\t\t[] 0 setdash\r\n\t\t\ttext stringwidth pop width exch sub text length div 0 text ashow\r\n\t\t} if\r\n\tgrestore\r\n} def\r\n\r\n/boxprim {\t\t\t\t% xcorner ycorner xsize ysize\r\n\t\t4 2 roll\r\n\t\tmoveto\r\n\t\t2 copy\r\n\t\texch 0 rlineto\r\n\t\t0 exch rlineto\r\n\t\tpop neg 0 rlineto\r\n\t\tclosepath\r\n} bind def\r\n\r\n/ellipse_path {\r\n\t/ry exch def\r\n\t/rx exch def\r\n\t/y exch def\r\n\t/x exch def\r\n\tmatrix currentmatrix\r\n\tnewpath\r\n\tx y translate\r\n\trx ry scale\r\n\t0 0 1 0 360 arc\r\n\tsetmatrix\r\n} bind def\r\n\r\n/endpage { showpage } bind def\r\n/showpage { } def\r\n\r\n/layercolorseq\r\n\t[\t% layer color sequence - darkest to lightest\r\n\t\t[0 0 0]\r\n\t\t[.2 .8 .8]\r\n\t\t[.4 .8 .8]\r\n\t\t[.6 .8 .8]\r\n\t\t[.8 .8 .8]\r\n\t]\r\ndef\r\n\r\n/layerlen layercolorseq length def\r\n\r\n/setlayer {/maxlayer exch def /curlayer exch def\r\n\tlayercolorseq curlayer 1 sub layerlen mod get\r\n\taload pop sethsbcolor\r\n\t/nodecolor {nopcolor} def\r\n\t/edgecolor {nopcolor} def\r\n\t/graphcolor {nopcolor} def\r\n} bind def\r\n\r\n/onlayer { curlayer ne {invis} if } def\r\n\r\n/onlayers {\r\n\t/myupper exch def\r\n\t/mylower exch def\r\n\tcurlayer mylower lt\r\n\tcurlayer myupper gt\r\n\tor\r\n\t{invis} if\r\n} def\r\n\r\n/curlayer 0 def\r\n\r\n%%EndResource\r\n%%EndProlog\r\n%%BeginSetup\r\n14 default-font-family set_font\r\n% /arrowlength 10 def\r\n% /arrowwidth 5 def\r\n\r\n% make sure pdfmark is harmless for PS-interpreters other than Distiller\r\n/pdfmark where {pop} {userdict /pdfmark /cleartomark load put} ifelse\r\n% make '<<' and '>>' safe on PS Level 1 devices\r\n/languagelevel where {pop languagelevel}{1} ifelse\r\n2 lt {\r\n    userdict (<<) cvn ([) cvn load put\r\n    userdict (>>) cvn ([) cvn load put\r\n} if\r\n\r\n%%EndSetup\r\nsetupLatin1\r\n%%Page: 1 1\r\n%%PageBoundingBox: 36 36 44 44\r\n%%PageOrientation: Portrait\r\n0 0 1 beginpage\r\ngsave\r\n36 36 8 8 boxprim clip newpath\r\n1 1 set_scale 0 rotate 40 40 translate\r\nendpage\r\nshowpage\r\ngrestore\r\n%%PageTrailer\r\n%%EndPage: 1\r\n%%Trailer\r\n%%Pages: 1\r\n%%BoundingBox: 36 36 44 44\r\nend\r\nrestore\r\n%%EOF\r\n"

[t-kalinowski]

@kevinushey do you have any guesses about why the code in the IDE might return that error?

[t-kalinowski]

Making sure the graphviz is on the PATH in the IDE doesn't fix the issue.

i.e., downloading from https://www.graphviz.org/download/ and then running:

PATH <- Sys.getenv("PATH")
PATH <- paste0(c(
  r"(C:\Users\kalin\Documents\windows_10_msbuild_Release_graphviz-7.1.0-win32\Graphviz\bin)",
  PATH),
  collapse = .Platform$path.sep)
Sys.setenv(PATH = PATH)

library(reticulate)
py_run_string("import pydot; d = pydot.Dot(); d.create()")

[kevinushey]

This might be related:

rstudio/reticulate#518

[ggeeoorrgg]

I can only reproduce this in the RStudio IDE R console...

Yes, you are most likely right. In my case the issue occurs in RStudio IDE. I ran this code in another IDE (VSC) and it works OK. So this is certainly a bug in RStudio.

[t-kalinowski]

Fixed on main now.

[ggeeoorrgg]

For some reason utils.py hasn`t included in keras 2.11.1 Windows binaries on CRAN. So after library updating this issue... occurs again.

[t-kalinowski]

Hi @ggeeoorrgg, this fix for this isn't on CRAN yet. It should go out in the next 2 weeks or so. For now you can install the development version of keras:

remotes::install_github("rstudio/keras")