Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples in the manual as MWE #640

Open
pgf-tikz-bot opened this issue Jul 3, 2018 · 36 comments · May be fixed by #883
Open

Examples in the manual as MWE #640

pgf-tikz-bot opened this issue Jul 3, 2018 · 36 comments · May be fixed by #883
Assignees

Comments

@pgf-tikz-bot
Copy link

Migrated from SourceForge
Author: krichter722
Timestamp: 2018-07-03 19:40:40.026000

I'm aware that the following is a huge pile of work, however I hope that you devs don't close it immediately because of that. It's intended to make this awesome software easier to use for beginners and trained users which tend to forget or can't keep up with the latest major changes:

It'd be nice to have a link to a Short, Self Contained, Correct (Compilable), Example for every code sample so that if you click on a link to a code sample like

\begin{tikzpicture}
\draw[step=.5cm,gray,very thin] (-1.4,-1.4) grid(1.4,1.4);
\draw (-1.5,0) -- (1.5,0);
\draw (0,-1.5) -- (0,1.5);
\draw (0,0) circle[radius=1cm];
\draw (3mm,0mm) arc[start angle=0,end angle=30,radius=3mm];
\end{tikzpicture}

on page xy you're redirected to the plain text file https://sourceforge.net/p/pgf/wiki/samples/001.tex which contains a SSCCE which can be directly passed to the compiler.

The manual describes what the user needs to do in a satisfying way, starting with

\documentclass{article}% say
\usepackage{tikz}
\begin{document}
We are working on
\begin{tikzpicture}
\draw (-1.5,0) -- (1.5,0);
\draw (0,-1.5) -- (0,1.5);
\end{tikzpicture}.
\end{document}

However, it's sometimes hard to follow all the packages which have been introduced and settings which have been changed over the course of the chapter where the snippet is shown (which can be dozens of pages long). The plain text links would avoid this and reduce the number of mistakes can make while using the snippet almost to zero.

This would also improve the incapacity of most PDF viewers to copy code accurate, i.e. without introducing unwanted linebreaks and replacing spaces with linebreaks, even though that's not PGF's job.

If you think that's a good idea regardless of the effort, then you might want to keep those SSCCEs coming piece by piece, maybe through merge requests. I can provide some, however, I often find myself trying to get the snippets to compile for hours which is the main reason why I'm suggesting this.

The SSCCEs could be provided for LaTeX, TeX, ConTeX and LuaLaTeX.

@hmenke hmenke removed the open label Apr 2, 2019
@hmenke hmenke assigned ghost and Mo-Gul Apr 6, 2019
@hmenke hmenke changed the title Add SSCCE links to manual Examples in the manual as MWE Apr 18, 2019
@hmenke hmenke pinned this issue Apr 18, 2019
@hmenke
Copy link
Member

hmenke commented Apr 25, 2019

@tallmarmot @Mo-Gul I have now more or less finished the extractor script. You can give it a go using the following bash script

#!/bin/bash
mkdir -p mwe
texlua extract.lua text-en/ mwe/
cd mwe
n=1
N=$(ls -1q *.tex | wc -l)
for i in *.tex; do
    echo -n "Processing $n/$N $i"
    if [ -f "${i%%.tex}.pdf" ]; then
        echo -n " Skipped." # skip existing
    else
        lualatex -interaction=batchmode -halt-on-error "$i" > /dev/null
        if [ "x$?" == "x1" ]; then
            echo -n " Failed!"
        fi
    fi
    echo ""
    n=$((n+1))
done

Save this script as, say build.sh in doc/generic/pgf/ next to extract.lua. Then you can run it using bash build.sh. Since this typesets all the examples individually, it takes a lot of time. The output will look like

[...snip...]
Processing 59/2128 pgfmanual-en-base-animations-8.tex
Processing 60/2128 pgfmanual-en-base-animations-9.tex
Processing 61/2128 pgfmanual-en-base-arrows-17.tex Failed!
Processing 62/2128 pgfmanual-en-base-arrows-8.tex Failed!
Processing 63/2128 pgfmanual-en-base-arrows-9.tex Failed!
Processing 64/2128 pgfmanual-en-base-decorations-11.tex
Processing 65/2128 pgfmanual-en-base-decorations-12.tex
Processing 66/2128 pgfmanual-en-base-decorations-1.tex
[...snip...]

As you can see not all of the examples typeset out-of-the-box. For the failed ones, you can inspect the log to find the error. As you can see from the extractor script

local preamble = [[
\usetikzlibrary{3d,arrows,arrows.spaced,arrows.meta,bending,babel,calc,
fit,patterns,plotmarks,shapes.geometric,shapes.misc,shapes.symbols,
shapes.arrows,shapes.callouts,shapes.multipart,shapes.gates.logic.US,
shapes.gates.logic.IEC,circuits.logic.US,circuits.logic.IEC,
circuits.logic.CDH,circuits.ee.IEC,datavisualization,
datavisualization.polar,datavisualization.formats.functions,er,automata,
backgrounds,chains,topaths,trees,petri,mindmap,matrix,calendar,folding,
fadings,shadings,spy,through,turtle,positioning,scopes,
decorations.fractals,decorations.shapes,decorations.text,
decorations.pathmorphing,decorations.pathreplacing,decorations.footprints,
decorations.markings,shadows,lindenmayersystems,intersections,
fixedpointarithmetic,fpu,svg.path,external,graphs,graphs.standard,quotes,
math,angles,views,animations,rdf,perspective}
\usetikzlibrary{graphdrawing}
\usegdlibrary{trees,circular,layered,examples,force,phylogenetics,routing}
]]

I currently just include all libraries in the preamble. This of course has to change. We only want to use the libraries which are necessary for each example.

hmenke added a commit to hmenke/pgf that referenced this issue Apr 25, 2019
@Mo-Gul
Copy link
Contributor

Mo-Gul commented Apr 29, 2019

@hmenke, thank you very much for providing the extractor script. I gave it a try and it works perfectly fine. And you are right, it takes a lot of time. Since I don't have clue about Bash scripts: Is there a chance to parallize the compilation?

@tallmarmot, what do you think about the extractor script? Do you see a chance to modify it according to "our understanding"?

@hmenke
Copy link
Member

hmenke commented Apr 29, 2019

Here is a script which runs in parallel on all available processors.

#!/bin/bash
mkdir -p mwe
texlua extract.lua text-en/ mwe/
cd mwe

ls -1q *.tex | sort | xargs -I @ -P `nproc` bash -c '
    i="@"
    str="Building $i"
    if [ -f "${i%%.tex}.pdf" ]; then
        str="$str Skipped."
    else
        lualatex -interaction=batchmode -halt-on-error "$i" > /dev/null
        if [ "x$?" == "x1" ]; then
            str="$str Failed!"
        fi
    fi
    echo $str'

@Mo-Gul
Copy link
Contributor

Mo-Gul commented May 9, 2019

Works like a charm. Thank you for that. The only downside is that now one doesn't have a clue about the progress. But I think one has to live with this minor issue.

@hmenke
Copy link
Member

hmenke commented May 9, 2019

I could add a counter again but that would require some sort of locking mechanism because xargs runs in parallel and I don't think bash variables are atomic.

@hmenke
Copy link
Member

hmenke commented Jun 24, 2019

@Mo-Gul I needed this for something else, so I had a look at how to lock file descriptors in Bash. Here is a new script with counter.

#!/bin/bash
mkdir -p mwe
texlua extract.lua text-en/ mwe/
cd mwe

touch /tmp/pgfmanuallock
echo 1 > /tmp/pgfmanualcount
ls -1q *.tex | sort | xargs -I @ -P `nproc` bash -c '
    i="@"
    { flock -x 200;
        n=$(cat /tmp/pgfmanualcount);
        echo $((n+1)) > /tmp/pgfmanualcount;
    } 200> /tmp/pgfmanuallock
    str="Building $n/$(ls -1q *.tex | wc -l) $i"
    if [ -f "${i%%.tex}.pdf" ]; then
        str="$str Skipped."
    else
        lualatex -interaction=batchmode -halt-on-error "$i" > /dev/null
        if [ "x$?" == "x1" ]; then
            str="$str Failed!"
        fi
    fi
    echo $str'

@hmenke
Copy link
Member

hmenke commented Jun 24, 2019

I have also run the script to see how many examples currently fail even in the presence of all libraries and there are luckily only few (181 out of 2128).

grep Failed build.log
Building 62/2128 pgfmanual-en-base-arrows-8.tex Failed!
Building 61/2128 pgfmanual-en-base-arrows-17.tex Failed!
Building 63/2128 pgfmanual-en-base-arrows-9.tex Failed!
Building 75/2128 pgfmanual-en-base-images-3.tex Failed!
Building 77/2128 pgfmanual-en-base-images-4.tex Failed!
Building 128/2128 pgfmanual-en-base-patterns-4.tex Failed!
Building 133/2128 pgfmanual-en-base-plots-6.tex Failed!
Building 161/2128 pgfmanual-en-base-quick-3.tex Failed!
Building 217/2128 pgfmanual-en-base-transformations-31.tex Failed!
Building 218/2128 pgfmanual-en-base-transformations-32.tex Failed!
Building 219/2128 pgfmanual-en-base-transformations-33.tex Failed!
Building 220/2128 pgfmanual-en-base-transformations-34.tex Failed!
Building 221/2128 pgfmanual-en-base-transformations-35.tex Failed!
Building 222/2128 pgfmanual-en-base-transformations-36.tex Failed!
Building 235/2128 pgfmanual-en-base-transparency-11.tex Failed!
Building 245/2128 pgfmanual-en-base-transparency-9.tex Failed!
Building 293/2128 pgfmanual-en-dv-axes-76.tex Failed!
Building 294/2128 pgfmanual-en-dv-axes-78.tex Failed!
Building 295/2128 pgfmanual-en-dv-axes-79.tex Failed!
Building 296/2128 pgfmanual-en-dv-axes-80.tex Failed!
Building 297/2128 pgfmanual-en-dv-axes-81.tex Failed!
Building 301/2128 pgfmanual-en-dv-axes-85.tex Failed!
Building 303/2128 pgfmanual-en-dv-axes-87.tex Failed!
Building 304/2128 pgfmanual-en-dv-axes-88.tex Failed!
Building 305/2128 pgfmanual-en-dv-axes-89.tex Failed!
Building 308/2128 pgfmanual-en-dv-axes-92.tex Failed!
Building 323/2128 pgfmanual-en-dv-main-15.tex Failed!
Building 339/2128 pgfmanual-en-dv-stylesheets-10.tex Failed!
Building 340/2128 pgfmanual-en-dv-stylesheets-11.tex Failed!
Building 343/2128 pgfmanual-en-dv-stylesheets-6.tex Failed!
Building 344/2128 pgfmanual-en-dv-stylesheets-7.tex Failed!
Building 345/2128 pgfmanual-en-dv-stylesheets-9.tex Failed!
Building 354/2128 pgfmanual-en-gd-algorithm-layer-1.tex Failed!
Building 355/2128 pgfmanual-en-gd-algorithm-layer-2.tex Failed!
Building 480/2128 pgfmanual-en-library-circuits-25.tex Failed!
Building 482/2128 pgfmanual-en-library-circuits-27.tex Failed!
Building 500/2128 pgfmanual-en-library-circuits-47.tex Failed!
Building 502/2128 pgfmanual-en-library-circuits-49.tex Failed!
Building 515/2128 pgfmanual-en-library-circuits-61.tex Failed!
Building 516/2128 pgfmanual-en-library-circuits-62.tex Failed!
Building 518/2128 pgfmanual-en-library-circuits-64.tex Failed!
Building 520/2128 pgfmanual-en-library-circuits-66.tex Failed!
Building 521/2128 pgfmanual-en-library-circuits-67.tex Failed!
Building 522/2128 pgfmanual-en-library-circuits-68.tex Failed!
Building 523/2128 pgfmanual-en-library-circuits-69.tex Failed!
Building 525/2128 pgfmanual-en-library-circuits-70.tex Failed!
Building 526/2128 pgfmanual-en-library-circuits-71.tex Failed!
Building 527/2128 pgfmanual-en-library-circuits-72.tex Failed!
Building 528/2128 pgfmanual-en-library-circuits-73.tex Failed!
Building 529/2128 pgfmanual-en-library-circuits-74.tex Failed!
Building 670/2128 pgfmanual-en-library-fpu-2.tex Failed!
Building 731/2128 pgfmanual-en-library-perspective-10.tex Failed!
Building 732/2128 pgfmanual-en-library-perspective-11.tex Failed!
Building 735/2128 pgfmanual-en-library-perspective-14.tex Failed!
Building 740/2128 pgfmanual-en-library-perspective-4.tex Failed!
Building 741/2128 pgfmanual-en-library-perspective-5.tex Failed!
Building 742/2128 pgfmanual-en-library-perspective-6.tex Failed!
Building 743/2128 pgfmanual-en-library-perspective-7.tex Failed!
Building 744/2128 pgfmanual-en-library-perspective-8.tex Failed!
Building 745/2128 pgfmanual-en-library-perspective-9.tex Failed!
Building 782/2128 pgfmanual-en-library-rdf-23.tex Failed!
Building 783/2128 pgfmanual-en-library-rdf-24.tex Failed!
Building 811/2128 pgfmanual-en-library-shapes-12.tex Failed!
Building 810/2128 pgfmanual-en-library-shapes-11.tex Failed!
Building 814/2128 pgfmanual-en-library-shapes-15.tex Failed!
Building 816/2128 pgfmanual-en-library-shapes-17.tex Failed!
Building 820/2128 pgfmanual-en-library-shapes-21.tex Failed!
Building 822/2128 pgfmanual-en-library-shapes-23.tex Failed!
Building 824/2128 pgfmanual-en-library-shapes-25.tex Failed!
Building 826/2128 pgfmanual-en-library-shapes-27.tex Failed!
Building 829/2128 pgfmanual-en-library-shapes-2.tex Failed!
Building 833/2128 pgfmanual-en-library-shapes-33.tex Failed!
Building 840/2128 pgfmanual-en-library-shapes-3.tex Failed!
Building 841/2128 pgfmanual-en-library-shapes-40.tex Failed!
Building 844/2128 pgfmanual-en-library-shapes-43.tex Failed!
Building 846/2128 pgfmanual-en-library-shapes-45.tex Failed!
Building 851/2128 pgfmanual-en-library-shapes-4.tex Failed!
Building 852/2128 pgfmanual-en-library-shapes-50.tex Failed!
Building 856/2128 pgfmanual-en-library-shapes-54.tex Failed!
Building 861/2128 pgfmanual-en-library-shapes-59.tex Failed!
Building 862/2128 pgfmanual-en-library-shapes-5.tex Failed!
Building 866/2128 pgfmanual-en-library-shapes-63.tex Failed!
Building 869/2128 pgfmanual-en-library-shapes-66.tex Failed!
Building 871/2128 pgfmanual-en-library-shapes-68.tex Failed!
Building 874/2128 pgfmanual-en-library-shapes-70.tex Failed!
Building 875/2128 pgfmanual-en-library-shapes-71.tex Failed!
Building 882/2128 pgfmanual-en-library-shapes-78.tex Failed!
Building 885/2128 pgfmanual-en-library-shapes-80.tex Failed!
Building 887/2128 pgfmanual-en-library-shapes-82.tex Failed!
Building 888/2128 pgfmanual-en-library-shapes-83.tex Failed!
Building 890/2128 pgfmanual-en-library-shapes-85.tex Failed!
Building 893/2128 pgfmanual-en-library-shapes-88.tex Failed!
Building 899/2128 pgfmanual-en-library-shapes-93.tex Failed!
Building 904/2128 pgfmanual-en-library-shapes-98.tex Failed!
Building 1099/2128 pgfmanual-en-module-parser-2.tex Failed!
Building 1100/2128 pgfmanual-en-module-parser-3.tex Failed!
Building 1161/2128 pgfmanual-en-pgfkeysfiltered-10.tex Failed!
Building 1165/2128 pgfmanual-en-pgfkeysfiltered-2.tex Failed!
Building 1166/2128 pgfmanual-en-pgfkeysfiltered-5.tex Failed!
Building 1167/2128 pgfmanual-en-pgfkeysfiltered-6.tex Failed!
Building 1168/2128 pgfmanual-en-pgfkeysfiltered-7.tex Failed!
Building 1169/2128 pgfmanual-en-pgfkeysfiltered-9.tex Failed!
Building 1170/2128 pgfmanual-en-pgfsys-animations-10.tex Failed!
Building 1171/2128 pgfmanual-en-pgfsys-animations-11.tex Failed!
Building 1172/2128 pgfmanual-en-pgfsys-animations-12.tex Failed!
Building 1173/2128 pgfmanual-en-pgfsys-animations-13.tex Failed!
Building 1174/2128 pgfmanual-en-pgfsys-animations-14.tex Failed!
Building 1175/2128 pgfmanual-en-pgfsys-animations-15.tex Failed!
Building 1176/2128 pgfmanual-en-pgfsys-animations-17.tex Failed!
Building 1177/2128 pgfmanual-en-pgfsys-animations-18.tex Failed!
Building 1178/2128 pgfmanual-en-pgfsys-animations-19.tex Failed!
Building 1179/2128 pgfmanual-en-pgfsys-animations-20.tex Failed!
Building 1180/2128 pgfmanual-en-pgfsys-animations-21.tex Failed!
Building 1181/2128 pgfmanual-en-pgfsys-animations-22.tex Failed!
Building 1182/2128 pgfmanual-en-pgfsys-animations-3.tex Failed!
Building 1183/2128 pgfmanual-en-pgfsys-animations-4.tex Failed!
Building 1184/2128 pgfmanual-en-pgfsys-animations-5.tex Failed!
Building 1185/2128 pgfmanual-en-pgfsys-animations-6.tex Failed!
Building 1186/2128 pgfmanual-en-pgfsys-animations-7.tex Failed!
Building 1187/2128 pgfmanual-en-pgfsys-animations-8.tex Failed!
Building 1188/2128 pgfmanual-en-pgfsys-animations-9.tex Failed!
Building 1217/2128 pgfmanual-en-tikz-actions-38.tex Failed!
Building 1218/2128 pgfmanual-en-tikz-actions-39.tex Failed!
Building 1220/2128 pgfmanual-en-tikz-actions-40.tex Failed!
Building 1231/2128 pgfmanual-en-tikz-actions-50.tex Failed!
Building 1357/2128 pgfmanual-en-tikz-arrows-38.tex Failed!
Building 1358/2128 pgfmanual-en-tikz-arrows-39.tex Failed!
Building 1360/2128 pgfmanual-en-tikz-arrows-40.tex Failed!
Building 1361/2128 pgfmanual-en-tikz-arrows-41.tex Failed!
Building 1362/2128 pgfmanual-en-tikz-arrows-42.tex Failed!
Building 1363/2128 pgfmanual-en-tikz-arrows-43.tex Failed!
Building 1364/2128 pgfmanual-en-tikz-arrows-44.tex Failed!
Building 1365/2128 pgfmanual-en-tikz-arrows-45.tex Failed!
Building 1366/2128 pgfmanual-en-tikz-arrows-46.tex Failed!
Building 1367/2128 pgfmanual-en-tikz-arrows-47.tex Failed!
Building 1368/2128 pgfmanual-en-tikz-arrows-48.tex Failed!
Building 1369/2128 pgfmanual-en-tikz-arrows-49.tex Failed!
Building 1371/2128 pgfmanual-en-tikz-arrows-50.tex Failed!
Building 1373/2128 pgfmanual-en-tikz-arrows-52.tex Failed!
Building 1372/2128 pgfmanual-en-tikz-arrows-51.tex Failed!
Building 1413/2128 pgfmanual-en-tikz-coordinates-19.tex Failed!
Building 1546/2128 pgfmanual-en-tikz-graphs-153.tex Failed!
Building 1665/2128 pgfmanual-en-tikz-matrices-23.tex Failed!
Building 1729/2128 pgfmanual-en-tikz-pics-11.tex Failed!
Building 1730/2128 pgfmanual-en-tikz-pics-12.tex Failed!
Building 1732/2128 pgfmanual-en-tikz-pics-16.tex Failed!
Building 1735/2128 pgfmanual-en-tikz-pics-2.tex Failed!
Building 1736/2128 pgfmanual-en-tikz-pics-3.tex Failed!
Building 1737/2128 pgfmanual-en-tikz-pics-4.tex Failed!
Building 1738/2128 pgfmanual-en-tikz-pics-5.tex Failed!
Building 1739/2128 pgfmanual-en-tikz-pics-6.tex Failed!
Building 1740/2128 pgfmanual-en-tikz-pics-7.tex Failed!
Building 1811/2128 pgfmanual-en-tikz-shapes-121.tex Failed!
Building 1812/2128 pgfmanual-en-tikz-shapes-122.tex Failed!
Building 1865/2128 pgfmanual-en-tikz-shapes-55.tex Failed!
Building 1949/2128 pgfmanual-en-tikz-transparency-22.tex Failed!
Building 2045/2128 pgfmanual-en-tutorial-chains-10.tex Failed!
Building 2046/2128 pgfmanual-en-tutorial-chains-11.tex Failed!
Building 2047/2128 pgfmanual-en-tutorial-chains-12.tex Failed!
Building 2048/2128 pgfmanual-en-tutorial-chains-13.tex Failed!
Building 2049/2128 pgfmanual-en-tutorial-chains-14.tex Failed!
Building 2050/2128 pgfmanual-en-tutorial-chains-15.tex Failed!
Building 2051/2128 pgfmanual-en-tutorial-chains-16.tex Failed!
Building 2053/2128 pgfmanual-en-tutorial-chains-18.tex Failed!
Building 2054/2128 pgfmanual-en-tutorial-chains-19.tex Failed!
Building 2056/2128 pgfmanual-en-tutorial-chains-20.tex Failed!
Building 2058/2128 pgfmanual-en-tutorial-chains-22.tex Failed!
Building 2059/2128 pgfmanual-en-tutorial-chains-23.tex Failed!
Building 2060/2128 pgfmanual-en-tutorial-chains-24.tex Failed!
Building 2062/2128 pgfmanual-en-tutorial-chains-26.tex Failed!
Building 2061/2128 pgfmanual-en-tutorial-chains-25.tex Failed!
Building 2065/2128 pgfmanual-en-tutorial-chains-4.tex Failed!
Building 2066/2128 pgfmanual-en-tutorial-chains-5.tex Failed!
Building 2067/2128 pgfmanual-en-tutorial-chains-6.tex Failed!
Building 2068/2128 pgfmanual-en-tutorial-chains-7.tex Failed!
Building 2069/2128 pgfmanual-en-tutorial-chains-8.tex Failed!
Building 2070/2128 pgfmanual-en-tutorial-chains-9.tex Failed!
Building 2087/2128 pgfmanual-en-tutorial-map-11.tex Failed!
Building 2090/2128 pgfmanual-en-tutorial-map-15.tex Failed!
Building 2091/2128 pgfmanual-en-tutorial-map-16.tex Failed!
Building 2128/2128 pgfmanual-en-xxcolor-1.tex Failed!

That means for all the other examples you can go ahead and check which libraries are actually needed. Let's choose for example

pgfmanual-en-base-decorations-1.tex
\documentclass{article}
\usepackage{fp,pgf,tikz,xcolor}
\usetikzlibrary{3d,arrows,arrows.spaced,arrows.meta,bending,babel,calc,
  fit,patterns,plotmarks,shapes.geometric,shapes.misc,shapes.symbols,
  shapes.arrows,shapes.callouts,shapes.multipart,shapes.gates.logic.US,
  shapes.gates.logic.IEC,circuits.logic.US,circuits.logic.IEC,
  circuits.logic.CDH,circuits.ee.IEC,datavisualization,
  datavisualization.polar,datavisualization.formats.functions,er,automata,
  backgrounds,chains,topaths,trees,petri,mindmap,matrix,calendar,folding,
  fadings,shadings,spy,through,turtle,positioning,scopes,
  decorations.fractals,decorations.shapes,decorations.text,
  decorations.pathmorphing,decorations.pathreplacing,decorations.footprints,
  decorations.markings,shadows,lindenmayersystems,intersections,
  fixedpointarithmetic,fpu,svg.path,external,graphs,graphs.standard,quotes,
  math,angles,views,animations,rdf,perspective}
\usetikzlibrary{graphdrawing}
\usegdlibrary{trees,circular,layered,examples,force,phylogenetics,routing}
\begin{document}
\makeatletter
\tikz \draw decorate[decoration=zigzag] {(0,0) -- (3,0)};
\end{document}

For this we only need \usetikzlibrary{decorations,decorations.pathmorphing}, so I go to the corresponding location in the manual and add the libraries to the options of the codeexample.

\begin{codeexample}[libraries/tikz={decorations,decorations.pathmorphing}]
\tikz \draw decorate[decoration=zigzag] {(0,0) -- (3,0)};
\end{codeexample}

It will show up in the manual like this:
test
For now this is only a preliminary solution and the details might change in the future, but if you could add all the required libraries for each codeexample that would be a great help. Right now, libraries/tikz and libraries/pgf are available, so it is best to start with those examples which do not require \usegdlibrary or \usepackage in the preamble.

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 24, 2019

@hmenke, great work (as always). It seems that the both keys libraries/tikz and libraries/pgf are not enough (for automated testing). We need another key that holds \tikzset stuff and doesn't show up in the manual. This is needed e.g. for the tutorial examples. Have a look at e.g.

\begin{codeexample}[]
\begin{tikzpicture}[node distance=5mm]
\node (dot) [terminal] {.};
\node (digit) [terminal,right=of dot] {digit};
\node (E) [terminal,right=of digit] {E};
\draw [help lines] let \p1 = (dot.base),
\p2 = (digit.base),
\p3 = (E.base)
in (-.5,\y1) -- (3.5,\y1)
(-.5,\y2) -- (3.5,\y2)
(-.5,\y3) -- (3.5,\y3);
\end{tikzpicture}
\end{codeexample}

(which corresponds to the extracted pgfmanual-en-tutorial-chains-4.tex) where the terminal style is no longer defined/repeated ...

The definitions of the styles are given at the beginning of the file

\tikzset{
nonterminal/.style={
% The shape:
rectangle,
% The size:
minimum size=6mm,
% The border:
very thick,
draw=red!50!black!50, % 50% red and 50% black,
% and that mixed with 50% white
% The filling:
top color=white, % a shading that is white at the top...
bottom color=red!50!black!20, % and something else at the bottom
% Font
font=\itshape
},
terminal/.style={
% The shape:
rounded rectangle,
minimum size=6mm,
% The rest
very thick,draw=black!50,
top color=white,bottom color=black!20,
font=\ttfamily},
skip loop/.style={to path={-- ++(0,#1) -| (\tikztotarget)}}
}

Maybe you have an idea how to add grab/store them and then either state the needed styles or just all of them. But in the latter case there is needed some kind of a reset thus that not all following examples contain stuff that is not needed any more.

To make it not too easy there is a need to grab several \tikzsets. E.g. for the given example the definition of the styles is updated later in the file

\tikzset{terminal/.append style={text height=1.5ex,text depth=.25ex}}
\tikzset{nonterminal/.append style={text height=1.5ex,text
depth=.25ex}}

Also we need to grab definitions

\def\matrixcontent{
% First row:
\& \& \& \& \& \& \& \& \& \& \& \node (plus) [terminal] {+};\\
% Second row:
\node (p1) [point] {}; \& \node (ui1) [nonterminal] {unsigned integer}; \&
\node (p2) [point] {}; \& \node (dot) [terminal] {.}; \&
\node (p3) [point] {}; \& \node (digit) [terminal] {digit}; \&
\node (p4) [point] {}; \& \node (p5) [point] {}; \&
\node (p6) [point] {}; \& \node (e) [terminal] {E}; \&
\node (p7) [point] {}; \& \&
\node (p8) [point] {}; \& \node (ui2) [nonterminal] {unsigned integer}; \&
\node (p9) [point] {}; \& \node (p10) [point] {};\\
% Third row:
\& \& \& \& \& \& \& \& \& \& \& \node (minus)[terminal] {-};\\
}
\begin{codeexample}[pre={\tikzset{ampersand replacement=\&,point/.style={coordinate}}}]
\begin{tikzpicture}[skip loop/.style={to path={-- ++(0,#1) -| (\tikztotarget)}},
hv path/.style={to path={-| (\tikztotarget)}},
vh path/.style={to path={|- (\tikztotarget)}}]
\matrix[row sep=1mm,column sep=2mm] { \matrixcontent };
\graph {
(p1) -> (ui1) -- (p2) -> (dot) -- (p3) -> (digit) -- (p4)
-- (p5) -- (p6) -> (e) -- (p7) -- (p8) -> (ui2) -- (p9) -> (p10);
(p4) ->[skip loop=-5mm] (p3);
(p2) ->[skip loop=5mm] (p5);
(p6) ->[skip loop=-11mm] (p9);
(p7) ->[vh path] (plus) -> [hv path] (p8);
(p7) ->[vh path] (minus) -> [hv path] (p8);
};
\end{tikzpicture}
\end{codeexample}

(which corresponds to the extracted pgfmanual-en-tutorial-chains-15.tex).

If my above suggestion cannot be implemented (easily), maybe there is a possibility to abuse the codeexamples pre key (see previous code snippet) for our needs?

@ghost
Copy link

ghost commented Jun 24, 2019

We also need something that allows us to load pgfmodules (e.g. for nonlinear transformations) unless we agree on changing the corresponding libraries in such a way that they always load the corresponding modules automatically. Whether or not we need the \tikzset I do not have a strong opinion on because after all at the corresponding place in the manual this is explained very well. However, one conceivable way to go would be a key "other" which can be used to add everything else like \usepgfmodule or \tikzset such that all commands run through. (If I would know how I can check out the current version of the repository and commit changes, I would be able to test all the above. Is there a tutorial which is not entirely stupid and allows one to see how that works?)

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 24, 2019

@tallmarmot, you are right. Just having a look at the manual you don't need to "catch" the \tikzsets. But for the automatic testing using the above bash script you need them ... (That I already tried to say in my previous comment, but maybe it wasn't clear enough.)

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 25, 2019

Ok, for now I use the pre key and added all styles and definitions. This works fine. Only drawback is that styles and definitions containing hashes need to be provided in double form, e.g.

\begin{codeexample}[pre={\tikzset{
skip loop/.style={to path={-- ++(0,##1) -| (\tikztotarget)}},
hv path/.style={to path={-| (\tikztotarget)}},
vh path/.style={to path={|- (\tikztotarget)}}}}]

But of course then also in the extracted files the hashes are doubled, but need to be in single form. In all these cases I added comment lines on top, thus the user has an idea what he has to do to make the example work. Of course this doesn't help for the automatic testing. These lines start with % !!! (and in the code of course with pre={ % !!!).

Also I have found out that

\begin{codeexample}[width=4cm]
\begin{minipage}{3.5cm}\raggedright
\color{red}Red text,%
\begin{colormixin}{25!white}
washed-out red text,
\color{blue} washed-out blue text,
\begin{colormixin}{25!black}
dark washed-out blue text,
\color{green} dark washed-out green text,%
\end{colormixin}
back to washed-out blue text,%
\end{colormixin}
and back to red.
\end{minipage}%
\end{codeexample}

isn't extracted. When I remove the optional argument it is. Is this a bug in extract.lua?

@hmenke
Copy link
Member

hmenke commented Jun 25, 2019

width={4cm} should help as a workaround.

@hmenke
Copy link
Member

hmenke commented Jun 25, 2019

Fixed in 0a6f197

@hmenke
Copy link
Member

hmenke commented Jun 26, 2019

Following the request by @tallmarmot I have changed the way libraries are listed. I have removed the libraries/tikz and libraries/pgf options and replaced it with a single option preamble in which you can put anything that should go in the preamble, e.g.

\begin{codeexample}[preamble={\usetikzlibrary{decorations,decorations.pathmorphing}}]
\tikz \draw decorate[decoration=zigzag] {(0,0) -- (3,0)};
\end{codeexample}

Right now the content of preamble is simply piped through \detokenize. I might add more elaborate formatting in the future but I think this approach is flexible enough for now. Remember, the main objective currently is to identify which libraries are required to make each example compile. Pretty printing in the manual is only secondary at this point.

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 27, 2019

@hmenke, it would be nice if you could add some more features

  1. Possibility to empty stuff collected by setup code.
  2. An option for codeexample that the collected setup code can be ignored in this codeexample.

To 1: Background is that sometimes styles or definitions are only needed for a section and not for the whole chapter. Simply enclosing this codeexample in (curly) braces like

{
\begin{codeexample}[setup code,hidden]
    ....
\end{codeexample}
...
}

will nonetheless add this collected stuff to a codeexample after the closing (curly) brace.

To 2: Sometimes there are simple examples to introduce some stuff before this is applied to the "real" example. Of course for these all the collected stuff is not needed and might confuse users when they see all these definitions.

@hmenke
Copy link
Member

hmenke commented Jun 27, 2019

You have clearly not understood what this task is about. Let me make this clear:
The aim is to annotate the examples in the manual with the necessary \usetikzlibrary etc. such that a user reading the manual can easily infer which libraries and packages to include in their document to obtain the demonstrated functionality.
The aim is not to make the examples copy-pastable or anything like that. Also the purpose of the extract.lua is not to generate the most minimal example from the code in the manual, but to extract a compilable example that we can use for regression testing.

@hmenke
Copy link
Member

hmenke commented Jun 18, 2020

@Mo-Gul I've put some stuff in place so that you can add options to the Lua examples.

A table of examples can be converted as such:

-  examples = {[["
-    \tikz \graph [spring electrical layout, horizontal=0 to 1]
-      { 0 [electric charge=1] -- subgraph C_n [n=10] };
-  "]],[["
-    \tikz \graph [spring electrical layout, horizontal=0 to 1]
-      { 0 [electric charge=5] -- subgraph C_n [n=10] };
-  "]],[["
-    \tikz \graph [spring electrical layout, horizontal=0 to 1]
-      { [clique] 1 [electric charge=5], 2, 3, 4 };
-  "]]
- }
+  examples = {
+    {
+      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
+      code = [["
+        \tikz \graph [spring electrical layout, horizontal=0 to 1]
+          { 0 [electric charge=1] -- subgraph C_n [n=10] };
+      "]]
+    },{
+      code = [["
+        \tikz \graph [spring electrical layout, horizontal=0 to 1]
+          { 0 [electric charge=5] -- subgraph C_n [n=10] };
+      "]]
+    },{
+      code = [["
+        \tikz \graph [spring electrical layout, horizontal=0 to 1]
+          { [clique] 1 [electric charge=5], 2, 3, 4 };
+      "]]
+    }
+  }

Sometimes though there are single examples, which are given as a string. These have to be wrapped in two tables (note the double braces in the example):

-  examples = [["
-    \tikz \graph [spring electrical layout, horizontal=0 to 1]
-      { 0 [electric charge=1] -- subgraph C_n [n=10] };
-  "]]
+  examples = {{
+      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
+      code = [["
+        \tikz \graph [spring electrical layout, horizontal=0 to 1]
+          { 0 [electric charge=1] -- subgraph C_n [n=10] };
+      "]]
+    }}

Please also disregard the code in tex/generic/pgf/graphdrawing/lua/pgf/gd/force/jedi. This doesn't seem to be interfaced and doesn't even compile.

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 18, 2020

Thank you for the additions. I'll have a look at them. But it seems that I am not able to test them, because

  • currently the Lua examples (examples = as you provided an example) are not extracted by the extract.lua script.

  • Currently "only" the Lua examples given as codeexamples are extracted and these also in a way that they can't be TeXed. For example

-- \begin{codeexample}[preamble={\usetikzlibrary{graphs,graphdrawing}
-- \usegdlibrary{force}}]
-- \tikz \graph [random seed=10, spring layout] {
-- a -- {b, c, d} -- e -- f -- {g,h} -- {a,b,e};
-- };
-- \end{codeexample}

is extracted as

\documentclass{standalone}
\usepackage{fp,pgf,tikz,xcolor}
\usetikzlibrary{graphs,graphdrawing}
--    \usegdlibrary{force}
\begin{document}

-- \tikz \graph [spring layout, node distance=7mm] { subgraph C_n[n=3] };
-- \tikz \graph [spring layout]                    { subgraph C_n[n=3] };
-- \tikz \graph [spring layout, node distance=15mm]{ subgraph C_n[n=3] };
--
\end{document}

Could you fix this as well, i.e. remove the leading --?

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 18, 2020

For the record: From the files in https://github.com/pgf-tikz/pgf/tree/master/doc/generic/pgf/text-en the extracted codeexamples

  • pgfmanual-en-base-images-3.tex
  • pgfmanual-en-base-scopes-5tex
  • pgfmanual-en-dv-visualizers-24.tex
  • pgfmanual-en-gd-algorithm-layer-2.tex
  • pgfmanual-en-gd-algorithm-layer-3.tex
  • pgfmanual-en-library-rdf-26.tex
  • pgfmanual-en-tikz-actions-53.tex
  • pgfmanual-en-tikz-graphs-5.tex

don't compile.

@Mo-Gul
Copy link
Contributor

Mo-Gul commented Jun 19, 2020

@hmenke, applying the changes as you have stated in your comment above for multiple examples in one go show the options stuff only at the first example (see the screenshot of another instance).

multiple example output

So either you see a possibility to state the options once which then are applied to all the following codes, e.g. as

   examples = {
+      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
     {
-      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { 0 [electric charge=1] -- subgraph C_n [n=10] };
       "]]
     },{
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { 0 [electric charge=5] -- subgraph C_n [n=10] };
       "]]
     },{
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { [clique] 1 [electric charge=5], 2, 3, 4 };
       "]]
     }
   }

for your given code above or I add the options to every code instance of the examples, i.e.

   examples = {
     {
       options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { 0 [electric charge=1] -- subgraph C_n [n=10] };
       "]]
     },{
+      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { 0 [electric charge=5] -- subgraph C_n [n=10] };
       "]]
     },{
+      options = [["preamble={\usetikzlibrary{graphs,graphdrawing} \usegdlibrary{force}}"]],
       code = [["
         \tikz \graph [spring electrical layout, horizontal=0 to 1]
           { [clique] 1 [electric charge=5], 2, 3, 4 };
       "]]
     }
   }

How shall I proceed?

Mo-Gul added a commit to Mo-Gul/pgf that referenced this issue Jun 21, 2020
So far this is done as @hmenke suggested, i.e. the libraries are only shown for the first example if multiple examples are given.
@Mo-Gul Mo-Gul linked a pull request Jun 21, 2020 that will close this issue
1 task
@hmenke hmenke removed this from the 3.1.6 milestone Aug 26, 2020
@projetmbc
Copy link

Hello.

The first example of phylogenetics with graphdrawing on page 485 should also indicate the use of the 1st style \pgfgdset{..}.

@hansonchar
Copy link

hansonchar commented Jun 9, 2024

@Mo-Gul

So either you see a possibility to state the options once which then are applied to all the following codes ...

Currently such apply-all feature is not yet supported, but it doesn't seem difficult to achieve. Will post a small PR on this.

hansonchar added a commit to hansonchar/pgf that referenced this issue Jun 10, 2024
hansonchar added a commit to hansonchar/pgf that referenced this issue Jun 10, 2024
@hansonchar
Copy link

The generated pdf can be downloaded at https://github.com/hansonchar/pgf/actions/runs/9441252330

hansonchar added a commit to hansonchar/pgf that referenced this issue Jun 10, 2024
hansonchar added a commit to hansonchar/pgf that referenced this issue Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

5 participants