Skip to content

Commit fbedc77

Browse files
dcuartiellesdcuartielles
committed
added export as HTML or PDF
1 parent ac16852 commit fbedc77

File tree

6 files changed

+233
-7
lines changed

6 files changed

+233
-7
lines changed

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Bash Site Generator
22

3-
The **Bahs Site Generator** is a piece of software to help in the creation of courses which will be rendered as **Markdown** files, including an index, etc.
3+
The **Bash Site Generator** is a piece of software to help in the creation of courses which will be rendered as **Markdown** files, including an index, etc.
44

55
The generator is a collection of scripts that can be executed step by step to:
66

‎build/config/config.conf

+7
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ NUM_PAGES=0
2121
VERBOSE=0
2222
DEFAULT_TEMPLATE="basic1"
2323

24+
## Export
25+
TYPE_FILE="HTML"
26+
EXPORT_FOLDER="export"
27+
TEMP_EXPORT_FILE="export.md"
28+
PDF_EXPORT_FILE="site.pdf"
29+
2430
## Data handling
2531
SEPARATOR='¤'
2632
PARAMETER_SEPARATOR=','
@@ -45,3 +51,4 @@ FIELD_NAMES=("OBJECTIVES" "INTRODUCTION" "PARTS" "VIDEO" "DESCRIPTION" "CIRCUIT"
4551
FIELD_TYPES=("text" "text" "text" "video" "text" "image" "code" "text" "license")
4652
##FIELD_PROPERTIES=("true,text" "true,text" "true,text" "true,video" "true,text" "true,image,local" "true,code,C,true" "true,text" "true,license")
4753
DEFAULT_CONTENT=("${EMPTY_STR}" "${EMPTY_STR}" "${EMPTY_STR}" "${EMPTY_VID}" "${EMPTY_STR}" "img.png" "CodeListing" "${EMPTY_STR}" "${LICENSE_EMBED}")
54+
EXPORT_TYPES=("HTML" "PDF")

‎build/export_site.sh

+168
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
#!/bin/bash
2+
3+
## 20220225 This export script takes the rendered website and uses pandoc
4+
## to convert to other formats such as HTML or PDF
5+
6+
## The possible parameters are:
7+
## ** -l: locale (default en)
8+
## ** -t: type of file (default HTML)
9+
## ** -r: render folder (default .. a.k.a. root)
10+
## ** -p: pandoc parameters
11+
## ** -e: export folder (default export)
12+
13+
## Load utils file
14+
UTILS_PATH='./utils.sh'
15+
16+
## Include the functions
17+
source "${UTILS_PATH}"
18+
19+
## Load config file (https://wiki.bash-hackers.org/howto/conffile#secure_it)
20+
CONFIG_PATH='./config/config.conf'
21+
22+
## Check for malformed config instructions
23+
configReturn=(checkConfigFile)
24+
25+
## Otherwise go on and source it:
26+
source "${CONFIG_PATH}"
27+
28+
while getopts ":l:t:r:p:e:" opt; do
29+
case $opt in
30+
l) LOCALE="$OPTARG"
31+
;;
32+
t) TYPE_FILE="$OPTARG"
33+
;;
34+
r) RENDER_FOLDER="$OPTARG"
35+
;;
36+
p) PANDOC_PARAMETERS="$OPTARG"
37+
;;
38+
e) EXPORT_FOLDER="$OPTARG"
39+
;;
40+
\?) echo "Invalid option -$OPTARG" >&2
41+
exit 1
42+
;;
43+
esac
44+
45+
case $OPTARG in
46+
-*) echo "Option $opt needs a valid argument"
47+
exit 1
48+
;;
49+
esac
50+
done
51+
52+
## Folders used
53+
LOCALE_FOLDER=${LOCALE}
54+
CURRENT_FOLDER="${RENDER_FOLDER}/${SITE_FOLDER}/${LOCALE_FOLDER}"
55+
CURRENT_EXPORT_FOLDER="${EXPORT_FOLDER}/${SITE_FOLDER}/${LOCALE_FOLDER}"
56+
IMAGE_FOLDER="${RENDER_FOLDER}/${IMG_FOLDER}"
57+
58+
## Data and index files
59+
INDEX_FILE="${CURRENT_FOLDER}/${SITE_INDEX}"
60+
61+
## Step 1: check the type of file to work with, exit if not in the list
62+
typeExists=$(elementInWhere "${TYPE_FILE}" "${EXPORT_TYPES[@]}")
63+
[[ $? -ne 0 ]] && { echo "$TYPE_FILE file format not supported"; exit 99; }
64+
65+
## Step 1.1: Copy the render folder to a temporary one to ease the work
66+
## But first, create the folder if it doesn't exist
67+
[ ! -d "${TEMP_FOLDER}" ] && mkdir ${TEMP_FOLDER}
68+
[ ! -d "${TEMP_FOLDER}/${SITE_FOLDER}" ] && mkdir ${TEMP_FOLDER}/${SITE_FOLDER}
69+
cp -R ${CURRENT_FOLDER} ${TEMP_FOLDER}/${SITE_FOLDER}
70+
cp -R ${IMAGE_FOLDER} ${TEMP_FOLDER}
71+
72+
## Since you are at it, create the export folder and add the images there
73+
[ ! -d "${EXPORT_FOLDER}" ] && mkdir ${EXPORT_FOLDER}
74+
[ ! -d "${EXPORT_FOLDER}/${SITE_FOLDER}" ] && mkdir ${EXPORT_FOLDER}/${SITE_FOLDER}
75+
[ ! -d "${CURRENT_EXPORT_FOLDER}" ] && mkdir ${CURRENT_EXPORT_FOLDER}
76+
cp -R ${IMAGE_FOLDER} ${EXPORT_FOLDER}
77+
78+
## Step 2: if HTML, search for all Markdown files and convert them one by one
79+
if [[ ${TYPE_FILE} == "HTML" ]]; then
80+
## List all of the Markdown files into an array
81+
markdownFiles=$(find "${TEMP_FOLDER}" | grep "\.md$")
82+
readarray -t MARKDOWN_FILES <<<"$markdownFiles"
83+
84+
## Iterate through the array
85+
for fileIndex in "${!MARKDOWN_FILES[@]}"
86+
do :
87+
echo -e "** Converting: ${MARKDOWN_FILES[$fileIndex]}"
88+
89+
## Extract file name without extension
90+
filename=$(basename -- "${MARKDOWN_FILES[$fileIndex]}")
91+
filename="${filename%.*}"
92+
## extension="${filename##*.}"
93+
##echo -e "** Filename: ${filename}"
94+
95+
## Extract the path to the file
96+
fileFolder=$(dirname "${MARKDOWN_FILES[$fileIndex]}")
97+
98+
## Remove the first folder block
99+
fileFolder=${fileFolder#$TEMP_FOLDER}
100+
101+
## Add the export folder name
102+
fileFolder=$EXPORT_FOLDER/$fileFolder
103+
104+
## Fix the index block to link to HTML files
105+
## TODO: this will not allow me to include links to ANY Markdown Files
106+
## It could be a good idea to rethink this in the future
107+
##if [[ "${filename}.md" == "${SITE_INDEX}" ]]; then
108+
sed -i "s/.md)/.html)/g" "${MARKDOWN_FILES[$fileIndex]}"
109+
##fi
110+
111+
## Create the page folder, this trick allows two levels of depth
112+
## such as the created by the generator, having more depth would
113+
## require a different subfolder generator capable of detecting how
114+
## many subfolders to create along the way
115+
[ ! -d "${fileFolder}" ] && mkdir ${fileFolder}
116+
##[ ! -d "${CURRENT_EXPORT_FOLDER}/${filename}" ] && mkdir ${CURRENT_EXPORT_FOLDER}/${filename}
117+
118+
pandoc -s ${MARKDOWN_FILES[$fileIndex]} --metadata pagetitle="${filename}" -o ${fileFolder}/${filename}.html
119+
##pandoc ${MARKDOWN_FILES[$fileIndex]} -o ${CURRENT_EXPORT_FOLDER}/${filename}/${filename}.html
120+
done
121+
fi
122+
123+
## Step 3: if PDF, remove the indexing part of all files and compose them together
124+
if [[ ${TYPE_FILE} == "PDF" ]]; then
125+
## List all of the Markdown files into an array
126+
markdownFiles=$(find "${TEMP_FOLDER}" | grep "\.md$")
127+
readarray -t MARKDOWN_FILES <<<"$markdownFiles"
128+
129+
## Iterate through the array
130+
for fileIndex in "${!MARKDOWN_FILES[@]}"
131+
do :
132+
## Work only if not dealing with the index file, which should be
133+
## left out of the PDF generation
134+
135+
## Extract file name without extension
136+
filename=$(basename -- "${MARKDOWN_FILES[$fileIndex]}")
137+
filename="${filename%.*}"
138+
139+
if [[ "${filename}.md" != "${SITE_INDEX}" ]]; then
140+
echo -e "** Adding: ${MARKDOWN_FILES[$fileIndex]}"
141+
142+
## Append the file at the bottom of the temp file
143+
cat "${MARKDOWN_FILES[$fileIndex]}" >> "${TEMP_FOLDER}/${TEMP_EXPORT_FILE}"
144+
fi
145+
done
146+
147+
## Remove all of the index blocks
148+
## TODO: fix this to work with other locales by using information from the
149+
## template.csv file instead of having it hardcoded in here
150+
##sed -z -i 's/## Index\(.*\)##/##/g' "${TEMP_FOLDER}/${TEMP_EXPORT_FILE}"
151+
sed -i '/## Index/,/##/{//!d}' "${TEMP_FOLDER}/${TEMP_EXPORT_FILE}"
152+
sed -z -i 's/## Index//g' "${TEMP_FOLDER}/${TEMP_EXPORT_FILE}"
153+
154+
## TODO: deal with the video blocks by searching for a screenshot
155+
156+
pandoc "${TEMP_FOLDER}/${TEMP_EXPORT_FILE}" --pdf-engine=xelatex -o "${EXPORT_FOLDER}/${PDF_EXPORT_FILE}"
157+
158+
## Open the file for inspection
159+
xdg-open "${EXPORT_FOLDER}/${PDF_EXPORT_FILE}"
160+
fi
161+
162+
## XXX
163+
164+
## We finished, delete the temporary folder
165+
[ -d "${TEMP_FOLDER}" ] && rm -fR ${TEMP_FOLDER}
166+
167+
## Done
168+
echo -e "** Done"

‎build/render_site.sh

-2
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,6 @@ while getopts ":l:r:c:f:" opt; do
4848
esac
4949
done
5050

51-
52-
5351
## Folders used
5452
LOCALE_FOLDER=${LOCALE}
5553
CURRENT_FOLDER="${SITE_FOLDER}/${LOCALE_FOLDER}"

‎docs/BUILD.md

+26-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Build the project
22

3+
## Install dependencies
4+
5+
If you are willing to export your sites to anything that is not Markdown, you will need to use the *pandoc* universal translation tool. It will allow converting the generator's outputs into HTML, PDF, LaTeX, amongst others.
6+
7+
`sudo apt install pandoc`
8+
39
## Build the templates
410

511
Prior to creating a course, you need to define the templates you will be using. The call for the template generation goes as follows:
@@ -47,18 +53,34 @@ You need to add your content to the *en/pages.csv* file. Make sure you include t
4753

4854
Once the templates have been created, render the markdown of the site by simply calling:
4955

50-
`./render_site.sh en .. config pages.csv`
56+
`./render_site.sh -l en -r .. -c config -f pages.csv`
5157

5258
inside the *build* folder. It will create all of the folders, *MD*, and code files based on templates. From there you will have the opportunity of modifying the content once more. Use the editor of your choice.
5359

5460
The parameters for the *render_site.sh* script are:
5561

56-
* locale: en, es, etc.
57-
* name of the folder with templates and the like, typically *config*
58-
* name of the *CSV* file containing the course
62+
* l, locale: en, es, etc.
63+
* r, render: folder where the site will be rendered into
64+
* c, config: name of the folder with templates and the like, typically *config*
65+
* f, file: name of the *CSV* file containing the course
5966

6067
You will have to call it once per locale, which also means you should need the actual *CSV* file for the corresponding language.
6168

69+
## Export your site to other formats
70+
71+
The way information is built in the form of Markdown files requires some considerations depending on which kind of outputs you might expect from the generator. For example, if you are looking at having an HTML site, you will have to convert each one of the *\*.md* files separately, index included. On the other hand, if you are looking at making a printing-ready PDF, you might have to download all of the images, compose everything into a single file, and then call the *pandoc* script with the options to generate a *TOC* (table of contents).
72+
73+
The export script is wrapping the call to *pandoc* after making some preparation work with the file. Call it as follows:
74+
75+
`./export_site.sh -t HTML -r ..`
76+
77+
The parameter for the *export_site.sh* script are:
78+
79+
* l, locale: en, es, etc.
80+
* t, type of output: HTML, PDF
81+
* r, render: folder where the site was rendered into
82+
* p, pandoc parameters: a string containing the non-default settings for *pandoc*
83+
6284
## Note
6385

6486
* The *CSV* separator in use is the **¤** symbol in order to have commas within the text

‎docs/LOGS.md

+31
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,37 @@ Read here the full design logs, day by day, for the Bash Site Creator from the d
66

77
See what happened in the second month of the year.
88

9+
### 20220227: added PDF export to export_site
10+
11+
* included the *PDF* export feature
12+
* TODO: include offline images for both *HTML* and *PDF* exports
13+
* TODO: include screencap of video file for PDFs + clickable URL
14+
15+
### 20220226: added index correction to export_site in HTML
16+
17+
* *export_site.sh* is now looking into all of the files and linking to *HTML* files instead of Markdown ones, this removes the possibility of having files with the extension *\*.md*
18+
* TODO: create a nice CSS for this ... maybe extract the one from Github pages
19+
20+
### 20220225: scripting pandoc
21+
22+
* *pandoc* is now officially a dependency to this project, therefore I have updated *BUILD.md* to include dependencies
23+
* created *export_site.sh*, a script to wrap the operations with *pandoc* after making all the needed preparations to the Markdown files
24+
* worked out the basic operations for exporting as *HTML*
25+
26+
### 20220224: testing pandoc
27+
28+
* *pandoc* has some serious superpowers, it can use your own CSS, it can generate the TOC in a PDF, etc
29+
* this also means that, after some testing, I will need to produce a tool which will be searching for materials and produce a preliminary file that could be sent to *pandoc*, e.g. I cannot put a video in a PDF, I will have to extract a screenshot of the video and add the clickable URL as a caption. Yet another example, I might have to locally download remote images with *Curl* prior to producing a compressed file in HTML with all of the assets for offline distribution of the content (which is one of my goals)
30+
* PDF files should be created from a single massive Markdown file, where contents should be sorted for *pandoc* to create the TOC
31+
* rendering HTML will require making my own tool looking for all *\*.md* files in a folder structure to be rendered as HTML
32+
33+
### 20220223: research on exporting
34+
35+
* I want mainly two different kinds of exports: HTML (in order to make the system independent from Github when publishing), and PDF (because a lot of people use it to distribute materials to their students). I came to just two possible solutions: *cmark* and *pandoc*
36+
* [*cmark*](https://github.com/commonmark/cmark) is a C tool, thus the fastest one currently existing capable of converting Markdown to HTML, it is considered the standard tool to benchmark your own against. The issue is that it does not produce PDFs
37+
* [*pandoc*](https://pandoc.org/) is the Swiss knife of format conversions from CLI, does not only HTML and PDF, but also LaTeX, AsciiDoc, Docx, etc.
38+
* I think I have to sacrifice performance for versatility, mainly because I will otherwise have to produce my own PDF export tool. The disavantage is obvious: I will need Linux to run my tool (which I do anyway, since I made things in *bash* ¯\\_(ツ)\_/¯)
39+
940
### 20220222 (or 22022022): pre-alpha Twosday release
1041

1142
* this is the very first release, pre-alpha 0.0.1 of the **Bash Site Generator**, it has been two months of work to make it to this point with small developments day after day

0 commit comments

Comments
 (0)