Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create pull request/add yaml to load data #1842

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ also specify classes on headers now
- Make `ignored_content` work with nested paths and directories
- `zola serve/build` can now run from anywhere in a zola directory
- Add XML support to `load_data`
- Add YAML support to `load_data`
- `skip_prefixes` is now checked before parsing external link URLs
- Add `render` attribute to taxonomies configuration in `config.toml`, for when you don't want to render
any pages related to that taxonomy
Expand Down
34 changes: 32 additions & 2 deletions components/templates/src/global_fns/load_data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use libs::tera::{
from_value, to_value, Error, Error as TeraError, Function as TeraFn, Map, Result, Value,
};
use libs::url::Url;
use libs::{nom_bibtex, serde_json, toml};
use libs::{nom_bibtex, serde_json, serde_yaml, toml};
use utils::de::fix_toml_dates;
use utils::fs::{get_file_time, read_file};

Expand Down Expand Up @@ -47,6 +47,7 @@ enum OutputFormat {
Bibtex,
Plain,
Xml,
Yaml,
}

impl FromStr for OutputFormat {
Expand All @@ -60,6 +61,7 @@ impl FromStr for OutputFormat {
"bibtex" => Ok(OutputFormat::Bibtex),
"xml" => Ok(OutputFormat::Xml),
"plain" => Ok(OutputFormat::Plain),
"yaml" => Ok(OutputFormat::Yaml),
format => Err(format!("Unknown output format {}", format).into()),
}
}
Expand All @@ -74,6 +76,7 @@ impl OutputFormat {
OutputFormat::Bibtex => "application/x-bibtex",
OutputFormat::Xml => "text/xml",
OutputFormat::Plain => "text/plain",
OutputFormat::Yaml => "application/x-yaml",
})
}
}
Expand Down Expand Up @@ -208,7 +211,7 @@ fn add_headers_from_args(header_args: Option<Vec<String>>) -> Result<HeaderMap>
}

/// A Tera function to load data from a file or from a URL
/// Currently the supported formats are json, toml, csv, bibtex and plain text
/// Currently the supported formats are json, toml, csv, yaml, bibtex and plain text
#[derive(Debug)]
pub struct LoadData {
base_path: PathBuf,
Expand Down Expand Up @@ -388,6 +391,7 @@ impl TeraFn for LoadData {
OutputFormat::Json => load_json(data),
OutputFormat::Bibtex => load_bibtex(data),
OutputFormat::Xml => load_xml(data),
OutputFormat::Yaml => load_yaml(data),
OutputFormat::Plain => to_value(data).map_err(|e| e.into()),
};

Expand All @@ -406,6 +410,13 @@ fn load_json(json_data: String) -> Result<Value> {
Ok(json_content)
}

/// Parse a YAML string and convert it to a Tera Value
fn load_yaml(yaml_data: String) -> Result<Value> {
let yaml_content: Value =
serde_yaml::from_str(yaml_data.as_str()).map_err(|e| format!("{:?}", e))?;
Ok(yaml_content)
}

/// Parse a TOML string and convert it to a Tera Value
fn load_toml(toml_data: String) -> Result<Value> {
let toml_content: toml::Value = toml::from_str(&toml_data).map_err(|e| format!("{:?}", e))?;
Expand Down Expand Up @@ -1084,6 +1095,25 @@ mod tests {
)
}

#[test]
fn can_load_yaml() {
let static_fn = LoadData::new(PathBuf::from("../utils/test-files"), None, PathBuf::new());
let mut args = HashMap::new();
args.insert("path".to_string(), to_value("test.yaml").unwrap());
let result = static_fn.call(&args.clone()).unwrap();

assert_eq!(
result,
json!({
"key": "value",
"array": [1, 2, 3],
"subpackage": {
"subkey": 5
}
})
)
}

#[test]
fn is_load_remote_data_using_post_method_with_different_body_not_cached() {
let _mjson = mock("POST", "/kr1zdgbm4y3")
Expand Down
9 changes: 9 additions & 0 deletions components/utils/test-files/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
key: "value"
array:
- 1
- 2
- 3
subpackage:
subkey: 5

9 changes: 6 additions & 3 deletions docs/content/documentation/templates/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,8 @@ The method returns a map containing `width`, `height` and `format` (the lowercas

### `load_data`

Loads data from a file, URL, or string literal. Supported file types include *toml*, *json*, *csv*, *bibtex* and *xml* and only supports UTF-8 encoding.
Loads data from a file, URL, or string literal. Supported file types include *toml*, *json*, *csv*, *bibtex*, *yaml*
and *xml* and only supports UTF-8 encoding.

Any other file type will be loaded as plain text.

Expand Down Expand Up @@ -293,7 +294,9 @@ The snippet below outputs the HTML from a Wikipedia page, or "No data found" if
{% if data %}{{ data | safe }}{% else %}No data found{% endif %}
```

The optional `format` argument allows you to specify and override which data type is contained within the specified file or URL. Valid entries are `toml`, `json`, `csv`, `bibtex`, `xml` or `plain`. If the `format` argument isn't specified, then the path extension is used. In the case of a literal, `plain` is assumed if `format` is unspecified.
The optional `format` argument allows you to specify and override which data type is contained within the specified file or URL.
Valid entries are `toml`, `json`, `csv`, `bibtex`, `yaml`, `xml` or `plain`. If the `format` argument isn't specified, then the
path extension is used. In the case of a literal, `plain` is assumed if `format` is unspecified.


```jinja2
Expand All @@ -302,7 +305,7 @@ The optional `format` argument allows you to specify and override which data typ

Use the `plain` format for when your file has a supported extension but you want to load it as plain text.

For *toml*, *json* and *xml*, the data is loaded into a structure matching the original data file;
For *toml*, *json*, *yaml* and *xml*, the data is loaded into a structure matching the original data file;
however, for *csv* there is no native notion of such a structure. Instead, the data is separated
into a data structure containing *headers* and *records*. See the example below to see
how this works.
Expand Down