Skip to content

marshal: don't escape quotes unnecessarily#991

Merged
pelletier merged 2 commits intopelletier:v2from
virtuald:fewer-multiline-escapes
Aug 21, 2025
Merged

marshal: don't escape quotes unnecessarily#991
pelletier merged 2 commits intopelletier:v2from
virtuald:fewer-multiline-escapes

Conversation

@virtuald
Copy link
Copy Markdown
Contributor

Only 3 consecutive quotation marks need to be quoted. We choose to quote all quotation marks in a sequence if there are 3 or more consecutive present.

Fixes #990


Strictly speaking, this would break backwards compatibility if anyone was using a textual comparison of the output of Marshal (as opposed to the semantic content). Could hide it behind an additional tag/option if that seems like a concern.

I've not used benchstat before, but I think there's a slight performance penalty, but worth it for me :)


Results of ./ci.sh benchmark -d v2:

goos: darwin
goarch: arm64
pkg: github.com/pelletier/go-toml/v2/benchmark
cpu: Apple M3 Max
                                  │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/v2.RfSXGXt4GH │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/HEAD.Cu81ZqvZ38 │
                                  │                             sec/op                             │                  sec/op                    vs base               │
UnmarshalDataset/config-2                                                              9.800m ± 1%                                 9.979m ± 0%  +1.83% (p=0.002 n=10)
UnmarshalDataset/canada-2                                                              38.80m ± 3%                                 38.32m ± 1%       ~ (p=0.123 n=10)
UnmarshalDataset/citm_catalog-2                                                        12.00m ± 2%                                 11.51m ± 1%  -4.09% (p=0.000 n=10)
UnmarshalDataset/twitter-2                                                             4.718m ± 4%                                 4.651m ± 0%  -1.43% (p=0.000 n=10)
UnmarshalDataset/code-2                                                                46.55m ± 1%                                 45.94m ± 1%  -1.32% (p=0.000 n=10)
UnmarshalDataset/example-2                                                             86.56µ ± 2%                                 86.24µ ± 0%  -0.37% (p=0.015 n=10)
Unmarshal/SimpleDocument/struct-2                                                      273.6n ± 2%                                 274.1n ± 0%       ~ (p=0.447 n=10)
Unmarshal/SimpleDocument/map-2                                                         385.2n ± 3%                                 383.6n ± 0%       ~ (p=0.315 n=10)
Unmarshal/ReferenceFile/struct-2                                                       24.67µ ± 2%                                 24.62µ ± 0%       ~ (p=0.105 n=10)
Unmarshal/ReferenceFile/map-2                                                          35.21µ ± 2%                                 35.18µ ± 0%       ~ (p=0.469 n=10)
Unmarshal/HugoFrontMatter-2                                                            5.932µ ± 1%                                 5.929µ ± 0%       ~ (p=0.644 n=10)
Marshal/SimpleDocument/struct-2                                                        203.7n ± 1%                                 205.0n ± 1%  +0.64% (p=0.011 n=10)
Marshal/SimpleDocument/map-2                                                           259.0n ± 0%                                 259.6n ± 0%       ~ (p=0.078 n=10)
Marshal/ReferenceFile/struct-2                                                         17.76µ ± 2%                                 17.66µ ± 0%  -0.52% (p=0.035 n=10)
Marshal/ReferenceFile/map-2                                                            21.55µ ± 0%                                 21.53µ ± 0%       ~ (p=0.631 n=10)
Marshal/HugoFrontMatter-2                                                              4.117µ ± 0%                                 4.112µ ± 0%       ~ (p=0.643 n=10)
geomean                                                                                53.03µ                                      52.79µ       -0.45%

                                  │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/v2.RfSXGXt4GH │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/HEAD.Cu81ZqvZ38 │
                                  │                              B/s                               │                    B/s                     vs base               │
UnmarshalDataset/config-2                                                             102.1Mi ± 1%                                100.2Mi ± 0%  -1.80% (p=0.002 n=10)
UnmarshalDataset/canada-2                                                             54.10Mi ± 3%                                54.79Mi ± 1%       ~ (p=0.110 n=10)
UnmarshalDataset/citm_catalog-2                                                       44.36Mi ± 2%                                46.25Mi ± 1%  +4.27% (p=0.000 n=10)
UnmarshalDataset/twitter-2                                                            89.32Mi ± 4%                                90.61Mi ± 0%  +1.45% (p=0.000 n=10)
UnmarshalDataset/code-2                                                               54.99Mi ± 1%                                55.72Mi ± 1%  +1.33% (p=0.000 n=10)
UnmarshalDataset/example-2                                                            89.24Mi ± 2%                                89.57Mi ± 0%  +0.37% (p=0.015 n=10)
Unmarshal/SimpleDocument/struct-2                                                     38.35Mi ± 2%                                38.29Mi ± 0%       ~ (p=0.470 n=10)
Unmarshal/SimpleDocument/map-2                                                        27.24Mi ± 3%                                27.35Mi ± 0%       ~ (p=0.305 n=10)
Unmarshal/ReferenceFile/struct-2                                                      202.6Mi ± 2%                                203.1Mi ± 0%       ~ (p=0.093 n=10)
Unmarshal/ReferenceFile/map-2                                                         142.0Mi ± 2%                                142.1Mi ± 0%       ~ (p=0.469 n=10)
Unmarshal/HugoFrontMatter-2                                                           87.79Mi ± 1%                                87.82Mi ± 0%       ~ (p=0.616 n=10)
Marshal/SimpleDocument/struct-2                                                       56.17Mi ± 1%                                55.81Mi ± 1%  -0.64% (p=0.011 n=10)
Marshal/SimpleDocument/map-2                                                          44.18Mi ± 0%                                44.08Mi ± 0%       ~ (p=0.085 n=10)
Marshal/ReferenceFile/struct-2                                                        110.5Mi ± 2%                                111.1Mi ± 0%  +0.53% (p=0.035 n=10)
Marshal/ReferenceFile/map-2                                                           88.77Mi ± 0%                                88.88Mi ± 0%       ~ (p=0.617 n=10)
Marshal/HugoFrontMatter-2                                                             121.2Mi ± 0%                                121.3Mi ± 0%       ~ (p=0.616 n=10)
geomean                                                                               74.15Mi                                     74.49Mi       +0.45%

                                  │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/v2.RfSXGXt4GH │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/HEAD.Cu81ZqvZ38 │
                                  │                              B/op                              │                  B/op                    vs base                 │
UnmarshalDataset/config-2                                                             5.452Mi ± 0%                              5.452Mi ± 0%       ~ (p=0.645 n=10)
UnmarshalDataset/canada-2                                                             76.24Mi ± 0%                              76.24Mi ± 0%       ~ (p=0.303 n=10)
UnmarshalDataset/citm_catalog-2                                                       32.87Mi ± 0%                              32.87Mi ± 0%       ~ (p=0.324 n=10)
UnmarshalDataset/twitter-2                                                            11.95Mi ± 0%                              11.95Mi ± 0%       ~ (p=0.883 n=10)
UnmarshalDataset/code-2                                                               20.28Mi ± 0%                              20.28Mi ± 0%       ~ (p=0.987 n=10)
UnmarshalDataset/example-2                                                            180.7Ki ± 0%                              180.7Ki ± 0%       ~ (p=0.204 n=10)
Unmarshal/SimpleDocument/struct-2                                                       805.0 ± 0%                                805.0 ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/SimpleDocument/map-2                                                        1.106Ki ± 0%                              1.106Ki ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/struct-2                                                      15.49Ki ± 0%                              15.49Ki ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/map-2                                                         32.39Ki ± 0%                              32.39Ki ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/HugoFrontMatter-2                                                           7.429Ki ± 0%                              7.429Ki ± 0%       ~ (p=1.000 n=10)
Marshal/SimpleDocument/struct-2                                                         232.0 ± 0%                                232.0 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/SimpleDocument/map-2                                                            248.0 ± 0%                                248.0 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/struct-2                                                        27.05Ki ± 0%                              27.05Ki ± 0%       ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/map-2                                                           23.98Ki ± 0%                              23.98Ki ± 0%       ~ (p=1.000 n=10) ¹
Marshal/HugoFrontMatter-2                                                             5.266Ki ± 0%                              5.266Ki ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                                               70.92Ki                                   70.92Ki       -0.00%
¹ all samples are equal

                                  │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/v2.RfSXGXt4GH │ /var/folders/4x/hhbzwffd2d5c3n6z2b09b91c000_0t/T/HEAD.Cu81ZqvZ38 │
                                  │                           allocs/op                            │                allocs/op                 vs base                 │
UnmarshalDataset/config-2                                                              219.2k ± 0%                               219.2k ± 0%       ~ (p=1.000 n=10) ¹
UnmarshalDataset/canada-2                                                              670.4k ± 0%                               670.4k ± 0%       ~ (p=1.000 n=10) ¹
UnmarshalDataset/citm_catalog-2                                                        178.0k ± 0%                               178.0k ± 0%       ~ (p=1.000 n=10)
UnmarshalDataset/twitter-2                                                             55.65k ± 0%                               55.65k ± 0%       ~ (p=1.000 n=10) ¹
UnmarshalDataset/code-2                                                                1.001M ± 0%                               1.001M ± 0%       ~ (p=1.000 n=10) ¹
UnmarshalDataset/example-2                                                             1.326k ± 0%                               1.326k ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/SimpleDocument/struct-2                                                       8.000 ± 0%                                8.000 ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/SimpleDocument/map-2                                                          12.00 ± 0%                                12.00 ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/struct-2                                                        158.0 ± 0%                                158.0 ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/map-2                                                           619.0 ± 0%                                619.0 ± 0%       ~ (p=1.000 n=10) ¹
Unmarshal/HugoFrontMatter-2                                                             142.0 ± 0%                                142.0 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/SimpleDocument/struct-2                                                         7.000 ± 0%                                7.000 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/SimpleDocument/map-2                                                            8.000 ± 0%                                8.000 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/struct-2                                                          226.0 ± 0%                                226.0 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/map-2                                                             344.0 ± 0%                                344.0 ± 0%       ~ (p=1.000 n=10) ¹
Marshal/HugoFrontMatter-2                                                               73.00 ± 0%                                73.00 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                                                 987.5                                     987.5       +0.00%
¹ all samples are equal

virtuald and others added 2 commits June 23, 2025 01:01
Only 3 consecutive quotation marks need to be quoted. We choose to quote
all quotation marks in a sequence if there are 3 or more consecutive
present.

Fixes pelletier#990
@pelletier pelletier added the feature Issue asking for a new feature in go-toml. label Aug 21, 2025
@pelletier pelletier merged commit 6d56ac8 into pelletier:v2 Aug 21, 2025
11 checks passed
@pelletier
Copy link
Copy Markdown
Owner

Thanks! Sorry for the delay.

Maks1mS pushed a commit to stplr-dev/stplr that referenced this pull request Mar 25, 2026
…374)

This PR contains the following updates:

| Package | Type | Update | Change | OpenSSF |
|---|---|---|---|---|
| [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) | require | minor | `v2.2.4` → `v2.3.0` | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/pelletier/go-toml/badge)](https://securityscorecards.dev/viewer/?uri=github.com/pelletier/go-toml) |

---

> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/23) for more information.

---

### Release Notes

<details>
<summary>pelletier/go-toml (github.com/pelletier/go-toml/v2)</summary>

### [`v2.3.0`](https://github.com/pelletier/go-toml/releases/tag/v2.3.0)

[Compare Source](pelletier/go-toml@v2.2.4...v2.3.0)

This is the first release built largely with the help of AI coding agents. Highlights include the complete removal of the unsafe package. go-toml is now fully safe Go code, with a geomean overhead of only \~1.4% vs v2.2.4 and zero additional allocations on benchmarks. This release also adds omitzero struct tag support, improves UnmarshalText/Unmarshaler handling for tables and array tables, and fixes several bugs including nil pointer marshaling, leap second handling, and datetime unmarshaling panics.

<!-- Release notes generated using configuration in .github/release.yml at v2.3.0 -->

#### What's Changed

##### What's new

- marshal: don't escape quotes unnecessarily by [@&#8203;virtuald](https://github.com/virtuald) in [#&#8203;991](pelletier/go-toml#991)
- Add `omitzero` tag support by [@&#8203;NathanBaulch](https://github.com/NathanBaulch) in [#&#8203;998](pelletier/go-toml#998)
- Support custom IsZero() methods with omitzero tag by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1020](pelletier/go-toml#1020)
- UnmarshalText fallbacks to struct unmarshaling for tables and arrays by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1026](pelletier/go-toml#1026)
- \[unstable] Support Unmarshaler interface for tables and array tables by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1027](pelletier/go-toml#1027)

##### Fixed bugs

- Add missing UnmarshalTOML call by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;996](pelletier/go-toml#996)
- Handle array table into an empty slice by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;997](pelletier/go-toml#997)
- Unwrap strict errors by [@&#8203;bersace](https://github.com/bersace) in [#&#8203;1012](pelletier/go-toml#1012)
- Fix leap second handling found by fuzz by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1019](pelletier/go-toml#1019)
- Fix nil pointer map values not being marshaled by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1025](pelletier/go-toml#1025)
- Fix panic when unmarshaling datetime values to incompatible types ([#&#8203;1028](pelletier/go-toml#1028)) by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1029](pelletier/go-toml#1029)
- Fix parser error pointing to wrong line at EOF without trailing newline by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1041](pelletier/go-toml#1041)

##### Documentation

- Improve Unmarshaling README by [@&#8203;heckelson](https://github.com/heckelson) in [#&#8203;1016](pelletier/go-toml#1016)
- Create AGENTS.md guidelines file by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1017](pelletier/go-toml#1017)

##### Other changes

- Unsafe package removal by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1021](pelletier/go-toml#1021)
- Bump CI and test scripts to Go 1.26 by [@&#8203;pelletier](https://github.com/pelletier) in [#&#8203;1030](pelletier/go-toml#1030)

#### New Contributors

- [@&#8203;virtuald](https://github.com/virtuald) made their first contribution in [#&#8203;991](pelletier/go-toml#991)
- [@&#8203;NathanBaulch](https://github.com/NathanBaulch) made their first contribution in [#&#8203;999](pelletier/go-toml#999)
- [@&#8203;bersace](https://github.com/bersace) made their first contribution in [#&#8203;1012](pelletier/go-toml#1012)
- [@&#8203;flyn-org](https://github.com/flyn-org) made their first contribution in [#&#8203;1013](pelletier/go-toml#1013)
- [@&#8203;heckelson](https://github.com/heckelson) made their first contribution in [#&#8203;1016](pelletier/go-toml#1016)

**Full Changelog**: <pelletier/go-toml@v2.2.4...v2.3.0>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At 12:00 AM through 04:59 AM and 10:00 PM through 11:59 PM, Monday through Friday ( * 0-4,22-23 * * 1-5 ), Only on Sunday and Saturday ( * * * * 0,6 ) (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My44Ni4xIiwidXBkYXRlZEluVmVyIjoiNDMuODYuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiS2luZC9EZXBlbmRlbmNpZXMiXX0=-->

Reviewed-on: https://altlinux.space/stapler/stplr/pulls/374
Co-authored-by: Renovate Bot <stapler-helper-bot@noreply.altlinux.space>
Co-committed-by: Renovate Bot <stapler-helper-bot@noreply.altlinux.space>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Issue asking for a new feature in go-toml.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multiline string marshalling creates unneccesary escapes

2 participants