Skip to content

new parser #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Nov 9, 2022
Merged

new parser #42

merged 32 commits into from
Nov 9, 2022

Conversation

nicolasparada
Copy link
Contributor

@nicolasparada nicolasparada commented Oct 20, 2022

Replacing the entire config parser with new code. This version does not use the participle library.

Instead, for the classic mode, it splits by line, and detects commands (@ prefix) and sections ([NAMED]). Then each property nested in a section is split by space. Simple.

It supports basic data types like string, integer, float and boolean. It does not try to parse regexes, dates, units, arrays, etc. Complex data types are left as string.

For the YAML parser, we use the yaml package with a custom unmarshaler to add support for sorted properties, so we don't loose track of the order of each property inside a plugin. This is mostly for testing so we can expect always the same order.

  • classic config parser
  • yaml config parser
  • tests between them both
  • support marshaling of multiline strings (|-)
  • semantic schema validation
  • json custom unmarshaler to preserve properties order too

@gitguardian
Copy link

gitguardian bot commented Oct 20, 2022

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id Secret Commit Filename
3751481 Generic High Entropy Secret e3f2eb0 tests/ini-to-json/11-s3-real.json View secret
3751481 Generic High Entropy Secret e3f2eb0 tests/ini-to-json/11-s3-real.yaml View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!

@nicolasparada nicolasparada changed the title [WIP] v2 new parser Nov 7, 2022
@nicolasparada
Copy link
Contributor Author

@niedbalski this is still a WIP but can you take a look?

Copy link
Contributor

@niedbalski niedbalski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work.

Looks generally good. I think the parsing schema for json might be a bit fragile.

  1. Did you remove the schema validator entirely from this parser?
  2. Any chance you can re-instate the original tests? I am particularly interested
    into booleans, multi-field, urls (with schema + port + uri) and the multiple-files ones.

@nicolasparada
Copy link
Contributor Author

  1. I'm bringing the validator back.
  2. I have already a good suite of tests, but sure I can add those :)

@nicolasparada nicolasparada marked this pull request as ready for review November 9, 2022 18:38
@niedbalski niedbalski merged commit eb2e002 into main Nov 9, 2022
@nicolasparada nicolasparada deleted the v2 branch November 24, 2022 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants