Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix scanner - ignore trailing tabs #711

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

eihli
Copy link

@eihli eihli commented Apr 25, 2023

Trying to address this TODO:

TODO: We need to make tab handling rules more sane. A good rule is
Tabs cannot precede tokens
BLOCK-SEQUENCE-START, BLOCK-MAPPING-START, BLOCK-END,
KEY(block), VALUE(block), BLOCK-ENTRY

I don't quite understand the "Tabs cannot precede tokens" statement though. My understanding of the YAML spec is that tabs can be treated the exact same as spaces - before/after token doesn't matter.

But the change I made here took to heart the "Tabs cannot precede tokens" statement and only ignores tabs when they directly precede other tabs/spaces followed by a newline.

Before:

(ghsync) ➜  pyyaml git:(fix-tabs) ✗ python setup.py test
running test
# ...
  File "/var/folders/p0/9kyh1lx53tb5_9tsft21ffrc0000gn/T/test_pyyaml89f6sqhb/pyyaml/yaml/scanner.py", line 258, in fetch_more_tokens
    raise ScannerError("while scanning for the next token", None,
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
  in "tests/data/trailing-tab.data", line 2, column 8
---------------------------------------------------------------------------
tests/data/trailing-tab.data:
foo:
  - bar
---------------------------------------------------------------------------
tests/data/trailing-tab.code:
{"foo": ["bar"]}
===========================================================================
TESTS: 1285
ERRORS: 1
error: Tests failed

@eihli
Copy link
Author

eihli commented Jul 12, 2023

Anyone know of who I can ping to get traction on this? I just ran into an issue again today.

@eihli
Copy link
Author

eihli commented Aug 15, 2023

Thanks. I'll sign this commit tonight.

@eihli
Copy link
Author

eihli commented Aug 29, 2023

@bitcoinfinancier Commit signed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant