Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add trailing whitespace test #2365

Merged
merged 2 commits into from
Jun 19, 2019
Merged

Conversation

wchargin
Copy link
Contributor

Summary:
Prevents regressions of nice fixes like #2364.

Test Plan:
Run the script, and note that it passes with no output. Then remove
README.md from the exceptions list and add nonexistent.txt, and
observe that the script fails with:

Superfluous trailing whitespace:
README.md:10:For in-depth information on the Graph Visualizer, see this tutorial: $
README.md:18:through setting up and using TensorBoard. There's an associated $
README.md:36:For more details, see $
README.md:69:For this, you need $

Stale exceptions (no whitespace problems; prune exceptions list):
nonexistent.txt

wchargin-branch: whitespace-hygiene-test

Summary:
Prevents regressions of nice fixes like #2364.

Test Plan:
Run the script, and note that it passes with no output. Then remove
`README.md` from the exceptions list and add `nonexistent.txt`, and
observe that the script fails with:

```
Superfluous trailing whitespace:
README.md:10:For in-depth information on the Graph Visualizer, see this tutorial: $
README.md:18:through setting up and using TensorBoard. There's an associated $
README.md:36:For more details, see $
README.md:69:For this, you need $

Stale exceptions (no whitespace problems; prune exceptions list):
nonexistent.txt
```

wchargin-branch: whitespace-hygiene-test
sys.exit(1)
result = []
while stdout:
(filename_raw, _unused_nul, rest) = stdout.partition(b"\0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since realistically we're not going to have newlines in filenames committed to our git repo, it seems simpler and more obvious to just do for line in stdout.splitlines(): filename_raw, line_number_raw, line_number = line.split(b"\0", 2).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, the correct version is sufficiently simple as to be
unobjectionable. But in the interest of compromise, I can switch this
over to the incorrect-but-fewer-lines version. (I’d rather cede this
point, which is under our control, than the other, which is under the
control of whoever clones the repository.)


def chdir_to_repo_root():
toplevel = subprocess.check_output(["git", "rev-parse", "--show-toplevel"])
toplevel = toplevel[:-1] # trim trailing LF
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.chdir(toplevel.rstrip())?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is again for whitespace-correctness—nothing wrong with

$ git clone -q gh:tensorflow/tensorboard "tensorboard "
$ cd "tensorboard "
$ pwd | xxd
00000000: 2f74 6d70 2f74 6d70 2e49 426d 7057 5446  /tmp/tmp.IBmpWTF
00000010: 7573 712f 7465 6e73 6f72 626f 6172 6420  usq/tensorboard 
00000020: 0a                                       .
$ python -c '
> import os
> import subprocess
> toplevel = subprocess.check_output(["git", "rev-parse", "--show-toplevel"])
> os.chdir(toplevel.rstrip())
> '
Traceback (most recent call last):
  File "<string>", line 5, in <module>
OSError: [Errno 2] No such file or directory: '/tmp/tmp.IBmpWTFusq/tensorboard'

It really costs us nothing here to do the right thing, and I’d love to
avoid proliferating even more programs that behave incorrectly on
filenames that have whitespace.

import sys


# Remove files from this list as whitespace errors are fixed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are any of these ones that should actually be longstanding exceptions? If not, why not just commit the whitespace fixes to these now and avoid ever checking in the exceptions list support and stale exceptions detection?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are any of these ones that should actually be longstanding exceptions?

Nope. (The only potential genuine exception that I could think of would
be Markdown hard-breaks, which should rarely be used anyway.)

why not just commit the whitespace fixes to these now and avoid ever
checking in the exceptions list support

Two benefits:

  • we avoid a megacommit that kills all the blame (existing such
    commits, like e1adbef and friends, are perennially frustrating in
    everyday TensorBoard development); and
  • in the case that we do need a genuine exception (say, some strange
    data format for a plugin’s test data), the path of least resistance
    will be to mark it as an exception rather than just disabling the
    test because there’s an urgent deadline.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running your script shows that there are only 88 affected individual lines. I don't think an 88-line delta is a megacommit and it's certainly nothing like a 11,000 line one. Blame impact would be localized only to affected lines.

(Though, even if we were talking about reformatting en masse (e.g. buildifiering all our build files) I think it's better to just get it over with. Doing it one touched line at a time will never actually fix everything so you never achieve the uniformity that's the desired goal anyway. Doing it file-by-file means that you still clobber the blame but now there's hundreds of such commits instead of one.)

The second benefit is entirely speculative.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point: I was scared off by the file count and didn’t look at the
delta. That does make it much less bad. I’m convinced; will send out a
fix.

The second benefit is entirely speculative.

Certainly.

wchargin-source: 7860a65430d548c78ef0e6d2032fbacfc8ead188
wchargin-branch: whitespace-hygiene-test
@wchargin wchargin merged commit 3ccafba into master Jun 19, 2019
@wchargin wchargin deleted the wchargin-whitespace-hygiene-test branch June 19, 2019 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants