Unicode text wrapping in comments and string literals.

[wada314]

This PR includes several fixes related to #3235 .

Clearing the terminology: Replaced some identifier's "chars" into "width".

Introduce unicode_width crate, which implements UAX#11: Replaced some of the str::len() calls into this unicode width method calls to support characters occupying != 1 width (e.g. CJK, combined chars).

Introduce unicode_categories crate just to make is_punctuation() method unicode supported: Without this CJK strings almost cannot be wrapped because it does not include whitespaces, and the old is_punctuation() method was only supporting ASCII delimiters.
But by this fix we may over-detect the punctuation characters. For example, do we really want to insert newline after like "§" or "¶"? I'm not sure.
I believe the next step is to support UAX#14, which is defining the line-breaking algorithm. Then we can completely solve this punctuation issue.

[wada314]

Hello, does someone have a time to review this PR? Thanks,

[scampi]

The changes look good to me.
The build failure with error_chain may have surfaced a bug with what you try to fix:

rustfmt --edition 2015 /home/travis/build/rust-lang/rustfmt/error-chain/tests/quick_main.rs /home/travis/build/rust-lang/rustfmt/error-chain/src/bin/has_backtrace.rs /home/travis/build/rust-lang/rustfmt/error-chain/examples/doc.rs /home/travis/build/rust-lang/rustfmt/error-chain/tests/tests.rs /home/travis/build/rust-lang/rustfmt/error-chain/examples/size.rs /home/travis/build/rust-lang/rustfmt/error-chain/examples/chain_err.rs /home/travis/build/rust-lang/rustfmt/error-chain/examples/all.rs /home/travis/build/rust-lang/rustfmt/error-chain/examples/quickstart.rs /home/travis/build/rust-lang/rustfmt/error-chain/src/lib.rs
thread 'main' panicked at 'byte index 31 is not a char boundary; it is inside '\u{ad}' (bytes 30..32) of `|| "env var was bork文字化ã"`', src/libcore/str/mod.rs:2098:5

[nrc]

LGTM too, thank you! COuld you investigate the error_chain bug please? Could you also squash the last few commits to make a coherent series please.

[wada314]

Thanks for the comments!

• I fixed the error_chain bug that I introduced: overflow.rs was using util.rs's first_line_width() method I modified in this PR. I changed first_line_width() to return "line width" instead of "str index", but overflow.rs was still using it as "str index".

• Filed a fixme issue Use Unicode UAX#14 to format multi-line string #3281 to use UAX#14 to decide line break position. Added that issue number into break_string() method comment (string.rs line 232).

• Squashed git commits (Am I doing it correctly? not sure...).

[pietroalbini]

Just as a note, the Travis CI build is stuck waiting for macOS builders, and that's a known issue. You can read more here. Sorry for the trouble!