#3427 Extend visualization of non-printables with clear and stable characters

Observation

macOS curiously renders the “Control Pictures" (U+2400) block characters differently depending on context.

Preceded by 00-08 | 0E-1F | 7f, 09-0D will render horizontally adjacent like other control characters
Preceded by any other character, they are rendered diagonally (NW to SE) – and much less legible to boot

ruby -e '((0..9).to_a + [127] + (11..31).to_a).each.with_index { |c, i| printf "%02X%s \x00%s ", c, c.chr, c.chr; puts if i & 7 == 7 }' | bat --color=never -A --tabs 1 | sed 's/·/ /g'

Proposals

1. Replace unstable Unicode characters

Use characters that don't exhibit confusing contextual behavior, and are also more legible and descriptive.

09 HT  ⇥ 21E5 "Rightwards Arrow to Bar"
0A LF  ⏎ 23CE "Return Symbol"
0B VT  ⤓ 2913 "Downwards Arrow to Bar"
0C FF  ↡ 21A1 "Downwards Two Headed Arrow"
0D CR  ← 2190 "Leftwards Arrow"

Note: HT already has some logic around ↹ and ├─┤ that would have to be optioned or reconsidered.

2. Display intuitive Unicode characters for other still-in-use characters

Albeit unaffected by the diagonal issue, these characters will also benefit from a clear iconography:

08 BS  ⌫ 232B "Erase to the Left"
1B ESC ⎋ 238B "Escape Symbol"
7F DEL ⌦ 2326 "Delete Symbol"

3. Extend visualization options

Provided:

Today only LF, CR, HT and ESC are ever really used as originally intended
Alternative meanings may apply (eg. \x1A terminates a Hayes AT SMS)
Triple abbrev Unicode characters are very slim and hard to read
- They might even overlap the next char (depending on font rendering)
Binary viewers like xxd display all non-printables as a dot .
- (Preventing clutter for a user most likely scanning for human-readable text)

Proposal (including a direly needed -c shorthand):

      -c, --nonprintable-notation <notation>
              p, period   Show as period
              c, caret    Show as caret notation (^@, ^A ... ^_)
              u, unicode  Show as Unicode (block U+2400)
              b, binary   Show relevant as Unicode, legacy as period
              d, default  Show relevant as Unicode, legacy as default

Other aspects

Leave a comment if you know anything about ...

Insights or theories on why macOS alters its font rendering
Corresponding behavior in other Operating Systems