Why doesn't a trailing tab typeset?

The file I used is

\catcode9=12

.
.       %
.
\bye

Line 3 has .<tab>, line 4 has .<tab>%, line 5 has .<tab><space>

> hexdump tabs.tex 
0000000 5c 63 61 74 63 6f 64 65 39 3d 31 32 0a 0a 2e 09
0000010 0a 2e 09 25 0a 2e 09 20 0a 5c 62 79 65 0a      
000001e

This is what pdftex typesets:

Only the <tab> followed by % survives, because TeX Live implementations of TeX remove trailing spaces and tabs from lines, irrespective of their catcodes. I tried to find the reference, but apparently this is to be considered folklore.

The space between the first two periods is added by the end-of-line.

Update 2019

With the 2019 release of TeX Live, the TeX engines no longer remove traling tab tokens, but only spaces and the output you get is

Some code findings for the "folklore" of egreg's answer.

TeX removes "blanks" at the end of an input line. This is done at a very early stage, just after reading the line before considering category codes and the input characters get tokenized.

Originally, these "blanks" are spaces only, but TeX distributions like TeX Live or MiKTeX extends them to include tabulators (horizontal tabulator). The snippets show the behavior for TeX, and pdfTeX. Not shown are XeTeX and LuaTeX that also remove spaces and tabulators at the end of input lines.

The code snippets come from TeX Live (2016).

• texk/web2c/tex.web:

  @ The |input_ln| function brings the next line of input from the specified
  file [...]
  Trailing blanks are removed from the line;
  [...]
  @p function input_ln(var f:alpha_file;@!bypass_eoln:boolean):boolean;
    {inputs the next line or returns |false|}
  var last_nonblank:0..buf_size; {|last| with trailing blanks removed}
  begin if bypass_eoln then if not eof(f) then get(f);
    {input the first character of the line into |f^|}
  last:=first; {cf.\ Matthew 19\thinspace:\thinspace30}
  if eof(f) then input_ln:=false
  else  begin last_nonblank:=first;
    while not eoln(f) do
      begin if last>=max_buf_stack then
        begin max_buf_stack:=last+1;
        if max_buf_stack=buf_size then
          @<Report overflow of the input buffer, and abort@>;
        end;
      buffer[last]:=xord[f^]; get(f); incr(last);
      if buffer[last-1]<>" " then last_nonblank:=last;
      end;
    last:=last_nonblank; input_ln:=true;
    end;
  end;
  

  The original TeX only removes spaces at the end of an input line. However, the Pascal version of input_ln will be overwritten by a more efficient C version, see the next code snippets.

• texk/web2c/tex.ch is a change file for tex.web:

  @x [3.31] l.933 - Do `input_ln' in C.
  @p function input_ln(var f:alpha_file;@!bypass_eoln:boolean):boolean;
  [...]
  end;
  @y
  We define |input_ln| in C, for efficiency. [...]
  @z
  

• texk/web2c/lib/texmfmp.c:

  /* Read a line of input as efficiently as possible while still looking
     like Pascal.  We set `last' to `first' and return `false' if we get
     to eof.  Otherwise, we return `true' and set last = first +
     length(line except trailing whitespace).  */
  
  #ifndef XeTeX /* for XeTeX, we have a replacement function in XeTeX_ext.c */
  boolean
  input_line (FILE *f)
  {
    [...]
  
    /* Trim trailing whitespace.  */
    while (last > first && ISBLANK (buffer[last - 1]))
      --last;
  
    [...]
  }
  

• texk/kpathsea/c-ctype.h:

  #ifndef isblank
  #define isblank(c) ((c) == ' ' || (c) == '\t')
  #endif
  
  #define ISBLANK(c) (isascii (c) && isblank ((unsigned char)c))
  

  isblank tests for space and tabulators, therefore both are removed at the end of an input line.

• texk/web2c/ChangeLog:

  Thu Oct 16 20:39:27 1997  Olaf Weber  <...>
  
      * `tex.ch`: [...]  Also, various changes
      for e-TeX (small rearrangements, introduces Init..Tini, remove
      tabs and trailing blanks).  From Peter Breitenlohner
      <...>.
  

  The change is very old, two decades ago in the last century.

REVISED ANSWER

I discovered that even though I was copy and pasting a TAB into TeXworks, the editor itself was doing a conversion to spaces in my ORIGINAL ANSWER. Thus, I used a different editor which I knew would preserve the TAB character in the file, and it shows that the keyboard TAB behaves like the ^^I "TeX TAB"...almost.

If the TAB is not at the line end, then the "keyboard-TAB" and "TeX-TAB" behave identically. If however, the TAB is at the end-of-line, the "keyboard TAB" is treated as a space, whereas the "TeX TAB" is still treated however redefined by TeX.

Conclusions:

1. Keyboard TABS and TeX-TABs (^^I) seem to be treated the same, except at the end of input lines.

2. Keyboard TABS are removed at the end-of-line (what David said), whereas ^^I TABS are not.

3. Use the TeX TAB ^^I to denote TABS in code, as editors are prone to do auto-conversion on your keyboard TABS otherwise.

The MWE (WARNING: copy/pasting this MWE into your editor may result in a conversion of the tabs into spaces):

With the TAB as defined by \TeX\par
% TWO EXPLICIT SPACES
x\ \ x\par%
% THE FOLLOWING PUTS AN EMPTY GROUP AFTER THE "KEYBOARD-TAB"; RESULT = 2 SPACES
x   {}
x\par%
% THE FOLLOWING TRAILS WITH A KEYBOARD TAB (WHAT THE OP TRIED); RESULT = 1 SPACE
x   
x\par
% THE FOLLOWING TRAILS WITH A "TeX-TAB"; RESULT = 1 SPACE
x^^I
x\par

With the TAB as catcode 12:\par
\catcode`\^^I=12 %
% TWO EXPLICIT SPACES
x\ \ x\par%
% THE FOLLOWING PUTS AN EMPTY GROUP AFTER THE "KEYBOARD-TAB"; RESULT = TAB GLYPH + SPACE
x   {}
x\par%
% THE FOLLOWING TRAILS WITH A KEYBOARD TAB (WHAT THE OP TRIED); RESULT = 1 SPACE
x   
x\par
% THE FOLLOWING TRAILS WITH A "TeX-TAB"; RESULT = TAB GLYPH PLUS SPACE
x^^I
x\par

\catcode`\^^I=\active %
\def^^I{\space}
With the TAB as an active space

% TWO EXPLICIT SPACES
x\ \ x\par%
% THE FOLLOWING PUTS AN EMPTY GROUP AFTER THE "KEYBOARD-TAB"; RESULT = 2 SPACES
x   {}
x\par%
% THE FOLLOWING TRAILS WITH A KEYBOARD TAB (WHAT THE OP TRIED); RESULT = 1 SPACE
x   
x\par
% THE FOLLOWING TRAILS WITH A "TeX-TAB"; RESULT = 2 SPACES
x^^I
x\par

\catcode`\^^I=\active %
\def^^I{Q}
With the TAB as an active Q

% TWO EXPLICIT SPACES
x\ \ x\par%
% THE FOLLOWING PUTS AN EMPTY GROUP AFTER THE "KEYBOARD-TAB"; RESULT = Q +  SPACE
x   {}
x\par%
% THE FOLLOWING TRAILS WITH A KEYBOARD TAB (WHAT THE OP TRIED); RESULT = 1 SPACE
x   
x\par
% THE FOLLOWING TRAILS WITH A "TeX-TAB"; RESULT = Q + SPACE
x^^I
x\par

\bye

ORIGINAL ANSWER (AS FOOLED BY MY EDITOR)

The original answer was removed, because I was fooled by my editor when pasting a TAB into the input file...the editor did an auto-conversion to spaces.

---

I should note the TeXbook talks about the TAB character, as noted in my comments to the OP, on pages 8, 45, 369-370, and 391.