PlainTeX: Float(s) lost with custom output routine

When output routine is called by TeX \outputpenalty is 10000 (except in the endgame) and TeX has the command \supereject which uses the value -20000. Every penalty value less than -10000 can be used for the output routine and it is suggested to use values -10001, -10002, etc. if the output routine shall be notified why it is called.

The TeXbook, Appendix D, page 400 states:

Sometimes an output routine needs to know why it was invoked, so there's a problem of communicating information from the rest of the program. TeX provides general \mark operations, but marks don't always yield the right sorts of clues. Then there's \outputpenalty, which can be tested to see what penalty occurred at a breakpoint; any penalty of -10000, -10001, -10002, or less, forces the output routine to act, hence different penalty values can be used to pass different messages.

OK; you use -30000 to trigger the output routine if a box was filled with a float. In the output routine this outputpenalty is checked and the box is placed in one of two lists. Now TeX has read the seventh float and it sees that the material that was collected exceeds what it needs. If you activate tracingpages you see the line

% t=565.88889 plus 60.0 g=542.21104 b=* p=-30000 c=*

which has infinite cost. So TeX breaks earlier; the previous lines are

% t=493.88889 plus 60.0 g=542.21104 b=52 p=0 c=52#

% t=517.88889 plus 60.0 g=542.21104 b=7 p=0 c=7#

% t=541.88889 plus 60.0 g=542.21104 b=0 p=150 c=150

and the break is made with cost c=7 and the outputpenalty is set to 10000. The box bx@G is filled but it is not emptied in the output routine. @currbox gets new content and the connection to bx@G is lost.

On the new page TeX starts with

%% goal height=542.21104, max depth=4.0

% t=13.88889 plus 60.0 g=542.21104 b=10000 p=150 c=100000#

% t=37.88889 plus 60.0 g=542.21104 b=10000 p=-30000 c=-30000#

because there is the line \float ABCD7 \vskip20pt\endfloat to finish. But now @currbox contains bx@F not bx@G. This box is permantly lost and will never be used again as it wasn't placed on any list. bx@F appears later twice in freelist.

How to fix this? Create a newif \if@currboxfilled and set it true in \endfloat after \egroup. Next put \if@currboxfilled before \ifx\@deferlist\@empty. Then replace the \else after the then-case of the -30000 by:

    \global\@currboxfilledfalse
    \fi
%
    \else
    \if@currboxfilled
       \expandafter\@additem\expandafter\@deferlist\@currbox
       \global\@currboxfilledfalse
    \fi

If TeX is called with an outputpenalty different from -30000 but a box was filled then it is placed on the @deferlist.

Thanks to @Udo Wermuth I it is clear now that the macro \@currbox is multiply used in the output routine. So in this case it gets another meaning. If I use an own macro for each case, the problem disappears, too:

Full MWE:

\font\testfont=cmr17 at 20 pt

\testfont
\normalbaselineskip24pt
\normalbaselines

\catcode`\@=11

\parskip\z@
\raggedbottom

\newdimen\fullhsize
\newdimen\fullvsize

\fullhsize\hsize
\fullvsize\vsize

\newdimen\@colroom
\@colroom\fullvsize

\newinsert\bx@A
\newinsert\bx@B
\newinsert\bx@C
\newinsert\bx@D
\newinsert\bx@E
\newinsert\bx@F
\newinsert\bx@G
\newinsert\bx@H
\newinsert\bx@I
\newinsert\bx@J

\newcount\@topnum
\newcount\maxtopnum \maxtopnum\thr@@
\newdimen\@toproom
\newdimen\maxtoproom \maxtoproom.4\fullvsize

\newcount\@botnum
\newcount\maxbotnum \maxbotnum\tw@
\newdimen\@botroom
\newdimen\maxbotroom \maxbotroom.3\fullvsize

\newcount\@dbltopnum
\newcount\maxdbltopnum \maxdbltopnum\tw@
\newdimen\@dbltoproom
\newdimen\maxdbltoproom \maxdbltoproom.4\fullvsize

\newcount\@botnum
\newcount\maxbotnum \maxbotnum\tw@
\newdimen\@botroom
\newdimen\maxbotroom \maxbotroom.3\fullvsize

\newcount\@dblbotnum
\newcount\maxdblbotnum \maxdblbotnum\tw@
\newdimen\@dblbotroom
\newdimen\maxdblbotroom \maxdblbotroom.3\fullvsize

\newif\if@insert
\newif\if@trybottom

\def\@empty{}

\def\@freelist{\bx@A\bx@B\bx@C\bx@D\bx@E\bx@F\bx@G\bx@H\bx@I\bx@J}
\let\@toplist\@empty
\let\@botlist\@empty
\let\@deferlist\@empty

\def\@additem#1#2{% #1: list, #2: token
    \expandafter\gdef\expandafter#1\expandafter{#1#2}}

\def\@takenextbox#1#2{% #1: macro for storing the box, #2: list
    \def\@tempa{#1}%
    \expandafter\@@takenextbox#2!%
    \glet#2=\@tempb}

\def\@@takenextbox#1#2!{\expandafter\gdef\@tempa{#1}%
    \def\@tempb{#2}}

\def\float#1{%
    \@takenextbox\@currbox\@freelist
    \global\count\@currbox#1\relax
    \global\setbox\@currbox\vbox\bgroup
    \parindent\z@ \parskip\z@}

\def\endfloat{\par\vskip\z@\egroup
    \penalty-40000\relax
    \dimen@\prevdepth
    \vbox{}
    \prevdepth\dimen@
    \penalty-30000\relax}

\def\@nextboxcnt{% type (count) of the next float on \@deferlist
    \expandafter\expandafter\expandafter
    \count\expandafter\@firstofmany\@deferlist!}

\def\@nextboxht{% height of the next float on deferlist
    \expandafter\expandafter\expandafter
    \ht\expandafter\@firstofmany\@deferlist!}

\def\@firstofmany#1#2!{#1}

\output{\myoutput}
\def\myoutput{%
    \ifnum\outputpenalty=-40000\relax
    \global\setbox\@ne\vbox{\unvbox\@cclv}%
    \else
    \ifnum\outputpenalty=-30000\relax
    \setbox8\box\@cclv
    \unvbox\@ne
    \ifvoid\footins\else\insert\footins{\unvbox\footins}\fi
    %
    \ifcase\count\@currbox\or%1
    \@inserttrue
    \ifx\@deferlist\@empty \else \@insertfalse\fi
    \ifx\@botlist\@empty \else \@insertfalse\fi
    \ifnum\@topnum<\maxtopnum \else \@insertfalse\fi
    \ifnum\@topnum>\z@
      \ifdim\maxtoproom<\dimexpr\@toproom+\ht\@currbox\relax \@insertfalse\fi\fi
    \if@insert
    \expandafter\@additem\expandafter\@toplist\@currbox
    \global\advance\@colroom-\ht\@currbox
    \global\advance\@toproom\ht\@currbox
    \global\advance\@topnum\@ne
    \else
    \expandafter\@additem\expandafter\@deferlist\@currbox
    \fi
    %
    \or%2
    \@inserttrue
    \@trybottomfalse
    \ifx\@deferlist\@empty \else \@insertfalse\fi
    \ifx\@botlist\@empty \else \@insertfalse\fi
    \ifnum\@topnum<\maxtopnum \else \@insertfalse\fi
    \ifnum\@topnum>\z@
    \ifdim\maxtoproom<\dimexpr\@toproom+\ht\@currbox\relax \@insertfalse\fi\fi
    \if@insert
    \expandafter\@additem\expandafter\@toplist\@currbox
    \global\advance\@colroom-\ht\@currbox
    \global\advance\@toproom\ht\@currbox
    \global\advance\@topnum\@ne
    \else
    \ifx\@deferlist\@empty \@trybottomtrue
    \else \expandafter\@additem\expandafter\@deferlist\@currbox\fi
    \fi
    %
    \if@trybottom
    \@inserttrue
    \ifdim\ht\@currbox>\@colroom \@insertfalse\fi
    \ifnum\@botnum<\maxbotnum \else \@insertfalse\fi
    \ifnum\@botnum>\z@
      \ifdim\maxbotroom<\dimexpr\@botroom+\ht\@currbox\relax \@insertfalse\fi\fi
    \if@insert
    \expandafter\@additem\expandafter\@botlist\@currbox
    \global\advance\@colroom-\ht\@currbox
    \global\advance\@botroom\ht\@currbox
    \global\advance\@botnum\@ne
    \else
    \expandafter\@additem\expandafter\@deferlist\@currbox
    \fi
    %
    \fi
    %
    \or%3
    \@inserttrue
    \ifx\@deferlist\@empty \else \@insertfalse\fi
    \ifdim\ht\@currbox>\@colroom \@insertfalse\fi
    \ifnum\@botnum<\maxbotnum \else \@insertfalse\fi
    \ifnum\@botnum>\z@
      \ifdim\maxbotroom<\dimexpr\@botroom+\ht\@currbox\relax \@insertfalse\fi\fi
    \if@insert
    \expandafter\@additem\expandafter\@botlist\@currbox
    \global\advance\@colroom-\ht\@currbox
    \global\advance\@botroom\ht\@currbox
    \global\advance\@botnum\@ne
    \else
    \expandafter\@additem\expandafter\@deferlist\@currbox
    \fi
    %    
    \fi
    %
    \else
    \shipout\vbox{\makeheadline\pagebody\makefootline}%
    \advancepageno
    %
    \global\@colroom\fullvsize
    \global\@topnum\z@
    \global\@toproom\z@
    \global\@botnum\z@
    \global\@botroom\z@
    %
    \ifx\@deferlist\@empty\else
    % Top floats, do not use \@currbox here!
    \loop
    \@inserttrue
    \ifnum\@nextboxcnt<\z@ \@insertfalse\fi
    \ifnum\@nextboxcnt>\tw@ \@insertfalse\fi
    \ifnum\@topnum<\maxtopnum \else \@insertfalse\fi
    \ifnum\@topnum>\z@
       \ifdim\maxtoproom<\dimexpr\@toproom+\@nextboxht\relax \@insertfalse\fi\fi
    \if@insert
    \@takenextbox\@nextbox\@deferlist
    \expandafter\@additem\expandafter\@toplist\@nextbox
    \global\advance\@colroom-\ht\@nextbox
    \global\advance\@toproom\ht\@nextbox
    \global\advance\@topnum\@ne
    \fi
    \ifx\@deferlist\@empty \@insertfalse\fi
    \if@insert
    \repeat
    %
    \fi
    %
    \ifx\@deferlist\empty\else
    % Bottom floats:
    \loop
    \@inserttrue
    \ifnum\@nextboxcnt<\tw@ \@insertfalse\fi
    \ifdim\@nextboxht>\@colroom \@insertfalse\fi
    \ifnum\@botnum<\maxbotnum \else \@insertfalse\fi
    \ifnum\@botnum>\z@
       \ifdim\maxbotroom<\dimexpr\@botroom+\@nextboxht\relax \@insertfalse\fi\fi
    \if@insert
    \@takenextbox\@nextbox\@deferlist
    \expandafter\@additem\expandafter\@botlist\@nextbox
    \global\advance\@colroom-\ht\@nextbox
    \global\advance\@botroom\ht\@nextbox
    \global\advance\@botnum\@ne
    \fi
    \ifx\@deferlist\@empty \@insertfalse\fi
    \if@insert
    \repeat
    %
    \fi
    %
    \ifnum\outputpenalty=-\@MM \dosupereject\fi
    \fi
    \fi
    \ifnum\outputpenalty<-30000\relax
        \global\vsize\maxdimen\else\global\vsize\@colroom\fi}

\def\pagebody{\vbox to\fullvsize{\boxmaxdepth\maxdepth \pagecontents}}

\def\pagecontents{\ifx\@toplist\@empty\else% Do not use \@currbox here!
    \loop
    \@takenextbox\@nexttopbox\@toplist
    \expandafter\@additem\expandafter\@freelist\@nexttopbox
    \box\@nexttopbox
    \unless\ifx\@toplist\@empty\repeat
    \fi
    %
    \dimen@=\dp\@cclv \unvbox\@cclv % open up \box255
    %
    \ifx\@botlist\@empty\else
    \loop
    \@takenextbox\@nextbotbox\@botlist
    \expandafter\@additem\expandafter\@freelist\@nextbotbox
    \box\@nextbotbox
    \unless\ifx\@botlist\@empty\repeat
    \fi
    \ifvoid\footins\else % footnote info is present
    \vskip\skip\footins
    \footnoterule
    \unvbox\footins\fi
    \ifr@ggedbottom \kern-\dimen@ \vfil \fi}

1. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

\float1 ABCD1 \vskip20pt\endfloat

\float1 ABCD2 \vskip20pt\endfloat

\float1 ABCD3 \vskip20pt\endfloat

\float1 ABCD4 \vskip20pt\endfloat

\float1 ABCD5 \vskip20pt\endfloat

\float1 ABCD6 \vskip20pt\endfloat

2. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.

\float1 ABCD7 \vskip20pt\endfloat% works now

\float1 ABCD8 \vskip20pt\endfloat

3. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

4. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

5. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

\bye