In LuaTeX, what is the “identity” linebreak_filter? (Or: how can we use `tex.linebreak` to match the default line-breaking algorithm?)

You need to reset (at least) \prevdepth something like this, otherwise \prevdepth gets the "start of list" value of -1000pt and the following paragraph is set with no space before it.

\directlua{
  function shouldbenoop(head, isDisplay)
    h,t =tex.linebreak(head)
    tex.prevdepth= t.prevdepth
   return h
  end
}
\directlua{callback.register('linebreak_filter', shouldbenoop)}

\hsize3cm
\parindent0pt

g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g 

\showthe\prevdepth
g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g


a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a

a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a

\bye

I am not deleting this for sentimental reasons, but from informed comment and comment and from the answer by @FrankMittlebach and @DavidCarlisle it is expected that \prevdepth (and \prevgraf) settings must be restored by the callback. I have not read the manual but it seems from that further comment by @ShreevatsaR that one needs to gather the information from other sources as well.


This is too long but is a comment: in the LaTeX case too, something is broken. Here is my test file:

\documentclass{article}

\begin{document}
\directlua{
  function shouldbenoop(head, isDisplay)
    return tex.linebreak(head)
  end
}
%\directlua{luatexbase.add_to_callback('linebreak_filter', shouldbenoop, 'This should be same as the default?')}

\parskip 0pt

\def\x{\rule{2cm}{5pt} }

\def\y{\x\x\x\x}
\def\z{\y\y}

\z

\z g g g 

\z

\thispagestyle{empty}
\showoutput

\end{document}

The g g g in second paragraph cause the third to move down, when the \directlua is executed. The baselineskip glue computed by TeX should be 4.94pt but turns out to be 7pt (making the distance from bottom of g to next baseline exactly 7pt+5pt=12pt).

Here is a diff of the logs, cut down to the relevant things. At top of each portion the result with the \directlua executed, at bottom, with it commented out. Clearly the last paragraph is positioned lower in former case due to the descenders of letter g: TeX's mechanism is broken for maintaining a constant base line.

*** 84,90 ****
  ...\hbox(0.0+0.0)x345.0, direction TLT
  ..\glue 25.0
  ..\glue(\lineskip) 0.0
! ..\vbox(550.0+0.0)x345.0, glue set 477.88895fil, direction TLT
  ...\write-{}
  ...\glue(\topskip) 5.0
  ...\hbox(5.0+0.0)x345.0, glue set - 1.0, direction TLT
--- 83,89 ----
  ...\hbox(0.0+0.0)x345.0, direction TLT
  ..\glue 25.0
  ..\glue(\lineskip) 0.0
! ..\vbox(550.0+0.0)x345.0, glue set 479.94873fil, direction TLT
  ...\write-{}
  ...\glue(\topskip) 5.0
  ...\hbox(5.0+0.0)x345.0, glue set - 1.0, direction TLT
***************
*** 168,174 ****
  ....\glue(\parfillskip) 0.0 plus 1.0fil
  ....\glue(\rightskip) 0.0
  ...\glue(\parskip) 0.0
! ...\glue(\baselineskip) 7.0
  ...\hbox(5.0+0.0)x345.0, glue set - 1.0, direction TLT
  ....\localpar
  .....\localinterlinepenalty=0
--- 167,173 ----
  ....\glue(\parfillskip) 0.0 plus 1.0fil
  ....\glue(\rightskip) 0.0
  ...\glue(\parskip) 0.0
! ...\glue(\baselineskip) 4.94
  ...\hbox(5.0+0.0)x345.0, glue set - 1.0, direction TLT
  ....\localpar
  .....\localinterlinepenalty=0
***************

Image of output with \directlua executed:

enter image description here

The last paragraph is moved down.


And \the\prevdepth indeed produces 0.0pt in the lua filter case and 2.06pt without it. (the kind of things one expects as collateral damage from using a \special, like happens with using \color etc... but here the \showoutput does not show such a thing, so must be more in the innards of luatex).