What is the LaTeX3 project doing to make LaTeX run faster?

The LaTeX project does spend a lot of time making sure that latex goes as fast as possible but none of the things you suggest are relevant to latex code; you are suggesting changes to the tex language in which latex is written.

As you can see if you look through github issues, a lot of thought goes into optimising the core expl3 programming constructs, whether it's quicker to have multiple \expandafter or \fi-delimited argument or to use \expanded or whatever in each case.

Also LaTeX releases this year have preloaded two largish packages into the format, expl3 (in February) and xparse (in October) this can make a fairly noticeable improvement in startup time as locating package files and reading the data off the filesystem can take significantly longer than processing the tex code within the file.

Note you can build a custom format pre-loading the packages you use which can also speed up start up time a lot.

The kind of check pointing that you mention is asking about the underlying tex system so not addressable within LaTeX. It is the same as other programming languages. A Web page author can avoid inefficient JavaScript in their page to make the page load faster, they can not re-write the JavaScript engine in all possible browsers in which that code may run, which is the equivalent of what you are asking here.

The actual checkpointing is hard as page breaking is asynchronous, it is feasible at forced page breaks from \clearpage which is exactly what the LaTeX \include system does, it saves the state of all LaTeX counters at that point so if on the next run you skip chapters 1-3, the page numbering is preserved and the draft document starts with chapter 4. But to do that automatically and to save more state, such as the definitions of all macros, not just the values of all counters, would require changes to the tex system not to latex.

You mention that later changes can affect earlier ones due to auxiliary files, but that is actually the easier case, just consider a long paragraph that spans over two or more pages. Adding a comma in the last line can change the line breaking of the entire paragraph so changing earlier pages without any auxiliary files being involved.

Many tex systems these days are fast enough that latex is set up to run continuously in the background as the file is edited and update the display whenever the pdf is successfully remade, if your build is slow you should look to your build system, are you including high resolution images or re-setting complicated tikz on every run? If you arrange to save these things to more occasional "full" builds you can usually get things to run at a reasonable speed.


Putting this here as a bit of a provocative manifesto:

Popping back into LaTeXworld lately I've contemplating finally finishing the LaTeX book that I'd started in the late 80s/early 90s when I taught the TUG LaTeX classes.

LaTeX 2e was first released in 1994 as a transitional step to the eventual release of LaTeX 3. 26 years later, there still isn't a 1.0 release of LaTeX 3. In the interim, we've seen the rise of HTML and the web, the dominance of PDF as a format for representation of printed material (and now there is a plan to have PDF extended with "liquid mode" that allows reflowing of PDF text for smaller screens).

In the meantime, the TeX engine has been extended multiple times, the little-used TeX-XeT, some early efforts to support large Asian character sets, and we have in widish use pdfTeX, XeTeX, LuaTeX along with an assortment of abandoned engines. Worst of all, it seems that none of pdfTeX, XeTeX or LuaTeX can serve as the one TeX to rule them all, each with some limitations that can require users to switch engines depending on their needs.

As I've thought about it, the problem at its root is TeX itself. It's what would be referred to in contemporary software engineering parlance, as a tightly-coupled monolith. Worse still, it's a tightly-coupled monolith with numerous compromises baked in because of the limitations of 1970s computing hardware. It seems that the vast majority of what work has been done with LaTeX 3 has been geared towards dealing with the limitations of TeX as a programming language.

On top of that, there's been an explosion of questionable, if not outright harmful practices from the greater LaTeX community. Ideally, a document should be translated from one document class to another structurally similar class (naming-wise, the choice of "class" to name document classes is unfortunate, but understandable) should not require changing anything after the preamble, better still, nothing but the \documentclass command itself. All the appearance should be handled through the document class and packages should be employed to provide document structure enhancements or new capabilities). There are numerous violations of this. The memoir class is a mess, claiming to be a replacement for article, report and book (this reminds me of the mess that's PHP where the same data structure acts as an array and an associative array and as a consequence manages to merge the worst aspects of both in one inefficient construct) and at the same time, providing a number of bits of functionality that belong in packages rather than the document class. On the flipside, packages like geometry and fancyhdr fall into a category that LaTeX2e doesn't really define, bits of common code that would be helpful to document class writers but shouldn't really be exposed to document authors.

Given the ultimate failure of NTS and ExTeX, I'm not hopeful for a resolution to any of these issues.

Edit 20 October 2020 I've done something stupid. I'm starting a new project. It will probably never amount to anything.