What is the LaTeX equivalent of ConTeXt \testfeatureonce to benchmark performance

With expl3 you can use \tex_resettimer:D and \tex_elapsedtime:D, which exist for both pdfTeX and since TL 2019 for XeTeX (they are the primitives \(pdf)resettimer and \(pdf)elapsedtime renamed). For LuaTeX you can use the Lua functions from l3kernel. Most of this is a rudimentary version of what's in the l3benchmark package (see Joseph's answer), released shortly after this question was asked.

\documentclass{article}
\usepackage{xparse}
\usepackage{tikz}

\ExplSyntaxOn
\sys_if_engine_luatex:TF
  {
    \cs_new:Npn \__tfo_start_timer:  { \lua_now:n { l3kernel.resettimer() } }
    \cs_new:Npn \__tfo_elapsed_time: { \lua_now:n { l3kernel.elapsedtime() } }
  }
  {
    \cs_new_eq:NN \__tfo_start_timer: \tex_resettimer:D
    \cs_new_eq:NN \__tfo_elapsed_time: \tex_elapsedtime:D
  }
\cs_new:Npn \__tfo_output:n #1
  {
    \iow_term:x
      {
        >~#1~feature~tests~done~
        (\fp_eval:n { round ( \__tfo_elapsed_time: / 65536 , 3 ) }s)
      }
  }
\cs_new:Npn \__tfo_run_n_times:nn #1 #2
  {
    \if_int_compare:w #1 > 0 \scan_stop:
      \exp_args:No \__tfo_run_n_times:nnw { \int_value:w \__int_eval:w #1 - 1 } { #2 }
    \fi:
  }
\cs_new:Npn \__tfo_run_n_times:nnw #1 #2 \fi:
  {
    \fi:
    #2
    \__tfo_run_n_times:nn { #1 } { #2 }
  }
\NewDocumentCommand\TestFeatureOnce
  { m m }
  {
    \__tfo_start_timer:
    \__tfo_run_n_times:nn { #1 } { #2 }
    \__tfo_output:n { #1 }
  }
\ExplSyntaxOff

\begin{document}

\TestFeatureOnce{10000}
    {\setbox0\hbox
        {\tikz \draw (0,0) -- (1cm, 1cm);}}

\end{document}

Edit: Bug fix. A large number of tests (~5000) would exceed TeX's input stack size by piling up tons of \fi:s, so I added a helper macro to make sure everything is executed outside the \if...\fi so that an arbitrary number of runs is possible.

With jfbu's idea to convert scaled seconds, one can even do it in plain (pdf, Xe, or Lua)TeX:

\input tikz.tex

\begingroup% pdfTeX
  \expandafter\ifx\csname pdfresettimer\endcsname\relax
  \else
    \global\let\resettimer\pdfresettimer
    \global\let\elapsedtime\pdfelapsedtime
  \fi
\endgroup
\begingroup% LuaTeX
  \expandafter\ifx\csname directlua\endcsname\relax
  \else
    \gdef\resettimer{\directlua{pdfelapsedtimer_basetime = os.clock()}}
    \gdef\elapsedtime{\directlua{tex.print((os.clock()-pdfelapsedtimer_basetime)*65536)}}
  \fi
\endgroup
% For XeTeX the primitives are already called \resettimer and \elapsedtime
\catcode`@=11
% Stolen from latex.ltx
\begingroup
  \catcode`P=12
  \catcode`T=12
  \lowercase{%
    \def\x{\def\rem@pt##1.##2PT{##1\ifnum##2>\z@.##2\fi}}}%
  \expandafter\endgroup\x
\def\strip@pt{\expandafter\rem@pt\the}%
%
\let\TFO@start@timer\resettimer
\let\TFO@elapsed@time\elapsedtime
\def\TFO@output#1{%
  \immediate\write17{%
    > #1 feature tests done (\strip@pt\dimexpr\TFO@elapsed@time sp\relax s)
  }%
}
\long\def\TFO@runNtimes@nn#1#2{%
  \ifnum#1>0\relax
    \expandafter\TFO@runNtimes@nnw\expandafter{\number\numexpr#1-1}{#2}%
  \fi
}
\long\def\TFO@runNtimes@nnw#1#2\fi{%
  \fi
  #2%
  \TFO@runNtimes@nn{#1}{#2}%
}
\long\def\TestFeatureOnce#1#2{%
  \TFO@start@timer
  \TFO@runNtimes@nn{#1}{#2}%
  \TFO@output{#1}%
}
\catcode`@=12

\TestFeatureOnce{1000}
    {\setbox0\hbox
        {\tikz \draw (0,0) -- (1cm, 1cm);}}

\bye

For all I know, there's no such macro in the LaTeX2e base (perhaps LaTeX3 has something equivalent?). But we can look up the definition of \testfeatureonce in the ConTeXt source code. The relevant definitions are in the syst-aux module.

That implementation makes use of the eTeX primitives \pdfresettimer and \pdfelapsedtime to reset an internal timer and get the elapsed time since the last reset, respectively. I didn't find a proper source for this, but it seems 65536 equals one second in the value returned by \pdfelapsedtime.

A reimplementation in pure LaTeX code might look like this:

\documentclass{article}
\usepackage{tikz}

\makeatletter
\let\resettimer=\pdfresettimer
\let\elapsedtime=\pdfelapsedtime

\newcount\c@syst@helpers@test@feature@n
\newcount\c@syst@helpers@test@feature@m

\newcommand\testfeature[2]{%
    \c@syst@helpers@test@feature@m=#1\relax
    \def\syst@helpers@test@feature@step{%
        \advance\c@syst@helpers@test@feature@n by 1\relax
        \ifnum\c@syst@helpers@test@feature@n>\c@syst@helpers@test@feature@m\else
            #2\expandafter\syst@helpers@test@feature@step
        \fi
    }%
    \retestfeature
}

\newcommand\retestfeature{
    \bgroup
    \ifcase\interactionmode \let\wait\relax \fi
    \resettimer
    \c@syst@helpers@test@feature@n=0\relax
    \syst@helpers@test@feature@step
    \wait
    \egroup
}

\newcommand\testfeatureonce[2]{%
    \begingroup
    \let\wait\relax
    \testfeature{#1}{#2}%
    \endgroup
}

\makeatother

\begin{document}

\testfeatureonce{1000}
    {\setbox0\hbox
        {\tikz{ \draw (0,0) -- (1cm, 1cm); }}}

\the\elapsedtime

\end{document}

Code using the new l3benchmark package (released to CTAN 2018-10-26):

\documentclass{article}
\usepackage{l3benchmark}
\ExplSyntaxOn
\cs_new_protected:Npn \testfeatureonce #1#2
  {
    \benchmark_once:n
      { \prg_replicate:nn {#1} {#2} }
  }
\ExplSyntaxOff
\usepackage{tikz}

\begin{document}
\testfeatureonce{1000}
    {\setbox0\hbox
        {\begin{tikzpicture} draw (0,0) -- (1cm, 1cm); \end{tikzpicture}}}
\end{document}

The benchmark code can also auto-determine how many cycles to do:

\documentclass{article}
\usepackage{l3benchmark}
\ExplSyntaxOn
\cs_new_protected:Npn \testfeatureonce #1#2
  { \benchmark:n {#2} }
\ExplSyntaxOff
\usepackage{tikz}

\begin{document}
\testfeatureonce{1000}
    {\setbox0\hbox
        {\begin{tikzpicture} draw (0,0) -- (1cm, 1cm); \end{tikzpicture}}}
\end{document}

(The code itself is of course much the same as the other answers here.)

Tags:

Performance

Context

Recent Posts