(How) Can I write a .dtx file without having to comment out *everything*?

If you look at the history of doc, then it is understandable why the current system works as it works. Initially the idea as to write a .sty file with the comments imbedded, but in a way that it would directly be processable by LaTeX. And that required putting every bit of documentation behind % signs.

Only later docstrip appeared (to strip the comments because back then processing all those unnecessary comment lines took noticable extra time). Even later the <guards> appeared and with them none-sequencial generation of target files. At that time the name of the files changed to .dtx and one could then have changed the documentation part to work without % --- but that didn't happen.

Having said this, one can with a little modification run doc in that form, e.g.,

%<*documentation>
\documentclass{article}

\usepackage{doc}

\makeatletter
\def\inlinecode{\macro@code
   \frenchspacing \@vobeyspaces
   \xinline@code}
\let\endinlinecode\endmacrocode

\begingroup
\catcode`\|=\z@ \catcode`\[=\@ne \catcode`\]=\tw@
\catcode`\{=12 \catcode`\}=12
\catcode`\\=\active
|gdef|xinline@code#1\end{inlinecode}[#1|end[inlinecode]]
|endgroup
\makeatother

\begin{document}
\noindent some text
\begin{inlinecode}
%</documentation>
\def\foo{bar}
%<*documentation>
\end{inlinecode}
more text
\end{document}
%</documentation>

To the processable code out of this file one would need a docstrip install file like

\input docstrip

\generate{\file{x.sty}{\from{how-can-i-write-a-dtx-file-without-having-to-comment-out-everything.dtx}{}}}

\endbatchfile

But it isn't fully satisfying, as the documentation guards show up in the documentation and without some serious changes to docstrip and doc I don't see a way to get rid of those:

enter image description here

perhaps it would look a little better in print if we call the guard "code" (but then * and / are in the wrong order) --- either way it is suboptimal.

Update

If one wants to get rid of the documentation guards, one simple solution is of course to use docstrip to generate a separate documentation file as it was suggested already by the OP. To do this all one has to do is to additionally provide the line

\generate{\file{packagedoc.tex}{\from{how-can-i-write-a-dtx-file-without-having-to-comment-out-everything.dtx}{documentation}}}

in the .ins file and then run LaTeX on the resulting file. However, to fully utialize the features of doc, e.g., the code indexing or the code line numbering it is imortant to use the macrocode environment. As this environment uses the special syntax with % \end{macrocode} it can't be used directly at least not if one wants to avoid putting the % in. Therefore the code above defining the inlinecode environment as an alternative is essential.

Perhaps that bit of code should be added to doc to allow for this approach.


Ok, so maybe it's not possible to do this elegantly using the 'standard' way of running the .dtx file through LaTeX to get the docs. But I have an alternative solution here which seems just as nice.

With this setup, there is only one file (the '.ins file' is included). You run it through LaTeX and it generates package.sty and package.tex. To get the actual documentation you still need to run package.tex itself through LaTeX. (A sacrifice I am willing to make.)

Here is a minimal working example (and a reasonable skeleton for anyone wanting to do the same):

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%  Copyright, license, etc... %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% \CheckSum{3}
%%
%% \CharacterTable
%%  {Upper-case    \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z
%%   Lower-case    \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z
%%   Digits        \0\1\2\3\4\5\6\7\8\9
%%   Exclamation   \!     Double quote  \"     Hash (number) \#
%%   Dollar        \$     Percent       \%     Ampersand     \&
%%   Acute accent  \'     Left paren    \(     Right paren   \)
%%   Asterisk      \*     Plus          \+     Comma         \,
%%   Minus         \-     Point         \.     Solidus       \/
%%   Colon         \:     Semicolon     \;     Less than     \<
%%   Equals        \=     Greater than  \>     Question mark \?
%%   Commercial at \@     Left bracket  \[     Backslash     \\
%%   Right bracket \]     Circumflex    \^     Underscore    \_
%%   Grave accent  \`     Left brace    \{     Vertical bar  \|
%%   Right brace   \}     Tilde         \~}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%<*driver>                                                                     %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\input docstrip.tex

\keepsilent

\usedir{tex/latex/package}

\nopreamble

\generate{
    \file{package.tex}{\from{\jobname.dtx}{d}}
    \file{package.sty}{\from{\jobname.dtx}{p}}
}

\obeyspaces
\Msg{*************************************************************}
\Msg{*                                                           *}
\Msg{* To finish the installation you have to move the following *}
\Msg{* file into a directory searched by TeX:                    *}
\Msg{*                                                           *}
\Msg{*     package.sty                                           *}
\Msg{*                                                           *}
\Msg{* To produce the documentation run the following file       *}
\Msg{* through LaTeX:                                            *}
\Msg{*                                                           *}
\Msg{*     package.tex                                           *}
\Msg{*                                                           *}
\Msg{* Happy TeXing!                                             *}
\Msg{*                                                           *}
\Msg{*************************************************************}

\endbatchfile

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%</driver>                                                                     %
%<*d>                                                                          %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\ProvidesFile{package.tex}

\documentclass{ltxdoc}

\usepackage{package}[2012/11/24]

\EnableCrossrefs         
\CodelineIndex
\RecordChanges

\changes{v0.0.1}{2012/11/16}{initial version}
\changes{v0.0.2}{2012/11/24}{put the package into a .dtx file}

\GetFileInfo{package.sty}

\DoNotIndex{\newcommand,\def,\xdef,\edef,\gdef,\NewDocumentCommand}

\title{The \textsf{package} package\thanks{This document
  corresponds to \textsf{package}~\fileversion, dated \filedate.}}
\author{Author Name \\ \texttt{[email protected]}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{document}                                                               %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}                                                         %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Put text here.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Usage}                                                                %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Put text here.

\DescribeMacro{\dummyMacro}
This macro does nothing.

\DescribeEnv{dummyEnv}
This environment does nothing.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Implementation}\StopEventually{}                                      %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The following piece of code contains the package meta-info:

\begin{verbatim}
%</d>%%%%%%%%%%%%%%%%%%%%%%%

    \NeedsTeXFormat{LaTeX2e}[2011/06/27]
    \ProvidesPackage{package}
        [2012/11/24 v0.0.2 description of the 'package' package]

%<*d>%%%%%%%%%%%%%%%%%%%%%%%
\end{verbatim}

\begin{macro}{\dummyMacro}
    This is a dummy macro.

    \begin{verbatim}
%</d>   %%%%%%%%%%%%%%%%%%%%%%%

        \newcommand{\dummyMacro}{}

%<*d>   %%%%%%%%%%%%%%%%%%%%%%%
    \end{verbatim}
\end{macro}

\begin{environment}{dummyEnv}
    This is a dummy environment.

    \begin{verbatim}
%</d>   %%%%%%%%%%%%%%%%%%%%%%%

        \newenvironment{dummyEnv}{}{}

%<*d>   %%%%%%%%%%%%%%%%%%%%%%%
    \end{verbatim}

    Some more information here... Very useful stuff.
\end{environment}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Finale\end{document}                                                          %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Some remarks:

  • The '.ins' part is terminated with \endbatchfile, so everything that comes afterwards is ignored on the first run. It is also delimited with %<*driver>, so it is itself ignored on subsequent runs.
  • The copyright / licensing / file integrity stuff is automatically included in every generated file.
  • In pure literate programming style, I made the package code 'submissive' to the documentation by my use of indentation. But you can choose to indent differently.

The only thing I couldn't get working was the macrocode environment. Perhaps it depends on the standard docstrip setup somehow? It was too hard to decipher its definition. So I'm using verbatim here to typeset the package code, but any code-typesetting technique (such as listings) would work.

Any feedback would be appreciated!


Edit: The previous version didn't work because I indented the guards. The %<*bla> code cannot have whitespace to its left. :-( Oh well. I also just noticed that the verbatim environment keeps all whitespace. (But there are solutions to that.)


Here is the skeleton of the dtx of a package I have written and which will appear in a few days on CTAN (the current version on CTAN does not yet have this structure of the dtx file)

%<*none>
% This skeleton PKG.dtx file is one way to not have everything commented
% out in the documentation part. It produces the .sty file (and also an
% .ins file) when one does latex PKG.dtx.
%
% It *does not* use DocInput!
% 
% It is also possible to not produce the .sty file, and still be able to
% compile the doc, using some boolean flag, and doing an \input which
% will get only the code, the only thing is that the log file will
% complain that we have required package `' and that package `PKG' was
% used.
%
% Here we rather first output the .sty file, and then do a normal
% \usepackage 
%
% [I am of course assuming that compiling the doc for package PKG 
%  does require the use of PKG.sty]
%
\begingroup
\input docstrip.tex
\askforoverwritefalse
\generate{
\file{PKG.ins}{\from{PKG.dtx}{ins}}
\file{PKG.sty}{\from{PKG.dtx}{package}}
\nopreamble\nopostamble
\file{test.txt}{\from{PKG.dtx}{test}}}
\endgroup
\iffalse
%</none>
%<*ins>
\input docstrip.tex
\askforoverwritefalse
\generate{\file{PKG.sty}{\from{PKG.dtx}{package}}}
\endbatchfile
%</ins>
%<*test>
This is a another file which can be produced by the latex run
%</test>
%<*none>
\fi
\documentclass{ltxdoc}
% \OnlyDescription
\usepackage{PKG} % <- often needed to produce its own documentation!
\begin{document}
I am the best packaging package.

\makeatletter
\StopEventually{\check@checksum\end{document}}
\makeatother

\MakePercentIgnore

% \catcode`\<=0 \catcode`\>=11 \catcode`\*=11 \catcode`\/=11 
%
% \let</none>\relax
% \def<*package>{\catcode`\<=12 \catcode`\>=12 \catcode`\*=12 \catcode`\/=12}
%
%</none>
%<*package>
%    \begin{macrocode}
\ProvidesPackage{PKG}
 [2012/11/24 v1.04 easy packaging (jfB)]
%    \end{macrocode}
% Let us point out the importance of this piece of code with 31
% \cs{expandafter}'s
%    \begin{macrocode}
\endinput
%    \end{macrocode}
% \MakePercentComment
%</package>
\CharacterTable
 {Upper-case    \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z
  Lower-case    \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z
  Digits        \0\1\2\3\4\5\6\7\8\9
  Exclamation   \!     Double quote  \"     Hash (number) \#
  Dollar        \$     Percent       \%     Ampersand     \&
  Acute accent  \'     Left paren    \(     Right paren   \)
  Asterisk      \*     Plus          \+     Comma         \,
  Minus         \-     Point         \.     Solidus       \/
  Colon         \:     Semicolon     \;     Less than     \<
  Equals        \=     Greater than  \>     Question mark \?
  Commercial at \@     Left bracket  \[     Backslash     \\
  Right bracket \]     Circumflex    \^     Underscore    \_
  Grave accent  \`     Left brace    \{     Vertical bar  \|
  Right brace   \}     Tilde         \~}

\CheckSum{2}

\Finale
%%
%% End of file

The actual situation in my dtx file is more complicated, I simplified things here to get that skeleton. I agree that having all those % is indeed a problem, although with emacs there is a mode which does the LaTeX syntax highlighting nevertheless. So I was also interested in finding a way not to have all those %'s.

Oups, I forgot to say that the ins stuff is purely optional. It is only to produce an .ins file, but it is not used, as the .sty file itself is directly output.