Generating PDF without any intermediary files (stdin/stdout)

A (PDF|Xe)TeX run accesses much more files than only the document source and the PDF created. Apart from the executables (which already include the format) there are document classes, packages, fonts, and the program creates (apart from the output PDF) at least a log file, but most often also an .aux file, sometimes some more files (depending on packages and use) to be used on a second or later run.

So I don't think this is really doable.

My idea would be to put all those files on a special memory-only file system (with an own directory at least for each output to allow better caching).

If all your files use the same template, another option to reduce accessing multiple files would be to create a new format which includes everything common to all files.

If we can assume a UNIX-like system running the compilations, what about using mktemp (with or without a RAMdisk-type filesystem)?

#!/bin/sh
TDIR=`mktemp -d --tmpdir=/tmp`
cd /path/to/source
cp file.tex other.sty references.bib ${TDIR}
cd ${TDIR}
pdflatex file.tex
# plus any other commands required for bibtex, 
# to display the pdf file,
# or move it to its final destination
cd ..
rm -r ${TDIR}

On re-reading your post, if this is designed to stream the final PDF through a web server or similar, then the concept can still be the same, but you might have to find mktemp equivalents in Perl, Python, or whatever your server's CGI language is.

I've done this for myself (in accordance with the suggestions above:

#!/bin/bash -e

STATUS500="Status 500 internal server error"
STATUS404="Status 404 file not found"


function removeTempDirectory {
    [ -d "$DIR" ] && rm -Rf $DIR
}

function die {
    removeTempDirectory
    printf "$*\n\n" 1>&2
    trap - EXIT
    exit 1
}

PDFLATEX=`which pdflatex`
PDFLATEX_OPTIONS="-halt-on-error"
[ -x "$PDFLATEX"  ] || die $STATUS500

DIR=`mktemp -d 2>/dev/null || mktemp -d -t 'cgilatex'`
[ -d "$DIR" ] || mkdir -p $DIR || die $STATUS500

[ -f $PATH_TRANSLATED ] || die $STATUS404

$PDFLATEX -halt-on-error \
    -interaction=nonstopmode \
    -output-directory=$DIR \
    $PATH_TRANSLATED &> /dev/null || \
    die $STATUS500

printf "Content-Type: application/pdf\n\n"
cat $DIR/*.pdf

removeTempDirectory
exit

and the in httpd.conf (for apache):

Options +Indexes +ExecCGI
Action pdflate-action /cgilatex/cgilatex.cgi
AddHandler pdflate-action .tex

hope it helps