Fundamental differences : PSTricks, TikZ/PGF and others
PostScript is a complete programming language, and as such is very powerful. To convert PostScript into output, you need an appropriate interpreter: the most common one in use today is GhostScript. The PDF format is related to PostScript, but via the more limited subset provided by the EPS (Encapsulated PostScript) format. As such, PDF viewers do not have the ability to directly render PostScript: it must be converted into the correct PDF directives. (PDFs are not programs in that sense: one of the reasons for the creation of the PDF format is that it is a 'known quantity'. A PostScript document may require arbitrary amounts of complex calculations to display, which may be an issue for some devices and workflows.)
TeX does not include a PostScript interpreter. In the dvips route to creating a PDF from a .tex source, TeX writes the PostScript instructions to the .dvi file as \special instructions, which are then interpreted by GhostScript in the ps2pdf step.
The same is true for newer TeX engines: a PostScript interpreter is a large overhead, and integrating directly into say pdfTeX would not therefore be particularly sensible. In direct PDF mode, pdfTeX creates the appropriate PDF instructions for creating 'special effects', while in .dvi mode it acts in the same way as Knuth's TeX. Some of the issues here are covered in Why doesn't pdfTeX support PStricks directly? and How to use PSTricks in pdfLaTeX?. (Note that XeTeX does take a somewhat hybrid route, which allows PSTricks use with direct PDF production. It's important to note also that XeTeX works via an extended .dvi format, and so this is in some ways more like dvips than direct PDF production.)
Turning to PSTricks and TikZ, the differences here are pretty fundamental. PSTricks always uses the same driver back-end, and that back-end is a powerful programming language in its own right. As such, PSTricks can leave a lot of work to the driver level, and also can in principal include an interface for anything that PostScript can do. On the other hand, TikZ is a 'driver independent' system, and as such leaves the driver-dependent code to the lowest 'layer'. TikZ therefore cannot use any special features of PostScript: all of the programming is done in TeX macros, and only the raw 'draw X' instructions are written to the output file.
The fact that TikZ supports multiple drivers also means that there are limitations on what can be implemented. Each driver has a certain range of capabilities, and supporting all drivers is not necessarily easy. (For example, transparencies are difficult with XeTeX due to the way xdvipdfmx works.) As such, it tends toward a common subset of features which work with most or all of the drivers, rather than having lots of 'this only works with X' comments.
A similar consideration applies to interfaces: PSTricks can rely on the way PostScript handles input, while TikZ has to abstract any features to a generic interface that works for PostScript, PDF and other output formats. (Remember TikZ works to some extend with SVG and so on.)
As Joseph already said, PDF is a subset of PostScript, but with some more new features. The biggest limitation of PDF is that it cannot do any calculation of values. And this is the only important difference to PostScript. TeX doesn't know anything about graphical elements. Everything has to be done with specials, called \pdfliteral for PDF or \special for PostScript or \directlua for LuaTeX. An example for pdflatex and latex:
\documentclass{article}
\usepackage{ifpdf}
\begin{document}
\ifpdf
foo\pdfliteral{1 0 0 RG 2 w 0 -1 m 10 10 l 10 0 l S }bar
\else
foo\special{" 1 0 0 setrgbcolor 2 setlinewidth 0 -1 moveto
10 10 lineto 10 0 lineto 3.14 3.14 exp 0 rlineto stroke }bar
\fi
\end{document}
Running it with pdflatex :
The lines must be defined by a special command, called \pdfliteral. Now we want extend the line just for fun to (ππ,0). There is no chance on PDF level to get the value of ππ. For PostScript it is no problem we have to write 3.14 3.14 exp and get the value. When running the example with latex->dvips->psp2df we get
With PSTricks I can create animations of a pendulum (see http://pstricks.blogspot.de/, in the middle of the page).
Edit: This is a simple example where ππ could be calculated on TeX level with the different math packages, but that is not possible if I want to draw a line to the value of \int_1^{27/π} (x^{π/7}+1/x) dx ...
Since I don't use tikz and haven't used pstricks for 20 years or so, I'll not talk in detail about either of those but about a package that I do know rather more about, namely psfrag which has many of the same issues.
psfrag replaces text in an image by TeX typeset text. However the way it does this relies entirely on the programming features of PostScript. TeX has no knowledge of the replacement, this makes it rather hard to support the package with any other back end.
The way psfrag works is that TeX typesets the "new" text off the page somewhere and the PostScript show operator that typesets a string is redefined to check its supplied string and if it is a string for which a replacement is requested, the latex version is translated into place. The position of the original text string in the included graphic might be calculated by arbitrarily complex postscript code. Originally this whole replacement would the happen in the printer although as @Joseph notes these days it probably more often happens in ghostscript ps2pdf. What is possible of course if using pdflatex is that you use the psfrag syntax but essentially you just write out a small document with just the image and use ps2pdf to generate a pre-computed pdf image with the replacement and include that into pdflatex. I believe there are packages which more or less hide the details these days.
pstricks has all the issues of psfrag, plus extra. If it is being used to generate a postscript-programmed figure, then it is possible to arrange things such that the appropriate postscript is written to a file, converted to pdf and included back into pdflatex. However pstricks can do much more, it can (essentially) apply postscript operators to the text stream being typeset and apply various effects, similarly it can pass positional information from the typesetting to the postscript engine. This makes it much harder (I would guess) to be able to make small "encapsulated" fragments of postscript to be included as self standing fragments of pdf back into the main document.
The whole point of pdf is that it implements the rendering model of postscript but without the computational cost. The original distiller was a postscript to postscript conversion that "distilled" complicated postscript to a simple subset of predefined postscript commands (understood by Adobe Illustrator) that set of predefined commands got normalised and standardised and in the end changed and pdf became a separate language.
For example you can draw the Mandelbrot set in about two lines of postscript that would tie up an early apple laserwriter for an hour. The equivalent pdf would take ages to distill but produce a (much bigger) pdf that essentially was just a big list of lines and points that renders in no time at all.
Thus the lack of computational power in a PDF back end compared to a PostScript back end is not a design failing; it was the main design criterion for PDF as a format. Any particular functionality one can produce in a non-postscript based PDF generator by implementing that functionality in the generator (that is, in TeX or lua or whatever) but to implement the full power of pstricks you would have to fully implement postscript and that is a major undertaking.