Does anybody know a good source of (free) ornaments, decorations, frames, backgrounds etc.?

Now with pgfornament.sty. The idea comes from the psvectorian package of Patrick Fradin and a lot of things are inpired from it. I added another package tikzrput to emulate the macro \rput from pstricks.

You can find it here

Update

I made a new version of the example e02 from psvectorian that you can see in Herbert's answer. In this example I used the rputversion from pgf and some style to see how to fill the symbol.

enter image description here

\documentclass{scrartcl} 
\usepackage[utf8]{inputenc} 
\usepackage[T1]{fontenc} 
\usepackage[dvipsnames]{xcolor}
\usepackage{tikzrput}
\usepackage[object=vectorian]{pgfornament} 
\begin{document}  
\tikzset{pgfornamentstyle/.style={draw = Periwinkle,
                                  fill = SpringGreen}}   
\unitlength=1cm   

\begin{center}   
\begin{picture}(10,10)%
  \color{blue}%
   \put(0,0){\framebox(10,10){%
   \rput[tl](-3,5){\pgfornament[width=6cm]{71}}%
   \rput[bl](-3,-5){\pgfornament[width=6cm,,symmetry=h]{71}}%
   \rput[tl](-5,5){\pgfornament[width=2cm]{63}}%
   \rput[tr](5,5){\pgfornament[width=2cm,,symmetry=v]{63}}%
   \rput[bl](-5,-5){\pgfornament[width=2cm,,symmetry=h]{63}}%
   \rput[br](5,-5){\pgfornament[width=2cm,,symmetry=c]{63}}%
   \rput[bl]{-90}(-5,3){\pgfornament[width=6cm]{46}}%
   \rput[bl]{90}(5,-3){\pgfornament[width=6cm]{46}}%
   \rput(0,0){\Huge \color{MidnightBlue} Ornaments}%
   \rput[t](0,-0.5){\pgfornament[width=5cm]{75}}%
   \rput[b](0,0.5){\pgfornament[width=5cm]{69}}%
   \rput[tr]{-30}(-1,2.5){\pgfornament[width=2cm]{57}}%
   \rput[tl]{30}(1,2.5){\pgfornament[width=2cm,symmetry=v]{57}}}}% 
\end{picture} 
\end{center}
\end{document} 

Another example

enter image description here

\documentclass{scrartcl}
\usepackage[utf8]{inputenc} 
\usepackage[T1]{fontenc} 
\usepackage[dvipsnames]{xcolor} 
\usepackage[object=vectorian]{pgfornament}
\usetikzlibrary{calc}
\definecolor{fondpaille}{cmyk}{0,0,0.1,0}

\begin{document}
\pagecolor{fondpaille}
\color{Maroon} 
\begin{tikzpicture}[every node/.style={inner sep=0pt}]   
\node[text width=8cm,align=center](Text){%
In visions of the dark night\\
I have dreamed of joy departed-\\
But a waking dream of life and light    Hath left me broken-hearted.\\
\bigskip
Ah! what is not a dream by day\\
To him whose eyes are cast \\
On things around him with a ray \\
Turned back upon the past? \\
\bigskip        
That holy dream- that holy dream,\\
While all the world were chiding,\\
Hath cheered me as a lovely beam\\
A lonely spirit guiding.\\
\bigskip        
What though that light, thro' storm and night,\\
So trembled from afar- \\
What could there be more purely bright \\
In Truth's day-star? \\
\vspace{24pt}
 A Dream  (1827) by Edgar Allan Poe 
} ;
\node[shift={(-1cm,1cm)},anchor=north west](CNW)  at (Text.north west)
                 {\pgfornament[width=2cm]{61}};
\node[shift={(1cm,1cm)},anchor=north east](CNE)   at (Text.north east)
                 {\pgfornament[width=2cm,symmetry=v]{61}}; 
\node[shift={(-1cm,-1cm)},anchor=south west](CSW) at (Text.south west)
                 {\pgfornament[width=2cm,symmetry=h]{61}}; 
\node[shift={(1cm,-1cm)},anchor=south east](CSE)  at (Text.south east)   
                 {\pgfornament[width=2cm,symmetry=c]{61}};  
\pgfornamenthline{CNW}{CNE}{north}{87}
\pgfornamenthline{CSW}{CSE}{south}{87}
\pgfornamentvline{CNW}{CSW}{west}{87}
\pgfornamentvline{CNE}{CSE}{east}{87} 
\end{tikzpicture}
\end{document} 

with the PSTricks package psvectorian you can create something like this:

enter image description here

\documentclass[]{book}
\usepackage[svgnames]{pstricks}
\usepackage{psvectorian}
\begin{document}

\begin{pspicture}(-5,-5)(5,5)
\renewcommand*{\psvectorianDefaultColor}{blue}%
\psframe[linewidth=0.4pt,fillstyle=solid,fillcolor=Beige](-5,-5)(5,5)%
%
\rput[tl](-3,5){\psvectorian[width=6cm]{71}}
\rput[bl](-3,-5){\psvectorian[width=6cm,flip]{71}}
%
\rput[tl](-5,5){\psvectorian[width=2cm]{63}}
\rput[tr](5,5){\psvectorian[width=2cm,mirror]{63}}
\rput[bl](-5,-5){\psvectorian[width=2cm,flip]{63}}
\rput[br](5,-5){\psvectorian[width=2cm,flip,mirror]{63}}
% 
\rput[bl]{-90}(-5,3){\psvectorian[width=6cm]{46}}
\rput[bl]{90}(5,-3){\psvectorian[width=6cm]{46}}
%
\rput(0,0){\Huge Ornaments}
\rput[t](0,-0.5){\psvectorian[width=5cm]{75}}
\rput[b](0,0.5){\psvectorian[width=5cm]{69}}
%
\rput[tr]{-30}(-1,2.5){\psvectorian[width=2cm]{57}}
\rput[tl]{30}(1,2.5){\psvectorian[width=2cm,mirror]{57}}
\end{pspicture}

\end{document} 

You can use web-o-mints and put variety of ornamental letters as borders. Some examples:

\PassOptionsToPackage{usenames,x11names,dvipsnames,svgnames}{xcolor}
\documentclass[]{scrartcl}
\usepackage{tikz}
\usetikzlibrary{calc,positioning}
\usepackage{geometry}
\geometry{a4paper,tmargin=1in,bmargin=1in,
lmargin=1in,rmargin=1in,heightrounded}
%
\newcommand\deco[3][black]{%
  \par\vspace{1ex}
  %\begin{center}
  \textcolor{#1}{%
  \fontsize{#2}{#2}\usefont{U}{webo}{xl}{n}#3
  }
  %\end{center}
  \vspace*{1ex}\par
}
%
\begin{document}
\begin{tikzpicture}[remember picture,overlay]
 \node[rotate=90,align=center] at ($(current page.west)+(.8cm,0.14cm)$){{\deco[DarkGreen]{15.1pt}{ NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN}}};
 \node[rotate=90,align=center] at ($(current page.east)+(-.8cm,0.14cm)$){{\deco[DarkGreen]{15.1pt}{ \rotatebox[origin=c]{180}{NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN}}}};
 \node[rotate=0,align=center] at ($(current page.north)+(0.1cm,-.8cm)$){{\deco[DarkGreen]{14.35pt}{\rotatebox[origin=c]{0}
 {NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN\hspace{2pt}}}}};
 \node[rotate=0,align=center] at ($(current page.south)+(0.1cm,.9cm)$){{\deco[DarkGreen]{14.35pt} {\rotatebox[origin=c]{180}{\hspace{2pt}NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN}}}};
 \draw [rounded corners=8pt,line width=4pt,draw=DarkRed]($(current page.south west)+(.4cm,.5cm)$) rectangle ($(current page.north east)+(-.4cm,-.39cm)$);
\end{tikzpicture}

\end{document}

enter image description here

Second example (with background):

\PassOptionsToPackage{usenames,x11names,dvipsnames,svgnames}{xcolor}
\documentclass[]{scrartcl}
\usepackage{wallpaper}
\usepackage{tikz}
\usetikzlibrary{calc,positioning}
\usepackage{geometry}
\geometry{a4paper,tmargin=1in,bmargin=1in,
lmargin=1in,rmargin=1in,heightrounded}

\newcommand\deco[3][black]{%
  \par\vspace{1ex}
  \textcolor{#1}{%
  \fontsize{#2}{#2}\usefont{U}{webo}{xl}{n}#3
  }
  \vspace*{1ex}\par
}

\begin{document}

\TileWallPaper{4cm}{4cm}{N-00.png}% 
\begin{tikzpicture}[remember picture,overlay]
 \node[rotate=90,align=center] at ($(current page.west)+(.8cm,0.14cm)$){{\deco[DarkGreen]{15.1pt}{ \rotatebox[origin=c]{0}{LKIJLKIJLKIJLKIJLKIJLK}}}};
 \node[rotate=90,align=center] at ($(current page.east)+(-.8cm,0.14cm)$){{\deco[DarkGreen]{15.1pt}{ \rotatebox[origin=c]{180}{LKIJLKIJLKIJLKIJLKIJLK}}}};
 \node[rotate=0,align=center] at ($(current page.north)+(0.1cm,-.8cm)$){{\deco[DarkGreen]{14.25pt}{\rotatebox[origin=c]{0}
 {LKIJLKIJLKIJLKIJ\hspace{2pt}}}}};
 \node[rotate=0,align=center] at ($(current page.south)+(0.1cm,.9cm)$){{\deco[DarkGreen]{14.25pt} {\rotatebox[origin=c]{180}{\hspace{2pt}LKIJLKIJLKIJLKIJ}}}};
 \draw [rounded corners=8pt,line width=4pt,draw=DarkRed]($(current page.south west)+(.4cm,.5cm)$) rectangle ($(current page.north east)+(-.4cm,-.39cm)$);
\end{tikzpicture}
This is my text.
\end{document} 

enter image description here

These take at least two compilation runs to settle down. Some manual adjustments will be necessary to get the finishing touch. (We call it procrastination ;-)). The background image N-00.png is here:

enter image description here

Niceframe:

Another options that is not mentioned here is niceframe package.:

\PassOptionsToPackage{usenames,x11names,dvipsnames,svgnames}{xcolor}
\documentclass[landscape]{article}
\usepackage{wallpaper}
\usepackage{niceframe}
\usepackage{geometry}
\geometry{a4paper,tmargin=.5cm,bmargin=.5cm,
lmargin=.5cm,rmargin=.5cm}


\begin{document}

\vspace*{-5.05mm}
\hspace*{-6mm}
\centering
\scalebox{3.015}{\color{DarkGreen} %% green!30!red!80!blue!80
\begin{minipage}[.329835\textheight]{.339835\textwidth}
%\font\border=umrandb
%\generalframe{\border\char'165}{\border\char'151}{\border\char'164}%
%             {\border\char'150}                  {\border\char'150}%
%             {\border\char'166}{\border\char'151}{\border\char'167}
\font\border=umranda
\generalframe{\border\char'136}{\border\char'137}{\border\char'140}
             {\border\char'145}                  {\border\char'141}
             {\border\char'144}{\border\char'143}{\border\char'142}
{\centering
\vspace*{1.145cm} %% Change this for vertical alignment
\vspace{44.8mm}} %% Change this for vertical alignment
\end{minipage}
}
\end{document}

enter image description here