How to create these nice operators?

It's encouraged to define macros for these types of things:

\documentclass{article}
\usepackage[export]{adjustbox}

\newcommand{\dstar}{\mathbin{{\ooalign{${<}\mkern-3.5mu{>}$\cr\hidewidth\adjustbox{raise=-.4ex,scale=.75}{*}\hidewidth}}}}
\newcommand{\ddollar}{\mathbin{{\ooalign{${<}\mkern-2mu{>}$\cr\hidewidth\adjustbox{scale=.7}{\$}\hidewidth}}}}
\newcommand{\ddollarl}{\mathbin{{<}\mkern-4mu\adjustbox{scale=.7}{\$}}}
\newcommand{\ddollarr}{\mathbin{\adjustbox{scale=.7}{\$}\mkern-4mu{>}}}

\begin{document}

$f \dstar x$

$f \ddollar x$

$f \ddollarl x$

$f \ddollarr x$

\end{document}

Note that the above macros will be sufficient for inline \displaystyle and \teststyle use, but not for sub-/superscripts (\scriptstyle and smaller).

Choose better names. Depending on the fonts used some adjustment might be necessary.

\documentclass{article}
\usepackage{amsmath}

\newcommand{\rstar}{
  \mathbin{
    <
    \mathrel{
      \mspace{-1.7mu}
      \vcenter{
        \hbox to 0pt{\hss$\scriptstyle*$\hss}
      }
      \mspace{-1.7mu}
    }
    >
  }
}
\newcommand{\rdollar}{
  \mathbin{
    <
    \mathrel{
      \mspace{-1mu}
      \vcenter{
        \hbox to 0pt{\hss$\scriptstyle\$$\hss}
        \kern0.005ex
      }
      \mspace{-1mu}
    }
    >
  }
}
\newcommand{\ldollar}{
  \mathbin{
    <
    \mathrel{
      \mspace{-1mu}
      \vcenter{
        \sbox0{$\scriptstyle\$$}
        \hbox to .5\wd0{\hss\box0}
        \kern0.005ex
      }
    }
  }
}
\newcommand{\gdollar}{
  \mathbin{
    \mathrel{
      \vcenter{
        \sbox0{$\scriptstyle\$$}
        \hbox to .5\wd0{\box0\hss}
        \kern0.005ex
      }
      \mspace{-1mu}%
    }
    >
  }
}

\begin{document}

$f\rstar x$

$f\rdollar x$

$f\ldollar x$

$f\gdollar x$

\end{document}

^{Look, ma! No percent!}

I think your second "very hacky" approach is not too bad at all. Instead of using \! for a negative space of fixed width, I used \hspace with a negative size argument. Also, I had to use \scalebox to shrink the dollar sign, which was taller than the angle braces. Here's the code:

\documentclass[border=10pt]{standalone}

\usepackage{mathtools}

\begin{document}

$<\hspace{-3.6pt}\mathclap{*}\hspace{-3.6pt}>$
$<\hspace{-3.6pt}\mathclap{\scalebox{.71}{\$}}\hspace{-3.6pt}>$
$<\hspace{-3.6pt}\mathclap{\scalebox{.71}{\$}}$

\end{document}

And here's the result:

Note the dollar sign isn't quite exactly the same font as you have. Also, to use <$ in text, you may want to add a horizontal space (for example with \) after it to make everything look nice.