Is there a name for a group-like structure under a unary operation?

To me the right context seems to be that the set $M$ consisting of the differentiation operator and its powers is a monoid, and the set you gave is a set with the monoid $M$ acting upon it.

So I would advise you to see semigroup and monoid actions.

If $f$ is a function, and $D$ means "differentiate", and I change your set notation for an ordered 4-tuple (because I'll bet you want order to matter), then what you've got is

$$(D^0 f, D^1 f, D^2 f, D^3 f)$$

where $f$ is $\sin$. A dynamicist would say that you've identified a 4-cycle for the action of $D$ on (differentiable) functions. And as others have noted, there's a monoid structure at play here, because $D^n D^m = D^{n+m}$.

The set $S$ is a torsor for the cyclic group of order $4$.

Given a group $G$, a $G$-torsor is a nonempty $G$-set $T$ with the property that $$\forall x,y \in T \, \exists! g \in G \, gx=y.$$

For $$G=C_4=\{e,g,g^2,g^3\},$$ one can define an action of $G$ on $S$ by letting $g^nf$ be $f^{(n)}$ (the $n$th derivative of $f$, where the $0$th derivative of a function is simply the function itself) for $n \in \{0,1,2,3\}$ and $f \in \{\sin,\cos,-\sin,-\cos\}$.

One can easily check that this action does in fact make $S$ into a $G$-torsor.