A set without the empty set

By definition, the empty set is a subset of every set, right?

Yes.

Then how would you interpret this set: $A\setminus\{\}$?

The set $A\setminus\{\}$ is the set of members of $A$ which are not members of $\{\}$. However, $\{\}$ has no members, so $A\setminus\{\}=A$.

On one hand it looks like a set without the empty set, on the other hand, the empty set is in every set...

If you wish to remove the empty set from $A$, you should do $A\setminus\{\{\}\}$.

On one hand it looks like a set without the empty set, on the other hand, the empty set is in every set...

The empty set is not a member of every set, it is a subset of every set. $A\subseteq B$ means that for all $x\in A$: $x\in B$. If $A=\{\}$, regardless of what kind of set $B$ is, this statement is always true. This is because there are no $x\in\{\}$.

Just write it $$ A\setminus\{\}=\Big\{a~\mid~ a\in A \text{ and } a \not \in \{\} \Big\} =A $$

The notation $A-\{\}$ roughly translates to "the set $A$ without the elements of $\{\}$." The difference is that the empty set is not an element of $A$ and this notation just means you're not removing any elements from your original set.