Are the integers closed under addition... really?
When it is said that "$X$ is closed under binary operation $\circ$", it means that for any $a$ and $b$ in $X$, $a \circ b$ is in $X$. It is easy to prove (by a simple induction) that any finite sum is therefore closed in $X$.
However, infinite sums are defined with a limit (of the partial sums), which means they don't just depend on the operation $\circ$, but also require a topological structure defined on $X$. Now the integers $\mathbb{Z}$ do have a standard topological structure in addition to their algebraic structure, it's the discrete topology, and it comes from the order on $\mathbb{Z}$. However, in this system, there is actually no limit of the sequence of partial sums $1$, $1 - 2$, $1 - 2 + 3$, ... (*) and so no infinite sum. In fact, an infinite sum of integers can only have a limit if all but finitely many of its terms are $0$. Another subtle flaw is that when you took a "derivative", that means you passed from $\mathbb{Z}$ to $\mathbb{R}$, and evaluated a function on $\mathbb{R}$ on the right side, to obtain a "sum" for the left (which may be a valid technique, giving a form of "divergent summation", but it's important to remember this is actually a generalization beyond the usual and strict "limit" definition of infinite summation). So you left the integers, and thus it is no surprise you'd get a non-integer result. The important point to remember though is that infinite sums and finite sums are not the same thing -- one is purely algebraic, the other leverages additional (topological) structure on the set in question, and closure is strictly algebraic.
Finally, it's important to note that this series doesn't have a sum in $\mathbb{R}$ either in the strict, limit sense.
Summation is defined to be finite. Infinite series are just limit of a sequence which is defined as partial sums.
The integers, if so, are closed under finite sums. And therefore by definition, closed under summation.
The series representation $$ 1+z+z^2+\ldots=\frac{1}{1-z} $$ is only valid for $|z|<1$ (in the sense that the left-hand side converges only in that region of the complex plane). So you can't assert this equality (or any of its derivatives) at $z=-1$: the sum $1-1+1-1+\ldots \neq 1/2$, at least not without an agreed-upon convention for the meaning or value of a non-convergent series. The problem doesn't have to do with infinity in this case, and the closure of $\mathbb{Z}$ is safe: the sum of an infinite series of integers will be an integer if the sum converges at all. (Of course, it can only converge if there are only finitely many nonzero terms.) In other cases, though, a group that is closed under finite summation may well not be closed under "infinite summation" (e.g., $\mathbb{Q}$ doesn't contain all of its limit points); you could reasonably say that "infinity is the problem" there.