Riemann rearrangement theorem
A complex series converges if and only if the real and imaginary parts converge, and an identical statements holds when taking absolute values. Then if $\sum_{n=1}^{\infty} c_{n}$ is convergent, so are $\sum_{n=1}^{\infty} a_{n}$ and $\sum_{n=1}^{\infty} b_{n}$ where $c_{n} = a_{n} + ib_{n}$. If $\sum_{n=1}^{\infty} |c_{n}|$ diverges, at least one of $\sum_{n=1}^{\infty} |a_{n}|$ of $\sum_{n=1}^{\infty} |b_{n}|$ diverges, and suppose without loss of generality that only one converges, and that it is the former. Then we can force the real part of our series, $\sum_{n=1}^{\infty} a_{n}$ to converge to whatever we want by the original Riemann Rearrangement Theorem. If $\sum_{n=1}^{\infty} |b_{n}|$ converges, we might be stuck with a fixed sum for the complex part, but we can still hit the entire horizontal line through $i\sum_{n=1}^{\infty} b_{n}$.
If both the complex and real parts are conditionally convergent, the whole situation becomes more complicated...
This is a really nice question (+1)! It's a very beautiful result, despite of the a-bit-lengthy proof.
So it really intrigues me, and I spent some time and found this paper gave answers to it:
P. Rsenthal, The remarkable theorem of Levy and Steinitz, Amer. Math. Monthly 94 (1987) 342-351.
According to the paper, the proof was made early 20th Century. The purpose of that paper was to "make this beautiful result more widely known". I put down the major part of the proof from the paper below, and anyone who's interested in details or extra discussions could check the paper directly. The proof is LONG, but each step itself is crystal clear; plus I put additional explanation wherever I feel needed.
In sum, to answer the question asked here: the set of all sums of rearrangements of a given series of complex numbers is (1) the empty set, or (2) a single point, or (3) a line in the complex plane, or (4) the whole complex plane. This result is from the $\mathbb R^2$-case of The Levy-Steinitz Theorem, i.e. those are all a translate of a subspace of $\mathbb R^2$.
Formal statement of THE LEVY-STEINITZ THEOREM:
The set of all sums of rearrangements of a given series of vectors in a finite-dimensional real Euclidean space $\mathbb R^n$ is either the empty set or a translate of a subspace, i.e. a set of the form $v + M$, where $v$ is a given vector and $M$ is a linear subspace.
Here comes the complete proof.
Step 1: THE POLYGONAL CONFINEMENT THEOREM
Polygonal Confinement Theorem: For each dimension n there is a constant $C_n$ s.t. whenever $\{v_i: i = 1, . . ., m \}$ is a finite family of vectors in $\mathbb R^n$ which sums to $0$ and satisfies $\|v_i\|<1\, \forall i$, there is a permutation $P$ of $(2,.. ., m)$ with the property that $${\Bigg\| v_1+\sum_{i=2}^j v_{P(i)} \Bigg\| \le C_n}$$ for every j. Moreover, we can take $C_1=1$ and $C_n \le \sqrt{4C_{n-1}^2+1},\, \forall\,n >1.$
Proof. The case n = 1 is easy. If, for example, $v_1 > 0$, we can choose $P(2)<0$, and keep choosing negative $v's$ until the sum of all the chosen vectors becomes negative. Then choose the next v to be positive, and keep choosing positive v's until the sum of all the chosen vectors becomes positive. Continue in this manner until all the v's are used. Since $\|v_i\| \le 1$ for all i, it is clear that each partial sum in this arrangement is within distance $1$ of $0$. Hence, $C_1 = 1$.
The general case is proven by induction. Assume that $n > 1$ and that $C_{n-1}$ is known to be finite, and consider a collection $\{v_i\}$ of vectors satisfying the hypotheses.
Since $\{v_i\}$ is finite there are a finite number of possible partial sums of the $v's$ that begin with $v_1$; let $L$ be such a partial sum with maximal length among all such partial sums. Then $L = v_1 + u_1 + ... +u_s$, where $\{u_l,..., u_s\} \subset \{v_i\}$. Let $\{w_1,..., w_t\}$ denote the other $v's$, so that $L + w_1 + ... + w_t= 0$.
We use the notation $(u|v)$ to denote the Euclidean inner product of $u$ and $v$.
We begin with a proof that the $\{u_i\}$ point in the same general direction as $L$, while the $\{w_i\}$ point in the opposite direction.
Claim (a): $(u_i|L) \ge 0,\, \forall i$. Suppose that $(u_i|L) < 0$, for some $i$. Then $$\Bigg( (L-u_i)\Bigg| \frac{L}{\|L\|} \Bigg)=\|L\|-\frac{1}{\|L\|}(u_i|L)>\|L\|$$ So $\|L-u_i\|>\|L\|$, which contradicts the assumption that $L$ is a longest such partial sum.
Claim (b): $(v_1|L) \ge 0$. For if $(v_1|L) < 0$, then $$\bigg(\frac{-L}{\|L\|} \bigg | (v_1+w_1+...+w_t) \bigg)=\bigg(\frac{-L}{\|L\|} \bigg | (v_1 - L) \bigg )= \|L\|-\frac{1}{\|L\|}(L|v_1)>\|L\|$$ so $v_1+w_1+...+w_t$ would be longer than $L$ - contradiction.
Claim (c): $(w_i|L) \le 0$ for all i. For if there was an $i$ with $(w_i|L) > 0$, then $$\bigg((L+w_i)\bigg|\frac{L}{\|L\|}\bigg)=\|L\|+\frac{(w_i|L)}{\|L\|}>\|L\|$$ Therefore, $\|(L+w_i)\|>\|L\|$. But $\|L+w_i\|$ is the length of a partial sum of the required kind. Thus this contradicts with $\|L\|$ being the longest.
Now, we use the inductive hypothesis in the $(n - l)$-dimensional space. $$L^{\perp}=\{v \in \mathbb R^n:(v|L)=0\}$$
We let $v'$ denote the component of a vector $v$ in $L^{\perp}$, i.e. $$v'=v-\frac{(v|L)}{(L|L)}L$$ Then $L=v_1+u_1+...+u_x$, implies $v_1'+u_1'+...+u_x'=0$. For a similar reason, $w_1'+...+w_t'=0$. By the induction hypothesis, there exists a permutation $Q$ of $(1,2,...,s)$ such that $$\bigg \| v_1' + \sum_{i=1}^{j}u{Q(i)}'\bigg \| \le C_{n-1},\, j = 1, 2, ..., s\,(*)$$ and there exists a permutation $R$ of $(2,...,t)$ such that $$\bigg \| w_1'+\sum_{i=2}^{j}w_{R(i)'}\bigg \| \le C_{n-1 },\, j=2,3,...,t\, (**)$$
Define $R(1)=1$.
Now the idea is to keep the above orders within the $u's$ and $w's$ (which will keep the components in $L^{\perp}$ of partial sums from being too large) and alternately "feed in" $u's$ and $w's$ to keep the components along $L$ of length at most 1 (as in the proof of the case n = 1).
More precisely, since $(v_1|L) \ge 0$ and $(w_i|L) \le 0$, we can choose a smallest $r$, say $r_1$, such that $$(v_1|L) + \sum_{i=1}^{r_1}(w_{R(i)}|L) \le 0$$ Then choose a smallest $s_1$ such that $$(v_1|L) + \sum_{i=1}^{r_1}(w_{R(i)}|L) + \sum_{i=1}^{s_1}(w_{R(i)}|L) \le 0$$ Then choose a smallest $r_2$ such that $$(v_1|L) + \sum_{i=1}^{r_1}(w_{R(i)}|L) + \sum_{i=1}^{s_1}(w_{R(i)}|L) + \sum_{i=r_1+1}^{r_2}(w_R(i)|L)\le 0$$ And so on. Arrange the vectors $\{v_i\}$ in the order of $$(v_1, w_{R(1)},...,w_{R(r_1)}, u_{Q(1)},...,u_{Q(s_1)},w_{R(r_1+1)},...,w_{R(r_2)},...)$$ In this arrangement, clearly the components along the direction of $L$ of each partial sum have norm at most $1$. The choice of the arrangements $Q$ and $R$ by the induction hypothesis insures that the components orthogonal to $L$ of the partial sums have norms at most $C_{n-1} + C_{n-1}$ (By *, and **). Hence, the norm of each partial sum is at most $\sqrt{(2C_{n-1})^2+1}$. Q.E.D.
Step 2: THE REARRANGEMENT THEOREM
First, the lemma below is a consequence of the Polygonal Confinement Theorem.
Lemma 1. If $\{v_i: i = 1,...,m\}\subset \mathbb R^n$ and $\|\sum_{i=1}^{m}v_i\| \le \epsilon$ for all $i$, then there is a permutation $P$ of $(1,2,...,m)$ such that $$\|v_{P(1)}+v_{P(2)}+...+v_{P(r)}\| \le \epsilon(C_n + 1),\,1 \le r \le m$$
Proof. Define $v_{m+1}=-v_1-...-v_m$ so that $\sum_{i=1}^{m+1}v_i=0$. By the Polygonal Confinement Theorem, there is a permutation $P$ of (2,...,m+1) such that $$\bigg\| \frac{1}{\epsilon}v_1 + \sum_{i=2}^{r}\frac{1}{\epsilon}v_{P(i)} \bigg\|\le C_n$$ for all r.
Then $\|v_1 + \sum_{i=1}^rv_{P(i)}\| \le \epsilon C_n$ for all r. Let $P(1)=1$.
Now order the $\{v_i\}$ according to $P$, but omit $v_{m+1}$; since $\|v_{m+1}\| \le \epsilon$ this omission changes the norms of the partial sums by at most $\epsilon$. Hence in this rearrangement, all the partial sums have norm at most $\epsilon C_n + \epsilon$. This proves the Lemma.
The Rearrangement Theorem. In $\mathbb R^n$, if a subsequence of the sequence of partial sums of a series of vectors converges to $S$, and if the sequence of terms of the series converges to $0$, then there is a rearrangement of the series that sums to $S$.
Proof. Let $\{v_i\}_{i=1}^{\infty}$ be a sequence of vectors in $\mathbb R^n$. For each m let $S_m=\sum_{i=1}^mv_i$. We assume that $\{S_{m_k}\} \rightarrow S$ for some subsequence $\{S_{m_k}\}$, and we must show how to rearrange the $\{v_i\}$ so that the entire sequence of partial sums converges to $S$. The ideas is to use Lemma 1 to obtain rearrangements of each of the families $(v_{m_k+1},...,v_{m_{k+1}-1})$ so that all the partial sums of these families are small. Then $S_m$ is close to $S_{m_k}$ if m is between $m_k$ and $m_{k+1}$.
Let $\delta_k=\|S_{m_k}-S\|$; then $\{\delta_k\} \rightarrow 0$. Now $$\bigg\| \sum_{i=m_k+1}^{m_{k+1}-1}v_i \bigg\|=\bigg\| \sum_{i=1}^{m_{k+1}}v_i-\sum_{i=1}^{m_k}v_i-v_{m_{k+1}} \bigg\| < \delta_{k+1} + \delta_k + \|v_{m_{k+1}}\|$$ For each $k$ let $$\epsilon_k=max\{\delta_{k+1}+\delta_k, sup\{\|v_i\|: i \ge m_k\}\}$$ Then $\{\epsilon_k\}\rightarrow 0$, and $$\bigg\| \sum_{i=m_k+1}^{m_{k+1}-1}v_i \bigg\| < 2 \epsilon_k$$
By Lemma 1, for each $k$ there is a permutation $P_k$ of $(m_k+1,...,m_{k+1}-1)$ such that $$\bigg\| \sum_{i=m_k+1}^{r}v_{P_k(i)} \bigg\| \le 2\epsilon_k(C_n+1)$$ for $r=m_k+1,...,m_{k+1}-1$
Now arrange the $\{ v_i\}$ as follows. Keep $v_{m_k}$ in position $m_k$ for each $k$. Then order the $v_i$ for $(m_k + 1) \le i \le (m_{k+1}-1)$ according to $P_k$. In this arrangement, if $m_k+1 \le m \le m_{k+1}-1$ then $S_m - S_{m_k}$ is a sum of the form $\sum_{i=m_k+1}^{m}v_{P_k(i)}$ with $m<m_{k+1}$, and hence has norm at most $2\epsilon_k(C_n+1)$. Since $\{S_{m_k}\}\rightarrow S$ and $\{\epsilon_k\} \rightarrow 0$, it follows that $\{S_m\}\rightarrow S$. Q.E.D.
Step 3 (Final Step): THE LEVY-STEINITZ THEOREM
We need another consequence of the Polygonal Confinement Theorem as below.
Lemma 2: If $\{v_i\}_{i=1}^m \subset \mathbb R^n,\, w=\sum_{i=1}^mv_i,\,0 < t < 1$, and $\|v_i\| \le \epsilon$ for all $i$, then either $\|v_1-tw\| \le \epsilon\sqrt{C_{n-1}^2+1}$ or there is a permutation $P$ of $(2,3,...,m)$ and an $r$ between $2$ and $m$ such that $\|v_1+\sum_{i=2}^{r} v_{P(i)}-tw\| \le \epsilon\sqrt{C_{n-1}^2+1}$. (Actually the two scenarios could be unified as permutations of $\{v_{P(i)}\}_{i=1}^r$ )
Proof. Suppose $w \ne0$ (otherwise the result is trivial). Consider the case $n=1$. By multiplication through $-1$ if necessary, we can assume that $w>0$ (since $n=1$, $w$ is a real number); let $s$ denote the smallest $i$ such that $$v_1+v_2+...+v_i > tw$$ Then since $$v_1+v_2+...+v_{s-1} \le tw$$ and $|v_s|\le \epsilon$, it follows that $$|v_1+v_2+...+v_s-tw|\le \epsilon\,(***)$$ Thus in the case $n=1$, the Lemma holds with $C_{n-1}=C_0$ being defined to be $0$. Note also that, in the case $n=1$, no rearranging is necessary to get an appropriate partial sum.
Now consider the general case of $\mathbb R^n$ for $n>1$. Since $w=\sum_{i=1}^m v_i$, the projections $\{v_i'\}$ of the $\{v_i\}$ onto $\{w\}^{\perp}$ add up to $0$. Since $\|v_i\|\le \epsilon$ for all $i$, the Polygonal Confinement Theorem yields a permutation $P$ of $(2,...m)$ such that $$\bigg\| \frac{1}{\epsilon}v_1' + \frac{1}{\epsilon}v_{P(2)}'+...+\frac{1}{\epsilon}v_{P(j)}' \bigg\| \le C_{n-1},\, j=2,3,...,m$$ Also, $$\bigg ( v_1 \bigg | \frac{w}{\|w\|} \bigg )+\bigg ( v_{P(2)} \bigg | \frac{w}{\|w\|} \bigg )+...+\bigg ( v_{P(m)} \bigg | \frac{w}{\|w\|} \bigg )=\|w\|$$ and $|\frac{(v_i|w)}{\|w\|}| \le \epsilon$ for all i. Hence, the case $n=1$ (***) yields an $r$ such that $$\bigg | \bigg ( v_1 \bigg | \frac{w}{\|w\|} \bigg )+\bigg ( v_{P(2)} \bigg | \frac{w}{\|w\|} \bigg )+...+\bigg ( v_{P(r)} \bigg | \frac{w}{\|w\|} \bigg ) - t\|w\| \bigg | \le \epsilon$$
The bounds on the components on $w$ and $w^{\perp}$ (notice that $tw$ has nothing on $w^{\perp}$) yield a bound on the vector, so $$\|v_1+v_{P(2)}+...+v_{P(r)}-tw\|^2 \le \epsilon^2C_{n-1}^2 + \epsilon^2$$ which is the Lemma. Q.E.D.
Now we can finally prove the main theorem.
The Levy-Steinitz Theorem. The set of all sums of rearrangements of a given series of vectors in $\mathbb R^n$ is either the empty set or a translate of a subspace.
Proof. Let $S$ denote the set of all sums of convergent rearrangements of the series $\sum_{i=1}^{\infty}v_i$. We must show that S, adjusted by a vector, is a subspace. Suppose $S$ is not empty, then the series converges to some point(s), therefore $\|v_i\| \rightarrow 0$. By replacing $v_1$ by $v_1 - v$, where $v$ is any element of $S$, we can assume that $0 \in S$.
Next we show that if $0,\,s_1,\,s_2 \in S$, so is $s_1+s_2$.
Let $\{\epsilon_m\}$ be a sequence of positive numbers that converges to $0$. Since an arrangement converges to $s_1$, there exists a finite set $I_1$ of positive integers such $1\in I_1$ and $\|\sum_{i\in I_1}v_i-s_1\|<\epsilon_1$. Since an arrangement converges to 0, there is a finite set $J_1 \supset I_1$ such that $\|\sum_{i\in J_1}v_i - 0\|<\epsilon_1$, and finite set $K_1 \supset J_1$ such that $\|\sum_{i \in K_1}v_i - s_2\| < \epsilon_1$. There is also a finite set $I_2$ containing both $K_1$ and $\{2\}$ such that $\|\sum_{i\in I_2}v_i-s_1\|<\epsilon_2$. And so on. Note we were only talking about finite sum above, and finite sums are interchangeable, and thus we do not care the order of elements in the integer sets above.
By above procedure, we inductively construct sets $I_m$, $J_m$, and $K_m$ of positive integers such that $$\{1,...,m-1\} \subset K_{m-1} \subset I_m \subset J_m \subset K_m,$$ $$\bigg \| \sum_{i\in I_m}v_i - s_1\bigg \|<\epsilon_m,\,\, \bigg \| \sum_{i\in J_m}v_i - 0\bigg \|<\epsilon_m,\,\,\bigg \| \sum_{i\in K_m}v_i - s_2\bigg \|<\epsilon_m\,(****)$$ For each $m$, starting at $m=1$, arrange the indices in $J_m$ so that those in $I_m$ come at the beginning, and then arrange the indices in $K_m$ so that those in $J_m$ come at the beginning. Then arrange the indices of $I_{m+1}$ so that those of $K_m$ come at the beginning. Thus there is a permutation $P$ of the set of positive integers and increasing sequences $\{i_m\},\{j_m\},\{k_m\}$ such that $i_m < j_m < k_m < i_{m+1}$, and $$\bigg \| \sum_{i=1}^{i_m}v_{P(i)} - s_1\bigg \|<\epsilon_m,\,\, \bigg \| \sum_{j=1}^{j_m}v_{P(j)} - 0\bigg \|<\epsilon_m,\,\,\bigg \| \sum_{k=1}^{k_m}v_{P(k)} - s_2\bigg \|<\epsilon_m$$ for each $m$.
Note that $$\bigg \| \sum_{i=j_m+1}^{k_m}v_{P(i)} - s_2 \bigg \|=\bigg \| \sum_{i=1}^{k_m}v_{P(i)} - \sum_{j=1}^{j_m}v_{P(j)} - s_2 \bigg \| < \epsilon_m + \epsilon_m$$ It follows that $$\bigg \| \sum_{i=1}^{i_m}v_{P(i)} + \sum_{i=j_m + 1}^{k_m}v_{P(i)}-(s_1 + s_2) \bigg \|<3\epsilon_m$$ For each $m$, rearrange the vectors in $\{v_{P(i)}: i=i_m,...j_m,...,k_m\}$ by interchanging the vectors $\{v_{P(i)}: i = i_m+1,...,j_m\}$ with the vectors $\{v_{P(i)}: i=j_m+1,...,k_m\}$. In this new arrangement, the above shows that there is a subsequence of the sequence of partial sums that converges to $s_1+s_2$. Since we are assuming $S \ne \emptyset$, $\{v_{P(i)}\}\rightarrow 0$. So the Rearrangement Theorem implies that there is another arrangement that converges to $s_1+s_2$. Therefore, $(s_1+s_2) \in S$.
It remains to be shown that $s \in S$ implies $ts \in S$ for all real number $t$. The additivity of $S$ implies this for $t$ as a positive integer, so it suffices to consider the cases $t \in (0,1)$ and $t=-1$.
We start the the arrangement $P$ used above to show the additivity of $S$. Fix $t \in (0,1)$. As shown above, $$\bigg \| \sum_{i=j_m+1}^{k_m}v_{P(i)}-s_2 \bigg \|<2\epsilon_m$$ for each m. Let $\delta_m=sup\{||v_{P(i)}||: i = j_m+1,...,k_m\},\,w=\sum_{i=j_m+1}^{k_m}v_{P(i)}$, notice $\delta_m \rightarrow 0$, as $m \rightarrow \infty$ and let $$u_m=\sum_{i=j_m+1}^{k_m}v_{P(i)}-s_2=w-s_2$$ By Lemma 2, there is a permutation $Q_m$ of $\{P(j_m+1),...,P(k_m)\}$ and an $r_m$ so that $$\bigg \| \bigg (\sum_{i=j_m+1}^{r_m}v_{Q_m(P(i))} \bigg )-t(s_2+u_m) \bigg \| \le M\delta_m,\, M=\sqrt{C_{n-1}^2+1}$$ Then $$\bigg \| \bigg( \sum_{i=j_m+1}^{r_m}v_{Q_m(P(i))} \bigg )-ts_2 \bigg \| \le M\delta_m + 2\epsilon_m$$ Now $$ \bigg \| \sum_{i=1}^{j_m}v_{P(i)} + \sum_{i=j_m+1}^{r_m}v_{Q_m(P(i))} -ts_2 \bigg \| \le M\delta_m + 3\epsilon_m$$ so in this arrangement, a subsequence of the sequence of partial sums converges to $ts_2$. The Rearrangement Theorem yields $ts_2 \in S$.
Last but not the least, we need to show that $-s_2 \in S$. Notice that by (****) $$\bigg\| \bigg( \sum_{i=1}^{j_{m+1}}v_{P(i)} - 0 \bigg) - \bigg ( \sum_{i=1}^{k_m}v_{P(i)} - s_2\bigg ) \bigg\| $$ $$=\bigg \| \sum_{i=1}^{j_{m+1}}v_{P(i)}-\sum_{i=1}^{k_m}v_{P(i)} - (0-s_2) \bigg \| < \epsilon_{m+1} + \epsilon_m$$ So $$\bigg \| \sum_{i=k_m + 1}^{j_{m+1}}v_{P(i)} -(-s_2) \bigg \| < \epsilon_{m+1} + \epsilon_m$$
Then $$\bigg \| \sum_{j=1}^{j_m}v_{P(i)} + \sum_{i=k_m + 1}^{j_{m+1}}v_{P(i)} - (-s_2) \bigg \| < \epsilon_{m+1} + 2\epsilon_m$$ thus, there is an arrangement with a subsequence of the sequence of partial sums converging to $(-s_2)$. By the Rearrangement Theorem, $(-s_2) \in S$. Q.E.D.
Now, we are finally DONE. Thank you for reading the long way down and hope you enjoyed it - at least I did :)