Is there a limitation on Gauss' law?
Gauss's law is always fine. It is one of the tenets of electromagnetism, as one of Maxwell's equations, and as far as we can tell they always agree with experiment.
The problem you've uncovered is simply that "a uniform charge density of infinite extent" is not actually physically possible, and it turns out that (i) it is not possible to express it as the limit of a sequence of sensible physical situations, and (ii) it is not possible to provide a proper mathematical formalization for it. It's a bit of a bummer, because you can do this perfectly with infinite line and surface charges, but bulk charges just don't work like this.
This might seem a bit strange (and, really, it should), so let's take another look at what you mean when you say "space is filled" with charge of density $\rho$. Could you implement this in real life? Of course not! You can only fill up some finite volume $V$. Your hope then is that as $V$ gets bigger and bigger, the field inside it stabilizes to some sort of limit.
The problem is that for this procedure to make sense, you need the limiting procedure to be independent of the detailed shape of $V$ as you scale it up, for surely if you're in the centre of the slab and the field has mostly converged, the answer can't depend on details of a boundary that's very far away.
For a line and a surface charge, this works perfectly. You can calculate the field for a finite line charge, and the limit doesn't depend on which end goes to infinity faster as long as they both do. You can also prove that the field of increasing patches of surface charge does not depend too much on the shape of the patches if they are big enough. For bulk charges, though, you've just proved that it doesn't work: if you displace $V$, you get a different answer. Hence, the problem for an infinite spread of bulk charge doesn't make sense, and it's not the limit of sensible physical systems that are "big enough".
Another way of showing that the "big enough" property doesn't make sense is that there is nothing to compare the charge's size with. For line and surface charges, this is perfectly fine, and in fact all they are is models for a finite line / surface charge of length / radius $L$, whose field is tested at a point a distance $d$ from the charge. The distribution is "infinite" if $L/d\gg 1$, or in other words the models are good if the point is much nearer to the source than the source's size. For a bulk charge, there's no meaningful distance $d$, and hence no meaningful dimensionless parameter to take a limit over, and this in turn is what drives the meaninglessness of the situation.
Finally, let me put this a bit more mathematically, in a way that sort of makes it have an answer. Another way to phrase the problem "space is filled with a uniform bulk charge of density $\rho_0$" is as the simple differential equation $$\nabla\cdot \mathbf E=\rho_0/\epsilon_0.$$ This is a perfectly reasonable question to ask, except that you're missing boundary conditions, so the solution won't be (anywhere near) unique. However, boundary conditions don't make sense if your domain is all of space, so you need something else, and what turns out to do the job is to demand answers which share the symmetry properties of the charge - both the translation symmetries and all the point symmetries.
For line and surface charges, this actually works almost perfectly. The coupled symmetry and differential equation demands have, fortunately, unique solutions: the translation symmetries and the differential equation rule out everything except uniform fields, which are then ruled out by the point symmetries.
For a bulk charge, on the other hand, you get a fundamental linear dependence and a uniform field, which cannot be ruled out by the translational symmetry: $$\mathbf E=\frac{\rho_0}{3\epsilon_0}\mathbf r + \mathbf E_0=\frac{\rho_0}{3\epsilon_0}(\mathbf r-\mathbf r_0).$$ This form is sort of translation invariant, except that now you have to re-choose $\mathbf r_0$ every time you translate, which can't be quite right. And if you try to impose any point symmetries, you'll need to put $\mathbf r_0$ at every point with an inversion symmetry - and there you lose out, because it cannot be done.
To rephrase this last bit in your terms, the inversion symmetry requires that the field be zero at every point, but this is not consistent with the differential equation. You always have "infinitesimal" bits of charge at $\vec r$ and $-\vec r$ producing infinitesimal bits of field which cancel each other out, so the field should be zero at every point. This is indeed inconsistent with Gauss's law - but you can simply chalk it up to the fact that the problem is inconsistent.
If you don't specify the boundary, there will be many solutions (including the unsymmetric ones)
The problem of this question is that solving Maxwell's differential equations necessarily involves specifying the boundary conditions which usually can be chosen the obvious ones, however here such a boundary simply does not exist.
Usually, it is straightforward to define what is this "background" field. Obviously, you can just go infinitely far from other charges and measure the field. It makes sense to set this field to zero. However, in our case, there just does not exist a place infinitely far and thus there is no sense in assuming anything about the "background". For example vacuum equations allow for a solution of constant $\vec E$. Adding this solution to the one derived by the OP just shifts the "centre". This resolves the paradox of violating translational symmetry pointed out in many other answers.
In fact, there exist many solutions which are even less symmetric. The simplest one is perhaps the solution $E_x = x \rho / \epsilon_0$, $E_y = E_z = 0$, which also satisfies all equations.
Note: You can't assume isotropy and symmetry of electric field, as this is based on the argument that electric field is completely defined by charges (which is true usually). However as can be seen, this assumption is not true.
Let's be really clear about the exact kind of mistake you are making.
Consider the numbers $1,-1,1,-1,1,-1, \dots$. If you wanted to sum them you could argue that $1-1+1-1+1-1+\dots=(1-1)+(1-1)+(1-1)+\dots=0+0+0+\dots=0$.
Your friend could argue that
$$1-1+1-1+1-1+\dots=1+(-1+1)+(-1+1)++\dots=1+0+0+\dots=1.$$
The problem is that an infinite series isn't something you actually compute so the rules of computation don't apply, it is a limit of things you compute, and the details of how you limit are part of the thing you describe.
So let's look at the case of uniform charge. You could pick any point $P$, any magnitude $E$ and any direction $\hat{r}$. Then, you could try to argue that the electric field at $P$ must be $\vec{E}=E\hat{r}$. Just imagine a sphere with radius $R=E3\epsilon_0/\rho$ located at $P-R\hat{r}$, try to argue that all the charge outside the sphere cancels, and that the charge inside the sphere matters and that therefore the field is $\vec{E}=E\hat{r}$. You are doing the same thing as that infinite sum. You are choosing the cancel what you want to get a result. But the problem didn't tell you to cancel in a particular way, that was all you, and that matters. It matters the order you add things up when there are an infinite magnitude of positive and negative things. Assuming the order didn't matter is simply a flaw in reasoning.
The problem is your argument, you argued that you could ignore something that actually produced an infinite amount of field in the positive $x$ direction and an infinite amount of field in the negative $x$ direction (and similarly for $y$ and $z$). You argued that you could ignore it and cancel them by pairing them up in a manner of your choosing. And you argued that your choices were OK, and acted as if your choices didn't affect the answer you got. They did, and it wasn't OK. Just as when you and your friend argued that the "sum" is zero or one, you are both wrong in the sense that there isn't a sum of an infinite number of terms, there are only limits of finite sums summed in a particular order, and that sometimes there aren't even limits of those sums.
Hopefully you see your mistake now, and see that it is a mistake in argument, you have not computed something, you have argued that a limit should be something when in reality there is not a limit to have any value.
Physically there should be a field due to every charge, and if there were a finite number of charges, then rightly there is a total field. But when you imagine a fictional continuous charge distribution there might not be a total field. You could apply Gauss's law to get the field due to every charge, but then in your case you might not end up with a finite number of fields, so there might not be a justified argument for their being a total field.
Infinite sums don't exist the same way that finite sums exist, integrals are just fancy symbols for certain kinds of limits (of finite sums) that sometimes exist, and a total field is also just a limit that sometimes exists when you don't have a finite number of charges. An infinite "sum" in particular sometimes matters the order in which you add, a finite sum doesn't matter the order in which you add. That's because an infinite "sum" isn't just a sum, it is a limit. When you bring up a total field based on an infinite "sum" without saying how the limit is taken, then you have a problem from the get go, before you can even applied Gauss' law to any alleged total field.