Magic $\mathbb{Z}\times\mathbb{Z}$-square
I'll try to show that we can make a square such that all rows and columns give zero sum. (Where "sum" is meant in the sense described in the question.) I.e., this is the answer to the stronger variant.1
The description is a bit informal, but I hope it could be clear how the construction goes.
We will proceed by induction and choose values $j(k,l)$ with $-N\le k,l \le N$. I.e., after $N$ steps the square of the size $(2N+1)\times(2N+1)$ will be filled. Moreover, will do it in such fashion that the row sums and column sums of the square are zeroes. (And in such way that no numbers in our table repeat.)
Base step. Let us start by putting $j(0,0)=0$.
In the first step of induction we want to add some other numbers into this three by three square $$ \begin{array}{|c|c|c|} \hline \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \hphantom{0} & 0 & \hphantom{0} \\\hline \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \end{array} $$
Now we simply choose two distinct positive integers $a$, $b$ and add also $-b$, $-a$ in the opposite positions so that we get zeroes in the middle row and the middle column. $$ \begin{array}{|c|c|c|} \hline \hphantom{-a} & a & \hphantom{-a} \\\hline b & 0 & -b \\\hline \hphantom{-a} &-a & \hphantom{-a} \\\hline \end{array} $$
We simply choose any integer $x>\max(a,b)$. If we then add $x$ to the topleft corner, we have only one possibility what to do in the other positions in order to get sum equal to zero
$$ \begin{array}{|c|c|c|} \hline x & a & -a-x \\\hline b & \hphantom{a+}0\hphantom{+b} & -b \\\hline -b-x &-a & a+b+x \\\hline \end{array} $$
The condition $x>\max(a,b)$ implies that all numbers in this table are distinct.
Inductive step. We assume that we already have a square where row/columnns add up to zero. We want to add two more rows (at the top and at the bottom) and two more columns (left and right).
The construction in the inductive step will be somewhat similar to what we did in the base step.
We already have a "zero square" in the middle. $$ \begin{array}{|c|ccc|c|} \hline \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\ \hphantom{0} & \hphantom{0} & 0 & \hphantom{0} & \hphantom{0} \\ \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \end{array} $$
We first add numbers above/below the square and also to the left and to the right. (I.e., only the four corners will be missing.) We will do this in such way that above the square we put some positive integers that neither these numbers, nor their opposites, have been used so far. And below the square we put their opposites. Similarly on the left and right side. After this the row and column sums are zeroes, with the possible exception of the first and last row/column - where we're going to add the missing values. Let us denote the sum of all numbers added "above" as $A$ and the sum of all numbers added on the left as $B$. We can also require that $A\ne B$. (If needed, we simply modify one of the numbers on the left.) $$ \begin{array}{|c|ccc|c|} \hline \hphantom{0} & \hphantom{0} & A & \hphantom{0} & \hphantom{0} \\\hline \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\ B & \hphantom{0} & 0 & \hphantom{0} & -B \\ \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline \hphantom{0} & \hphantom{0} & -A & \hphantom{0} & \hphantom{0} \\\hline \end{array} $$ Again we choose some $x$ which is larger than the absolute values of the numbers we have used so far. (This will ensure that the numbers in the corners will be distinct from the existing ones.) The we have only one possibility what to add in the remaining positions. $$ \begin{array}{|c|ccc|c|} \hline x & \hphantom{0} & A & \hphantom{0} & -x-A \\\hline \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\ B & \hphantom{0} & 0 & \hphantom{0} & -B \\ \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline -B-x & \hphantom{0} & -A & \hphantom{0} & x+A+B \\\hline \end{array} $$
Continuing in this way, we fill the whole $\mathbb Z\times\mathbb Z$ table in a way which fulfills the requirements stated in the question.
1Ben Barber suggested some approach in the comments to the original question. I do not understand those comments well enough to be able to judge whether the solutions proposed there is similar to this one. However, one of the comments suggests that we are in fact able to get a bijection - which seems more difficult (and more interesting).
With the unusual definition of convergence, and infinite space, there's so much flexibility that it's difficult to go wrong. Here's one way to proceed.
Enumerate the rows and columns arbitrarily and suppose we are currently trying to fill some row or column $r$ having previously filled $n$ rows and columns such that the desired property holds. Fill additional cells of $r$ (with distinct integers that have not been used so far) until
- The filled in cells of $r$ form an interval centred about $0$.
- The sum of the entries of these cells is $0$.
- Every integer from $-n$ to $n$ appears somewhere in the part of the grid filled so far.
Once this is done (and it remains to check that it can be), fill in the remaining entries of $r$ by pairs $a$, $-a$ where $a$ is a power of the $(n+1)$st prime $p_{n+1}$.
(1) and (3) are easy to ensure. Once they are in place the filled in cells have some sum $s$. We seek some $b$ such that $b$ and $-(b+s)$ are both unused; placing them in the cells at either end of the filled in interval brings the sum so far to $0$.
To see that there exists such a $b$, let $A$ be the set of absolute values of integers used so far. $A$ contains only finitely many elements that are not powers of the first $n$ primes, so $A$ has zero density and its complement must contain an interval of length $|s|+1$. The endpoints of this interval, with the correct signs added, can be taken as $b, -(b+s)$.
Continuing in this way eventually fills the entire grid using every integer exactly once.