What does this notation in this SAT-Test question mean?

You're always allowed to invent notation, as long as you explain what it means. Tests like the SAT like to do this to test if you're really comfortable with the ideas in abstract sense, and not tied to any particular notation.

But I have some objections to the way this is worded. The first sentence

$\fbox{k} = (-k, \frac{k}{2})$, where $k$ is an integer

seems to imply that $k$ is a single unknown value, so $\fbox{k}$ is a single point. But then there are an infinite number of lines through that point, so the question should be "which of these lines goes through that point?". The wording of the question seems to indicate that $\fbox{k}$ is a function defined on the integers, but then the question should say something like "... passing through all points $\fbox{k}$".

It appears that $\fbox{k}$ denotes a point (in this case, the point $(-k, k/2)$ for some integer $k$). It is not a notation I have ever seen before -- I would expect something like $P_k = (-k, k/2)$) -- but there is no accounting for taste.

(Although it wasn't in your question, the correct answer to the SAT question is then E: the line $y = -\frac{1}{2}x$ contains the point $ \fbox{k} = (-k, k/2)$ for every integer $k$.)