Quadruples of integers $(a,b,c,d)$ such that for any two distinct elements, $n,m$, $nm+1$ is a square

From John Omielan's answer, we see that we are looking for a number $P=abcd$ that can be expressed as the product of two numbers of the form $k^2-1$ in three different ways.

Note that $P\le N(N-1)(N-2)(N-3)$, so bounding $N$ or $P$ doesn't make much difference.

You can start looking for numbers of the form $P=(r^2-1)(s^2-1)<N^4$. This has a cost $O(N^2/2)$, which it is fair better than brute force. Then look for numbers of the form $k^2-1$ which divide $P$. You needn't look for divisors greater that $\sqrt P$, so it has a cost $O(\sqrt[4]P)=O(N)$. Total cost is then $O(N^3/2)$.

If you find two more divisors of $P$ $d_1$ and $d_2$ of the form $k^2-1$, just check if $P/d_1$ and $P/d_2$ are also of this form.

To find $a,b,c,d$, we assume that we already have the six divisors $d_1<d_2<\ldots<d_6$.

Now, $d_1=ab$, $d_2=ac$. Then $d_1/d_2=b/c$, but $bc=d_3$, so $$b=\sqrt{\frac{d_1d_3}{d_2}}$$ Now, from $d_1=ab$ you get $a$, etc.


Greg Martin shows that the number of such quadruples up to $N$ is asymptotic to $CN^{1/3}\log N$, where $C={2^{4/3}\over3\Gamma(2/3)^3}$, which is about $0.33828$.


This is only a partial answer, but I believe it might help provide you with a direction to pursue to check on this further. Your requirement of each pairwise products plus $1$ to be perfect squares leads to the following set of equations

$$ab + 1 = s_1^2 \iff ab = s_1^2 - 1 \tag{1}\label{eq1}$$

$$ac + 1 = s_2^2 \iff ac = s_2^2 - 1 \tag{2}\label{eq2}$$

$$ad + 1 = s_3^2 \iff ad = s_3^2 - 1 \tag{3}\label{eq3}$$

$$bc + 1 = s_4^2 \iff bc = s_4^2 - 1 \tag{4}\label{eq4}$$

$$bd + 1 = s_5^2 \iff bd = s_5^2 - 1 \tag{5}\label{eq5}$$

$$cd + 1 = s_6^2 \iff cd = s_6^2 - 1 \tag{6}\label{eq6}$$

Next, \eqref{eq1} times \eqref{eq6} gives

$$abcd = (s_1^2 - 1)(s_6^2 - 1) \tag{7}\label{eq7}$$

Also, \eqref{eq2} times \eqref{eq5} gives

$$abcd = (s_2^2 - 1)(s_5^2 - 1) \tag{8}\label{eq8}$$

In addition, \eqref{eq3} times \eqref{eq4} gives

$$abcd = (s_3^2 - 1)(s_4^2 - 1) \tag{9}\label{eq9}$$

Combining \eqref{eq7}, \eqref{eq8} and \eqref{eq9} gives

$$abcd = (s_1^2 - 1)(s_6^2 - 1) = (s_2^2 - 1)(s_5^2 - 1) = (s_3^2 - 1)(s_4^2 - 1) \tag{10}\label{eq10}$$

Thus, you're basically looking for $3$ cases where the product of two distinct squares $- 1$ are equal to each other. I did some checking of this issue myself, and also a few brief online searches (note I had some difficulty determining what the most appropriate search terms to use are), but I wasn't able to find anything more about research, if any, which has done into under what types of conditions, and what sorts of limitations (e.g., are there an infinite number of such values), to have this occurrence of even $2$ of these products, much less $3$ of them, equaling each other occur.