Can't ssh to server after Debian upgrade to Buster without previous login from "trusted machine"
connection refused suggests that sshd is not running.
It might be just a matter of time: when the login prompt appears on the console, it does not guarantee that all the system services have completed startup yet.
sshd might be also be stuck waiting on /dev/[u]random, especially if the system is located in a network segment with very little network traffic. In this case, the system has very few sources of true randomness available and has difficulties gathering up enough truly random bits for initially seeding the kernel's random number generator. Logging onto the system console will provide some randomness in the form of the lowest bits of keyboard interrupt call times. If the system has some form of hardware RNG, enabling it might fix this problem.
To diagnose, just type a few lines of nonsense to the console login prompt without actually logging in. If sshd responds normally after that, the kernel was probably starved of randomness and unable to seed the kernel RNG, and that caused the startup of sshd to be delayed.
Or it might be some sort of systemd dependency bug that only allows sshd to start up when the login-related processes get started first. That would be one of the reasons why Debian Buster is still the testing distribution: if this turns out to be the cause, please send a bug report.
Thanks to telcoM's answer about a Hardware RNG I did an apt search and found the package rng-tools5 and installed it:
sudo apt-get install rng-tools5
This resolved the issue on my Intel NUC.
Editor's note: My issue on Dell PowerEdge T20 with Xeon CPU was also resolved with this.
Additional notes:
After installation of the package, please do check if there is a random source with:
rngd -vIn my case, there is no TPM device, but the CPU has
rdrandcapability:Unable to open file: /dev/tpm0 Available entropy sources: DRNG
The package haveged solved the problem.
The thread: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912087