How to get a second PSU to power only hard drives if the CPU and motherboard connectors are not used?
You can turn an ATX power supply on by using the "paperclip technique".
Find the 24-pin connector. If the wires are colored, then you'll have a slightly easier time, if they are all one color (e.g: black), then be careful about which pins you use.
Orient the connector so that you are looking "in" to the holes (i.e: as if you were the socket, with the wires coming out of the back), and with the clip on the top.
Starting at the top left pin, and counting right, you want to connect (aka "short") the third and fourth pins (i.e: "Ground" and "Power On").
This will cause the power supply to enable its main outputs (instead of just the standby 5v supply). The power supply's fan may spin, and any connected peripherals should power up.
As mentioned in the comments, if this is going to be a permanent installation, then you would be much better off using an adapter as shown below that uses the motherboard's signal to control both power supplies.
If you're comfortable with it, then you can make one of these yourself - depending on which route you take (crimping, housings, etc... vs. cutting, splicing and soldering) it may require permanent modifications to one of the power supplies - make sure you label it as such.
In any case, ensure that the signals are properly isolated, and you don't leave any bare wires exposed.
The product pictured below is ThermalTake's AC-005-CNONAN-P1.
Regarding Grounding - in PCs, the "Ground" / 0v is typically bonded with Mains Earth and the Chassis, meaning that the two supplies should be at the same potential. If there is any doubt, or if this is a permanent installation, then it would also be highly advisable to connect a number of the ground / 0v signals from each power supply together.
If you are using two separate wall sockets to power each supply, then you may introduce a ground loop, as each socket may take its ground from different points... If possible, use the same or neighbouring wall sockets for both supplies.
If you do not, and the supplies are not at the same potential, then current will flow through equipment (e.g: hard disks) via the power and data cables.
You can get 1 of those PSU 24 pin jumpers if you don't want to use the paper clip method.
In your situation, I'd avoid the "paperclip jumper" and simply cut both appropriate lines, and join them properly with solder and heat shrink. This is less likely to fall out.
Do please label it well.
I did something similar years ago when time was cheap and parts were not. Ended up with twelve 3.2 GB IDE hard drives in a large AT case, which had an AC power switch. It was used as a cache disk for a squid server.
It worked fine if the drives were turned on first, or at the same time as the board, and up to ~5 seconds after the board. Any more than that and the host was booting past that part of the POST.
Since the host was ATX based (Pentium2 from memory) I planned on using a small relay running off the Power LED connector, which would switch mains voltage on to the other unit.
However I was dissuaded from this by management, who pointed out that mains electrical work was outside my job description and that I was not licensed for such work. In the end I just made sure to power on the drive rack before the host. I left in ~2006 and that host ran on for another 5 years before being destroyed by an earthquake. I must have some something right.