Should the UPS output be exactly same in normal and battery backup modes? Or it is normal to have a slight variation in voltage or something?

Should the UPS output be exactly same in normal and battery backup modes ?

Of course the AC line power from the electrical utility is true sinusoidal with an accurate frequency of 50 or 60 Hertz.
The typical (standby and line-interactive) UPS will pass through the line power when in normal mode.

Typically only a high-quality UPS will output true sinusoidal AC power when in battery mode.
A UPS of lesser quality will convert the DC battery power to a simulated sinusoidal AC wave.
A UPS of low quality will convert the DC battery power to just a square AC wave.

The typical power supply unit of a PC should be able to cope with low-quality AC waveform such as simulated sinusoidal or square wave.
Or maybe not; YMMV.
See When do I need a pure sine wave UPS? and Sinewave vs Simulated Sinwave - Which is Best? .

Interestingly the web page for your UPS does not bother to specify the type of output that it produces in battery mode. That could be an indicator that it does not produce a true sine wave, and therefore the answer to your question would be "no".

I think that only a qualified technician can test the UPS and say whether it is faulty or not. It is always possible, if it is not new, that its battery has weakened over time.

Otherwise, the only way you will ever know is if your new PSU also starts misbehaving.

As mentioned by @sawdust in his answer, it could be that your computer power supply was a PFC (Power Factor Correction) one which requires a truer sine-wave UPS to function. These PFC power supplies are higher efficiency, and are getting more common, especially in higher power supplies in gaming rigs, etc. These power supplies are known to shut down if they don't get sufficiently sine-wave voltage. The computer power supply that you listed is a PFC model, and if your UPS is a lesser-cost one that isn't producing a true-enough sine wave, that could likely be your problem, and possibly the cause of death of your old power supply. To be safe, if you have a PFC power supply, you need to make sure your UPS is one that's listed as PFC compatible.