Is the gravitational force for an object on the surface of earth always pointed to the center?
The force vector would only point towards the centre of the Earth if it were spherically symmetric and non-rotating (except perhaps at the equator or poles). There are significant deviations from this and local gravity can deviate by tenths of a degree from a line towards the centre of the Earth.
e.g. see https://www.wolframalpha.com/widgets/view.jsp?id=e856809e0d522d3153e2e7e8ec263bf2
I believe that the rotational effects (which are easily predicted) are usually somewhat bigger than the effects caused by local geography.
These deviations were used historically to estimate the mean density of the Earth by Neville Maskelyne among others, who used the deviation of a plumbline caused by the mountain Schiehallion.
A large skyscraper's mass:
$$M=2.5\times 10^{8}\,\mathrm{kg}$$
For the benefit of the doubt, let's assume a centre-of-mass at the ground, and that you are standing at the building's wall, possibly 50 metres away.
$$r=50\,\mathrm{m}$$
The skyscrapers gravity on a $m=75\,\mathrm{kg}$ person is:
$$F_{g,\text{skyscraper}}=G\frac{mM}{r^2}=0.0005\,\mathrm N$$
Compare this with the $F_{g,\text{earth}}=mg=735\,\mathrm N$ gravity from Earth. The skyscraper is adding a perpendicular component, so the total force of gravity on the person is a vector, which now is slightly turned:
$$F_g=\begin{pmatrix}735\\0.0005\end{pmatrix}\,\mathrm N$$
The angle in degrees between this vector $\begin{pmatrix}735\\0.0005\end{pmatrix}\,\mathrm N$ and the original vector $\begin{pmatrix}735\\0\end{pmatrix}\,\mathrm N$ is:
$$\theta=3.9\times 10^{-5} ~^\circ$$
This is a very, very small deflection. Just the fact that gravity from Earth does not pull directly to the centre due to uneven density, centrifugal effects, altitude etc., might far outweigh this tiny deflection.
Geodesists typically distinguish between gravity and gravitation. Gravity is gravitation plus centrifugal acceleration and is measured by the acceleration of an object released at rest with respect to the rotating Earth, as measured from the surface of the rotating Earth.
A plumb bob points "down" -- the direction in which gravity (not just gravitation) pulls the plumb bob. The nominal angular difference between "down" and toward the center of the Earth reaches a maximum of about 0.1924° at about 45° latitude. This nominal angular difference results from the more-or-less ellipsoidal shape of the Earth. "Down" nominally points in the direction of the inward normal to the ellipsoid.
Even on eliminating centrifugal acceleration from consideration, leaving gravitation only, the resultant acceleration only points to the center of the Earth at the equator and the poles. The Earth's rotation results in a bulge of material around the equator, the equatorial bulge. The primary effect of this bulge on gravitation is called "J2". At a constant distance from the center of the Earth, the equatorial bulge makes gravitation stronger than spherical gravitation above the equator and weaker above the poles. The gravitational bulge also results in a component of acceleration that is orthogonal to the direction to the center of the Earth. This effect is greatest at about 45° latitude, where the angle between the nominal gravitation vector and the vector toward the center of the Earth is about 0.9°.
The Earth is not quite an ellipsoid. It has mountains and pockets of material the are more dense or less dense than nominal. (Mountains are for the most part huge pockets of material that are less dense than nominal.) These regions of varying density affect which way gravity (and gravitation) point locally. The angular difference between "down" as observed by a plumb bob and "down" as calculated by assuming an ellipsoidal Earth is called the deflection of the vertical. This deflection is very small, rarely over an arc minute.