How do you disable the window preview in Ubuntu dock (GNOME sidebar)?

On your Terminal run:

gsettings set org.gnome.shell.extensions.dash-to-dock click-action 'skip'

Or you can try(This will allow you to cycle through the open windows by clicking on the icon):

gsettings set org.gnome.shell.extensions.dash-to-dock click-action 'cycle-windows'

To revert the action:

gsettings set org.gnome.shell.extensions.dash-to-dock click-action 'previews'

A perhaps more novice-friendly approach, that also allows you to see the other options that exist for this setting, would be using the Dash-to-Dock extension, which is actually what Ubuntu uses for its dock, albeit with less configuration options.

First, you need to be able to install GNOME Shell extensions, so make sure that you have chrome-gnome-shell and gnome-tweaks installed by running the following command in a terminal:

sudo apt install chrome-gnome-shell gnome-tweaks

Also, install the GNOME Shell integration extension for Chrome or for Firefox, depending on the browser of your choice.

Then, install Dash-to-Dock from the GNOME Shell extension website by turning the switch to ON and clicking Install.

After this, open Tweaks and go to Extensions → Dash to Dock → click the gear icon → Behavior and select the Click action option that you wish.