How can I get my monitor's maximum resolution without the proprietary AMD graphic driver installed?

I didn't see any errors in the Xorg.0.log file the radeon driver seems to work well so try to create a Modeline for the unsupported resolution. Open a terminal with Ctrl+Alt+t and type:

cvt 1366 768

It should return:

# 1368x768 59.88 Hz (CVT) hsync: 47.79 kHz; pclk: 85.25 MHz
Modeline "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync

Now try to create a new mode for xrandr using this Modeline:

xrandr --newmode "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync

Finally apply the new mode to your desired output:

xrandr --addmode DVI-0 1368x768_60.00

Note: Change DVI-0 by your current active output, check the result of the xrandr command to know which one to choose.

EDIT: How to force lightdm to use this specific resolution:

1. Create a shell script in /usr/bin called lightdmxrandr.sh containing:

   #!/bin/sh
   xrandr --newmode "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync
   xrandr --addmode DVI-0 1368x768_60.00
   xrandr --output DisplayPort-0 --off --output DVI-0 --mode 1368x768_60.00 --pos 0x0 --rotate normal --output HDMI-0 --off
   

2. Make /usr/bin/lightdmxrandr.sh executable:

   sudo chmod +x /usr/bin/lightdmxrandr.sh
   

3. Modify your lightdm settings:

   sudo gedit /usr/share/lightdm/lightdm.conf.d/50-ubuntu.conf
   

   to add the following line:

   display-setup-script=/usr/bin/lightdmxrandr.sh
   

   This will set the login screen resolution. To set the desktop resolution, do the following:-

4. Open Startup Applications. One can do so from the dash.

5. In the startup applications window select "Add". Then give a name of your choice. Then press browse and select lightdmxrandr.sh from /usr/bin. Now select "Add". Make sure there is a tick mark before the entry you just added and then close the window.

Now reboot. Both your login screen and desktop must have the 1366x768 resolution.

PS:- Credits for the last part of the answer go to this question here.