What is the sequence loading linux kernel module on startup? How priority is set to them?
They are not loaded automatically at start-up or any other time, although a lot of them do end up being loaded during boot. There are three different mechanisms by which this happens:
Userspace request: Which covers everything from init services to udev to the command-line. Init or udev are probably the most straightforward means if you want to load a specific module at boot time.
Hotplugged device: When you connect something to, e.g., USB or PCI, the kernel detects this and requests an appropriate module based on how the device identifies itself.
Needed protocol or other implementation: When the kernel needs to do something, such as read a filesystem, and discovers it lacks the knowledge to do so, it will request a module.
Notice that for the last two I used the phrase "request a module" -- this is because the kernel actually loads via a userspace daemon, kmod which executes /sbin/modprobe. According to Wolfgang Mauerer in Linux Kernel Architecture, there are only ~100 different points in the 2.6 kernel where it calls an internal request_module() function.
modprobe uses a database of installed MODULE_ALIAS's. These are specified in the module source code explicitly, or derived from it's MODULE_DEVICE_TABLE, which is a list of OEM device IDs that the module services.
Many systems are set up to use an initrd or initramfs. These are filesystem images that are loaded by the bootloader and made available to the kernel before it mounts the root partition. This allows drivers that are necessary to mount the root partition (disk drivers, filesystem drivers, device mapper or logical volume drivers, …) to be compiled as modules and loaded from the initrd/initramfs.
The startup scripts on the initrd (/linuxrc) or initramfs (/init) typically loads some modules and locates the root filesystem. Each distribution has its own setup. Ubuntu uses an initramfs which is assembled from components in the initramfs-tools package and regenerated for each kernel based on the necessary drivers to mount the root filesystem.
After the root filesystem is mounted, during the system boot, modules listed in /etc/modules (Debian, …) or /etc/modules.conf (Red Hat, Arch, …) are loaded. This file usually lists few modules if any. Most modules are loaded on demand.
When the kernel detects some hardware for which it lacks a driver, or certain other components such as network protocols or cryptographic algorithms, it calls /sbin/modprobe to load the module. For hardware drivers, the kernel passes names that encode the PCI id, the USB id, or other systematic designation of the hardware. There is a table in /lib/modules/$VERSION/modules.alias that maps these systematic designations to module names. This table is generated by depmod and read by modprobe.
If you have an extra kernel module that you compiled manually for a hardware device, drop it into /lib/modules/$VERSION/local (create the local subdirectory if it doesn't exist) and run depmod -a to regenerate the alias database. If the module is for some unusual feature that the kernel is unable to detect automatically, drop it into /lib/modules/$VERSION/local, run depmod -a to analyze its dependencies, and add the module name to /etc/modules.