/usr/bin vs /usr/local/bin on Linux

1. /bin (and /sbin) were intended for programs that needed to be on a small / partition before the larger /usr, etc. partitions were mounted. These days, it mostly serves as a standard location for key programs like /bin/sh, although the original intent may still be relevant for e.g. installations on small embedded devices.

2. /sbin, as distinct from /bin, is for system management programs (not normally used by ordinary users) needed before /usr is mounted.

3. /usr/bin is for distribution-managed normal user programs.

4. There is a /usr/sbin with the same relationship to /usr/bin as /sbin has to /bin.

5. /usr/local/bin is for normal user programs not managed by the distribution package manager, e.g. locally compiled packages. You should not install them into /usr/bin because future distribution upgrades may modify or delete them without warning.

6. /usr/local/sbin, as you can probably guess at this point, is to /usr/local/bin as /usr/sbin to /usr/bin.

In addition, there is also /opt which is for monolithic non-distribution packages, although before they were properly integrated various distributions put Gnome and KDE there. Generally you should reserve it for large, poorly behaved third party packages such as Oracle.

I recommend taking a look at the file system hierarchy man page:

man hier

which is also available online, for instance: http://linux.die.net/man/7/hier.  Relevant portions have been copied below.  Depending on your system, it may say something different.

Name

hier - description of the file system hierarchy

Description

A typical Linux system has, among others, the following directories:

/bin
This directory contains executable programs which are needed in single user mode and to bring the system up or repair it.
 
/sbin
Like /bin, this directory holds commands needed to boot the system, but which are usually not executed by normal users.
 
/usr/bin
This is the primary directory for executable programs.  Most programs executed by normal users which are not needed for booting or for repairing the system and which are not installed locally should be placed in this directory.
 
/usr/local/bin
Binaries for programs local to the site.
 
/usr/local/sbin
Locally installed programs for system administration.
 
/usr/sbin
This directory contains program binaries for system administration which are not essential for the boot process, for mounting /usr, or for system repair.

The Filesystem Hierarchy Standard entry in Wikipedia helped me answer the same question when I had it, plus it has a very explanatory table.

Excerpt from that page¹:

/bin        Essential command binaries that need to be available in single user mode; for all users, e.g., cat, ls, cp.
/usr/bin    Non-essential command binaries (not needed in single user mode); for all users.
/usr/local  Tertiary hierarchy for local data, specific to this host. Typically has further subdirectories, e.g., bin, lib, share
/usr/sbin   Non-essential system binaries, e.g., daemons for various network-services.
/sbin       Essential system binaries, e.g., fsck, init, route.

¹ _{Retrieved on June 19, 2019; permalink.}