What is the difference in usage between shell variables and environment variables?
Shell variables
Shell variables are variables whose scope is in the current shell session, for example in an interactive shell session or a script.
You may create a shell variable by assigning a value to an unused name:
var="hello"
The use of shell variables is to keep track of data in the current session. Shell variables usually have names with lower-case letters.
Environment variables
An environment variable is a shell variable which has been exported. This means that it will be visible as a variable, not only in the shell session that created it, but also for any process (not just shells) that are started from that session.
VAR="hello" # shell variable created
export VAR # variable now part of the environment
or
export VAR="hello"
Once a shell variable has been exported, it stays exported until it is unset, or until its "export property" is removed (with export -n in bash), so there's usually no need to re-export it. Unsetting a variable with unset deletes it (no matter if it's an environment variable or not).
Arrays and associative hashes in bash and other shells may not be exported to become environment variables. Environment variables must be simple variables whose values are strings, and they often have names consisting of upper-case letters.
The use of environment variables is to keep track of data in the current shell session, but also to allow any started process to take part of that data. The typical case of this is the PATH environment variable, which may be set in the shell and later used by any program that wants to start programs without specifying a full path to them.
The collection of environment variables in a process is often referred to as "the environment of the process". Each process has its own environment.
Environment variables can only be "forwarded", i.e. a child process can never change the environment variables in its parent process, and other than setting up the environment for a child process upon starting it, a parent process may not change the existing environment of a child process.
Environment variables may be listed with env (without any arguments). Other than that, they appear the same as non-exported shell variables in a shell session. This is a bit special for the shell as most other programming languages don't usually intermix "ordinary" variables with environment variables (see below).
env may also be used to set the values of one or several environment variables in the environment of a process without setting them in the current session:
env CC=clang CXX=clang++ make
This starts make with the environment variable CC set to the value clang and CXX set to clang++.
It may also be used to clear the environment for a process:
env -i bash
This starts bash but does not transfer the current environment to the new bash process (it will still have environment variables as it creates new ones from its shell initialization scripts).
Example of difference
$ var="hello" # create shell variable "var"
$ bash # start _new_ bash session
$ echo "$var" # no output
$ exit # back to original shell session
$ echo "$var" # "hello" is outputted
$ unset var # remove variable
$ export VAR="hello" # create environment variable "VAR"
$ bash
$ echo "$VAR" # "hello" is outputted since it's exported
$ exit # back to original shell session
$ unset VAR # remove variable
$ ( export VAR="hello"; echo "$VAR" ) # set env. var "VAR" to "hello" in subshell and echo it
$ echo "$VAR" # no output since a subshell has its own environment
Other languages
There are library functions in most programming languages that allows for getting and setting the environment variables. Note that since environment variables are stored as a simple key-value relationship, they are not usually "variables" of the language. A program may fetch the value (which is always a character string) corresponding to a key (the name of the environment variable), but will then have to convert it to an integer or whatever data type the language expects the value to have.
In C, environment variables may be accessed using getenv(), setenv(), putenv() and unsetenv(). Variables created with these routines are inherited in the same way by any process that the C program starts.
Other languages may have special data structures for accomplishing the same thing, like the %ENV hash in Perl, or the ENVIRON associative array in most implementations of awk.
Environment variables are a list of name=value pairs that exist whatever the program is (shell, application, daemon…). They are typically inherited by children processes (created by a fork/exec sequence): children processes get their own copy of the parent variables.
Shell variables do exist only in the context of a shell. They are only inherited in subshells (i.e. when the shell is forked without an exec operation). Depending on the shell features, variables might not only be simple strings like environment ones but also arrays, compound, typed variables like integer or floating point, etc.
When a shell starts, all the environment variables it inherits from its parent become also shell variables (unless they are invalid as shell variables and other corner cases like IFS which is reset by some shells) but these inherited variables are tagged as exported1. That means they will stay available for children processes with the potentially updated value set by the shell. That is also the case with variables created under the shell and tagged as exported with the export keyword.
Array and other complex type variables cannot be exported unless their name and value can be converted to the name=value pattern, or when a shell specific mechanism is in place (e.g.: bash exports functions in the environment and some exotic, non POSIX shells like rc and es can export arrays).
So the main difference between environment variables and shell variables is their scope: environment variables are global while non exported shell variables are local to the script.
Note also that modern shells (at least ksh and bash) support a third shell variables scope. Variables created in functions with the typeset keyword are local to that function (The way the function is declared enables/disables this feature under ksh, and persistence behavior is different between bash and ksh). See https://unix.stackexchange.com/a/28349/2594
1This applies to modern shells like ksh, dash, bash and similar. The legacy Bourne shell and non Bourne syntax shells like csh have different behaviors.
Shell variables are difficult to duplicate.
$ FOO=bar
$ FOO=zot
$ echo $FOO
zot
$
Environment variables however can be duplicated; they are just a list, and a list can have duplicate entries. Here's envdup.c to do just that.
#include <err.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
extern char **environ;
int main(int argc, char *argv[]) {
char **newenv;
int envcount = 0;
if (argc < 2) errx(64, "Usage: envdup command [args ..]");
newenv = environ;
while (*newenv++ != NULL) envcount++;
newenv = malloc(sizeof(char *) * (envcount + 3));
if (newenv == NULL) err(1, "malloc failed");
memcpy(newenv, environ, sizeof(char *) * envcount);
newenv[envcount] = "FOO=bar";
newenv[envcount+1] = "FOO=zot";
newenv[envcount+2] = NULL;
environ = newenv;
argv++;
execvp(*argv, argv);
err(1, "exec failed '%s'", *argv);
}
Which we can compile and run telling envdup to then run env to show us what environment variables are set...
$ make envdup
cc envdup.c -o envdup
$ unset FOO
$ ./envdup env | grep FOO
FOO=bar
FOO=zot
$
This is perhaps only useful for finding bugs or other oddities in how well programs handle **environ.
$ unset FOO
$ ./envdup perl -e 'exec "env"' | grep FOO
FOO=bar
$ ./envdup python3 -c 'import os;os.execvp("env",["env"])' | grep FOO
FOO=bar
FOO=zot
$
Looks like Python 3.6 here blindly passes along the duplicates (a leaky abstraction) while Perl 5.24 does not. How about the shells?
$ ./envdup bash -c 'echo $FOO; exec env' | egrep 'bar|zot'
zot
FOO=zot
$ ./envdup zsh -c 'echo $FOO; exec env' | egrep 'bar|zot'
bar
FOO=bar
$
Gosh, what happens if sudo only sanitizes the first environment entry but then bash runs with the second? Hello PATH or LD_RUN_PATH exploit. Is your sudo (and everything else?) patched for that hole? Security exploits are neither "an anecdotal difference" nor just "a bug" in the calling program.