Parallel execution of a program on multiple files

This is possible and does occur in reality. Use a lock file to avoid this situation. An example, from said page:

if mkdir /var/lock/mylock; then
    echo "Locking succeeded" >&2
    echo "Lock failed - exit" >&2
    exit 1

# ... program code ...

rmdir /var/lock/mylock

The two instances of your script can certainly interact in this way, causing the command to run twice. This is called a race condition.

One way to avoid this race condition would be if each instance grabbed its input file by moving it to another directory. Moving a file (inside the same filesystem) is atomic. Moving the input files may not be desirable, and this is already getting a bit complicated.

mkdir staging-$$ making-$$
for input in folder/*; do
  if mv -- "$input" "$staging" 2>/dev/null; then
    bin/myProgram "$staging" >"$output"
    mv -- "$output" "$destination"
    mv -- "$staging" "$input"

A simple way to process the files in parallel using a widely-available tool is GNU make, using the -j flag for parallel execution. Here's a makefile for this task (remember to use tabs to indent commands):

all: $(patsubst folder/%,done/%,$(wildcard folder/*))
done/%: folder/%
        ./bin/myProgram $< >[email protected]
        mv [email protected] $@

Run make -j 3 to run 3 instances in parallel.

See also Four tasks in parallel... how do I do that?