Accessing array index variable from bash shell script loop?

You can do this using List of array keys. From the bash man page:

${!name[@]}
${!name[*]}

List of array keys. If name is an array variable, expands to the list of array indices (keys) assigned in name. If name is not an array, expands to 0 if name is set and null otherwise. When @ is used and the expansion appears within double quotes, each key expands to a separate word.

For your example:

#!/bin/bash
AR=('foo' 'bar' 'baz' 'bat')
for i in "${!AR[@]}"; do
  printf '${AR[%s]}=%s\n' "$i" "${AR[i]}"
done

This results in:

${AR[0]}=foo
${AR[1]}=bar
${AR[2]}=baz
${AR[3]}=bat

Note that this also work for non-successive indexes:

#!/bin/bash
AR=([3]='foo' [5]='bar' [25]='baz' [7]='bat')
for i in "${!AR[@]}"; do
  printf '${AR[%s]}=%s\n' "$i" "${AR[i]}"
done

This results in:

${AR[3]}=foo
${AR[5]}=bar
${AR[7]}=bat
${AR[25]}=baz

Additional to jordanm's answer you can also do a C like loop in bash:

for ((idx=0; idx<${#array[@]}; ++idx)); do
    echo "$idx" "${array[idx]}"
done