What does `.[].foo[]` do in bash? Why does it match `..`?

The [ starts a set. A set is terminated by ]. But there is a way to have ] as part of the set, and that is to specify the ] as the first character. As an empty set doesn't make any sense, this is not ambiguous.

So your examples are basically all a dot followed by a set that contains a dot, therefore it matches two dots.

The later examples don't find any files and are therefore returned verbatim.

Only quoted strings are not subject to globbing:

$ echo ".[].aliases[]"
.[].aliases[]

But un-quoted strings are subject to globbing. An unquoted string that contains an * or a ? or (valid) [] (bracket expression) will be modified by the list of files that match it. In the same way as a * will transform into all the files in the matching directory and a ? will match files of only one character, a (valid) [] will match files with the characters inside the brackets. A dot is a valid character:

$ echo a[.]b
a[.]b

$ touch "a.b"
$ echo a[.]b
a.b

To be able to match a ] it should be the first character inside the brackets:

$ touch "a]b"
$ ls a[]]b
a]b

An empty bracket expression makes no sense (and is not expanded):

$ touch ab
$ ls a[]b
ls: cannot access 'a[]b': No such file or directory

That is why this works:

$ touch a]c abc afc azc a:c a?c aoc 
$ ls a[]bfz:?]c
abc  a:c  a?c  a]c  afc  azc

For [ the idea is similar:

$ touch a[c
$ ls a[[]c
a[c

but it could be at any position in a bracket expression:

$ ls a[]bf[z:?]c
abc  a:c  a?c  a[c  a]c  afc  azc

$ ls a[]bfz:?[]c
abc  a:c  a?c  a[c  a]c  afc  azc

The string you posted .[].foo[] will match a dot followed by either a ], a ., a f, a o or a [. It is similar to:

$ echo a[].foo[]c
a[c a]c afc aoc

And it will match as follows:

$ touch .] .f .o .[ .a .b .z

$ echo .[].foo[]
.. .[ .] .f .o

Note that the directory entry .. does not need to be created as it exists inside every directory by default. But a simple dot . won’t be matched by a glob as it needs to be matched explicitly (by actually using a dot).

But that will not match ..aliases as the bracket expression will only match one character. To match several characters you need to use a * (anything):

$ touch ..a ..l ..i ..aliases ..alias ..ali
$ echo .[].aliases[]
.. .[ .] .a

$ echo .[].aliases[]*
.. .[ .] .a ..a ..ali ..alias ..aliases ..i ..l