Duplicate & switch case

Python, 56 54 bytes

lambda s:''.join(c+c.swapcase()*c.isalpha()for c in s)

Test it on Ideone.

JavaScript ES6, 70 68 66 64 bytes

Saved 2 bytes thanks to @Kevin Lau - not Kenny

Saved 2 bytes thanks to @Cᴏɴᴏʀ O'Bʀɪᴇɴ

s=>s.replace(/[A-Z]/gi,l=>l+l[`to${l<"a"?"Low":"Upp"}erCase`]())

Explanation

This uses a really hacky:

l[`to${l<"a"?"Low":"Upp"}erCase`]()

which ungolfed is:

l[`to${
   l < "a" ?
   "Low" : 
   "Upp"
}erCase`]()

Basically l < "a" checks if the code point of the letter is less then the code point of a (therefore being an uppercase letter). If it is it'll do to + Low + erCase which becomed l['toLowerCase']() and makes the character lowercase. ` quotes allow string formatting so essentially you can think of:

`to${l < "a" ?"Low" : "Upp"}erCase`

as: "to" + (l<"a" ? "Low" : "Upp") + "erCase" which generates the function to call (make the string upper or lower case). We put this in square brackets [ ... ] which lets us access a property given its name as a string. This returns the appropriate function and then we just call it.

Jelly, 5 bytes

żŒsQ€

Try it online!

How it works

żŒsQ€  Main link. Argument: s (string)

 Œs    Yield s with swapped case.
ż      Zip s with the result.
   Q€  Unique each; deduplicate each pair of characters.