How does the leading dollar sign affect single quotes in Bash?

It causes escape sequences to be interpreted.

$ echo $'Name\tAge\nBob\t24\nMary\t36'
Name    Age
Bob     24
Mary    36

After those sequences are expanded, the result is single-quoted, as if the dollar sign had not been present.

Using $ as a prefix tells BASH to try to find a variable with that name. $' is a special syntax (fully explained here) which enables ANSI-C string processing. In this case, the single tick isn't "take value verbatim until the next single tick". It should be quite safe to use. The drawbacks are it's BASH only and quite uncommon, so many people will wonder what it means.

The better way is to use single quotes. If you need a single quote in a string, you need to replace it with '\''. This ends the previous single quoted string, adds a single quote to it (\') and then starts a new single quoted string. This syntax works with any descendant of the Bourne shell, it's pretty easy to understand and most people quickly recognize the pattern.

The alternative is to replase each single tick with '"'"' which translates to "terminate current single quoted string, append double quoted string which contains just a single tick, restart single quoted string". This avoid the escape character and looks nicely symmetric. It also works the other way around if you need a double quote in a double quoted string: "'"'".