Minecraft castle fractal

Jelly, 43 36 35 bytes

ḶṚ3*µ5B¤xЀṁ€Ṁ×\Ṛ©1,‘xS$¤ṁ×®ị“*# ”Y

Just a start, I'm sure this could be shorter.

Try it online!

*For n > 5, your browser might wrap the output but if you copy-paste it into a non-wrapping editor, you will see the proper output.

Explanation

ḶṚ3*µ5B¤xЀṁ€Ṁ×\Ṛ©1,‘xS$¤ṁ×®ị“*# ”Y  Input: integer n
Ḷ                                    Create the range [0, n)
 Ṛ                                   Reverse it
  3*                                 Raise 3 to the power of each
    µ                                Begin a new monadic chain on the powers of 3
     5B¤                             Nilad. Get the binary digits of 5 = [1, 0, 1]
        xЀ                          Duplicate each of [1, 0, 1] to a power of 3 times
             Ṁ                       Get the maximum of the powers of 3
           ṁ€                        Reshape each to a length of that value
              ×\                     Cumulative products
                Ṛ©                   Reverse and save the result
                  1,‘xS$¤            Niladic chain.
                  1                    Start with 1
                    ‘                  Increment it
                   ,                   Pair them to get [1, 2]
                       $               Operate on [1, 2]
                      S                  Sum it to get 3
                     x                   Repeat each 3 times to get [1, 1, 1, 2, 2, 2]
                         ṁ           Reshape that to the saved table
                          ×®         Multiply elementwise with the saved table
                            ị“*# ”   Use each to as an index to select from "*# "
                                  Y  Join using newlines
                                     Return and print implicitly

JavaScript (ES7), 132 125 bytes

n=>[...Array(n)].map((_,i)=>[...Array(3**~-n)].map((_,j)=>/1/.test((j/3**i|0).toString(3))?" ":`*#`[j/3+i&1]).join``).join`\n`

Where \n represents the literal newline characer. ES6 version for 141 bytes:

f=
n=>[...Array(n)].map((_,i)=>[...Array(Math.pow(3,n-1))].map((_,j)=>/1/.test((j*3).toString(3).slice(0,~i))?" ":`*#`[j/3+i&1]).join``).join`
`
;

<input type=number min=1 oninput=o.textContent=f(+this.value)><pre id=o>

Python 2, 142 138 136 bytes

r=range
def f(n):
 for i in r(n+1):
  s="";d=i%2<1
  for k in r(3**i):s+="#*"[(6+d-1+k*(d*2-1))%6<3]
  exec"s+=len(s)*' '+s;"*(n-i);print s

This is the piece of code from here, and then edited for this challenge.

Will post an explanation later.

Also, BTW, two spaces are tabs.

Edit 1: 4 bytes saved thanks to @DJMcMayhem.

Edit 2: 2 bytes saved thanks to @daHugLenny.