Fizzbuzz in any base

JavaScript (ES6),  117  116 bytes

Outputs comma-delimited digits, each digit being expressed as a decimal quantity (e.g. \$19_{20}\$ is \$19\$ and \$21_{20}\$ is \$1,1\$).

b=>(g=n=>n>1?g(n-1)+`
`+((s=n%(b+2>>1)?'':'Fizz',n%(b/3+3.9|0)?s:s+'Buzz')||(g=n=>n?[...g(n/b|0),n%b]:s)(n)):1)(5e3)

Try it online!

(limited to 100 so that TIO's output does not blow up)

Jelly,  42 38 34 33 29  32 bytes

+3 to adhere to strict formatting rules

5ȷɓ;8Ä:2,3‘ḍȧ"“Ƈד=%»ḟ0Fȯb@K¥ð€Y

A full program which prints 5000 lines of text, each line containing a series of integers (the digits) or one of fizz, buzz, or fizzbuzz (works fine beyond base 62).

Try it online!

How?

Note that
\$\lfloor b÷2+1\rfloor\ = \lfloor b÷2\rfloor+1\$
...and
\$\lceil b÷3+3\rceil = \lceil b÷3+2\rceil+1 = \lceil (b+6)÷3\rceil+1 = \lfloor (b+8)÷3\rfloor+1\$

updating...

5ȷɓ;8Ä:2,3‘ḍȧ"“Ƈד=%»ḟ0Fȯb@ð€ - Link: integer, b
5ȷ                            - 5*10³ = 5000
  ɓ                        ð€ - for €ach n in [1,2,...,5000] get this f(b,n):
    8                         -   eight
   ;                          -   concatenate      -> [b,8]
     Ä                        -   cumulative sums  -> [b,b+8]
       2,3                    -   pair literal        [2,3]
      :                       -   integer division -> [b//2, (b+8)//3]
          ‘                   -   increment        -> [b//2+1, (b+8)//3+1]
           ḍ                  -   divides n?       -> [n is fizzy?, n is buzzy?]
              “Ƈד=%»         -   list of dictionary strings = ['fizz','buzz']
             "                -   zip with:
            ȧ                 -     logical AND    -> [0,0], ['fizz',0], [0,'buzz'],
                              -                       or ['fizz','buzz']
                      0       -   zero
                     ḟ        -   filter discard   -> [], ['fizz'], ['buzz'],
                              -                       or ['fizz','buzz']
                       F      -   flatten          -> [], ['fizz'], ['buzz'],
                              -                       or ['fizzbuzz']
                          @   -   using swapped arguments:
                         b    -     (n) to a list of digits in base (b)  (say, [nb])
                        ȯ     -   logical OR       -> [nb], ['fizz'], ['buzz'],
                              -                       or ['fizzbuzz']

Charcoal, 40 bytes

ＮθＥ…·¹×⁵φ∨⁺⎇﹪ι⊕÷θ²ωFizz⎇﹪ι÷⁺¹¹θ³ωBuzz⍘ιθ

Try it online! Link is to verbose version of code. Explanation:

Ｎθ                                      Input `b` into variable `q`
   ¹                                    Literal 1
 …·                                     Inclusive range to
      φ                                 Predefined variable 1000
    ×                                   Multiplied by
     ⁵                                  Literal 5
Ｅ                                       Map to
           ι                            Current value
          ﹪                             Modulo
              θ                         Input value
             ÷                          Floor divide
               ²                        Literal 2
            ⊕                           Incremented
         ⎇                              If nonzero
                ω                       Then predefined empty string
                 Fizz                   Otherwise literal `Fizz`
        ⁺                               Concatenated with
                       ι                Current value
                      ﹪                 Modulo
                            θ           Input value
                         ⁺              Plus
                          ¹¹            Literal 11
                        ÷               Integer divided by
                             ³          Literal 3
                     ⎇                  If nonzero
                              ω         Then predefined empty string
                               Buzz     Otherwise literal `Buzz`
       ∨                                Logical Or
                                    ι   Current value
                                   ⍘    Converted to base
                                     θ  Input value
                                        Implicitly print each result on its own line