Numbers that are palindromes in N bases

Mathematica, 80 71 bytes

Thanks to JungHwan Min for saving 9 bytes!

Do[#!=Length[i~IntegerReverse~Range[2,i-2]~Cases~i]||Echo@i,{i,3,∞}]&

(∞ is the three-byte character U+221E.) Pure function taking a nonnegative integer as input. i~IntegerReverse~Range[2,i-2] creates a list of the reversals of the number i in all the bases from 2 to i-2; then Length[...~Cases~i] counts how many of these reversals are equal to i again. #!=...||Echo@i halts silently if that count is not equal to the input, and echoes i if it is equal to the input. That procedure is embedded in a straightforward infinite loop.

Jelly,  18  15 bytes

-1 thanks to caird coinheringaahing.

ṄµbḊŒḂ€S’⁼³µ¡‘ß

Try it online! - the online interpreter will timeout at 60 seconds then flush it's output (unless it has a cached copy), offline it will print each in turn.

How?

Evaluates numbers from n up, printing them if they are in the sequence. Note that the first number in any output will be greater than n since otherwise the range of b is not great enough, so there is no need to seed the process with 3. Also note that the number of palindromes from base 2 to x_i-2 inclusive is just one less than the number of palindromes from base 2 to x_i.

ṄµbḊŒḂ€S’⁼³µ¡‘ß - Main link: n
            ¡   - repeat...
Ṅ               - ...action: print (the CURRENT n)
 µ         µ    - ...number of times: this monadic chain:
   Ḋ            -      deueue -> [2,3,4,...,n]
  b             -      n converted to those bases
    ŒḂ€         -      is palindromic? for €ach
       S        -      sum: counts the 1s
        ’       -      decrement: throw away the truthy base n-1 one
         ⁼³     -      equals input: Does this equal the ORIGINAL n?
             ‘  - increment n
              ß - calls this link as a monad, with the incremented n.