On the Windows commandline, how do I get a dynamic prompt that tells me where in the filesystem I am?

I believe you can use the prompt command.

PROMPT [text]

  text    Specifies a new command prompt.

Prompt can be made up of normal characters and the following special codes:

  $A   & (Ampersand)
  $B   | (pipe)
  $C   ( (Left parenthesis)
  $D   Current date
  $E   Escape code (ASCII code 27)
  $F   ) (Right parenthesis)
  $G   > (greater-than sign)
  $H   Backspace (erases previous character)
  $L   < (less-than sign)
  $N   Current drive
  $P   Current drive and path
  $Q   = (equal sign)
  $S     (space)
  $T   Current time
  $V   Windows version number
  $_   Carriage return and linefeed
  $$   $ (dollar sign)

If Command Extensions are enabled the PROMPT command supports
the following additional formatting characters:

  $+   zero or more plus sign (+) characters depending upon the
       depth of the PUSHD directory stack, one character for each
       level pushed.

  $M   Displays the remote name associated with the current drive
       letter or the empty string if current drive is not a network
       drive.

---

Example:

setx PROMPT $P$S$$$S

Update

Create a batch file containing the code below and run.

@echo off

set root=%~p0
for %%F in ("%root%.") do set "ParentFolder=%%~nF"
setx PROMPT %ParentFolder%$S$F$S

Result is:

Based on the wikipedia entry for AUTOEXEC.BAT prompt $P$G will do what you want. This is a holdover from good old DOS. God only knows why something like this was never made a default since the early days of ... DOS3.3 or so.