How do devices like the Game Boy Advance achieve their frame rate?
Other answers cover your question pretty well at an abstract level (hardware), but having actual experience with the GBA in particular I figured a more detailed explanation may be worth while.
The GBA had many drawing modes and settings which could be used to control how the graphics processor interpreted the video RAM, but one thing was inescapable: the frame rate. The graphic processor was drawing to the screen in a nearly (more on this below) constant loop. (This is likely the most relevant bit for your question.)
It would draw one line at a time taking a very short break between each. After drawing the last line for the frame it would take a break roughly equal to the time it takes to draw 30 lines. Then start again. The timing of each line, and the timing of each frame were all predetermined and set in stone. In a lot of ways the graphics processor was really the master of that system and you needed to write your games around its behavior, because it would continue doing what it did whether you were ready or not.
Roughly 75-80% of the time it was actively pushing to the screen. What frame rates could you accomplish if you were doing the same?
That 80% of the time was also what the CPU had to process user input, calculate game state, and load sprites/tiles to areas of VRAM that were currently off screen (or at least not included in the current line being drawn).
The 20% between frames, was all the CPU had to tweak video settings or RAM that would impact the whole next frame.
At the end of each line, the graphics processor would send a line sync interrupt to the CPU. This interrupt could be used to tweak settings on a few sprites, or a few background layers (this is how you can get an effect like a conical spotlight, by changing the size and location of one of the rectangular masks between each line drawn. As far as the hardware is concerned all those regions are rectangular.). You have to be careful to keep these updates small and finish before the graphic processor starts drawing the next line or you can get ugly results. Any time spent processing these interrupts also cut into that 80% of the CPU's processing time...
For games that got the most out of this system, neither the CPU nor the graphic processor ever took a real break; each were chasing the other around the loop updating what the other wasn't currently looking at.
The key feature of all the games consoles that distinguished them from early PCs and virtually all home computers(1) was hardware sprites.
The linked GBA programming guide shows how they work from the main processor point of view. Bitmaps representing player, background, enemies etc are loaded into one area of memory. Another area of memory specifies the location of the sprites. So instead of having to re-write all of video RAM every frame, which takes a lot of instructions, the processor just has to update the location of the sprites.
The video processor can then work pixel by pixel to determine which sprite to draw at that point.
However, this requires dual-port RAM shared between the two, and I think in the GBA the video processor is on the same chip as the main ARM and secondary Z80 processor.
(1) Notable exception: Amiga
"My question is - how did a device like the GBA achieve a frame rate of nearly 60fps?"
To answer just the question, they did it with a graphics processer. I'm pretty sure the Game Boy used sprite graphics. At a top level, that means that the graphics processor gets loaded things like an image of a background, and an image of Mario, and an image of Princess Peach, etc. Then the main processor issues commands like "show the background offset by this much in x and y, overlay Mario image #3 at this x, y position", etc. So the main processor is absolutely positively not concerned with drawing each pixel, and the graphics processor is absolutely positively not concerned with computing the state of the game. Each is optimized for what it needs to do, and the result is a pretty good video game without using a lot of computation power.