Self-referential list or table of buttons

Of course!

button := Button["Press me!",
  list = Drop[list, 1]];
list = Table[button, {5}];
Dynamic[list]

If you only want to remove one button, the solution is easy:

list = {Button[1], Button[2, list = Delete[list, 2]], Button[3]};
Dynamic@list

If you want to remove multiple unique buttons, you have to use some kind of identification for each button other than the actual position in the list, as that is changed when one of them is removed. Here I use a $position \rightarrow button$ identifier:

list2 = (# -> Button[#, list2 = DeleteCases[list2, _[#, _], 1]]) & /@ Range@4;
Dynamic@(Last /@ list2)

After v10, it is even simpler with associations:

a = AssociationMap[Button[#, a = KeyDrop[a, #]] &, Range@4];
Dynamic@Values@a

The crux of the problem is defining an identity for the buttons. As such a nice way is to keep them unevaluated in the list and define a structure to display them as buttons, this makes identifying them and deleting them simple:

 list = Table[button[i], {i, 5}];

 SetAttributes[display, HoldAll]
 display[list_] := list /. button[id_] :> 
     Button["#" <> ToString[id] <> " Press me!", list = DeleteCases[list, button[id]]]

 Dynamic[display[list]]

The above images shows the state after #4 was clicked and deleted. Using this method, the list is quite simply the line:

{button[1], button[2], button[3], button[4], button[5]}

Which is easy to handle and manipulate, and the buttons are just a visual representation of it. And it makes it very easy to put new buttons back, just run AppendTo[list,button[42]] and you get a new button, without having to retype all the code related to how it works in the dynamic display.