How is this SQL Server PK violation possible?
Now I have a SQL statement, which selects the MAX value, adds to it, and INSERTs that value into the same table -- all inside one statement. So in theory, it's not possible to get a PK violation on this statement (although I think could possibly get a deadlock). But, somehow, it IS getting one.
It definitely IS possible. This is due to your isolation level and how locking works.
Violation of PRIMARY KEY constraint 'PK_IDMaster'. Cannot insert duplicate key in object 'IDMaster'. The duplicate key value is (25309587).
This is due to locking; initially in the read committed isolation level will take a shared lock on the (in this case) key while reading what the maximum value is. After it figures it out, an X lock will be needed to insert into the table. That's the simple overview of it.
When you have a single session, this isn't a problem. When you have multiple sessions this is a huge problem as shared locks are compatible with each other. This means that multiple readers can obtain the same value.
don't know how this is possible, but apparently it is; because it just happened.
See above.
So How Do You Fix It?
Don't try to be smarter than SQL Server. Use the available constructs that fit the need - in SQL Server 2012 (major version 11) has both identity and sequences. Use them.
Purposely kill your concurrency by either using serializable or lock hints such as XLOCK. This will on purpose block other sessions which means you'll have less performance... and you're doing it on purpose... so yeah that seems bad but a possible solution.
Pre-create values (again trying to be smarter than SQL Server) and use XLOCK + READPAST to get a little better concurrency from a different table. Blah. Let me reiterate #1, don't try to be smarter than SQL Server [in this case].
Your theory is wrong. Reading from your "identity" table doesn't lock it, the insert into it locks it. While they are part of a single compound statement, they are still seperate actions. Two (or more) connections can be reading from the table during the time between the read and the insert. The insert uses a lock, so only one of the connections will actually be able to write the value they read into the table, other connections, will, as you have discovered, get a primary key violation error.
The best solution is to use one of the builtin methods for handling this: identity columns, sequences, or guids. Roll your own solutions should generally be avoided: you simply don't have the experience or user base that the database management systems do (and you are dealing with an abstraction, so even if you were one of the programmers for one of the major database management systems, you wouldn't be able to do the same kind of things).