Is it possible to simulate record locks in test methods?

I don't believe runas(User) is going to isolate the locking within the transaction. The sObject record locking spans the entire transaction, regardless of the runas statements.

You will probably need to simulate the concurrency UNABLE_TO_LOCK_ROW DmlException using Test.isRunningTest() and another type of exception. You could either cause an alternative type of DmlException or throw something specific to the test case. Neither is ideal as you are bringing elements of the test into the actual code.


Update:

It is now possible to directly throw a DMLException of your own making.

DmlException e = new DmlException();
e.setMessage('UNABLE_TO_LOCK_ROW');
throw e;

It is also possible to use the Stub API to simulate the return result of a public method.

In theory you could combine these two abilities to force what looks like an UNABLE_TO_LOCK_ROW DMLException in a test context without having to modify the actual code. You will need to ensure that the method to be stubbed is public and not static.

Proof of concept

Those with a sensitive testing disposition may want to look away. This is by no means the correct way to do dependency injection and inversion of control for stub testing. Among many other things, the stub for the class shouldn't be an inner class of the one being stubbed.

At the least it does demonstrate throwing a DMLException that looks like a locked record.

Class to be mocked:

public class DoesTheDml {
    // Must be public so it can be mocked!
    public void updateAccount(Id accountId) {
        Account accountToUpdate = [Select Id, Description from Account where Id = :accountId FOR UPDATE];
        accountToUpdate.Description = 'Updated';
        update accountToUpdate;
        System.assert(false, 'Bang! Actually interested in the mock firing');
    }

    public static DoesTheDml Instance {
        get {
            if(Instance == null) {
                Instance = new DoesTheDml();
            }
            return Instance;
        }
        private set {
            Instance = value;
        }
    }

    public static void updateTheAccount(Id accountId) {
        Instance.updateAccount(accountId);
    }

    private static DoesTheDml mockSettings;
    @TestVisible
    private static DoesTheDml createMock() {
        // Use static instance to avoid creating a new mock with each call
        if(mockSettings == null) {
            mockSettings = (DoesTheDml)Test.createStub(DoesTheDml.class, new DoesTheDmlMockProvider());
            System.debug('DoesTheDml.createMock() completed - new ' + mockSettings);
        } else {
            System.debug('DoesTheDml.createMock() completed - existing');
        }
        Instance = mockSettings;
        System.assertEquals(mockSettings, Instance);

        return mockSettings;
    }

    public class DoesTheDmlMockProvider implements System.StubProvider {
        public Object handleMethodCall(Object stubbedObject, String stubbedMethodName, 
            Type returnType, List<Type> listOfParamTypes, List<String> listOfParamNames, 
            List<Object> listOfArgs) {

            if(stubbedMethodName == 'updateAccount') {

                // Simulate the record locking result if a record is locked for update
                DmlException e = new DmlException();
                e.setMessage('UNABLE_TO_LOCK_ROW');
                throw e;

                //return null;
            } 

            System.assert(false, 'unmocked stubbedMethodName: ' + stubbedMethodName);

            return null;
        }
    }
}

Test class:

@IsTest
public class DoesTheDml_Test {
    @IsTest
    public static void updateTheAccountTest() {
        // Configure the Stub/Mock response
        DoesTheDml instance = DoesTheDml.createMock();

        Id mockAccountId = '001000000000001';

        try {
            DoesTheDml.updateTheAccount(mockAccountId);
            System.assert(false, 'Expected Exception');
        } catch (DmlException dmlEx) {
            System.assert(dmlEx.getMessage().contains('UNABLE_TO_LOCK_ROW'),
                'Expected UNABLE_TO_LOCK_ROW race condition');

            System.debug('We did it!');
        }
    }
}

Test methods do not actually run asynchronous operations asynchronously. All async requests (future calls, queueable Apex, batches, etc.) get queued up and are run synchronously when Test.stopTest is run.

So you will never be able to intentionally create a DML lock in a unit test (though when unit tests are run in parallel you may see them, particularly with custom settings, which should never actually be created or updated in a unit test - which is another story).

You'll want to test this the way you would any other exception handler, by throwing an error, or moving your handling code into a separate method and calling it directly.

Don't stress about keeping tests and solutions separate - that's a preference, not a religious dictate, and this is exactly the kind of scenario that Test.isRunningTest() exists to handle.