Customize the Duplicate Username error message in a Community

Its not in the controller. Since you have a Lightning Community, the error messages are controlled by Lightning Components. See this help page and the image below for more.

I eventually was able to determine the correct solution, though I didn't end up actually implementing it. The steps that needed to be taken were:

1. Duplicate the existing standard lightning self registration component (this was important -- I thought I could just edit the standard component, but apparently that's not feasible).
2. In the controller, find the line of code which calls a function to create a user in the system
3. Before the above-mentioned line of code, use a SOQL query to check if any user exists with the same username (this code should already be written since it is standard)
4. Wrap that segment of code in a try-catch or if-else statement. Add your custom error message there, such that if the SOQL query returns 1 or more records, your custom error is displayed to the user.
5. Add your new component to a new community page
6. Update your Community Administration settings to use your new custom page for registration
7. Test it out by registering a User with an existing username

That should do the trick!