Would a reverse-proxy authentication server be a secure setup?

Reverse Proxies are pretty common for what you are asking. Quite a few companies make servers designed for what your asking so you could use that as a reference.

For instance, I've used WebSeal (IBM ISAM) quite a bit at company's (seems popular for some reason around me). They have modules already build for OAuth, and most other type of authentication.

You can use these servers to:

• provide a single "Identity" across multiple systems with difference User stores.
• Provide a more secure method of Authentication for users, on legacy systems which may lack the features you want (e.g Provide OAuth authorizations for users, while the proxy uses Basic Auth for a back-end call.)
• Isolate systems by forcing connections through a single point
• Combinations for all of these.

Design notes:

• OAuth is an AUTHORIZATION protocol NOT an AUTHENTICATION protocol.
• When you are trying to establish identity, look to OpenID or JWT or SAML or run as your owned Identity Provider.

• When you are trying to authorize a request look at OAuth 2.0 or JWT.

• Use an ID Token for Identity (e.g OpenID spec, or if rolling your own look at JWT's)

• Have Web Applications use an authorization token to get a access token.

• Use access tokens to provide protected resources.