python logging print traceback only in debug

Log the exception at DEBUG level instead and set exc_info=True. logger.exception() is essentially a logger.error(..., exc_info=True) call, but you can log exception tracebacks at any level:

log.debug("could not open configuration file", exc_info=True)

It's the exc_info option that's important; from the documentation:

If exc_info does not evaluate as false, it causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) or an exception instance is provided, it is used; otherwise, sys.exc_info() is called to get the exception information.

You perhaps want to use printing (to stdout or stderr) to communicate with the end-user:

except FileNotFoundError as e:
    log.debug("could not open configuration file", exc_info=True)
    print("Could not open configuration file:", e.strerror, file=sys.stderr)
    sys.exit(1)

I included the system error message in the print output without the FileNotFoundError(...) representation.

If you use a command-line argument parser like argparse or click, then do use their user feedback API (which usually includes exiting too).

You can make the logging module produce user-level messages too, but if you want a single logger call to produce debug-friendly tracebacks in a file and user-friendly output on a console, you'd have to configure separate handlers for these use-cases with the console handler using a custom Formatter() class to override the formatException() method to alter how exceptions are shown. It's just much easier and clearer to separate logging and end-user communication.

I'd use a combination of exc_info and .getEffectiveLevel:

try:
    ...
except FileNotFoundError as ex:
   logger.error(ex, exc_info=log.getEffectiveLevel() == logging.DEBUG)

This way, the exception itself (FileNotFoundError) is always logged, but the stacktrace will only be logged if log level is debug.