ADVERTISEMENT

How to debug the stack trace that causes a subsequent exception in python?

You can use the with_traceback(tb) method to preserve the original exception's traceback:

try: 
    foo()
except TypeError as err:
    barz = 5
    raise ValueError().with_traceback(err.__traceback__) from err

Note that I have updated the code to raise an exception instance rather than the exception class.

Here is the full code snippet in iPython:

In [1]: def foo(): 
   ...:     bab = 42 
   ...:     raise TypeError() 
   ...:                                                                                                                                                         

In [2]: try: 
   ...:     foo() 
   ...: except TypeError as err: 
   ...:     barz = 5 
   ...:     raise ValueError().with_traceback(err.__traceback__) from err 
   ...:                                                                                                                                                         
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-a5a6d81e4c1a> in <module>
      1 try:
----> 2     foo()
      3 except TypeError as err:

<ipython-input-1-ca1efd1bee60> in foo()
      2     bab = 42
----> 3     raise TypeError()
      4 

TypeError: 

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-2-a5a6d81e4c1a> in <module>
      3 except TypeError as err:
      4     barz = 5
----> 5     raise ValueError().with_traceback(err.__traceback__) from err
      6 

<ipython-input-2-a5a6d81e4c1a> in <module>
      1 try:
----> 2     foo()
      3 except TypeError as err:
      4     barz = 5
      5     raise ValueError().with_traceback(err.__traceback__) from err

<ipython-input-1-ca1efd1bee60> in foo()
      1 def foo():
      2     bab = 42
----> 3     raise TypeError()
      4 

ValueError: 

In [3]: %debug                                                                                                                                                  
> <ipython-input-1-ca1efd1bee60>(3)foo()
      1 def foo():
      2     bab = 42
----> 3     raise TypeError()
      4 

ipdb> bab                                                                                                                                                       
42
ipdb> u                                                                                                                                                         
> <ipython-input-2-a5a6d81e4c1a>(2)<module>()
      1 try:
----> 2     foo()
      3 except TypeError as err:
      4     barz = 5
      5     raise ValueError().with_traceback(err.__traceback__) from err

ipdb> u                                                                                                                                                         
> <ipython-input-2-a5a6d81e4c1a>(5)<module>()
      2     foo()
      3 except TypeError as err:
      4     barz = 5
----> 5     raise ValueError().with_traceback(err.__traceback__) from err
      6 

ipdb> barz                                                                                                                                                      
5

EDIT - An alternative inferior approach

Addressing @user2357112supportsMonica's first comment, if you wish to avoid multiple dumps of the original exception's traceback in the log, it's possible to raise from None. However, as @user2357112supportsMonica's second comment states, this hides the original exception's message. This is particularly problematic in the common case where you're not post-mortem debugging but rather inspecting a printed traceback.

try: 
    foo()
except TypeError as err:
    barz = 5
    raise ValueError().with_traceback(err.__traceback__) from None

Here is the code snippet in iPython:

In [4]: try: 
   ...:     foo() 
   ...: except TypeError as err: 
   ...:     barz = 5 
   ...:     raise ValueError().with_traceback(err.__traceback__) from None    
   ...:                                                                                                                                                         
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-b090fb9c510e> in <module>
      3 except TypeError as err:
      4     barz = 5
----> 5     raise ValueError().with_traceback(err.__traceback__) from None
      6 

<ipython-input-6-b090fb9c510e> in <module>
      1 try:
----> 2     foo()
      3 except TypeError as err:
      4     barz = 5
      5     raise ValueError().with_traceback(err.__traceback__) from None

<ipython-input-2-ca1efd1bee60> in foo()
      1 def foo():
      2     bab = 42
----> 3     raise TypeError()
      4 

ValueError: 

In [5]: %debug                                                                                                                                                  
> <ipython-input-2-ca1efd1bee60>(3)foo()
      1 def foo():
      2     bab = 42
----> 3     raise TypeError()
      4 

ipdb> bab                                                                                                                                                       
42
ipdb> u                                                                                                                                                         
> <ipython-input-6-b090fb9c510e>(2)<module>()
      1 try:
----> 2     foo()
      3 except TypeError as err:
      4     barz = 5
      5     raise ValueError().with_traceback(err.__traceback__) from None

ipdb> u                                                                                                                                                         
> <ipython-input-6-b090fb9c510e>(5)<module>()
      3 except TypeError as err:
      4     barz = 5
----> 5     raise ValueError().with_traceback(err.__traceback__) from None
      6 

ipdb> barz                                                                                                                                                      
5

Raising from None is required since otherwise the chaining would be done implicitly, attaching the original exception as the new exception’s __context__ attribute. Note that this differs from the __cause__ attribute which is set when the chaining is done explicitly.

In [6]: try: 
   ...:     foo() 
   ...: except TypeError as err: 
   ...:     barz = 5 
   ...:     raise ValueError().with_traceback(err.__traceback__) 
   ...:                                                                                                                                                         
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-ee78991171cb> in <module>
      1 try:
----> 2     foo()
      3 except TypeError as err:

<ipython-input-2-ca1efd1bee60> in foo()
      2     bab = 42
----> 3     raise TypeError()
      4 

TypeError: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-5-ee78991171cb> in <module>
      3 except TypeError as err:
      4     barz = 5
----> 5     raise ValueError().with_traceback(err.__traceback__)
      6 

<ipython-input-5-ee78991171cb> in <module>
      1 try:
----> 2     foo()
      3 except TypeError as err:
      4     barz = 5
      5     raise ValueError().with_traceback(err.__traceback__)

<ipython-input-2-ca1efd1bee60> in foo()
      1 def foo():
      2     bab = 42
----> 3     raise TypeError()
      4 

ValueError: 

Yoel answer works and should be your go-to procedure, but if the trace is a bit harder to debug, you may instead use the trace module.

The trace module will print out each instruction executed, line by line. There is a catch, though. Standard library and package calls will also be traced, and this likely means that the trace will be flooded with code that is not meaningful.

To avoid this behavior, you may pass the --ignore-dir argument with the location of your Python library and site packages folder.

Run python -m site to find the locations of your site packages, then call trace with the following arguments:

python -m trace --trace --ignore-dir=/usr/lib/python3.8:/usr/local/lib/python3.8/dist-packages main.py args

Replacing the ignore-dir with all folders and the main.py args with a script location and arguments.

You may also use the Trace module directly in your code if you want to run a certain function, refer to this example extracted from https://docs.python.org/3.0/library/trace.html:

import sys
import trace

# create a Trace object, telling it what to ignore, and whether to
# do tracing or line-counting or both.
tracer = trace.Trace(
    ignoredirs=[sys.prefix, sys.exec_prefix],
    trace=0,
    count=1)

# run the new command using the given tracer
tracer.run('main()')

# make a report, placing output in /tmp
r = tracer.results()
r.write_results(show_missing=True, coverdir="/tmp")