Python Debugging Hub

Debugging code with print statements, while often criticized, remains a widely adopted approach to Python debugging and can have benefits over a debugger. A few well-placed print statements in a complex codebase can give you a high-level overview of what's happening in the code over time. Debugger workflows, on the other hand, tend to focus on the state of a program at specific moments. While debuggers offer tools that are in many ways more powerful and flexible, the humble print statement can also be a valuable Python debugging tool.

To make your print debugging as effective as possible, consider three points: what to print, where to print, and how to print. We'll dive into each of these below.

What To Print

Print statements used for Python debugging usually do one of two things: show the current value(s) of one or more variables and indicate the current position in the code. The simplest way we can do this is manually:

x = 1
print("example.py:2 --> x = ", x)
if x == 1:
    x += 1

When we run this script, it will produce the following output:

example.py:2 --> x = 1

We can save ourselves some effort by using a few Python built-ins to supply this information instead of writing it out manually. For example:

import inspect

x = 1
print(f"{__file__}:{inspect.currentframe().f_lineno} --> {x = }")
if x == 1:
    x += 1

Running this code will produce the following output:

/home/user/example.py:4 --> x = 1

Here, we've used Python's built-in __file__ variable to get the filename and the inspect module to get the current line number instead of specifying them manually.

We've also used a special piece of f-string syntax, the self-documenting expression, to print out the name and value of x. We can do this with variables of any type and even expressions. Some examples:

x = 1
y = 2
print(f"{x = }") # will print "x = 1"
print(f"{y=}") # will print "y=2" (note the lack of spaces)
print(f"{x + y = }") # will print "x + y = 3"

This approach allows us to use the same line to display the value of x anywhere in our code, and easily change it to print the value of another variable instead:

print(f"{__file__}:{inspect.currentframe().f_lineno} --> {x = }")

print(f"{__file__}:{inspect.currentframe().f_lineno} --> {y = }")

We can also use functionality from the inspect module to return the name of the current function:

import inspect

def my_function():
    x = 1
    print(f"{__file__}:{inspect.currentframe().f_lineno} in function {inspect.currentframe().f_code.co_name} --> {x = }")
    if x == 1:
        x += 1

my_function()

# will print:
# /home/user/example.py:5 in function my_function --> x = 1

To print the values of multiple variables, you could add additional self-documenting expressions to the f-string or create additional print statements. But this approach is tedious if you want to print many variables, and won't help in cases where the names of variables are unknown, like when debugging someone else's code or using a new library. Fortunately, Python comes with several built-in functions to show the values of all variables in a given scope.

To get a dictionary containing all variables in the current function, call the locals() function:

def debug_me():
    x = 1
    y = 2
    z = 3
    print(locals())

debug_me() # will print "{'x': 1, 'y': 2, 'z': 3}

Similarly, to get a dictionary containing all variables defined in the top-level scope, call the globals() function at any point in the code. This will also print the values of built-in global variables such as __name__. Calling locals() outside of a function definition will produce the same output as globals().

Finally, to get the attributes of an object, use vars():

class MyObject:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

my_object = MyObject(1, 2, 3)
print(vars(my_object)) # will print "{'x': 1, 'y': 2, 'z': 3}

Calling vars() without arguments will produce the same output as globals().

Where To Print

We usually want to insert print statements at key points in the code, such as before, after, and during loops, at the start and end of functions, and within conditional branches. For example:

print("Starting loop")
for item in items:
    if accepted(item):
        print(f"Processing item: {item}")
        process(item)
    else:
        print(f"Discarding item: {item}")
print("Loop ended")

Depending on the nature of the bug you may not need all of these print statements, but the above code will produce a detailed log of each processed and discarded item.

Another valuable place to print is when catching exceptions, especially in cases where broad classes of exceptions are being caught and ignored.

try:
    risky_operation()
except Exception as e:
    print(f"Exception occurred: {e}")

For more on printing the details of exceptions, such as stack traces, please see this answer.

How To Print

As discussed above, we can use the inspect module and self-documenting expressions to display information about the current location in the codebase and the values of variables. It's better to rely on these techniques rather than manually constructing print statements to avoid mistakes like this:

print("x = ", y) # will display the wrong value for x

You can use pretty printing from Python's built-in pprint module to display complex data structures, like nested dictionaries, and lists, in a more readable format. For example, consider the following code:

data = [
    {"name": "Alice", "age": 30, "hobbies": ["reading", "cycling", "hiking"]},
    {"name": "Bob", "age": 25, "hobbies": ["painting", "fishing"]},
    {"name": "Charlie", "age": 35, "hobbies": ["swimming", "chess", "running"]},
]

print(data)

When executed, this script will produce the following difficult-to-read output:

[{'name': 'Alice', 'age': 30, 'hobbies': ['reading', 'cycling', 'hiking']}, {'name': 'Bob', 'age': 25, 'hobbies': ['painting', 'fishing']}, {'name': 'Charlie', 'age': 35, 'hobbies': ['swimming', 'chess', 'running']}]

If you replace print with pprint, for example:

data = [
    {"name": "Alice", "age": 30, "hobbies": ["reading", "cycling", "hiking"]},
    {"name": "Bob", "age": 25, "hobbies": ["painting", "fishing"]},
    {"name": "Charlie", "age": 35, "hobbies": ["swimming", "chess", "running"]},
]

from pprint import pprint
pprint(data)

The output is easier to read:

[{'age': 30, 'hobbies': ['reading', 'cycling', 'hiking'], 'name': 'Alice'},
 {'age': 25, 'hobbies': ['painting', 'gaming'], 'name': 'Bob'},
 {'age': 35, 'hobbies': ['swimming', 'chess', 'running'], 'name': 'Charlie'}]

You may also want to know when a given line was executed or measure the time between two points for benchmarking purposes. For that, you can use the methods in these two answers:

• Get the current time in Python
• Measure the elapsed time in Python

A standard breakpoint will pause the execution of a program on the line where it is set. A conditional breakpoint does the same thing, but only if a provided condition evaluates to True. If the condition is not met, the breakpoint is disregarded and execution continues.

Conditional breakpoints are incredibly useful in Python debugging for conducting focused investigations of specific edge cases in our program's logic. For example, if we want to debug code in a long-running loop without having to step through each iteration, or if we have a bug that appears only intermittently. In these cases, we can set conditional breakpoints to trigger only under specific conditions, so that we only pause execution at places that are relevant to our bug-squashing efforts.

Let's work through a simple example of how to use conditional breakpoints with the following script, which we'll save in a file named example.py:

def looper(x):
    import pdb; pdb.set_trace()  # standard breakpoint
    for i in range(10):
        x += i
        print(x) # we will break conditionally here

looper(5)

If we run this script, we should see output like the following:

$ python example.py
> /tmp/example.py(3)looper()
-> for i in range(10):
(Pdb)

Our script has started and executed up to the breakpoint set by pdb.set_trace(). Note that in modern versions of Python (3.7 and above), we can use the function breakpoint(), which will import pdb and call pdb.set_trace(). This is considered best practice as it allows breakpoints to be disabled, but for this example that is not essential.

The syntax for setting a standard breakpoint in pdb requires us to specify the filename and line number, as in the example below:

(Pdb) break example.py:5

To turn this breakpoint into a conditional breakpoint, we add a comma followed by a Python expression, as below:

(Pdb) break example.py:5, x > 10

After setting our conditional breakpoint, tell pdb to continue execution until the next breakpoint with the command c. It will then continue executing our program, running through each iteration of the for loop. Each time execution reaches the print function on line 5, pdb will evaluate our condition (x > 10). If it's false, execution will continue. If it's true, execution will pause. Here's what our pdb output thus far should look like:

> /tmp/pdbtest/example.py(3)looper()
-> for i in range(10):
(Pdb) b example.py:5, x > 10
Breakpoint 1 at /tmp/pdbtest/example.py:5
(Pdb) c
5
6
8
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here

As we can see, execution paused only on the fourth iteration of the loop (i = 3), when the value of x was set to 11 (8 + 3). With x only increasing in value from now on, our conditional breakpoint will now trigger on each subsequent iteration of the loop, as shown below:

> /tmp/pdbtest/example.py(3)looper()
-> for i in range(10):
(Pdb) b example.py:5, x > 10
Breakpoint 1 at /tmp/pdbtest/example.py:5
(Pdb) c
5
6
8
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
11
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
15
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
20
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
26
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
33
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
41
> /tmp/pdbtest/example.py(5)looper()
-> print(x) # we will break conditionally here
(Pdb) c
50

If this output is confusing to follow, consider using ipdb, the IPython debugger, in place of pdb. The ipdb debugging tool supports syntax highlighting and provides a more user-friendly default experience, while still understanding all the same commands used in pdb. It can be installed through PIP (pip install ipdb) and used as a drop-in replacement for pdb. For example:

def looper(x):
    import ipdb; ipdb.set_trace()  # standard breakpoint
    for i in range(10):
        x += i
        print(x) # we will break conditionally here

looper(5)

When executed, this script will now use ipdb instead of pdb. Running the script will now produce output like the following, showing more context around the current breakpoint by default:

$ python example.py
> /tmp/pdbtest/example.py(3)looper()
      2     import ipdb; ipdb.set_trace()  # standard breakpoint
  --> 3     for i in range(10):
      4         x += i
ipdb>

Exception handling in Python is done using try-except blocks. try-except blocks prevent your program from crashing when an error occurs. This is because your program will run the code in the except block if it encounters an error. Understanding how your code should run, and what potential states your application could get into that are incorrect is critical for setting up the correct try-except blocks. To do this, you wrap a block of code in a try block, which is followed by one or more except blocks. The try block contains the code that might raise an exception, while the except block(s) handle the exception if it occurs.

import traceback

try:
    result = 10 / 0
except ZeroDivisionError as e:
    print(f"Error occurred: {e}")
    traceback.print_exception(e)

In the example above, by importing the traceback library and calling the function traceback.print_exception(), it will print the stack trace when an exception occurs. A stack trace helps you trace back the path of execution for an exception by displaying the line numbers, functions, and methods involved. It's almost like having print statements throughout all of your code, but only printing when something actually goes wrong. You can also use the traceback.print_stack() to output the call stack/traceback at any point in the program, even when no exception has occurred. This can be useful to understand the code path for a specific place in your code when debugging Python.