Tutorial On Exception Hooks in Python

• How to invoke the pdb debugger in earlier versions of Python.
• How to make use of the new, built-in breakpoint() function introduced in Python 3.7.
• How to write your own breakpoint() function to simplify the debugging process in earlier versions of Python.
• How to use a post-mortem debugger

Let’s get started.

Tutorial overview

This tutorial is divided into three parts; they are:

• Setting Breakpoints in Python Code
  • Invoking the pdb Debugger in Earlier Versions of Python
  • Using the breakpoint() Function in Python 3.7
• Writing One’s Own breakpoint() Function for Earlier Versions of Python
• Limitations of the breakpoint() function

Setting breakpoints in Python code

We have previously seen that one way of debugging a Python script is to run it in the command line with the Python debugger.

In order to do so, we would need to use of the –mpdb command that loads the pdb module before executing the Python script. In the same command line interface, we would then follow this by a specific debugger command of choice, such as n to move to the next line, or s if our intention is to step into a function.

This method could become quickly cumbersome as the length of the code increases. One way to address this problem and gain better control over where to break your code, is to insert a breakpoint directly into the code.

Invoking the pdb debugger in earlier versions of Python

Doing so prior to Python 3.7 would require you to importpdb, and to call pdb.set_trace() at the point in your code where you would like to enter an interactive debugging session.

If we reconsider, as an example, the code for implementing the general attention mechanism, we can break into the code as follows:

from numpy import array
from numpy import random
from numpy import dot
from scipy.special import softmax

# importing the Python debugger module
import pdb

# encoder representations of four different words
word_1 = array([1, 0, 0])
word_2 = array([0, 1, 0])
word_3 = array([1, 1, 0])
word_4 = array([0, 0, 1])

# stacking the word embeddings into a single array
words = array([word_1, word_2, word_3, word_4])

# generating the weight matrices
random.seed(42)
W_Q = random.randint(3, size=(3, 3))
W_K = random.randint(3, size=(3, 3))
W_V = random.randint(3, size=(3, 3))

# generating the queries, keys and values
Q = dot(words, W_Q)
K = dot(words, W_K)
V = dot(words, W_V)

# inserting a breakpoint
pdb.set_trace()

# scoring the query vectors against all key vectors
scores = dot(Q, K.transpose())

# computing the weights by a softmax operation
weights = softmax(scores / K.shape[1] ** 0.5, axis=1)

# computing the attention by a weighted sum of the value vectors
attention = dot(weights, V)

print(attention)

Although functional, this is not the most elegant and intuitive approach of inserting a breakpoint into your code. Python 3.7 implements a more straightforward way of doing so, as we shall see next.

Using the breakpoint() function in Python 3.7 

Python 3.7 comes with a built-in breakpoint() function that enters the Python debugger at the call site (or the point in the code at which the breakpoint() statement is placed).

When called, the default implementation of the breakpoint() function will call sys.breakpointhook(), which in turn calls the pdb.set_trace() function. This is convenient because we will not need to importpdb and call pdb.set_trace() explicitly ourselves.

Let’s reconsider the code for implementing the general attention mechanism, and now introduce a breakpoint via the breakpoint() statement:

from numpy import array
from numpy import random
from scipy.special import softmax

# encoder representations of four different words
word_1 = array([1, 0, 0])
word_2 = array([0, 1, 0])
word_3 = array([1, 1, 0])
word_4 = array([0, 0, 1])

# stacking the word embeddings into a single array
words = array([word_1, word_2, word_3, word_4])

# generating the weight matrices
random.seed(42)
W_Q = random.randint(3, size=(3, 3))
W_K = random.randint(3, size=(3, 3))
W_V = random.randint(3, size=(3, 3))

# generating the queries, keys and values
Q = words @ W_Q
K = words @ W_K
V = words @ W_V

# inserting a breakpoint
breakpoint()

# scoring the query vectors against all key vectors
scores = Q @ K.transpose()

# computing the weights by a softmax operation
weights = softmax(scores / K.shape[1] ** 0.5, axis=1)

# computing the attention by a weighted sum of the value vectors
attention = weights @ V

print(attention)

One advantage of using the breakpoint() function is that, in calling the default implementation of sys.breakpointhook() the value of a new environment variable, PYTHONBREAKPOINT, is consulted. This environment variable can take various values, based on which different operations can be performed.

For example, setting the value of PYTHONBREAKPOINT to 0 disables all breakpoints. Hence, your code could contain as many breakpoints as necessary, but these can be easily stopped fromhalting the execution of the code without having to remove them physically. If (for example) the name of the script containing the code is main.py, we would disable all breakpoints by calling it in the command line interface as follows:

PYTHONBREAKPOINT=0 python main.py

Otherwise, we can achieve the same outcome by setting the environment variable in the code itself: