Talk to your Teddy (Debugging)

Presenter     Good evening. Tonight ‘Spectrum’ looks at one of the major problems in the world today – that old vexed question of what is going on. Is there still time to confront it, let alone solve it, or is it too late? What are the figures, what are the facts, what do people mean when they talk about things? Alexander Hardacre of the Economic Affairs Bureau.
Cut to equally intense pundit in front of a graph with three different coloured columns with percentages at the top. He talks with great authority.
Hardacre     In this graph, this column represents 23% of the population. This column represents 28% of the population, and this column represents 43% of the population.

One of the irritating things about writing programs is that, often, when you type your program in perfectly, it still doesn’t work.   You’ve got bugs!  (Apparently, one of the first errors in the operation of a computer occurred because a moth got stuck in amongst the componentry. Its removal thus started the act of debugging computer programs.)

The good news is that, in every tutorial I’ve done on this blog I have had at least one, and often many, bugs in the programs I’ve written – even the ones which are only 2 or 3 lines long!!!!  Getting things wrong is a natural part of writing programs.  We need to know how to get things right after they are wrong.  So in this tutorial I wanted to talk about bugs and debugging and things to look for when trying to get your programs running.

The two main causes of bugs we’re going to look at are syntax errors and logical errors.

Syntax errors

A syntax error means that you’ve said something to the computer which doesn’t make grammatical sense.  Thus, if you were to say to your sibling, “Yesterday I the car,” they’d have absolutely no idea what you were talking about.  That’s because this sentence doesn’t fit with English rules of grammar (it lacks a verb).   Similarly, Python programs have their own “grammar”.  If you deviate from that grammar the Python interpreter is unable to work out what you mean.

Often a syntax error is caused by a single typo somewhere in the code.  Here are some examples:

>>> def aFunction
File "<stdin>", line 1
def aFunction
^
SyntaxError: invalid syntax

Can you see here that the interpreter is trying to tell you where the problem is?  The caret ^ shows where Python thinks that there is a problem.  The interpreter is not always right, but this is a good place to start.

>>> def aFunction()
File "<stdin>", line 1
def aFunction()
^
SyntaxError: invalid syntax

First, the interpreter expects a function to have a list of parameters, so is looking for a pair of parentheses -> (). Then, it also expects to see a colon -> : to mark where the function’s code starts.

>>> def aFunction():
...
File "<stdin>", line 2

^
IndentationError: expected an indented block

Here, the interpreter sees that you’ve tried to define a function, but hasn’t found any code for the function.  Code for a function is indented, hence, if it’s not there, the interpreter thinks that there’s a problem with the indentation.

>>> def aFunction():
...   print "this is a function"
...

Note that you need to press return on a blank line to finish the function’s definition if you are in the interpreter.  This is not the case if you are using a text file for your program. Leaving out a colon or having the wrong indentation are perhaps two of the most frequent syntax errors.   As a refresher, Python uses indentation to mark off code blocks.  This means that all the code which is to be executed at the same code level must be indented to the same indent level.  There is no “right” level of indentation, as long as the indentation is consistent.   So these are both ok:

>>> def aFunction():
...   print "First line of code"
...   print "second line of code"
...
>>> def bFunction():
...      print "first line of code"
...      print "second line of code"
...
>>> aFunction()
First line of code
second line of code
>>> bFunction()
first line of code
second line of code

However, the function below won’t work because the level of indents (ie the number of spaces before the text starts) is different for each line within the same code block.  A code block is (always? <- not sure) marked off by a colon : in the previous line.

>>> def cFunction():
...     print "first line of code"
...        print "second line of code"
File "<stdin>", line 3
print "second line of code"
^
IndentationError: unexpected indent

Where you have a function you need to pass it the right number of arguments:

>>> range()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: range expected at least 1 arguments, got 0

>>> range(3)
[0, 1, 2]

The range() function is interesting in that it can take between 1 and 3 arguments (the second and third arguments are optional).  However if you pass no arguments, or more than 3, then the interpreter will complain to you.

Logical Errors

Logical errors are usually harder to diagnose.  When you write the program, because you have used the right syntax, the Python interpreter doesn’t tell you that there’s an error.  Nevertheless, when you run it, you get the wrong answer or the program crashes half way through.  Logical errors often happen with loops, for example, where you forget that the range() function starts at zero, or you have got some other offset incorrect.  They can also happen where you use a function, but don’t understand what it does, or what output it is supposed to produce.

When faced by a logical error, the usual way to debug is to break down the program into different components and to inspect each in turn.   When you’re more advanced, you can put logging messages into the code and Python will log the progress of the program to a separate log file.  For the time being however, the easiest thing to do is to put a print statement in.  You put a number of print statements in and sooner or later one of them will print the wrong thing.  Then you know that the error is before that print statement, but after the previous one.

Others

Usually if you make a mistake it will show up as one of these two errors.  However, sometimes you’ve just made a typing mistake but haven’t realised it – like typing * instead of + somewhere.

Lastly and Most Importantly – Talk to your Teddy

I’ve left the most important bit to last.  When you have a debugging problem that you really just can’t solve, get your teddy bear (I’m serious, get your bear), bring it to the monitor and explain (in words, out loud) to your teddy what the problem is and why it isn’t working.

This is a sure-fire, clinically proven way to solve whatever debugging problem you have.  As I mentioned earlier, I’m absolutely serious about sitting your teddy down beside you and talking (with your voice, not in your head) to it. Now, it doesn’t absolutely have to be a teddy.  It could be a Lego figure, or an action figure, a parent or a sibling (if they’re willing) or even a balloon with a face and ears drawn on in pen.

The important thing is that you explain aloud in words what the problem is, because that helps you identify it for yourself.

Review, Formatting Silly Sentences

Voice Over     Well it’s a straight fight here at Leicester…On the left of the Returning Officer (camera shoes grey-suited man) you can see Arthur Smith, the Sensible candidate and his agent, (camera pans to silly people) and on the other side is the silly candidate Jethro Walrustitty with his agent and his wife.
Officer     Here is the result for Leicester. Arthur J. Smith…
Voice Over     Sensible Party
Officer     30,612…
Jethro Q. Walrustitty…
Voice Over     Silly Party
Officer     32,108.

We have been covering lots of new concepts very quickly. So now is the time to slow down a bit and go over some of the things we’ve already done, to have a look at them in a little more depth. In particular I want to try to give you some direction on how to make these code samples your own.

I also noticed that I did some things in the last tute which we haven’t encountered yet.  In particular, I:

• assigned a function to an object r = random.randint.  Isn’t Python neat?  You can make these assignments, then the name r points at the function random.randint, so you can use r wherever you use the function.  Sometimes you can use this pattern to make your program run faster (honest!).  However, it might be confusing so I’ll stop it now!
• used tab characters to format a print out: ‘\t’.  There are some “control” characters which modify the usual output of a set of printed characters.  Of particular interest are ‘\n’ (which makes a new line) and ‘\t’ which adds  a tab space. Try printing some strings with \n and \t in them to see what they do.

In this tutorial we’re going to look at formatting strings. In the previous tute we constructed a string by adding (using the ‘+’ operator) them together. Building strings this way can be a little challenging. Another way to do it, is to use formatting strings. We put place markers into a string, and match the place markers with values to go in there. The place markers are identfied by a % sign in the string, and values are added at the end after  a % sign. Here is a simple example:

>>> print '%s'%'hi'
hi

The string ‘hi’ is inserted in place of the %s (%s (the s is important – not just the % by itself) says insert a string here).  There are a number of different options for formatting other than %s (eg %f) but we’re not going into them here.  Read the documentation for more details.

 >>> print '->%s<-'%'This is the value which is inserted'
 ->This is the value which is inserted<-

We can define the format string and the value differently:

>>> formatString="...tell that to the young people of today, and %s."
>>> print formatString%"they won't believe you"
...tell that to the young people of today, and they won't believe you.

Then you can use the same format string with a different value.  This is useful when you have to print something repetitive, where only a part of it changes:

>>> print formatString%"they will fall about laughing"
 ...tell that to the young people of today, and they will fall about laughing.

You can put multiple values into the format string, but the values you supply must:

• be in brackets ()  (this makes a thing called a tuple, but we haven’t seen them yet); and
• have a comma between each of the values; and
• there must be the same number of values as there are %s in the format string.

So, in the previous tute, we might have done something like this:
 >>> formatString="%s x %s = %s"
 >>> print formatString%(2,2,4)
 2 x 2 = 4

We can also pass variables to be printed as values to the formatting string:

 >>> i=2
 >>> j=3
 >>> print formatString%(i,j,i*j)
 2 x 3 = 6

So here’s the times table again:

 >>> for i in range(12):
 ...    for j in range(12):
 ...       print formatString%(i+1,j+1,(i+1)*(j+1))

Silly Sentences

Let’s adapt the silly sentence example from last tutorial.
Set up:
1. open your text editor,
2. open a new file
3. “save as” this empty file with the name “sillySentences110614.py”, save it in the p4k directory we’ve created to hold our stuff.

Now copy and paste this:

import random
nouns = ["apple","toy car","pumpernickel","dinner","pillow","homework","little finger","house","pencil","letterbox","hamster"]
verbs = ["ate","played","smelled","lost","forgot","dropped","hid","erased","infuriated","planted","ripped up","tripped over","dusted","talked to"]
v = len(verbs)-1
n = len(nouns)-1
numberOfSillySentences = 10
formatString = "I was very taken aback when Mrs Pepperpot %s my %s."

for i in range(numberOfSillySentences):
    verb = verbs[random.randint(0,v)]
    noun = nouns[random.randint(0,n)]
    print formatString%(verb,noun)

And save the file.  Can you see how much easier it is to get the spacing right?

Run the file by going to a console and typing:

> python sillySentences110614.py

Funny?

Now you need to make this code your own.

Exercise: First, change some of the verbs and nouns. You will see the pattern – for verbs you need to substitute doing words, and they need to be in the past – ate rather than eat.  For nouns you need to substitute words which name things.   Save the file, then run the program again from the console with:

> python sillySentences110614.py

Next, we’re going to change it to add some feelings in, but you have to supply the feelings:

Exercise: add some feeling words to the feelings arrray and run again:

import random
nouns = ["apple","toy car","pumpernickel","dinner","pillow","homework","little finger","house","pencil","letterbox","hamster"]
verbs = ["ate","played","smelled","lost","forgot","dropped","hid","erased","infuriated","planted","ripped up","tripped over","dusted","talked to"]
feelings=["very taken aback","very happy","quite sad"] # add more feelings to the array here 
v = len(verbs)-1
n = len(nouns)-1
f = len(feelings)-1 
numberOfSillySentences = 10
formatString = "I was %s when Mrs Pepperpot %s my %s."

for i in range(numberOfSillySentences):
  verb = verbs[random.randint(0,v)]  # choose a verb,noun, feeling  by calculating a random integer
  noun = nouns[random.randint(0,n)]  # between 0 and v,n,f (number of verbs,nouns, feelings)
  feeling = feelings[random.randint(0,f)]  
  print formatString%(feeling,verb,noun)  # substitute into the format string and print

Making this Code Your Own

Shopkeeper No, I’m afraid not actually guv, we’re fresh out of parrots. I’ll tell you what though … I’ll lop its back legs off, make good, strip the fur, stick a couple of wings on and staple on a beak of your own choice. (taking small box and rattling it) No problem. Lovely parrot.

While I’ve been giving you coding examples, I’ve assumed that generalising from them will be self evident.  I understand that that it is not.  In this tutorial we will look at ways of approaching the code with a view to giving you some guidance on how to make this code your own.

The first step is to be able to break the code down into logical units, and then to break those units into subunits.   The second step, once you have broken the code down, is to understand what each of the parts of the code is doing.  If you don’t know what the pieces are doing, it’s hard to change what they’re doing!  The final thing is this:

try, try, try

and:

take notice of what happens when you try, try, try.  Python tries to explain why things didn’t work like you thought they would.  Often you’ve just mistyped something.

Remember this: there is no reason why you can’t be a good programmer, even at a young age.

Remember this also: in all of these tutes there is code which I don’t get right the first time, or have to retype for some reason.

Let’s take an example:

>>> for i in range(10):
...     print i
...
0
1
2
3
4
5
6
7
8
9

This code has two lines, and two parts.  The first part (for i in range (10)) creates a loop.  The second part (print i) does something each time the loop executes.   Next I break the first part down and ask what range() does.   If you don’t know, use help (in my console this gets “paged”, if this happens to you, press ‘q’ when you’re finished reading it):

>>> help(range)
Help on built-in function range in module __builtin__:

range(...)
    range([start,] stop[, step]) -> list of integers

    Return a list containing an arithmetic progression of integers.
    range(i, j) returns [i, i+1, i+2, ..., j-1]; start (!) defaults to 0.
    When step is given, it specifies the increment (or decrement).
    For example, range(4) returns [0, 1, 2, 3].  The end point is omitted!
    These are exactly the valid indices for a list of 4 elements.

Based on the help file, range() gives us a list (of integers). Range tute is here.

Next, I confirm that I understand what it does by trying a concrete example:

>>> print range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

What the for statement does is makes the variable i point to each of these elements in turn (see the print out above).  So, how could we change this? Well, we could either change how the code is looping, or what it is doing in the loop.  Let’s change what it is doing:

>>> for i in range(10):
...   print 10-i ...
10
9
8
7
6
5
4
3
2
1

Now, instead of counting up, the loop counts down.  Let’s change the loop to be a multiplication table:

>>> for i in range(10):
...   print "7 x ",i," = ",7*i
...
7 x  0  =  0
7 x  1  =  7
7 x  2  =  14
7 x  3  =  21
7 x  4  =  28
7 x  5  =  35
7 x  6  =  42
7 x  7  =  49
7 x  8  =  56
7 x  9  =  63

Note: We need to use a comma (,) here in the print statement because i is a number and “7 x ” (etc) is a string.

Exercise: what would we need to change to make this give the times table for 7×1 through to 7×12?

Solution:

In order to start at 1, we need to add 1 to i.  Further for it to run to 12, we need to increase the range by 2:

>>> for i in range(10+2):
...   print "7 x ",i+1," = ",7*(i+1) # increase each i and put brackets in otherwise you'll get 1+ 7 x i
...
7 x  1  =  7
7 x  2  =  14
7 x  3  =  21
7 x  4  =  28
7 x  5  =  35
7 x  6  =  42
7 x  7  =  49
7 x  8  =  56
7 x  9  =  63
7 x  10  =  70
7 x  11  =  77
7 x  12  =  84

Exercise: How would you change this to print out the times table for all numbers from 1 through 12?

Solution:

>>> for i in range(12): ...   for j in range(12):
...     print i+1,"x",j+1," = ",(i+1)*(j+1)  # note the brackets
...   print   # just to make it more readable - try it without the print statement
[output omitted]

What I’ve done here is create two loops, one runs inside the other.  The print out will show you how i and j change in each loop.  See how it looks if you swap i and j in the first two lines:

>>> for j in range(12):
...   for i in range(12):
...     print i+1,"x",j+1," = ",(i+1)*(j+1)
...   print
...
[output omitted]

You can see that there’s an issue here with the length of the output.  So, let’s print two tables each time through the loop, and loop half as many times:

>>> for i in range(6):
...   for j in range(12):
...     print i+1,"x",j+1," = ",(i+1)*(j+1),'\t\t',i+7,"x",j+1," = ",(i+7)*(j+1) 
...   print

Now, I think that that middle of the loop looks cumbersome.  How about we simplify it by making a function:

>>> def xByy(x,y):
...   ''' given two numbers x and y, return a string
...       representing their product.  So, for example
...       for 3 and 4, this function returns a string:
...       "3 x 4 = 12"
...   '''
...   return str(x)+" x "+str(y)+" = "+str(x*y) # x and y are numbers so they need to be converted to strings first - str()
...

Test the function to see it works as you think it should:

>>> print xByy(3,4)
3 x 4 = 12

(3×4 is 12 isn’t it?)  Note, help works even on this newly created function – Try help(xByy)

Now we can “refactor” the code we had earlier:

>>> for i in range(6):
...   for j in range(12):
...     print xByy(i+1,j+1)+'\t\t'+xByy(i+7,j+1) # So, i+1 becomes x and j+1 is y, then i+7 and j+1
...   print

We could change it again to print more of the tables on a line:

>>> for i in range(4):
...   for j in range(12):
...     print xByy(i+1,j+1)+'\t\t'+xByy(i+5,j+1)+'\t\t'+xByy(i+9,j+1)# Adding another column
...   print

And so on.   The other thing we could do is recognise that range() is a list just like any other, so we could equally see what would happen if we used another list:

>>> names = ["John","Eric", "Terry","Graham","Terry","Michael","Carol","Connie","Ian"]
>>> for n in names:
...    print n
...
John
Eric
Terry
Graham
Terry
Michael
Carol
Connie
Ian

Realising this you could mix things up a little:

>>> import random
>>> r = random.randint  # You haven't seen this before, it is me cheating a little to save on typing.
>>> nouns = ["apple","toy car","pumpernickel","dinner","pillow","homework","little finger","house","pencil","letterbox","hamster"]
>>> verbs = ["ate","played","smelled","lost","forgot","dropped","hid","erased","infuriated","planted","ripped up","tripped over","dusted","talked to"]
>>> v = len(verbs)-1
>>> n = len(nouns)-1
>>> for i in range(10):
...   print "I was very taken aback when Mrs Pepperpot "+verbs[r(0,v)]+" my "+nouns[r(0,n)]+"."
...
I was very taken aback when Mrs Pepperpot infuriated my letterbox.
I was very taken aback when Mrs Pepperpot smelled my pumpernickel.
I was very taken aback when Mrs Pepperpot forgot my pencil.
I was very taken aback when Mrs Pepperpot smelled my toy car.
I was very taken aback when Mrs Pepperpot tripped over my house.
I was very taken aback when Mrs Pepperpot played my toy car.
I was very taken aback when Mrs Pepperpot lost my house.
I was very taken aback when Mrs Pepperpot forgot my pillow.
I was very taken aback when Mrs Pepperpot dropped my pumpernickel.
I was very taken aback when Mrs Pepperpot erased my house.

Exercise: What is going on here?  Why have I assigned, v = len(verbs)-1? etc?

Note: r(0,v) is shorthand for random.randint(0,v)

You can run this with a different list of verbs and nouns, just make sure, after you assign or change the verbs/nouns list that you reassign the values of v and n.

Exercise: change this code so that you have a list of feelings one of which is inserted at random – I was very x….

Final note: Syntax Errors

Often things will fail because you have a syntax error, which is computer speak for “you have made a typo”.  It is usually just a case of working through what you did to find the mistake.

Examples:

TypeError: ‘list’ object is not callable – this probably means you’ve used round brackets () when you were supposed to use square ones []

NameError: name ‘some_text’ is not defined – this probably means you are missing inverted commas around something which is meant to be a string “some_text”

TypeError: cannot concatenate ‘str’ and ‘int’ objects – you’ve tried to add a string and a number.  Convert the number to a string using str().

Conclusion

We’ve played around here with some basic building blocks of the language and found we can do some interesting stuff.  We have been able to vary things by breaking the parts of the program down into pieces, understanding what the pieces do, and, based on that understanding, changing them.