Perlcast Interview 14 - Interview with Richard Foley
Perlcast Interview 14 - Interview with Richard Foley
Recently, Perlcast interviewed Richard Foley about his book "Pro Perl Debugging." In the interview Richard talks about the often overlooked Perl debugger.
The audio of this interview can be found at http://www.perlcast.com/.
Josh: Welcome to another Perlcast interview. This interview, recorded on October 13th 2005, was with Richard Foley. In the interview we discuss Richard's book "Pro Perl Debugging." For this interview, Apress is giving away three copies of Richard's "Pro Perl Debugging" book. To get entered into the drawing, just email perlcast at gmail dot com before December 1st, and let me know that you want to be in the contest.
Josh: Today on Perlcast, I am speaking with Richard Foley. Richard is the author of "Pro Perl Debugging," published by Apress. He also previously authored "Perl Debugger Pocket Reference" and co-authored "RT Essential," both published by O'Reilly. Richard is a Perl and Oracle developer living in Germany. He's worked in both the aerospace and banking industries. He is an active member of the Perl community, with contributions to CPAN and the Perl source. He is also a member of the YAPC Europe committee. Richard, welcome to Perlcast.
Richard: Hi Josh.
Josh: Hi. Well, Richard let's dive on in and start asking some questions. First, we should probably start off with defining what debugging is. I'm sure that everyone of Perlcast's listeners does it in one form or another, just not possibly in the most efficient manner. Would you mind elaborating?
Richard: Debugging is basically the process of finding and eliminating software bugs. Well, bugs of any sort, really. The debugger can help you with that in a number of different ways. Most people probably use a print statement, generally, because it is a little easier. You don't have to learn a new syntax. The debugger can be quite formidable to inexperienced users. But it's really quite a fun tool, when you get to know to use it.
Josh: Whenever we talk about the Perl debugger, is that actually something that comes packaged with Perl?
Richard: Yeah. In fact, it has alway been there from version 1, right back to stone age of Perl, if you like. Every single version has had a debugger there. And it's alway been there stable to use. The instructions have always been there inside the perldoc, which again is something that I find that many programmers I meet don't actually use perldoc in the way it should be used, an instant online reference in middle of their programming tasks. They very often spend a lot of time running around looking on the web for information, when it is actually right there, accessible from their command line while they are programming. And it is the same with the debugger. It's always been there. People just don't use because they are unfamiliar with it.
Josh: Now, the only debuggers that I have really used have been in visual debugging environments, and there were breakpoints, watches, tracing, all these types of things. Does the Perl debugger come with these, and how do you actually use them, since I am pretty sure that it's not a GUI debugger.
Richard: Essentially, the debugger is a command line debugger, and yes, it does support all the standard debugging features, like breakpoints, watches and tracing, setting actions, and such. Essentially, what you can do with the debugger is that you can inside your program when it's running, and you can get it and stop its execution at any time, and then observe all the variables, or you can inspect the state of your program at any point in its execution, which can be very useful. In course to make it stop at a particular point when, say, let's say, for example, a line or file that you are parsing has a particular content, so you can set a breakpoint to watch a particular variable, and when this variable contains certain information, the debugger will stop your program and then you can look around at your leisure at the data and the state it's in at that moment in time. So it's very useful, rather then setting, for example, print statements everywhere, and you have to start your program at the beginning, run it ... wait, and at the end you might have several megabytes of debugging information, or logging information, that's been printed to a file somewhere, and then you need to grep through it for the right information, and you might not have enough information, why, then you need to go back and start digging around somewhere else, and printing out dumps of complex data structures, for example, just using the print command, which can get very messy. You can do all that sort of thing inside the debugger, without it running on ahead of you, or behind you. You just stop it, and you can just look around exactly where you are. It's very convenient.
Josh: Beyond looking, are you actually able to change the values of variables, midstream?
Richard: Yes, indeed. You are basically there inside your Perl program, and you can execute any Perl statement you want, and it will take in the context you are in. You can even overload some of the libraries you are using, and change the code you are about to call. Well, this is going to get a little bit advanced, possibly, but a more simple example would be just to simply change the variable you are using at that time, you would just execute it as a simple Perl statement.
Josh: You mentioned earlier that the perldoc, and also the debugger, kind of two tools that come with Perl, that people don't really pay attention to, or don't use as much or as good as they should, and so whenever it comes to it, using the Perl debugger, why do so many people not use it? Is it too slow, or too complicated?
Richard: It is of course a little bit slower using the debugger on a program, because it's doing some checks for log on your behalf. And it's a little bit more complicated, because to use it, you really have to learn a slightly different set of commands. In a way, the learning curve is a little steep, but it's very short, it's not really too complicated. Most of the commands are very intuitive, like "p" for "print" and "h" for "help," "b" for setting a breakpoint. It is a very friendly environment, but you do just have to step over the threshold. I use it quite often to try out simple programming constructs, foreach or map, for example, you know, it's like using "perl -e" on the command line. You can wrap up a very simple command and experiment with it, without writing a whole program and going out of the file, and then running the file from the command line. Which is convenience mechanism, really. And the more you use it, the more convenient it becomes.
Josh: And that is something that I did notice in the book that I didn't realize Perl had, and it's through the debugger, and that is a Perl type of console, similar to "perl -e," where you can have an interactive programming console, and, you said you don't really use Python so you don't know, but it sounds very similar to what happens when you just type "python" on the command line.
Richard: Being able to interactively work with your code is a very powerful mechanism, really. Because, you can just experiment on the fly with all sort of constructs. You know, simple constructs, or things like when you're unsure about how somethings evaluates in a list context, or scalar context, or whether it is a list or scalar context. So you can just experiment and find out.
Josh: Okay, so on the subject of the book, you divided into three primary sections. The first being a general overview of the debugger, the second takes you through usage scenarios and customizing the debugger, and the final is a reference section. So, I was just wondering why you chose to layout the book in this way.
Richard: It seemed a fairly natural spread for me, I think, I was hoping that the book would be able to appeal to both people that hadn't really used the debugger before, hadn't really had much experience in, and therefore the first several chapters are completely oriented toward the beginner. If you are a more experienced Perl user and, quite possibly, already a debugger user, but you weren't sure of all the details, then that is where middle of the book was going to come more into effect, to get you up to speed there, really. And of course you need to have a reference, and every books have references, goodness me. The debugger command, well, there's quite lot of them really, especially the options. We needed to have some form of definitive reference of all the various command possibilities and the various syntaxes. Which, while they are multiplely intuitive, some of them can be a bit obscure, especially when you get down to the options. You need to have the absolutely right name in some places. So, you sort of need to know where to look and that's where the online help, again, comes into hand there. In the debugger itself, there is a small screen, a help screen, that you can use. That you can bring up just with a simple "h" command at the prompt. And that gives you almost all the information you need to know at that point. To get more information, you would use "h" and then the command name itself.
Josh: You covered debugging modules and debugging objects separately in the book, and it seems like those two areas kind of overlap some in Perl because of the very loose object definitions. So, what is the difference when you're debugging those?
Richard: Essentially, a module is just a library. We've come to associate object oriented programming with little modules, but the two aren't strictly bound together in that way. I think treating a number of libraries as a debuggable entity is valid on its own, and there are things the debugger can help you with there. Equally when you use object oriented code it's entirely irrelevant whether you are actually using modules or you just got a whole bundle of code in a single file. You can still set up inheritance and member variables, classes and so on. And the debugger offers you certain commands to help you with your object oriented programming as well, like the "i" command, which just tells you a little bit about the inheritance of the object you're looking at. Or rather the executable statement you've got in front of you, but you would normally use it on an object.
Josh: Whenever I debug CGI scripts, typically I just print out to the screen and run them through the web browser, or something. So, you can actually debug [CGI scripts using, or] Perl CGI scripts, using the Perl debugger, but it does seem like it been a little bit tricky, if not impossible, honestly to me. So how would you go about doing that?
Richard: The debugger is a very useful tool when you are dealing with CGI scripts, just like any other Perl program. There's no particular magic there. You would just call your CGI script using "-d," where appropriate, at the end of the Perl path, and the debugger kicks in, takes control of the situation and offers you a prompt. The point where it gets tricky is if you are using Apache mod_perl, for example, where you are combining with a web server and you've got the debugger essentially already sitting inside, well Perl, already sitting inside C space, inside the web server. Even there, thanks to Doug MacEachern's work with the mod_perl code, Pierce provided a very simple interface which you can effectively switch on, and you can step inside mod_perl, as well. All you have to do is you have to get the server running in single user mode, and there's a small configuration file change you have to make in the Apache server config file. And then you can step through mod_perl programs, just like you can step through any other Perl programs. So, it's very, very sweet, and very, very, simple. It's not complicated at all. The only complicated about it is trying it the first time. And once you have done that you think "Oh, it's so easy, why didn't I do this before." And that's the case over and over again I found that with the debugger. That people always where saying "isn't it complicated" or "isn't it strange," you know, and "how can you possibly do this and do that." And it basically just makes your live much, much easier and much more straightforward. It's a very powerful tool that people seem to miss because it's essentially unfamiliar.
I was at one job here in Germany where there were some people on a programming team here that spent, I think probably months of man-hours, well maybe weeks. They were trying to get a tracing mechanism in their moderately complicated Perl code, and they were using caller all the time to essentially step in all the time and print out stack traces during the runtime execution of their program. And all of this, of course, comes in standard with Perl. You can get stack traces, printing out all the arguments going into subroutines, what the return values are, just with a switch, with the "-t," inside the Perl debugger. It was an awful lot of work, which would have been completely saved had they bothered to read the documentation, you know. Which isn't to say that none of us ever don't like to reinvent every now and then, but that was an lot of work for absolutely nothing. It wasn't even as good, of course, as the debugger solution.
Josh: Whenever you say "reinventing the wheel," I am not sure if this is a reinvention or an improvement, or not, but there are some debugger replacements, or at least supplements, that are available, and one of these is "Devel-ebug." And how does this module actually compare to the Perl debugger?
Richard: I've not yet time to recall on this to dive into that a great deal. But I believe that's Leon's newer module, or at least I know he has been working on one. And I think any approach to making debugging Perl programs is a good thing. And if there's several debuggers out there to choose from that can only be a good thing too. There have been various attempts in the past, I think, to replace the debugger, because it is a little bit crusty. It essentially had a bit of a rewrite when Perl 5 came in, and since it has only been extended and extended and extended, and as you can imagine that led to very spaghetti like bundle of code to maintain, which can lead to its own problems. Essentially, any further development tools that we can get our hands on can only be a good thing.
Josh: Is there any GUI to sit on top of the debugger to maybe make it easier for beginning users to use that, other than the command line where you have to remember the commands?
Richard: Yeah, the classic one is probably coming from the UNIX background, is the ptkdb, the Perl/Tk Debugger interface, which works very well with all the Linux platforms. You just have Tk installed and you can run that. And that's a very mouse oriented debugger interface, and quite friendly. The other one, quite well known, is the Komodo debugger and Perl IDE from ActiveState, which, because I work mostly on UNIX and Linux I don't have actually much experience with. But from what I have seen of it it's very sweet. Any of the GUI interfaces that are available. Excellent particularly for CGI debugging. I mean, this is the case where with ptkdb, for example, you can set a display variable where you are using it. A neat little trick is the Perl code that's running by a web server will send a window to wherever set the display variable to, and you can debug from there. So you can debug from a completely remote machine.
Josh: In the book, you also show how to debug threaded applications and POE based applications. So, what kind of special support does the debugger have to make debugging these types of applications easier?
Richard: Where POE is concerned, really, there is all the extra support needed, because it's simply a single running application that's pretending to be threaded, or forked. So it works with that quite happily. Where you are using actual fork calls, and some program do, the debugger does support that. The only problem at hand is setting up a new console. And than you have to a little bit of higgery-pokery to get the correct console running, the correct xterm. The support for forked programs is moderately limited, but it is actually there. What we actually need is for some fork experts to come along, and to write all the various chunks of code to support the debugger on those platforms. The main problem at the moment really is that the Perl debugger is running inside what it thinks is an xterm, and it doesn't really know how to create more, because you need a new input and output for the debugger itself, not the program as such. There is some minimal support there for forks.
Where threads are concerned, that was a problem until quite recently, because all the variables inside the Perl debugger were being shared among all the threaded programs, and no one really knew which variable was in which thread. And that led to some quite interesting results. But that was fixed quite recently, 5.8.5 I think. And you can now use the Perl debugger with threads as well.
Josh: One feature of the debugger that looks really neat was the ability to debug regular expressions. But before we dive into that, a non-debugger related tip for working with regexes was to avoid contracting the "Leaning Toothpick Syndrome," and what is that, and how do I avoid it?
Richard: Right, Leaning Toothpick Syndrome is when you normally have a regexp, you normally start with a forward slash and then you end with a forward slash. And so, if you got any forward slash in your regex, like for example in a typical path name on a UNIX like system, it can get very confusing very quickly, which is the start slash and which is the end slash, and which is the slash in the middle, and has it been backslashed yet to escape it. If you just use different delimiters at the end, maybe a comment mark or open and close brackets or braces, that can make the regex a lot easier to read. And then you don't have to backslash your forward slashes any more, and you no longer get the Leaning Toothpick Syndrome. So when you just have so many forward slashes and backslashes to try to keep them in sync, it get very bitterly confusing.
Josh: Regular expression debugging seem to produce a lot of data, but then again regexes are almost miniature programs within your script. So, what types of information and controls does the debugger provide for regular expressions?
Richard: The debugger itself doesn't actually provide terribly much regex support itself. It uses what support there is in Perl already. Perl has its own regex engine, that will give you a lot of information if you ask it correctly. It seemed moderately relevant to include a chapter on this in the book, because there was so little information about it anywhere else, and people do have problems with their regexes. So it's nice to see one thing exploded. Essentially, you've got a couple of choices with regexes. You can use the re pragma, use regular expression module then or pragma (use re), which will show you what Perl is going to be doing, or show you what the regex engine is going to be doing in terms of compiling and optimizing your regex. Or you can use the debugging flags. You have to use -Dr for that. But for that you need to have a special Perl build with debugging support, which, again, is not very complicated and shared to do that in the book. But its one more step you have to go through. But its quite nice to see your rexex actually exploded out, and walk through step by step.
Josh: When do I just use the print statement and when do I pull out the debugger? Is there something to look for, some kind of signals, to know that its time to actually invoke the debugger and get beyond the print?
Richard: If you don't turn around and just use it first off because its a convenient and a close friend, then yeah, you probably gonna be, maybe, at a stage where you are struggling with the print command. When you are saying something like "that just can't be right," and you are going round and round in circles, looking at data that is clearly not what you're expecting. And that really is a time when you should stop and say "Okay, I'm gonna step back and I'm just gonna force myself to go in there, to step inside the code, and to step through it and find out what the values of these variables actually are at the time, rather than assuming there are something of." And the debugger can help you do that. The nice thing about it is that it can do it without you actually altering your code in any way. One of the problems is, very often people will actually try to find a bug and help fix it. Finding it has involved, perhaps, change in the code in numerous places to put printstatements in, and then fixing it involves changing the code again and taking all that print statements out correctly, without affecting aversely, in any way, the running of the program. Sounds very straightforward, but I have seen a number of bugs appear just by people finding and fixing bugs actually introduce their own. And the debugger can help there, with just keeping you slightly arms-length. You are not changing the code in any way. You have complete control of what you are doing.
Josh: That was all the questions I had. Is there anything in the book that you think is really important that we should let people know about?
Richard: I think the last comment is a critical one: Don't change the source code. Don't change the code, just get in there and run your program and have a look around as it's running. It is a very powerful mechanism, and it's a very convenient mechanism. And it's a lot of fun as well.
Josh: Thank you. Well, the book is "Pro Perl Debugging." It's published by Apress, and is written by Richard Foley, who we've been talking to here on Perlcast. Richard, thank you very much for the interview.
I like to once again thank Richard for taking the time to be interviewed, and I hope that you enjoyed it as much as I did. The garageband.com artist featured in this interview was "The Lemmings" with their track "Evolution." And that's it for this interview. Thank you very much for listening.
BUGS AND LIMITATIONS
I tried to faithfully transcribe the interview, but quite likely got some things wrong. Corrections and other feedback are most welcome.
Ronald Blaschke (ron at rblasch dot org)
LICENCE AND COPYRIGHT
Copyright (c) 2005 Ronald Blaschke (ron at rblasch dot org). All rights reserved.
Content of this interview: Copyright (c) 2005 Perlcast (perlcast at gmail dot com).
You can redistribute and/or modify this document under the same terms as the audio itself.