Perlcast Interview 13 - Interview with Randal Schwartz

Perlcast Interview 13 - Interview with Randal Schwartz

On October 26th 2006 Perlcast had a chance to interview Randal Schwartz, a key figure in the Perl community and author of countless books, articles, and web pages about Perl. In the interview, we talk about Randal's books, business, and a few Perl'isms that can be attributed to him.

Josh: Hello, this is Josh McAdams, the host of Perlcast, with another Perlcast interview. On October 26th 2005 I had the chance to interview Randal Schwartz, a key figure in the Perl community. In the interview we talked about Randal's books, business and a few Perl'isms that can be attributed to him.

Josh: Today on Perlcast I am speaking with Randal Schwartz. Randal is the author of many Perl books, including "Programming Perl," "Learning Perl," "Learning Perl on Win32 Systems," "Learning Perl Objects, References & Modules," "Effective Perl Programming: Writing Better Programs With Perl," "Object Oriented Perl" and "Perls of Wisdom." Beyond that he is also the owner of Stonehenge Consulting and frequently shows up at conferences and Perl haunts on the web. Welcome to Perlcast, Randal.

Randal: Hi, good to be here Josh Thanks for having me.

Josh: Just from that intro you seem to be an amazingly busy man. Just getting this interview together I've had a tough time catching up with you between our two schedules, so I'm glad you've found the time to talk to me and I am sure all the listeners. Learning Perl: Why did you choose to update this book?

Randal: Learning Perl fourth edition is not a huge change over the third edition. The biggest problem with the third edition, which was a complete rewrite over the second edition, is that the third edition came out before the Alpaca book. And the unfortunate part about that is that it didn't refer to the Alpaca book in any way, shape or form. So when someone was finished with Learning Perl third edition they went "Oh that's great, I now know a bunch of stuff about Perl and now I just got to have to dig into that old Camel book or maybe attack the Perl Cookbook or, at the time, attack the Advanced Perl Programming first edition, which was way out of date, or maybe study those old scary man pages or stuff," but, you know, so there wasn't really any reference. And even though the Alpaca book came out I think about a year and a half later we hadn't updated the Learning Perl book to refer to the Alpaca book. ... The two books are actually images of our courses that we teach. We teach a week long course based on the Lama book and a week long course based on the Alpaca book. And of course our courses refer to each other, because we wrote both of them. We just didn't have any references to that Lama book. So when Brian came along and joined the company as a full partner one of the things he said was "You know that really got to get more aligned, we got to have the books point at each other and make a lot more sense out of it" and I agreed with him, so he kicked me in the tail a little bit, got the project up on the schedule for O'Reilly and shepherded the project to rewrite enough of Learning Perl 3 to become Learning Perl 4. And we are currently in the midst of rewriting the Alpaca to become Alpaca 2, which will have some of the things we sort of left out of the first edition and also some fore/back references, kind of little clean up, little reassignment of where things go in terms of beginning versus advanced.

Josh: In case somebody is not familiar with the Alpaca and the Lama and things like that. The Alpaca is Programming Perl and Learning Perl is the Lama? Or did I get this messed up?

Randal: No, no. Learning Perl is the Lama. The Alpaca is Learning Perl Objects, References & Modules. Neither of them are the Camel book, Programming Perl, which I created originally with Larry's help and then I shepherded the second edition. I was pulled off the third edition to focus entirely of the training sides of the books.

Josh: When you are talking about the "we" there, you said "you and Brian," you are talking about Brian d'foy.

Randal: Brian d'foy, yes. Brian foy, sorry, don't pronounce the "d" or you'll get in trouble with him. Brian foy. Brian is a new partner. Brian has actually been with me for a number of years and I have been with Brian even before that. Before he was working for me, I was working with him as the co-founders of the Perl Monks organization. The original board of directors of Perl Monks was me and Brian and Dave and one of Brian's co-workers. So I have been involved with Brian for a really long time and I was really happy to bring him on as an instructor and then made him full partner at Stonehenge this year.

Josh: You mentioned also that you had co-authored the update, the fourth version, of Learning Perl. And you have also written a book with Tom Phoenix. So, with all this dynamics of working together: I know you are on the east coast, Brian's here in Chicago, I'm not sure where's Tom's at. How did it go updating this book?

Randal: Now wait a second. You said I am on the east coast. I'm on the west coast. I am actually in the same city as Tom, but I only see him at conferences, believe it or not. We spend a lot of time online and email talking back and forth to each other, but I am actually, because of my travel schedule, I'm pretty much around the world a lot. So I only see Tom when we actually have a scheduled conference to work together with, even though he is just down the street in my same city. But we have essentially a virtual corporation. So I have office staff in Portland, where I live. And occasionally I'm home - I happen to be home right now. Tom is also in the same city. In terms of collaboration: I mean the whole purpose of the Internet was to let people collaborate on things and communicate and so we are pretty good at that too. We write books and book sections in POD and we ship stuff back and forth, we use a centralized change control system, so that we can keep track of each others writing. We have one senior person for each book, that's usually me, to go through and kind of make sure that ultimately everything is all seamless and makes sense from an editorial standpoint before O'Reilly even gets to look at it. It works out pretty well, actually. It's just a matter of recognizing that you don't have to be face to face with people to get things to happen.

Josh: Shifting gears a little bit, there are a handful of Perl'isms that can be contributed to you, and one that stands out is the "Just another Perl hacker" signatures. Could you give us a little history on that?

Randal: Yeah, I've been around Usenet since the real early days. 1981, I think, was one of my first posts. In fact, if you go to Google's history of Usenet, I am actually referred to there because I am talking about the upcoming Star Wars movies. So I have been around Usenet for a really long time. And somewhere around probably five years into that, I started signing my posts with Randal and something in quotes, some cute phrase in quotes, Schwartz, and the cute phrase would apply to whatever the newsgroup was. Like it was a group on Rollerskating, "Randal Skataholic Schwartz," or something like that. The signature was mostly the same, except it was tailored to this specific group. When I finally put most of that information about me and how to get a hold of me down into my signature block, the four lines after the double dash space, I would just sign above it then "Just another 'whatever' hacker," depending on the group that I was posting in. For example, if I was in the shell group I would say "Just another shell hacker." If it was in C I would say "Just another C hacker," and so on. Just to kind of, you know, say as a way of sort of personalizing in the same way that I have done with my double quoted messaging between my name. When I got to the Perl group, of course, as I started posting more and more messages to Perl group, every one of them was "Just another Perl hacker," "Just another Perl hacker," "Just another Perl hacker," and it got really boring to keep typing the same phrases. So one day I decided to get a little cute and say 'print "Just another Perl hacker,"' and that started the whole sequence. Because I think the next day then I said "Why, I can't duplicate that again," so I did something with the word reverse, I believe it was, where I printed the string in reverse and I just started to get more and more and more complicated, always trying to keep it in one line, but not always succeeding, a few of my better ones are multiple lines, and just got it more and more obfuscated, and this became sort of a way that I represented myself, where I distinguished myself on the newsgroup. Very interesting when I started, you know, working with Larry on the first Programming Perl Camel book. And because it identified me as being not just the guy that was there, but somebody who actually could play around with Perl in a nice sort of ways. And then that sort of got adopted by a few other people as being another sort of level of challenge like, okay, they wanted to come up with something that was sort of obfuscated and weird and so they decided that the moniker "Just another Perl hacker," should apply to the entire community, not just me. I'm "Just another Perl hacker number zero," as I like to call myself, 'cause I started the whole trend.

Josh: One thing that you did mention was that you've been on Usenet since 1981 and one of our listeners pointed out a post that you've put on one Usenet group and it was about audio versions of Perl. And, let's see, it was:

    "And I wouldn't want someone driving beside me listening to a tape on
    Perl (even of me :-), and then flipping down to look at the graphics." [1]

Cause you thought that only Perl audio would be useful if it had side aids. So what do you think about Perlcast?

Randal: I actually like what you've done with it because I think what you are doing is making it more personable than just simply being an instruction book on Perl. I don't think if I read Learning Perl into a tape recorder that - Oh, that's such an archaic term, isn't it - but you know what I mean, it's like saying somebody has a record of something, you know, it's now a CD or whatever. But anyway, if I were to read the contents of Learning Perl into an audio only recording it would very useless. But what you're doing with Perlcast is you are talking to people about things that they are doing and I think that really works. I've listened to all your previous casts up to this point. And I like what you are doing, I am actually honored to be, you know, a member of this elite club now, it's very good. Cause you are not teaching Perl, you're actually talking about what's happening. And what's happening doesn't require a lot of graphics.

Josh: Another one of the Perl'isms that you're responsible for, or at least is named after you, is the Schwartzian Transform. And so what is that and how did that become named after you?

Randal: Oh boy, again, a lot of my life is - in fact I think they gonna put this on my tombstone, on my grave - is ruled by serendipity, you know. "Serendipity strikes again" would be sort of my take on things, and that is that I end up with a lot of things that accidentally turn out to be really cool. And it seems like I almost count on that from time to time but that seems to be the way it went. And the Schwartzian Transform was an example of that kind of serendipity. When I was starting my first phase of teaching and really sort of rolling out instruction on the corporate world one of the things I would do was, of course, every break I would have I would go log on to Usenet and see if there any new posts, things that needed questions answered, and things like that. And unfortunately my breaks were always fairly short because the breaks priority one was being with the customer, being with the client, but, you know, I'd always tell them "Okay, I may be up here looking like I'm doing a corporate hostile takeover or something but really all I am doing is answering my email and checking up on Usenet so feel free to interrupt me." But anyway, so I'm reading through the different questions that had shown up on the day and one of the questions that showed up on this particular day - and you can go to the Google news and see the actual posting that triggered this - "why is sort so slow?" I think that was the subject line. And what they had done in this particular sort - they were sorting the password file if I recall - and they were sorting based on the gecos field, the real name field. And in the sort subroutine there was this split that was taking the entire text line and breaking it into its seven component pieces by splitting on colon. As I starred at this problem I knew exactly what was happening, I knew exactly what was going wrong because it's a kind of thing that I had already described within our coursewares being something you really don't want to do, you want to keep the sort subroutine down to something that is very, very easy to compute, because the number of times it will be invoked is O(n * log n) as you get more items, it's gonna be more and more expensive exponentially - well not quite exponentially, someone's gonna write in and say "That's not right Randal" but you know what I mean - it's gonna get more and more as it gets more and more. And that's not a good deal. Right, so I said "You know this is just not gonna scale well" and he was talking about having maybe 300 or 400 entries in his password file and went "Oh that is just gonna be amazingly slow." Now, I am an old Lisp hacker of all things. I've done a lot of work in Lisp and almost every language under this sun, eventually. But one of the things that you do when you are programming in Lisp - it seems to happen really frequently - is pipelining things. You end up taking one data structure and then mapping it through the list concatenator things in Lisp to create another data structure, then concatenate that end with a third data structure and so on. So, what I realized that happened was that I needed to somehow cache the results of that split and associate that with the original item, then sort based on that cached value and then extract the original line back. And in a period of about four or five minutes I sat there and worked through the stabs - I mean this was a quick thing, remember I'm down a break in my class. I'm just trying to get back to class. I quickly said "Here, try this" and I coded up what was basically the first use of the Schwartzian Transform in public. And it was nothing unusual for me. I was just putting together building blocks and relying on some old techniques that I had seen in Emacs Lisp before that. But it was just something that had to get done and I didn't provide any explanation. Well, it's interesting that I hadn't provided any explanation because when I came back a few hours later, the next time I had a chance to get online, Tom Christiansen had seen my post and essential scolded me for not providing an explanation and also pointed out a small bug I had in my code - I forgot to add the newline back on the strings - and went through step by step elaboration, first explaining why this first thing was slow, then improving it one step after an other, after another, through about six or seven or so, it's getting closer and closer what I had done. Finally, he gets to step seven or eight, or whatever it is that was exactly that I did and he introduces that particular step as "and now let's go back to the Schwartzian Transform." He sort of meant it as sort of generic term, meaning something that Randal had done in a posting. Now, he posted that message, put that out through on the net and I thanked him for that if I remember right, I'd post a little "thanks for the extra explanation." The next week somebody had a sorting question, again one Usenet, and somebody else followed up and said "oh that's simple, you need to use a Schwartzian Transform." And I chuckled at that, 'cause that's supposed to be a generic term, just meaning something Randal did, but he was using it like it was a proper noun. And than that started catching on. Other people posted questions, other people had answers and said "use the Schwartzian Transform." So it created a life of its own. It wasn't anything that I started. I only could image that Tom was thinking that it was gonna be the proper name for the sequence of map-sort-map to speed up the acceleration of sort. And I can tell that because in a following post Tom even said something that effect - I can't quote this exactly, but he said something the effect of "why, you know, I really didn't mean Schwartzian Transform to have all the magical, mystical qualities that people have been assigning to it, so I'm just gonna call this the Black Transform." And there he was making fun of the fact that "schwarz" in German is "black," but of course that never took off, that never stuck. People continued to call it the Schwartzian Transform and then it ended up in the FAQ being that and so I've got my bit of immortality that way, but for me it was just a quick thing I had done in five minutes, didn't expect it to have a life of its own, but now I'm a part of the Perl culture as a result.

Josh: That's absolutely amazing. That's a neat story.

Randal: But that's the way everything happens in the Perl culture. This is the amazing part of it. There are little bits of things being done by people all over the place in the Perl culture. And they create these little, sort of, pockets of things that just take a live of their own. I mean, the CPAN started as an idea of a few people coming together, saying "we need something like this," you know. The whole Pugs thing, you know, autrijus just always said and going "Boom, let's just make this thing work" and starting with one goal in mind and creating an entirely separate thing out of it, an entirely different relation. Perlmonks started as idea of one guy, Tim Vroom, said "Let me have like a little bulletin board system that Perl people can talk." And they have become much larger than anybody every imagined them to be. I think the Perl culture, while not being unique in this sort of taking a seed and making it grow, I think its continuely been the reason that Perl has expanded into the all the areas it has expanded into, because there is this encouragement to kind of put something out there, even if it is not quite right, even if it is not quite fully formed, or even if you don't think it's all that useful. At least have a place to register it on - Perlmonks or put it into CPAN or something - and let other people poke at it, and see if there is something useful to it. I imagine that when Andy Wardley created Template::Toolkit, he was just solving his little problem in the corner and now it has become the sort of brandade de-facto standard templating system that everybody seems to use for a lot of different projects. It just seems to be that way in the Perl culture. There have been a lot of very interesting individual contributions that all together created a great synergy toward we have, you know, this huge, you know, how many thousands of modules on CPAN and always downloadable scripts and, you know, I could even argue that Perl revolutionized and made way for the Web. You know, without Perl I don't imagine we be seeing "www" on the sides of buses quite yet. You know, I think this all had an impact as well. And again, that was just again the marriage of the right things. Tim Berners-Lee came along, said "I want to have, you know, hyperlinked documents" and the Perl guys said "Oh, by the way, we could move data into there, we could manage the log files, we could create interactivity with Perl" and it was a marriage made in heaven. So, it's the same sort of synergy. It just keeps working. Open Source is there, individual contributers are there, lots of cooperation's there and, yeah, you know, there's hitches with it, but overall I think we have gotten here because we have this wonderful cooperation, this wonderful sort of opportunity with Perl.

Josh: So is it the community, is it the language, what is it? Why did you end up choosing Perl in the beginning and sticking with it so long?

Randal: I got into Perl because it was solving my problems. I saw Perl 1, downloaded it off the net - I was using mostly a bad mix of, you know, sed and awk and grep in my shell scripts, and this is back in the early days, I'd say - well, let's see, Perl 1 is 1977 so probably... Oh no, it should be 1987, I've been using UNIX since 1977, so by the time '87 rolled around I was already pretty fluent in sed and awk and grep and shell and m4. I've written entire project management systems in m4, for example. You don't want to go there, you know. So I got fluent in a lot of different things and when I saw Perl 1 come along, the only thing that first attracted me to it was it was by Larry and I had - Larry Wall - and I had been using the rn newsreader as my primary newsreader for a number of years and I was impressed because the rn man pages were something like 30 or 40 pages long. And I thought anybody who can put together a free piece of software that has a man page that's 30 pages long and just give it away and it does that cool stuff and just works right. I think the best phrase in the rn man pages "it may not be any faster but it seems faster." You have to go dig up the old man page to find out the reference what that actually is about, but I love that phrase. It so describes Larry's philosophy: It just works. It works amazingly well. So here comes Perl 1, comes along, looks like it can replace some of my awk programs, but it just didn't quite have the flavor that would really sink in for me. But I downloaded it and played with it for a little while. Perl 2 comes along and it had like twice the size the man pages. And I went "Oh, this is interesting now." And I started writing different things in Perl and I went "Oh, this is simpler in Perl than it is in that complicated eight process launching thing that I had before." And then I went on to the UNIX newsgroups, the UNIX questions and UNIX wizards newsgroups and I started answering the questions that they had in Perl. And it's funny because after a while I started in, for about three or four months, I was like here's somebody had a question and I could do it in one or two lines of Perl. Somebody else had a question, I could do it in four or five lines in Perl. I started realizing how powerful Perl was, that almost everything people wanted to do, that would have been a ten or fifteen lines shell script could have done in a one or two line Perl program. In fact, I answered that so many times in the UNIX shell and the UNIX wizards newsgroups that there started to be people that would post a question and immediately post below that "No Perl, please." Cause I had done it so often that they now thought it was gonna be me popping up there again posting a Perl answer. So I would respond to those by first posting the eight or ten lines of shell script that took to solve their problem, and then the two lines of Perl. So again, just to show them that. So I really was one of the first champions of the Perl sort of "It just works." But I was only looking at that from the point of view being a user. I was an excited user, it got me to do the things I wanted to do and I was really happy that Larry was spitting this code out from this mythical location of the Jet Propulsion Lab in Pasadena. He was spitting this stuff out and I was able to magically use it and do what I wanted to do. Then when Perl 3 came along Larry asked for anybody that could be involved as alpha testers. You know, people who had an opportunity to download some of the early versions of the code and provide him feedback. It was kind of like the early version of p5p, because we were supposed to give feedback, we were supposed to try to compile it on machines he didn't have, and just kind of expand it out. And that was really great because I got to play with an early version, and Perl 3 had like three or four times the size of the man page. The man page had gone from like fifteen or twenty pages up to about sixty pages, if I recall correctly. The thing was that it was a much larger language, much richer. It had binary data, it had the ability to talk TCP/IP and pull stuff out into separate modules and things, separate included files. I really liked it, it really was like getting close to being like "let me just ditch C and use this for everything, for everything I was going to do low level stuff with." And something interesting happened when that Perl 3 was finally released out to the net. There was a complaint that, although the man page was nice for Perl 3, the language had become significantly harder, so Larry please would you write a book on the style of introduction to sed and awk from O'Reilly that sort of covered the intro to Perl, so people could understand where the small end of the language is, you know, where do you really start in this sixty page man page. So Larry said in his very kind and gentle ways, normal sort of response was "Well, when I get around to it I'll write something up." Well, I had known from conversing with Larry in email that Larry is getting around to it when he's not dealing with his wife, his four kids and the church quire and his day job and I am told he sleeps from time to time, so you subtract that out of a 168 hour week there's not much time left. So I volunteered, with my technical writing background, volunteered to help Larry with the online documentation. Somebody at O'Reilly and associates saw that message, replied back to me privately and said "Look Randal, if you're gonna write a book we might as well publish it." And so after a couple of rounds of conversation with O'Reilly I then said "You know, the only way I want to make this book happen is to bring Larry on as well because it will add more credibility and I'm gonna be involved with Larry on a lot of questions anyway." So I wrote Larry and invited him into the project. And I designed the book with Larry's assistance, I managed the project through there and that became the first Camel book nine months later. Now, having written the Camel book - the first authoritated book on Perl - I was thrust into the center of the Perl community, not just being a cheerleader over on the side saying "come and use Perl" as a happy user, I was now like one of the chief writers about Perl, and I'm still to this day. I've written, you know, all those books you have listed and over 270 magazine articles, which you didn't mention at all. So, my byline has been out there 22 million times so far. You know, it's because Perl works for me and I want to talk about it, and I want to have other people use it. I'm not a seller, though. I'm still like "Use the best thing for the best language, but if I could help you use Perl to use it, then that'll be my way." So, I'm very prolific. I'm online two hours a day answering email, talking on Perlmonks and talking on IRC, trying to get questions answered and designing the next courseware I'm gonna be teaching and designing the next book I'm gonna be writing and figuring out what magazine article I'm gonna write this month.

That was all on one breath, by the way.

Josh: That was good. Scared to ask the next one.

I didn't realize: 270 magazine articles. I did see that on Perlmonks you had over 5400 posts. And one listener wrote back and said you actually have more books than you do modules on CPAN. I was gonna ask why, because it's backwards to how a lot of coders are, a lot of time you see a lot of code and very little writing. And instead you have a whole lot of writing to explain how to use all the code. And you mentioned the technical writing background. Would you care to talk a little bit about that?

Randal: The primary reason I don't have very much in the CPAN is, generally I'm not spending a lot of time these days working for real contract clients. So, most of my work these days is writing and teaching and staying in that arena, and managing my company. I have people that I send out to do contracting, and actual code work needing down and dirty. Most of the code that I have written has been because it's now the end of the month and I've gotten the second letter from my editor for the monthly article that's now a week past due. And when I do that kind of an article, where I now have to come up with an idea, come up with some code that supports it, write the code, write it up as a module, that kind of thing, and than put that as part of my article, the problem with that is that that code belongs to the magazine now. So I can't just easily take that code and put that on the CPAN separately, because I have written the code as part of what I'm being paid for to write the magazine article. I have an agreement to take all of my magazine articles and put them all back on my website www.stonehenge.com, but only, sort of, intact. I can't then do something further with those articles. The copyright still belongs to the original magazine. And because I write the magazine article first and the code sort of falls out of that, I can't just take that code and put it in the CPAN. I can't even just take that code and give it to somebody else. I've had people write me and say "Can I use the code from this article?" and I have to say "I can't actually tell you yes. You'll have to talk to the magazine. I can't tell you yes because I don't own that copyright." Now, the few things that are in the CPAN are things where, slightly ahead of the pressure of writing the monthly magazine article, I actually come up with an idea and then I sit down and I code the idea and I play with it a bit and work it out and write up some doc for it, or maybe I'm doing this for a client that's let me put it in the CPAN, like CGI::Prototype is. You know, so I've got some headroom on, I've actually written the code and got into the CPAN. Now when the end of the month comes along I write a magazine article about the code that is already published, now I've got something both in the CPAN and and article about it and that's been wonderful and peachy keen when I could remember to do that. But that's the reason I've written thousands and thousands of published lines of code, it's just that they are all inside the magazine articles, and unfortunately I can not just lift those and put those into the CPAN. I'd really love to, but I can't, by the copyright reasons.

Josh: A lot of these articles you've written have been published recently - I say recently - within the last year in a book by Apress. Is that right?

Randal: Yeah, the "Perls of Wisdom" was a selection of about, I think somewhere between sixty and seventy of what my favorite 240 articles of that time had been. They went through and bought the rights from the other magazines, from the magazine publishers, so that they could republish them in a book form. So I don't even own those rights, so they had to go out and acquire the rights to do that. And then they consulted with me to say "Is this the best sixty out of those?" and I said "You know, if you pick those two you really doing the same thing twice because the second is just a better version of the first one." And so I was able to kind of guide them along, you know, and kind of what articles I had referred to the most in Questions & Answers and answers on the net. I mean that's one thing if you've gotten through Perlmonks a little bit you might have noticed that about every sixth or seventh post I say I wrote an article about that. Because it is. I mean, why repeat something in an entire Perlmonks post when I have entirely detailed 2000 word article that already describes the answer to one of their questions and that's also an indication that I'm hitting the pulse really well. I'm writing the articles because lot of the questions people have get answered by my articles. Anyway, so the Apress went out, bought the rights to the articles, consulted with me. They originally wanted me to go through and edit all those articles to kind of bring that up to date and I said "You know what, that will expand it from a good long weekend to four or five solid weeks of work." Because, you know, the problem with being a writer is that you can always go back a second time and go "I should completely rearrange this and change to that" and so on. I said "Here's what I'll do instead for you: Let's leave the articles historically intact, as they were originally published. So what you are reading in Perls of Wisdom is the original article as it was shown originally. And what I've done is in the case where I wrote an article saying "This is the way something worked, but I wish there was a module that do this." and then later on somebody actually had a module that did it, I've put a sidebar into it that says "Well know, if I wrote this today it would be this module and that module and the whole code would be from 100 lines to under 20 lines and here's how it would be." So I was able to put the sidebars in that really kind of bring some historical perspective to these articles. Also, some of the sidebars were about the process I used to write the article, like what was I thinking at that time, what was the problem that I was solving at that time, or maybe things that happened after the article was published. For example, I have an article in there about scrapping the Dilbert website, to pick up the Dilbert historical archive so that in one page you can pull up all the Dilbert comics at once. And I talk about how the moment that that article was published they changed the format. Coincidence? I don't know, but it's the only time the format has ever been changed in the entire history of that website was right after my article was published. So, the old version that was in the magazine would no longer work, you had to change two URLs to make it work in the new article. So, you know, did they do it to keep my program from scrapping it? I don't know, but that's my sidebar. I'd be able to say this in historical perspective now. And I really appreciate that Apress let me do that, because it sort of said: Okay, here's these great articles, people been reading them, one of my reviewers said there exactly the right length, and he tells me privately that that means, of course, the right length for sitting in the smallest room in your house. And so here's these articles that are great to read, nice and easy just go to the punch, get right to the point and then I put a little bit historical perspective with it by meddling sidebars on most of the articles and I think it's a great book. I'm glad it's out there, because a lot of people, you know, they know my website has them all, but you can't read my website on a train or plane unless you have a good wireless access on either of those and I'm glad Apress did that for me, it's really nice of them.

Josh: And so, all the articles, you have some in the Apress book, you have multiple magazines with articles, you have a lot of them reposted on your website, and also on Perlmonks and, I think, IRC and things like that, you use the alias of "merlyn," right?

Randal: I try to use "merlyn" everywhere I can. The places where I can't use merlyn, believe it or not - it started with AOL, of course, I tried to get the name merlyn on AOL and it was taken by the time I got there - so, of course, I went "Darn it, I'm the real merlyn," so I'm "realmerlyn" on AOL, and I'm also realmerlyn on anyplace I can't get merlyn, so that's the only handle I use, it's pretty rare to find me with any other alias than that.

Josh: Any significance behind choosing that one?

Randal: Yeah, In Highschool I read the book "The Crystal Cave" and fell in love with the whole Merlyn genre and, although, when I say "fall in love with that," I sort of read that book and stopped reading about it. So it's a little weired because I was fascinated by that particular aspect of it, and as I'm reading that book they spell Merlyn with a "y" and I kind of liked that as a distinction, so when people spell it with an "i" I say "That's the Disney spelling, mine is the other spelling, you know, the one with the 'y'," and that again has taken up by a lot of people as well, but it's also the reason that I use "Stonehenge" for my company name though, because of this whole Highschool fascination I had with Merlyn and the whole mystical thing of Stonehenge was selected because it was related to, in some literature Stonehenge is called "Merlyn's Dance," it's the stones that Merlyn set up. Of course, historically that would be wrong because, you know, theoretically Arthur's time was, you know, 2000 year after Stonehenge was built, but, you know, a little bit adjusting time here and there it's all good.

Josh: You mentioned "Stonehenge Consulting," which is your company, and they advertise Perl training, Perl consulting, Guru-On-Demand services. Could you tell us a little more about what you guys do?

Randal: Primarily, over the last ten years, we have been the Boutique Perl Training Company. By that I mean that we specialized almost exclusively in training Perl. We do a little bit of over stuff too but basically we do Perl training. Because that was needed and wanted from 1995 to 2003 or so. There were some consulting gigs, some contracting gigs along it, but primarily we were training. Starting the last couple of years the need for Perl training has kind of subsided a little bit, I still think we've got probably 75 percent of the national market at this point, and some of the international market, but not so much. But we're also looking more and more at what other ways our skilled people can be used to provide high level consulting services. For example, one of the things we've done with clients in the past is we'll go in for an entire week with one of our trainers - trainers/coders - and provide some large group trainings, like maybe for a day or two of that, some small group trainings, maybe one on three, one on one, kind of trainings, and then some code reviews, some design review, and just kind of open Q&A sessions. So we basically go in and try to infuse the kind of high level knowledge quickly so that, instead of having to have an on-site guru, we're basically having a Perl guru rented for a week. And we hope that through our osmosis process that, you know, with the immersion, with the large group trainings and the - sort of hands-on with the - small group trainings, that we leave the organization much greater enriched. At least we eliminate probably 75 per cent of their worst practices and give them a whole new set of best practices to work from. And that's been really successful, our clients reported really good results from that. So, the Guru-On-Demand session is something kind of like that, and we can scale that down to a one day show or a two week show or whatever. We also have some longer term clients that we do, basically on-retainer consulting. So we have some clients that pay us money every month and we guarantee a certain number of hours of consulting in response to that. And we've placed contractors for weeks or months at a time as well. So we're kind of, you know, whatever works, if it is about Perl, we can probably either have somebody on staff or we can find somebody real quickly to put that in, and, you know, our trainees are still pretty happy and still keeping us pretty busy, too. So that's kind of the scope of Stonehenge at this point.

Josh: From that it sounded like it was more not like you'd be handed some specs and code out a program in your closet and then hand it back to the company, but you are really there to share your knowledge and enrich their employees.

Randal: Yeah, we've done both of those. One of the advantages we have over, say, a traditional contracting and consulting house is that, because our senior team are all trainers, we're also professional communicators, as you might have told from this conversation already. I like to talk I suppose is the easy way to say that. But what we know is, we know how to talk about what we are doing, we know how to teach, we were trained to be up in front of a room or talk one-on-one about what we are actually getting across unlike, say, someone who, well, let's call them the traditional nerd, the traditional geek, where they know their stuff but they don't really know how to talk about their stuff. We know both how to do it and talk about it, so we leave a lot more behind than just, you know, 4500 lines of code: We actually have 4500 lines of code and a changed way of viewing the code instilled in every participant or organization.

Josh: You also tend to show up at tons of Perl related events. We were talking earlier and you said you'd actually been to two geek cruises within the last three weeks. You go to just about every conference, so could you tell us a little bit about, I mean, you're there to speak? What are you there to do?

Randal: I go to conferences for a variety of reasons: At OSCON I'm just sort of part of the woodwork now at this point. I try to get at least two or three talks accepted every year, although the competition is much more fierce the last couple of years. I try to get a tutorial accepted as well. I also have the exhibit booth at OSCON. And then there of course is the famous Stonhenge party that's there every year. But what I'm trying to do with the conferences is two way: I want to both be there to be presence that people are reminded "Hey, we're Stonehonge, we're over here, we've got services available for you, unlike some more competitors we're still in business, we're still kicking tail there and being active participants in the Perl community." But also I want to go to conferences where Perl is being discussed, so that I know what's happening next. So I know what people are interested in, and also I know what magazine articles to write about, I know what books to write, I know what courseware needs to be generated and to kind of converse on that level as well. Plus, you know, at a conference there's a lot of networking that happens. I meet a lot of fascinating people at conferences and that's been really cool. It's actually where I met Brian, that's where I met all the people that work around me and for me. It's a place to network, it's a place to really kind of figure out what's going on. It's also a great place to recognize that you're not alone. That it's not just you downloading some piece of software and then using it, and maybe you see a couple of websites about it, but that's it. When you go to a conference, even like YAPC or like the Perl Whirl cruises and things. When you to these things you start realizing there are other people that are solving completely different things than you are solving, or maybe they are solving almost exactly what you are solving, the same sort of problems. But you get to chat with them and it gets to become more of a "Okay, I'm part of a tribe. This is cool. I'm actually a member of something. It's not just me being somebody who's, you know, pushing one's results around using this particular technology," it's actually like "Okay, there are a lot of other people that are as crazy as I am about this stuff and want to keep using it and it's reinforcing, it helps me to sell it to my boss, it helps me to understand where I can go if I have trouble. It helps me to know I'm not alone if I have a concern or question."

And so, I'm there - back to me again - I'm there at conferences, so people know I fully welcome people sending me email. I do. I enjoy email. I don't enjoy spam. I enjoy questions, though, about Perl. I love getting those. I get maybe twenty a day. And it's great, because not only do I get to make a difference specifically for the person that I get to answer the question for, but at second it let's me know, it gives me the finger on the pulse of where people are having problems. Because that leads to my next magazine article, my next course, my next book, my next area of research if a lot of people are working on something, that kind of thing. And so it's not really altruistic, it's really great because it's really a two way street. They help me by sending me questions. And I think that's a remarkable sort of synergy. It's amazing. It's why I spend an hour every day being on Perlmonks and answering email and hitting Usenet as much as I can and so on. It gives me a place to contribute and be contributed to in a great sharing way.

Josh: Conference season is winding down and you've been to quite a few. What is exciting that you see in the Perl world? What's coming up that we need to be looking out for?

Randal: Until Pugs I was worried about Perl 6. Just being real honest with that. Pugs has reinvigorated an entire new community and trying to get Perl 6 off the ground. I know there's a lot to do, there still a lot to be done to make Perl 6 really work. I'm excited by that. Of course, I'm terrified by Perl 6, given that most of my business is about providing Perl training and Perl writing, because the moment Perl 6 really becomes official a lot of my stuff has to get rewritten and it can't all be rewritten in the same day.

But I am excited as a user of Perl. I mean, some of the stuff I'm seeing that are doing at Perl 6, some of the design features I'm seeing in Perl 6 are exciting me as much as that Perl 3 did when I was using Perl 2. And how Perl 2 did when I was using the shell. It's like here is something that's gonna solve more of my problems in a simpler way, I can write fewer lines of code and I've got more time at the end of the day to just sit around and surf the web and read Wikipedia until my fingers fall out. You know, it's a chance for me to get more effective at doing what I'm doing, so that I don't have to think about what I'm doing as often as I can. So, I love what's Pugs is doing, I think that's very important. I like where CGI::Prototype is going, CGI::Prototype being my MVC platform, a very lightweight platform to do web applications using essentially by bringing everything back to its core. I like that I have clients that are paying for me to make changes to that and yet letting me release to the CPAN. This is really cool synergy things going on there. I like that YAPCs are happening all over the world. This is exciting me. I can't go to all of them. I mean, you mentioned that I go to a lot of places but I can't go to every YAPC. There's too many of them now. You know, that's crazy, I love that there are more Perl conferences that I can make it to. That's great, that is incredible. I love that there is a lot more Perl books on the shelve that I can read. That is also very cool. That means that Perl is alive and kicking. I love seeing jobs.perl.org increase every month the number of postings. Not only increase every month, but increase the dollar value, the average dollars of what they are willing to pay people and jobs shifting for being Perl and eighteen other things to being just full time Perl programmer $80,000/year working for Dreamworks. That was just on there yesterday. That's amazing. Working for a movie production company to doing Perl full time for $80,000/year. Lovely stuff. This is great. I think Perl is bigger than ever and I'm still with it and I'm gonna stick with it probably as long as Larry does, and Larry says he has a long way to go.

Josh: You mentioned CGI::Prototype being one of the modules you are excited about. What about the heavier weight alternative - I think it's heavier - Catalyst? Have you paid any attention to that or noticed what those guys are doing?

Randal: When I first looked at Catalyst, when I was researching the playing field before I wrote CGI::Prototype, one of the things I didn't like about Catalyst is that it made a lot of decisions for me about what a particular application look like, and I didn't see the right hooks to kind of push it the way I wanted to push it. Same thing with Maypole, Catalyst actually came out of Maypole, and CGI::Application, also. I looked at all three of those as being the most touted solutions for this kind of stuff. It seemed to me that the people that created those applications were solving the most common problems, without having been able to step back and do a real systems thought and systems design. In other words, what parts of all the applications on the web are the same, and what parts are different, and how do you spell the parts that are the same once and how are you able to completely and easily spell the parts that are different once. Catalyst and Maypole and CGI::Application all let you write a narrow range of applications, but they don't have enough tweaks to completely include just about everything you'd want to do on the web. And so when I designed CGI::Prototype I said let me go back to fundamental principles: What really happens on a webhead. And by abstracting that all the way back to the beginning, and then using a nice framework of class prototypes so that I'd essentially get accessors for free and subclasses for cheap, and things like that, I was able to more rapidly develop applications using CGI::Prototype than any other framework that I have worked in, including having spent some time trying to work with Catalyst and Maypole to make that work. I think it's because I stayed minimal, I just stayed with what are the basics. And that's given me the framework that I need, and it's been really, really useful. My customers have gotten a lot more leverage out of me than I would have, had I try to be tweaking Maypole and trying to dig into details of it every time, or should it be Catalyst, and fixing the bugs that I saw in CGI::Application. So I am leveraged. And that's the whole point of having a good framework is that I'm not spending my time thinking about the framework, except when I'm tweaking it to make something better. When the customer says I need a new page to do this, I go: Okay, there's the td file, there's the pm file, let's see, we need these callbacks, there's the code that goes right there. We need this code reused from that other part of the application, great, include, include, include and it's done. I mean, the design is simple. You add a couple of files and you are in a new state, it's just right there. It's like Ruby on Rails. In fact, somebody said I should come up with a more clever name than CGI::Prototype. My current name I'm kicking around is Perl on Planks. So, we'll see. Maybe I'll just be titleing the new project to that. So you may see that come to the CPAN near you we'll have a clever name like that. But I like where it's going, the people that have been playing with it, Brian took it on for a client that we had. At first he groans at it. Then he sees through playing where it goes. You know, he says, every time I try to do something the right hook is there. And I go, yeah, that's because I sat back and thought about what hooks needed to be there. It's not accidental, I've been programming for 32 year professionally. I have a sense of where things need to go, I've been around the web since its beginning, I have a sense of things need to go in terms of design and I also have a systematic sense, I look at the systems, I look at the large picture all the time. It's one of my downfalls, it keeps you out of the details, but it is what I'm doing all the time. And the clients that are taking it on, I mean I love that I didn't have to create the mailing list or the Sourceforge site for it. Somebody else wanted CGI::Prototype so much that they just want out and created a FAQ for it, they created a mailing list, they created a Sourceforge website, somebody else is adding some of the features that I wanted in there, editing stuff on Sourceforge, working together as a team. And I don't have to do that, so I'm getting leverage for free. Enough people like it already that it's becoming the sort of community building around it. And that's like one of these accidental things again. I needed a framework, I shared it with people and now it's taken a life of its own. And the important thing is to not stand in its way but to actually let that process happen and things are resolved from there.

Josh: I've taken quite a bit of your time and I know you are very busy so I guess I'll wrap it up and not any more questions. Is there anything that you'd like to tell the audience?

Randal: Just I appreciate this opportunity, like I said I've been listening to your Perlcasts for a while. I feel like it's pretty cool that I get to hang out with some of the other people you interviewed already. So this is really nice. I have fun, I like having fun, I like the classes that I teach to have fun, I like the writings that I have to have fun. I think that's probably the most important thing about me and I think what I'm trying to do is also bring that level of energy and fun into the Perl community. I think people sort of report that. I get a bit short tempered after answering the 37th post that was coming out of the FAQ, like anybody else does. But I try to help people the same time I'm having fun. I'm privileged to be near enough to have to travel all around the world all the time. To go talk to people about Perl and about programming in general and computer security and things like that. I appreciate all that. I'm living a life that I never could have imagined as a kid, and I love it. And I'm gonna keep doing it for a long time. For me life's not about money, it's about having fun at the end of the day. So, I am not getting rich of anything I'm doing here, I'm trying to give back as much as I can, I'm just glad to part of all this.

Josh: I'm sure many people out in the Perl community appreciate it. Thank you very much, Randal.

Randal: Thank you very much Josh, it's been really great talking to you. Thank you.

Josh: I'd like to once again thank Randal for the interview and hope that you enjoyed it as much as I did. The GarageBand.com artist that wrapped this interview was Syfin with their track "Wasted." Thanks for listening and be sure to tuned to Perlcast for more Perl audio.

1. http://groups.google.com/group/comp.lang.perl.misc/msg/6bc54ee547612091

I tried to faithfully transcribe the interview, but quite likely got some things wrong. Corrections and other feedback are most welcome.

Ronald Blaschke (ron at rblasch dot org)

Copyright (c) 2006 Ronald Blaschke (ron at rblasch dot org). All rights reserved.

Content of this interview: Copyright (c) 2005 Perlcast (perlcast at gmail dot com).

You can redistribute and/or modify this document under the same terms as the audio itself.