What studying the humanities taught me about software development

This article was originally published on the NewCity weblog.

Back in my computer science days, I took a programming class where the professor decided to assign us a project to show us how real-world programming gets done (or ought to get done).

1. We discussed the problem we were trying to solve in extreme detail
2. The professor dictated the precise toolset we were supposed to use
3. We were required to create and turn in a series of planning documents before we ever wrote any code.
4. Then we were supposed to use those planning documents to write the program

…which should be super easy because we’d done all the planning up front — so it should just be a mechanical process from there, right?

Friends, I didn’t just fail that project. I failed that class.

A semester later I changed my major to English, and the coursework had a different approach for projects:

1. I’d decide what my writing project would be.
2. I’d go research the topic to get a better understanding of what I was supposed to address.
3. Then I’d write a rough draft, and …

OK, here’s a funny thing about writing. You have this idea in your head, but the task of putting it on paper forces you to think slowly and carefully about it. Sometimes I never got past the first draft before I changed my mind about what I was trying to say. Thinking in detail about a problem often changes our understanding of it.

But once I was comfortable with what I was trying to say, the work was still not mechanical. Every word you write is a decision. So for most writers, the process is always:

1. Write
2. Revise
3. Repeat

Up until the deadline. (There has to be a deadline. Otherwise, you don’t know when to stop.)

I was much more comfortable with this way of working. This felt natural to me. I could get ideas out of my head, see they were wrong, fix them, and move on.

Almost any writing advice you read will say the same thing: Get your ideas on paper first. Get the words out where you can stare at them, glare at them, and throw them away if they don’t work. Don’t care if it’s graceless; grace can come later. Write the perfect sentence in the next draft.

Here’s the thing about that computer science class: not many people passed it, and one of the problems we had was no matter how hard we looked at the problem space or made plans, the project kept changing underneath us. The professor “forgot” details, or made decisions in one week that contradicted the previous week’s decisions, throwing all the plans into disarray. It was frustrating and discouraging, but we were assured that’s what the “real world” was like.

But as much as I was distressed by the changing requirements, I was frustrated by my own lack of understanding. Was my own plan solid? Would it work? All these tools, all these processes, all these ideas — they were new to me. How did I know my plan would work before I did it?

There was no way.

But a paper — I might not have known before I started writing precisely what words I was going to use (and how many of each). Or even which sources I was going to cite. But I knew in broad strokes what I needed to accomplish. And it was just a matter of wrestling each project into shape, one revision at a time.

And when I graduated with an English degree and found myself in the technology industry anyway, I took that exploratory, write/revise process with me to work. That was 1996, the Web was brand new as a communications medium, and we were all figuring everything out on the fly. So I could get away with that.

My computer science professor was right, though — real-world programming, full of specification documents and entity relationship diagrams and UML charts — often involved a significant amount of planning up front, and then the expectation that this planning would make the execution easy-peasy.

But that process is only as good as the plan. When you forget details, or have new thoughts, or change your mind, or even the problem-space changes mid-execution, it can be tough to recover unless you have a lot of skill and the liberty to make use of it.

In fact, I’d say every project I’ve been involved in that was planned exhaustively up-front came out quite a bit different in the end than anyone expected — and it got there through a last-minute mad scramble to get everything merely functional. In other words, it ended up “write/revise” anyway — but with everyone working in crisis mode, just trying to get the project done.

I am not the only person to have noticed this. In the intervening years, programmers have come to understand that while there’s a significant degree of engineering in code, there’s also a lot of creative work and exploration. Plans work great when you know the problem, you know the tools, you can describe everything clearly, and you know nothing will change.

Most programming work isn’t like that. Any interesting software project involves a continuous process of transmuting the unknown into the known. You can’t do that with plans; plans only tell you what you think might happen. You can only find the truth by doing the work.

Craftspeople, musicians, artists, and writers have known this for millennia. It’s no surprise to me that we’re learning this about software as well.