AI Coding Weekend (I)

Giving the AI coding thing "the old college try," not that I tried that much in college.

I spent a good portion of the weekend expanding my understanding of coding with AI assistance. I want to jot down a few notes here on the experience and will probably do so from time to time. At the risk of provoking some Internet Ire, I am going to “set aside the moral and environmental issues”¹ — except to say that I am very worried about these, too.

Where I am right now

I’ve been using AI autocomplete with copilot for a few years now, except for an extended period of several months when I turned it off entirely. Its success rate feels disappointing, and I am frustrated that VS Code seems incapable of prioritizing suggestions from IntelliSense over Copilot in cases where IntelliSense has better context.

That’s a long way of saying IntelliSense rarely mucks up auto-completing my import statements, but Copilot often does.

Auto-complete during code refactors is handy, however, and not something traditional coding tools are able to accomplish.

I’ve since moved on to asking coding questions in the chatbot, then asking Copilot, in agent mode, to do some smaller coding tasks for me. The latter generally requires a fair amount of tweaking, but it is saving me a lot of tedious boilerplate typing. It has also, on occasion, surfaced features of the tools I’m using that I was unaware of.

In terms of this great post on learing AI-assisted coding, I am somewhere in step three, but about to skip step four entirely.

This weekend’s work

I added comments to this blog recently, but I do not yet have any comment management tool. I am working with the database through a web front-end that has some significant usability problems. My goal for this weekend was to create a content management tool that would make the task easier.

To my mind, this was a good project for AI assistance:

1. It’s software entirely for myself.
2. It’s a small but not trivial project.
3. It involves a lot of tedious and uninteresting labor.

I signed up for a Claude Pro account to get started. It dropped me into Claude chat, so asked it to get me started with agentic coding. It generated a syllabus of sorts for how to code AI agents, which is a reasonable, but incorrect, interpretation of what I asked.² I asked it if there was an IDE I could use or a way to integrate it into VS Code, and it explained: sorry, this chat window was it.

Fortunately I knew this to be broadly false, so I ended up doing a YouTube search on beginner Claude tutorials. This led me to the terminal-based Claude Code app and was enough to let me get started.

These misunderstandings and partly, but not mostly, true conversations can be expensive. Claude generated a ton of text to throw back at me both times. How expensive the conversations are is unclear, however. Anthropic does not give you a clear budget of “tokens” to use. They say a lot of words about the usage limits, but they all boil down to the Bartender’s Prerogative: we’ll cut you off when we think you’ve had enough.

The first attempt

Writing an app with a single prompt is not going to get you particularly far. This is less because “AI is bad” and more because anything complicated requires detail and precision. Single prompts don’t work when you’re talking to a human engineer, so there’s no reason to think they’ll work for an LLM.

/* TODO: write rant about natural-language programming. */

To this end, I started by asking Claude to help me write a specification. I took the simple approach here:

1. I explained what I want my app to be and do
2. I then told Claude to ask me clarifying questions

Claude asked a quite a few pertinent and useful questions about what I was trying to accomplish and then developed a respectable plan of attack, including task lists and Q&A checklists. We did a few rounds of this. Then: should it get started? Sure, why not. I turned it (mostly) loose.

Claude operates initially with a very low degree of trust; there’s a lot of Mother-may-I at the start. Mother, may I read this file? Mother, may I install this package? Your options are “yes,” “yes, and you don’t have to ask me this anymore,” and “no.” “No” halts progress until you provide alternative instructions.

I mostly turned it loose with little oversight. This is often my approach with human junior developers, too. It’s easier to provide constructive feedback after. And the results were pretty much what I expected.

Finally, Claude handed me its completed artifact and declaring something along the lines of “here is your awesome app, built just for you!” Just like an absolute greenhorn, though, it did this without even checking to see if the dev server would run.

It didn’t. Build errors all over the place.

I spent a little time on prompting fixes, and when that didn’t work I tried my hand at it myself. Here I was foiled because I’d told it to use a library I was unfamiliar with, so when that broke I was pretty much at a loss — at least for a weekend project. Easier to scrap it and start over.

The second attempt

I tweaked the spec from the previous project by hand, changing some of the technology requirements to simplify the stack, and then I turned it loose again. This was an improvement.

1. I was reasonably sure I could understand everything at the end.
2. When Claude did its victory lap, the dev server started.

(I think this latter is because we did a couple of rounds of “Here’s your thing!” and me saying “It won’t start, doof.” Eventually it seems to have decided checking to see if the dev server would run was a good idea.)

Most of the functionality didn’t work, though. Claude made the buttons but didn’t wire them up. “Why doesn’t this work? Oh, there’s literally no click event assigned.”

Rather than fight this, I scrapped things back and tried a different approach.

The third attempt (ongoing)

I am aware of spec kit, a set of scripts and prompts designed to walk you and an LLM through setting up a lot of project documentation. It’s a very waterfall approach, and some people are even starting to think LLMs will make waterfall work for the first time in history ever. Someday I will give spec kit a try, but Sunday was not that day.

<-- TODO: Write a rant about why waterfall is always doomed. (It's a people thing.) -->

I realized that I don’t write every bit of code and then try to run it for the first time at the end. I set up an automatically-updating dev server and confirm the results that I’m getting maybe a couple-hundred times an hour. It’s hard to have this workflow with Claude. But you know what might work?

That’s right. Agile methods.

The third attempt is taking this approach. I left the spec parts intact but removed the checklists. Now I am having Claude tackle problems one-at-a-time. Did Claude want to do this at first? No. It immediately built a massive specification document with to-do lists and QA checkpoints, then asked if it should do it.

I literally said “Don’t get ahead of me, we’re going to do this one feature at a time.” It pretended to be chagrined, and that’s how we’re working.

It’s also how I learned, very early on, that it had improperly wired up the CSS. “Hey, here’s your feature with beautiful professional typography and everything,” it said. I took a screenshot of the obviously broken and awful layout, showed it to Claude, and asked “are you sure?”

It said, “oops! I forgot to style the status markers.”

Really? That’s the only thing? Well, we’re learning.

Lol. I’m learning. Claude will forget everything soon enough.

Feelings at the end

Besides the big-picture worldwide-stakes arguments¹, I have a lot of concerns about AI-assisted development that are still not addressed.

1. Will we, as a matter of business, be able to afford this? This stuff isn’t cheap, and the companies selling it are going to expect to make a profit — and a huge one — someday soon. Anthropic not being able to be clear about plan quotas is not comforting.
2. Will AI coding save time? I suspect it may save some time in very small-team environments or auteur projects (where the developer is also the stakeholder), but the problem Frederick Brooks identified remains: the time-consuming part is deciding what to code, not doing the actual coding. While the typing goes really fast, we are lengthening the specification phase.
3. Will this change the nature of programming work into something I dislike? I enjoy programming because it lets me think about problems in close detail and explore intricate details that most people don’t think about. I don’t think I am alone in this. The value proposition of AI-assisted coding is that it takes away having to think about the detailed work (“the fun part”). In exchange, we get to do the technical product manager’s job (“the irritating part”).

Do I want to become a TPM? Well, I’m not one now, what do you think?