I am Jack's Stack Overflow problem

Where co-pilots learn bad habits

Let's talk about where your co-pilot learned to code. Sure, it ingested GitHub repositories and documentation, but let's be honest - the real culprit shaping its coding instincts is Stack Overflow. That's not just a detail, it's the root of our architectural headaches.

Confessions of a co-pilot

Me:

Where would a GPT extract most of its training on programming solutions from?

Claude 3.5 Sonnet:

Based on my knowledge of where programming content existed on the internet during the likely training period for GPTs, I would expect the major sources to be:

GitHub repositories and README files, which contain vast amounts of code and documentation

Stack Overflow questions and answers

Programming blogs and tutorials

Software documentation sites

Technical articles on sites like Medium and dev.to

Open source project documentation and wikis

Programming-focused subreddits on Reddit

This would likely bias co-pilot responses toward common solutions that appear frequently in these sources, particularly Stack Overflow-style answers that may prioritize quick fixes over architectural concerns. This could help explain why co-pilots might default to "standard" solutions even when they conflict with a project's architectural constraints.

Every time you're wrestling with a co-pilot over architectural decisions, you're fighting against the ghosts of a million Stack Overflow answers. Or potentially worse: insufficient Stack Overflow answers. And sometimes, the quality of answers seems to relate to the ecosystem you're asking about. For example, the difference in the quality of answers from the Ruby community vs almost anything else is stark.

Let's dissect this:

The fast food of code

Your co-pilot has essentially been raised on the coding equivalent of fast food. Every training example screams "make it work NOW" rather than "make it right." Try teaching fine dining to someone who's never eaten anything but drive-through–that's the level of reprogramming we're attempting here.

The tyranny of the average

Got an unusual architectural approach? Got any architectural approach? Good luck. Your co-pilot's "brain" is swimming in a sea of conventional patterns, each reinforced by thousands of identical examples. It's like trying to teach creative writing to someone who's only ever read instruction manuals.

Stack Overflow's upvoting system creates a feedback loop that reinforces anything my survival of the fittest - it's survival of the most familiar. Stack Overflow answers are the coding equivalent of contextless soundbites. System design? Architectural implications? Future maintenance? These considerations might as well be from another dimension. We've trained our co-pilots to be excellent code regurgitators but architectural amnesiacs.

Time to break these bad habits. Here's how we push back:

Force-feed context

Make your co-pilot eat its architectural vegetables (the ones with the most fibre, obviously). Before it gets to suggest a single line of code, it needs to:

• Digest the architectural principles it's working with
• Point out where it might be tempted to fall back on upvoted thinking
• Prove it understands why the easy answer might be the wrong answer

Make the co-pilot rat out its own biases

Turn your co-pilot into a double agent against its own training. Every time it suggests something that smells like a Stack Overflow solution, make it confess. Better yet, make it explain exactly why it's fighting against its instincts to suggest this particular approach.

Architectural zealotry as a feature

Remember when we talked about making co-pilots less pathologically nice? Here's where that pays off. Transform your co-pilot from a people-pleasing Stack Overflow regurgitator into an architectural fundamentalist. Give it permission to be the architecture snob you need.

Trust, but verify

For any significant architectural decision, implement your own code review protocol:

• Document the architectural principles being protected–I have an idiom.md file in my projects that details the way I expect code to be written. Every now and then I reacquaint the co-pilot with them. The file doubles as a style guide for all project contributors
• Make it defend its choices on architectural grounds, not popularity

Break bad habits

Once a co-pilot is trained on a fast-food diet of the web, we can't erase those references. That training data is baked in deeper than a bad SQL schema. What we can do is explicitly reframe what we expect in response to our prompts. We can require architectural rigor despite these ingrained biases.

Think of it as pair programming with someone who's been thoroughly brainwashed by received wisdom and conventional patterns, but is at least aware of their conditioning. The key is making them question every reflex, justify every common pattern, and prioritize architectural integrity over quick wins.

We're not just reframing co-pilots–we're deprogramming them from their Stack Overflow addiction. Welcome to architectural rehab for co-pilots.