We've reached the vibe coding limit with Replit/Claude

Our project set out to determine where the line between AI hype and human engineering reality currently lies - and we have found that limit. While AI coding can be stunningly productive, it can only take you so far without the rigorous skills of a human engineer.

This is not a post against AI coding tools. Understanding this boundary is crucial for building products responsibly and knowing where to focus human effort.

Defining the Vibe Coding Limit

The term Vibe coding - where you fully give in to the vibes, embrace exponentials, and forget that the code even exists - was coined by Andrej Karpathy, founding member of OpenAI.

Based on this, the inherent limit of vibe coding is the point at which you must stop and examine the code itself. Our attempts to extend this boundary - adding domain-specific AI agents, having multiple LLMs perform code review, and implementing a deep AI code audit - technically go beyond that vibe coding limit. However, I was willing to keep those in bounds because my Fox Mulder wants to believe. 

At Burrow, we define the Vibe Coding Limit as the point at which human engineering skills and review are required to maintain code quality.

Burrow has definitely gone beyond this limit.

Limitations of AI enforcement of best practices in vibe coding

We tried various tactics to extend the Spaghetti Code Limit (SCL) - the threshold where code complexity and cascading issues make feature implementation exponentially slower - without rigorous human engineering review. Every step, despite explicit instructions to the AI to enforce best practices, eventually reached its breaking point.

Human review consistently identified shortcomings that the AI had missed or actively created. A short list of what was discovered includes:

• Violations of SSOT: Redundant default weights were stored in multiple locations, violating the Single Source of Truth (SSOT) principle.
• Inefficient Data Calls: A request to improve search by adding a new parameter led to Replit/Claude Opus proposing a new server endpoint to retrieve that single parameter, even though it was already in the local cache. This violated KISS (Keep It Simple Stupid) and DRY (Don't Repeat Yourself).
• Repetitive Code: Claude implemented various "hooks" for similar operations instead of consolidating them, another violation of KISS and DRY.
• Failure to Comply: Portions of longer prompts, especially those related to the deep AI audit, were ignored, likely due to context limitations. This would have been missed without a close, human review.
• Context Overload: In addition to ignoring specific prompt instructions, our deep AI audit of the entire project missed many issues that were later found by narrowing the focus to a file-by-file review. The broad context proved too much for the AI to handle effectively.
• Objections to Simplification: When attempting to clean up poor code, Claude objected, arguing the cleanup was too much effort and suggesting we continue building on the fragmented codebase. We had to override this objection, explaining that failing to clean the code would make the overall product increasingly fragile.

AI Limitations are Not the Same as Useless

What is possible with vibe coding and AI-assisted coding is stunning. Period.

However, being overzealous about AI and insisting that it can do everything under the sun without human involvement is incorrect and potentially dangerous.

Everything has limits, and AI coding will continue to improve, making more and more possible. Knowing the limitations is something that should be celebrated as part of rational practice. Now that we understand some of these limitations, we can proceed responsibly and keep a close eye on how that boundary extends.

For Burrow, we know what the problem is and now have to figure out what to do next. Stay tuned.