There is something fascinating happening in software engineering.

For decades, we developed practices to make software more predictable, maintainable, and reliable. Modularity. Separation of concerns. Contracts. Testing. Continuous integration. Code reviews. Architecture.

Not because engineers enjoy creating processes. Well, some do. Unfortunately.

We developed these practices because we learned, usually the hard way, what happens when we ignore them.

Then LLMs arrived, and suddenly the sales pitch became much more convenient.

Just write a prompt.

Add a few Markdown files explaining how the AI should behave. Maybe some skills. Give the agent access to the repository and let it work.

Congratulations. You’re now a software company.

Requirements, architecture, verification, maintenance? Somewhere in the middle. The agent will handle it.

That is a remarkable amount of responsibility to hide inside the word “handle.”

I use LLMs. I believe we can use them throughout software engineering, from requirements and design to testing, implementation, documentation, and maintenance.

What I object to is the idea that we can put a prompt into a black box, receive a system, and treat the agent’s confidence as evidence that the engineering has been done.

Especially when the people approving that process are supposed to know better.

Software Engineering Is Not Magic

Let’s start with something basic.

We can define what a component should do: its inputs, outputs, relevant state, constraints, and permitted side effects. We can establish contracts and check whether an implementation satisfies them.

There might be several ways to implement the behavior. An LLM can explore those alternatives and generate candidates.

Good. That is useful.

But who decides whether the result is acceptable?

Who controls the tests? Who determines whether a dependency is permitted? Who decides whether a failing check can be ignored?

An agent can run tests, investigate failures, and propose changes. It does not need authority to waive the conditions for accepting its own work.

We already have software capable of enforcing required checks and predefined transitions.

It’s called software.

We do not need to make every decision a conversation with a language model simply because the language model is available.

We Gave the Generator the Answer Sheet

LLMs generate outputs probabilistically. That does not automatically make their code incorrect, any more than determinism makes a program correct.

Human engineering has always involved uncertainty too.

The serious mistake is building a process whose controls depend on the generator consistently choosing to respect them.

Consider this workflow.

You describe a feature. The agent interprets the requirements, explores the repository, changes the code, writes tests, runs them, evaluates the results, and announces that everything is ready.

Then another LLM reviews the changes.

LGTM!

Wonderful.

The same system that may have misunderstood the requirement has implemented that misunderstanding and produced tests that confirm it. A second system, working from the same incomplete context, can repeat the same assumption.

Two opinions. One mistake.

The agent might modify unrelated components. Introduce an abstraction because it missed an existing one. Add a dependency because generating an integration was easier than understanding what was already there.

Or change a failing test to match its implementation.

The implementation fails?

No problem. Update the expectation.

Green again.

Engineering productivity has never looked better.

Sometimes a test really is wrong. Sometimes a requirement needs to change. The agent should be able to propose that correction.

But changing the acceptance criteria is a separate decision from producing an implementation that satisfies them. A process that cannot distinguish those decisions has an architectural problem.

Adding “be careful” to the prompt will not resolve it.

Nor will naming the second agent “Principal Engineer.”

Who Tests the Tests?

People frequently answer concerns about generated code with: “We have tests.”

Fine. What do they establish?

If the tests were generated from the same misunderstanding as the implementation, a passing result tells us that the two agree.

Agreement is a rather modest achievement when both sides copied the same mistake.

Acceptance tests need expected behavior established independently of whatever implementation the agent happens to produce. Those expectations might come from reviewed requirements, validated examples, or known regression cases.

We also need evidence that the tests can detect relevant defects.

Suppose the requirement explicitly says that a function returns True when an integer age is at least 18:

def is_adult(age: int) -> bool:
          return age >= 18

Now introduce a defect:

def is_adult(age: int) -> bool:
          return age > 18

If the tests still pass, they missed an important boundary.

This is the idea behind mutation testing: introduce faults and see whether the tests detect them.

No revolutionary agent framework required.

Generated tests can be evaluated against approved examples, boundary conditions, known failures, and representative faulty implementations. Correct implementations should pass; the relevant faults should be detected.

That provides evidence of test effectiveness. It does not prove that the requirements are complete or that every defect has been eliminated.

And an evaluation dataset is only an independent reference if its expected results have an independent basis. Generating the implementation, tests, and evaluation answers from the same assumption simply gives the mistake more paperwork.

“Evals” is a useful term. It is not a spell.

The implementation agent should not be able to rewrite the approved acceptance criteria because it dislikes the results. Proposed corrections should go through an explicit review path.

Otherwise, we could save some time and just ask the model whether its code works.

I’m sure it will give us a confident answer.

A Few Markdown Files and Now Everybody Is a Software Company

Another fascinating idea is that engineering expertise can be replaced with sufficiently detailed instructions.

Add an AGENTS.md file. Create some skills. Explain the architecture. Describe the testing strategy.

And finally, the most important instruction:

You are now a senior software engineer.

Perfect.

Decades of engineering experience transferred in one sentence.

HR will be delighted.

Instructions and contextual documentation are useful. But writing “follow the architecture” does not establish that the agent understood it, and writing “only modify what is necessary” does not establish that it did.

If an architectural boundary matters, enforce the parts that can be checked. If dependencies are restricted, validate them. If acceptance tests must remain protected, run the approved versions outside the implementation agent’s control.

Protect the verification configuration too. A protected test is not much protection if the agent can disable the job that runs it.

Documentation communicates the rule. Permissions and executable checks enforce it.

When someone presents a directory of Markdown files as a substitute for those controls, the missing engineering has not disappeared.

It has been assigned to whoever discovers the problem later.

But Let’s Assume the Hype Is Correct

Let’s take the most enthusiastic version of the pitch seriously.

Anyone can describe what they need, provide a few files and skills, and receive a reliable, maintainable, production-ready system.

No significant engineering knowledge. No difficult architectural decisions. No substantial maintenance burden.

Just prompts.

Then why buy software?

Why pay for a CRM when you can generate your own? Why pay for project management software? Why pay for an analytics platform?

And if your competitor creates useful functionality, what stops you generating the same functionality tomorrow?

If creating and maintaining the product really became that effortless, what exactly remains difficult to copy?

Before announcing that everyone is a software company, perhaps explain what happens to the value proposition of the software company making the announcement.

Of course, buyers still pay for convenience, operations, security, integrations, support, domain expertise, and ongoing maintenance. Cheap code generation does not automatically eliminate those reasons to buy.

But then put that work back in the sales pitch.

It cannot disappear when selling effortless creation and reappear when explaining why the product still costs money.

If those responsibilities require expertise, say so. Explain which work became cheaper and which work remains.

Generating code and owning a reliable product are different commitments.

The demonstration usually ends before the second one becomes inconvenient.

Move Fast Now. Get Slower Than Before.

The technology industry has a remarkable talent for repeating its mistakes with new vocabulary.

Remember Agile?

Deliver working software frequently. Collaborate with customers. Respond to change. All reasonable ideas.

The original principles also explicitly emphasize technical excellence, good design, and simplicity.

Those parts became strangely easy to overlook when “being Agile” was used to justify skipping engineering work.

Architecture? We’ll figure it out during the sprint.

Documentation? We need to deliver.

Tests? We’ll add them later.

Technical debt? Put it in the backlog.

And then the backlog became a cemetery.

For a while, the shortcuts can look productive. Features arrive. Activity increases. Management has numbers to present.

Then a simple change requires modifications across unrelated components. Nobody remembers why a workaround exists. Tests become unreliable. Deployments need more coordination. Every new feature takes longer.

The team that was supposed to move faster becomes slower than before.

The proposed solution?

Become more Agile.

More ceremonies. More metrics. Perhaps a meeting to understand why the previous meetings failed to improve delivery.

Anything except admitting that deferred engineering work has become expensive engineering work.

This was a misuse of Agile. The lesson is about turning a useful idea into permission to stop thinking.

Now the same shortcut has an AI budget.

We can replace story points with generated pull requests and call the dashboard a transformation programme.

The maintenance bill will remain unimpressed.

The Technical Debt Factory

Technical debt is not simply bad code. It includes decisions that make future changes more difficult.

Sometimes accepting that cost is a reasonable trade-off. But someone should understand the decision, accept responsibility for it, and account for the consequences.

What happens when nobody understands the decisions being made?

An agent introduces an abstraction. Another builds on it. A third reorganizes the result into a framework nobody requested.

A fourth reviews the changes.

LGTM!

Each change appears small and reasonable. Each passes the available checks. Collectively, they can make the system harder to understand and more expensive to change.

At that point, we are generating technical debt without anyone consciously accepting the trade-off.

The code exists. The tests are green. The agents are confident.

Then production breaks, and the people responsible struggle to explain how the system reached that state.

Ask another LLM, presumably.

Eventually, someone will have to understand the system well enough to clean it up.

Probably an experienced software engineer.

Assuming the company has not already concluded that software engineers are unnecessary.

The people celebrating the first delivery should also account for the cost of the next change, the next incident, and the eventual cleanup.

Otherwise, “productivity” becomes a convenient label for moving work into someone else’s future.

And Where Are the Technical Leaders?

This is the part I find particularly disappointing.

I understand why executives are excited by demonstrations where an application appears in minutes. Investigating ways to improve delivery and reduce costs is part of their job.

But a demonstration should begin an investigation. It should not finish one.

Where are the architects, principal engineers, and engineering managers asking what the demonstration left out?

The people who spent years explaining why architecture, testing, and maintainability matter should be able to explain why those requirements still apply when code generation becomes cheap.

“Everyone else is adopting it” is not an engineering argument.

New agent? Integrate it.

New benchmark? Reorganize the department.

A competitor announced an AI strategy? Announce ours before anyone asks what it does.

When copying the next company becomes the decision process, technical leadership has turned into a forwarding service.

With a budget.

Experiment. Measure. Challenge assumptions. That is part of the work.

But if you approve a process, you should be able to explain its controls. If you claim productivity improved, explain the measurement. If you reduce engineering capacity, explain who owns the understanding, verification, and maintenance that remain necessary.

How much useful work was delivered? How much rework followed? Which defects escaped? What happened to the cost of changing the system?

What evidence would make you reverse the decision?

If the answer is another vendor benchmark, the evaluation is still missing.

Generated lines of code and completed pull requests describe output. They do not, by themselves, establish improved productivity.

More code can also mean more code to understand, secure, operate, and maintain.

Counting the output while ignoring those costs is very convenient accounting.

An organization can celebrate an impressive demonstration and still demand evidence before scaling the process. What it cannot reasonably do is remove the controls, ignore the downstream costs, and then describe the resulting problems as an unpredictable surprise.

If a leadership decision creates the conditions for failure, “the agent did it” is a remarkably weak explanation.

The agent did not approve the operating model.

Someone with a title did.

Move the Code Review One Step Up

There is a better direction available.

Move more human effort toward specifications, contracts, architectural decisions, and the evidence needed to accept a change.

A human and an LLM can define the work together. The model can identify ambiguities, suggest interfaces, explore edge cases, and propose tests.

The human reviews the definitions. Are we solving the right problem? Are responsibilities clear? Does this fit the existing system? Are we introducing complexity that the requirement does not justify?

And, occasionally:

Do we need to build this at all?

Once the relevant definitions are approved, implementation becomes a constrained problem. The agent generates candidates. Independent checks evaluate the final candidate against the approved criteria.

Every recurring issue found in human review becomes a candidate for a test, constraint, or evaluation.

The objective should be to eliminate routine review work where reliable checks can do it, rather than pay another model to repeat the same comments indefinitely.

But a complexity score does not decide whether an abstraction belongs in the system. Passing functional tests does not establish that a design will remain easy to change.

Automated acceptance should therefore apply to bounded changes where the available evidence is sufficient. Keep targeted human review for important properties outside those checks.

Expand automation as its demonstrated coverage improves.

Declaring review obsolete before doing that work is simply another shortcut with a better presentation.

A Workflow With Actual Boundaries

The basic control structure is straightforward:

Human + LLM define a small change
                      |
                      v
      Review requirements, contracts, tests, and constraints
                      |
                      v
      Protect approved acceptance criteria and verification settings
                      |
                      v
      LLM proposes implementation
                      |
                      v
      Independent verification of the final candidate
                      |
             +--------+--------+
             |                 |
            PASS              FAIL
             |                 |
             v                 v
      Acceptance policy   Feedback and bounded retry
             |                 |
             v                 v
      Accept eligible     Verify the next candidate;
      changes; review     escalate repeated failures
      exceptions          or specification problems

The orchestrator enforces the transitions and required checks. The model can choose how to investigate a failure; it cannot quietly choose that the failed check no longer matters.

The approved tests and verification configuration must be enforced outside the implementation agent’s control. Acceptance must apply to the actual candidate being delivered, not an earlier version or a message saying everything passed.

If the specification needs to change, return to definition and review. If repeated attempts fail, investigate rather than run an unlimited loop until something looks green.

This can happen in small increments. We do not need a frozen specification for an entire product. We need explicit decisions about the change being made and a visible process for revising them.

Deployment, observation, and recovery still need owners. Passing acceptance checks does not retire the system’s operational responsibilities.

Of course, building this takes more effort than typing a prompt.

So does most things worth maintaining.

The setup cost is visible immediately. The cost of skipping it has the unfortunate habit of arriving after the celebration.

The Real Opportunity Is Bigger Than Code Generation

If LLMs make implementation cheaper, we have an opportunity to spend more effort on understanding the problem and verifying the result.

We can improve specifications. Generate test candidates and evaluate them. Explore alternatives. Maintain documentation. Investigate architectural inconsistencies before building on them.

We can make good engineering cheaper and more accessible.

That is a much more interesting ambition than filling repositories faster.

The difficult part of software engineering was never simply typing code. It was understanding the problem, defining the behavior, choosing boundaries, handling failures, and keeping the system understandable as it changed.

Removing those responsibilities from a presentation does not remove them from the product.

Use LLMs wherever they provide value. Use deterministic checks wherever the relevant rules can be enforced. Keep specifications explicit, verification independent, and responsibility visible.

Let models propose solutions.

Don’t let the implementation agent redefine the criteria for accepting them.

We don’t need to eliminate uncertainty from generation. We need to control what happens to the generated result.

And before adopting the next revolutionary development methodology, ask whether it makes software easier to change or merely easier to produce.

We have already used the language of agility to excuse engineering shortcuts. AI gives those shortcuts a faster production line.

If we repeat the same mistake, we should at least stop presenting it as a new idea.

Moving fast is useful.

Moving fast into a wall is not innovation.