Notes on Coding with AI
Since 2023 I have written one article a year about AI, recording what I understood about it at the time and what actually using it felt like.
The 2023 piece was about getting acquainted with ChatGPT, AIGC and large language models. The 2024 piece discussed AIGC's comprehension and generative abilities and their likely consequences. The 2025 piece documented the first time I let an AI complete a real project largely on its own. (All three are in Chinese: 初识 ChatGPT, 我对于 AIGC 的一些看法, 一个 AI 独立完成的游戏自动化脚本.)
This year AI has moved from a question-answering and autocomplete tool to an agent that continuously reads a project, edits code and executes tasks. This is the fourth entry in that annual series, and the question has shifted from "can AI write code" to "how should a person work with AI to build software".
There is no longer any point in debating whether AI can participate in real development. For code generation, localised edits, research, verification and debugging, it has clear practical value.
Whether AI can replace developers is the question I have been sitting with all year.
AI changes more than how you type
When writing code myself, I generally did not finish the design before starting. Once the requirement was clear I would begin implementing, and directory layout, code organisation, module boundaries and API shape would be adjusted as I went. It sounds sloppy, but a person is continuously engaged, continuously correcting their own understanding of both the requirement and the code.
Carry that habit into AI-assisted development and something changes. Requirement analysis, trade-off evaluation and implementation detail can all be produced by the AI in one continuous stretch, and the developer never derives any of it. If you neither fully understood the requirement nor formed an expectation of the result, it is very easy to accept whatever runs.
So the analysis and design still have to be done by a person. Where it helps, sketch the flow, pseudocode or an interface draft first — get the goal, the boundaries and the overall structure straight, and only then have the AI implement it. Asking for whatever comes to mind, one thought at a time, tends to produce repeated rework: wasted time, wasted tokens, and often a worse result anyway.
This does not mean every requirement needs a long document. A document is just a vehicle for thinking. What matters is whether the developer actually understands the problem.
What AI is good at
Validating ideas quickly
AI is well suited to writing demos. Researching how a component library is used, verifying an API, reproducing an edge case, comparing a few technical approaches — all fine to delegate. The goal is clear, the scope is bounded, and even if the implementation has to be thrown away the cost is low.
Implementing modules with clear boundaries
I maintain a CLI project: a full set of dev and build commands for runtime micro-frontend tooling, with a number of Webpack, Babel and PostCSS plugins built in. A lot of that code was written by AI.
For a Webpack plugin, I state the requirement and the constraints and let the AI implement it. The plugin itself may be intricate, but within the overall pipeline it has a clean boundary and cohesive responsibility. Even if the generated code has problems, the blast radius stays inside one small module. Meanwhile the CLI's overall flow and its critical junctions are still mine, which is why I feel confident about the project.
What I am currently willing to hand over:
- Implementations with clear boundaries that can be verified independently.
- Work that is repetitive and tedious but low in decision density.
- Demos for research and exploration.
- Local detail, once the overall structure is settled.
The task type decides the payoff
Whether AI improves development efficiency cannot be discussed apart from the specific task. A task with a clear goal, bounded context and easily verified output is simply not the same problem, from the AI's point of view, as a business requirement that spans several systems and depends on a great deal of tacit knowledge.
A few dimensions for judging whether a task suits AI:
| Dimension | Better suited to AI | Needs human control |
|---|---|---|
| Context scope | Single file, single module | Cross-system, cross-team |
| Verification | Clear types, tests or runtime output | Correctness depends on business judgement |
| Cost of error | Easy to roll back, limited blast radius | Touches data, security, money or a core flow |
| Requirement state | Goal and boundaries already settled | The requirement is still being explored |
| Source of knowledge | Obtainable from code and docs | Depends on undocumented team experience |
The further a task sits to the left, the more directly AI pays off. The further right, the more the cost shifts into understanding context, forming a plan and verifying the result.
Why a complete business feature is harder
Hand a complete business requirement to an agent and the code will usually not match what the developer had in mind. The problem is that if the developer had nothing definite in mind either, and the output appears to run and passes verification, it is very easy to just merge it.
For bounded tasks with a clear goal, short-term efficiency usually does improve. For complex tasks in a mature codebase, the cost of understanding, verifying and reworking can cancel out the savings on generation.
Existing research shows the same split. In a controlled GitHub Copilot experiment, participants completed a fixed JavaScript HTTP server task 55.8% faster. But when METR ran a study in early 2025 with open-source developers working on codebases they knew well, AI tooling increased completion time by 19%. METR later noted that as agents spread, parallel work, task selection and participant self-selection make productivity harder to measure at all. None of these results generalise to every situation, but together they establish that "does AI improve productivity" has no single answer independent of task type.
Related research:
- The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- We are Changing our Developer Productivity Experiment Design
Over a longer horizon, each round of generation may bring its own style and design instincts, and locally reasonable does not add up to globally reasonable. The more serious case is when no engineer is genuinely familiar with a given module or a given period of the code, which makes subsequent maintenance and debugging much harder.
A simple example: one round of agent work produced a decent API, but the developer did not retain it and the knowledge base was not updated. A few rounds later another agent implements something similar, and redundancy accumulates. The problem is not which model was used. It is that the project lacks stable module boundaries, a way to discover existing capabilities, and a mechanism for keeping knowledge current.
Over-design and defensive code
In my experience, without explicit constraints, generated code also tends toward over-design. A single function ends up with far more parameters than the current requirement needs. The extra capability may well be thoughtfully considered — but the developer never thought about it, does not currently need it, and merges it because "this might be useful later".
Those designs are not necessarily wrong, but they violate the principle of evolving from the minimum. Every extra parameter, branch and abstraction becomes something that has to be understood and maintained.
Coding style needs to be stated
Developers also have to state their preferred coding style and design principles rather than leaving the AI to improvise.
Without project-level constraints, an AI may produce large all-in-one components while the project actually emphasises separating data, presentation and business logic. In frontend code, where implementation styles are especially flexible, explicit constraints matter more. Otherwise each generation round drifts toward a different structure and style, and maintainability erodes.
Documents do not substitute for understanding
To get a requirement done in one pass, the common practice now is to break it down, produce a requirement design and a high-level design, write the results into Markdown, and hand that to the AI. Those documents have their place, but their existence does not mean the developer has done the thinking.
It is entirely possible to not read the document carefully, not really understand the requirement, run a few prompts, invoke a few skills, and let the AI proceed to the next step. The resulting code may not match expectations, and the developer — lacking an understanding of the requirement and the design — will struggle to spot the important problems in review.
Documents also go stale. If design docs, the knowledge base and the code are each updated by a different process, they drift apart. For critical decisions, generating a document is not enough: you have to be explicit about which location is the source of truth, who maintains it, and how it is synchronised after a code change. Otherwise more documents just means more stale information for the AI to read.
We now have a fairly complete AI development process: product submits an AI-generated requirement document to a Git branch, developers pull the branch, produce requirement and high-level designs, review them, split into development tasks, let the AI implement, and finish with human inspection, verification and handoff to QA.
By that process, developers are mainly involved in the high-level design and the review. But when a requirement spans several frontend and backend modules and the owner is not familiar with the relevant code, judging whether the design is correct becomes very hard. AI can summarise a module quickly; it cannot replace the process by which a person absorbs information. A human brain cannot take on all that context in a short window. Even after reading the summary you still need time to build a mental model of the code.
Skip that, and the developer is bluffing: unable to judge whether the AI's design is sound, and unable to meaningfully check its implementation later.
Verification has the same problem. Frontend and backend may now be done by one person working with AI. If the developer does not understand how the backend constructs and processes data, they have to spend time learning it first. AI can generate SQL, hand you a curl command, or trigger a scheduled job — but those operations still contain decisions, and you cannot assume every answer is correct.
"Humans decide, AI implements" sounds reasonable, but it has a precondition: the human must understand the project's context. On teams adopting agents, the frontend/backend division is beginning to blur, and not every developer has full-stack, cross-stack and architectural ability. That is a real difficulty in the current practice, not a hypothetical one.
Generation speed is not delivery speed
What AI directly reduces is the cost of producing some of the code. Requirement clarification, design review, result verification, cross-stack integration, release and maintenance costs do not drop to zero alongside it. Evaluating AI's effect on efficiency means measuring the full cycle from requirement to stable delivery, not just how long the typing took.
DORA's research reflects the same distinction: higher AI adoption correlates with increased delivery throughput and with increased delivery instability. AI can produce more changes faster; teams still need automated tests, fast feedback, smaller batch sizes and reliable rollback to keep the risk contained.
Related research: DORA: Balancing AI tensions
Where a sense of control comes from
I once read a comment under a Chinese-language video on Bilibili. The commenter described doing engineering work with AI: constantly reading AI-generated documents and code, and then, when QA filed a pile of edge-case bugs, not understanding the implementation well enough to act — only able to hand reproduction steps back to the agent and wait for it to locate and fix each one. As the deadline approached, that inability to judge where the problem was became increasingly stressful.
Another reply argued that you can let AI handle the details, but a person still needs to know the rough flow of a feature, how data moves, and the overall structure. When the AI cannot fix something after several attempts, the person should go back into the code, find the cause, and then point the AI at the specific problem.
What both comments are circling is the developer's sense of control over the system.
If a requirement goes from analysis to implementation entirely through AI, and the person participated in none of the key decisions, it is very hard to feel confident in the result. If the person establishes the overall structure first — how data flows, how modules cooperate — and then has the AI fill in the local detail, then when something breaks they can at least locate which junction it broke at.
"Control" here does not mean hand-writing or line-by-line reviewing everything. For details with limited blast radius, type checking, tests and runtime output are adequate verification. For the flow, the boundaries and the key decisions, the person needs to keep understanding.
For me, being able to answer the following is the basic test of whether I actually own a requirement:
- Where does the data come from, which modules does it pass through, where does it end up written?
- Which interfaces, pieces of state and side effects are affected?
- What are the failure paths through the core flow?
- When something goes wrong, which junction do I check first?
- Which outcomes are guaranteed by types and tests, and which need human judgement?
- After release, how do I observe the effect, and how do I roll back on failure?
If none of those can be answered, then even with the code generated and the local tests passing, I would not call the requirement under control.
Do not outsource the thinking
Looking back, the plans and flows an AI produces after reading project context are often still worse than what I get from walking the code and thinking it through myself. Mine matches my expectations better, and it carries the context that the project takes for granted but has never written down.
This is how I read "sharpening the axe does not delay the woodcutting": one extra round of thinking up front, manually laying out the critical flows, is usually more reliable than a single generation pass followed by a result check. At minimum, by the time it reaches QA, you can still explain what the code does.
Do not hand everything to AI, and above all do not hand over the thinking. When you genuinely do not know how to approach something, having the AI propose options or brainstorm is fine — but the output still has to be read, understood, and where necessary validated with a demo.
If you did not understand one step and let the AI continue anyway, every subsequent step is built on that uncertainty. It is a lot like losing attention in a lecture: you missed one derivation, you keep listening, and you only accumulate more confusion.
The practical problem is that the industry mostly measures delivery speed, efficiency gains and headcount saved. Learning fundamentals and understanding AI-generated code are not the easiest things to quantify. But neglect them long enough and a developer's understanding of the project erodes, taking the basis for making good decisions with it.
Wanting everything fast
Another thing I have clearly noticed: since using AI, it is harder to settle into doing one thing. Running several agents at once and pushing multiple tasks forward looks like parallelism, but in practice it means constant context switching, and it is exhausting.
AI extends me into domains I do not know and gets me to a starting point quickly. In domains I do know, if it keeps interrupting a train of thought, it is interference.
Reinventing wheels
Development used to be expensive enough that when you needed a tool you searched for an existing product first, checked whether it fit, and only built your own when it did not.
Now, when an idea arrives, it is very easy to have AI generate one, then maintain it and use it yourself. Convenient for the individual; in aggregate, potentially an enormous amount of duplicated implementation.
AI lowers the cost of the first implementation. It does not eliminate dependency upgrades, security fixes, data migration, deployment or long-term maintenance. Whether reimplementation is worth it still requires comparing the purchase-and-integration cost of an existing product, the lifecycle cost of building your own, and the exit cost when you eventually stop using it.
Not all duplication is a problem, of course. For one-off scripts, personal demos and learning projects, generating one is often the cheapest option. What warrants caution is a temporary tool gradually creeping into a core workflow without anyone re-evaluating who maintains it and what it risks.
Where does the software process go
The development processes built up over the past few decades do not fully fit AI-assisted development, and we are still in the exploratory phase. Two kinds of change seem plausible:
- More disposable and personally-customised software. For that category, maintainability and extensibility may drop in priority; software becomes closer to an instant consumable.
- New paradigms in the vein of waterfall and agile, imposing clearer constraints on the human/AI division of labour, the inputs and outputs, and who is accountable for quality.
The current difficulty comes precisely from the boundary between human and AI being unclear.
Everything above concerns AI as it exists today, based on current large language models. Whether AGI with substantially greater autonomy, reliability and generality arrives, I cannot say. If it does, software engineering may change dramatically again. The AGI question is about scope of capability and degree of autonomy, not about whether the system uses a probabilistic model.
Given current capabilities, the most important thing in AI-assisted development is still thinking the problem through. Humans design the overall flow and the key structures and keep the deterministic parts in hand; the tedious, local and exploratory work goes to AI, with the blast radius of its output confined inside clear boundaries.
If requirement comprehension, design and architectural decisions are all delegated, and the developer is not familiar with the result either, then from the standpoint of executing the task, swapping in a different person to operate the AI may not make much difference.
Where does frontend go
This year happens to be my tenth in frontend. From building pages to working on frontend architecture, the role has changed a great deal. Under the influence of AI coding, "will frontend still exist" is being debated again — though the same argument long predates AI.
One common prediction is that frontend and backend both collapse into full-stack. In most web systems the business logic sits mostly on the backend, so the reasoning goes that backend failures are more likely to cause production incidents while frontend problems are easier to fix. But how severe an incident is depends on blast radius, whether data is recoverable, the release mechanism and the ability to degrade gracefully — not on which layer the bug is in.
Frontend has its own domains that take years to accumulate. In an app without hot-update capability, or where dynamic updates are restricted by platform review policy, a frontend bug that causes widespread crashes after a release can take as long to fix and hurt as much as a backend failure. Apple's App Review Guideline 2.5.2 restricts downloading, installing or executing code that changes an app's features.
Beyond that: accessibility, browser compatibility, rendering performance, complex interaction, SSR and hydration, multi-platform containers, audio and video, design systems — all carry a great deal of knowledge tied to a specific runtime environment. AI can help implement these. Cross-domain execution ability is not cross-domain professional judgement.
Related: Apple App Review Guidelines
Another claim is that the AI era will stop distinguishing product, design, frontend, backend and QA at all — everyone becomes a generalist, communication cost falls, delivery speeds up.
I am sceptical. One person's capacity is limited, and going deep in every domain simultaneously is hard. Modern software development specialised because product, design, frontend, backend and QA each accumulated a large body of knowledge that takes time to acquire. AI lowers the barrier to working across domains; it does not follow that professional depth and division of labour simply dissolve.
For a one-person company or a personal product, full-stack works fine. With AI, one person can get from idea to demo and even to production quickly. But as the product iterates, users accumulate and problems keep surfacing, one person struggles to cover every detail indefinitely.
My current view is that the more likely outcome is smaller specialist teams — developers in a given area taking on more than they used to, with AI's help — rather than every role merging into an undifferentiated full-stack one. How much smaller depends on business complexity, quality requirements and the team's own engineering maturity; there is no fixed ratio.
AI will change headcount, job boundaries and how people collaborate. Professional specialisation still has value.
That said
All the concerns above rest on one premise: that people still need to read and understand the project's code.
What if we no longer need to understand generated code line by line? If software can be continuously generated, verified, maintained and replaced by AI, and code is merely an intermediate artifact, then much of the current discussion about readability, maintainability and control may need revisiting.
Even so, not understanding every line does not mean not understanding the system. Attention would shift from concrete implementation to verifiable requirements and acceptance criteria, system behaviour, observability, security and compliance, release and rollback — and, finally, to who is accountable.
So the more precise question may be: once code becomes a machine-generated intermediate artifact, does a person still need to understand system behaviour and answer for the requirements, the verification criteria and what actually runs?
I do not have an answer to that.
At this stage I still hold to this: let AI extend your reach and speed up the work, but do not give up understanding the problem, and do not hand over the key thinking and the final judgement.
你要请我喝一杯奶茶？
版权声明：自由转载-非商用-保持署名和原文链接。
本站文章均为本人原创，参考文章我都会在文中进行声明，也请您转载时附上署名。
