What AI can (and can't) do with code

Beginner · 6 min · Set realistic expectations for AI code generation and name where verification must sit.

coding
evaluation

AI code generation is a fast writing assistant for code: it produces plausible, often runnable drafts of small, precisely described tasks — and it verifies nothing. Reliability comes from your verification, not the model’s confidence. Knowing where each side’s strength ends is the whole skill.

What it does well

Small, well-specified pieces — “a script that renames every .jpg in a folder to a numbered sequence” — short tasks with clear inputs and outputs.
Transformations — translate a snippet to another language, reformat, rename consistently.
Explanations — “what does this error mean” is one of the safest and most useful requests.
Boilerplate — repetitive structure with a known shape, written quickly.
Tests as drafts — first versions to review, not proof of anything.

The four limits that stay

No guarantee. Fluent code compiles, runs sometimes, is not necessarily correct — edge cases (empty inputs, odd file names, missing folders) are exactly where drafts fail quietly. Same mechanism as why AI makes things up, wearing a compiler.
No context — only what fits its context window. It cannot see your files, versions or settings; whatever you do not state gets assumed silently — the describe-don’t-guess rule applies twice here.
No responsibility. It cannot test in your environment or own the outcome. It will also never tell you it did not test.
No fresh knowledge by default — its training cutoff never moves. Libraries change; a plausible call may be two versions old. Version facts follow the search-vs-generation rule.

The cost center is review

Generation is fast; review is the bottleneck. Code you cannot read is code you cannot trust: either you understand what each line does, or you keep the task small enough to verify by behavior — a copy folder, fake files, known results. Both paths work. Skipping both does not.

A bad example

Ask: “Write a script that cleans up our production database.”

Vague input, destructive output, and “cleans up” is not a specification — it is a wish with a delete statement inside.

A better example

Ask: “Write a script that, given a folder path, lists every file older than 30 days with its size — read-only, delete nothing. I’ll run it on a test folder first.”

Start read-only, verify the list, and only then consider a second step that acts. Small steps you can check beat one big step you have to trust.

Your assistant writes a small script that looks fluent and compiles. What does the lesson say this proves?

The script was tested in your environment before it was handed over Nothing about correctness — fluent drafts are not guaranteed, and edge cases are where they fail quietly The library calls are current — a compiling script cannot use outdated APIs

Practice

Choose the right coding task

You are not a programmer. You want AI help with something small, and you are deciding which kind of task fits what code generation actually does well on a first pass.

Pick the task that fits AI code generation well — as a draft to review.

Hint

Which task has a small, checkable outcome?

Which tasks ask for guarantees or judgment rather than a draft?

Transfer

Personal automation: small scripts, run on copies, checked by behavior.
Learning: ask for explanations of code you find — a genuine study loop.
Anything production: AI drafts, a human reviews and owns.

Key takeaways

AI writes plausible drafts of small, precisely described tasks — and verifies nothing.
Four limits stay: no guarantee, no context, no responsibility, no fresh knowledge.
Review is the bottleneck: understand each line, or test by behavior on a copy.

Next: Describe, vet, never blind-run — the review pass that makes generated code safe to execute.