Linus Torvalds Used AI to Debug the Linux Kernel, but He Had to Keep It From Giving Up

Source

Linus Torvalds: drm/xe: Don't hand out the flat CCS storage as usable VRAM

In July I wrote about The Problem Wasn't the AI—Linus Torvalds pushing back on treating the Linux kernel as an anti-AI project and calling AI a useful tool that should be judged like any other.

It is unusual enough to see Linus Torvalds personally author a patch deep inside an Intel graphics driver.

The reason he ended up there is even more interesting.

Torvalds had been chasing a nasty Intel Xe graphics problem that could leave a machine with a black screen while GDM repeatedly restarted. The failure had become reliably reproducible on an Intel Battlemage G21 GPU with 16 GB of VRAM.

The eventual fix was almost embarrassingly small.

The Xe driver was calculating where usable VRAM ended and Intel's compression metadata storage began. It rounded that boundary up to a 128 KB boundary.

That is exactly the wrong direction when everything below the calculated address is going to be handed to a memory allocator.

Rounding upward meant a small piece of memory actually owned by the GPU's compression hardware was being advertised as ordinary usable VRAM.

On Torvalds' machine, the overlap was only 2 KB.

That was enough.

A Mesa virtual machine page table happened to land on the affected page during cold boot. The GPU's compression hardware then wrote its metadata into the same memory and corrupted the page table. The compositor subsequently faulted while trying to execute its batch buffer, GDM restarted it, and the process repeated indefinitely.

Restarting GDM could make the problem disappear because the next page table might be allocated somewhere else.

That kind of bug is miserable.

It depends on memory placement, hardware behavior and timing. The actual corruption happens somewhere completely different from where the failure eventually becomes visible.

Torvalds called the investigation a "debug session from hell."

It took 24 progressively instrumented patches and 18 kernel boots to narrow the failure down.

And AI helped him do it.

Not by handing him the answer.

Torvalds said the AI did much of the grunt work: adding debugging instrumentation, processing what came back and helping continue the investigation as they narrowed the problem.

The funny part is that the AI repeatedly wanted to give up.

Torvalds wrote that it several times concluded the problem was impossible or unsolvable and suggested they should simply write up a report.

He kept pushing it.

The AI kept adding more debugging code when instructed, kept analyzing the resulting information, and eventually they got enough evidence to isolate the bad memory-boundary calculation.

Torvalds even let the AI write the final commit message.

When the AI reached the wrong conclusion and said the problem could not be solved, the human knew better and kept going.

I can relate to Torvalds having to push the AI to keep working. I use these tools constantly, and one of my recurring frustrations is how readily they decide something is too difficult, take a shortcut, or ignore half of what I asked them to do. The human still has to recognize when the work isn't finished and insist on continuing. In that sense, stubbornness is becoming an increasingly useful engineering skill.

I really like this example because it is almost the opposite of what people usually mean when they complain about "AI coding."

The engineer understood the system.

The engineer reproduced the problem.

The engineer decided what needed to be measured next.

The AI handled repetitive instrumentation and helped process the results.

And the human ultimately took responsibility for the patch.

That last part is especially relevant because the Linux kernel now has explicit documentation for AI-assisted development.

The kernel's current guidelines say AI tools must follow the normal development and testing process, and they explicitly reserve the Signed-off-by certification for humans. The person submitting the change is responsible for reviewing AI-generated work, validating it and taking responsibility for the contribution.

That seems like exactly the right model.

AI is extraordinarily useful when paired with someone who understands the problem well enough to know when the model is wrong.

It is considerably less useful when the person operating it cannot distinguish a good answer from a confident dead end.

There is another interesting piece of history buried in this bug.

The incorrect round_up() was introduced in a September 2024 Xe driver fix. That earlier patch said Intel's specification required software to round the address upward to the nearest 128 KB boundary.

Two years later, the debugging session showed why applying that rule at this particular point in the calculation was wrong.

The resulting Linux fix rounds the usable-memory boundary down to the allocator's 4 KB page size instead, ensuring that none of the GPU's compression metadata area can be handed out as VRAM.

One incorrect rounding direction.

Two kilobytes of overlapping memory.

A corrupted GPU page table.

A machine that appeared to have a completely different graphics failure.

Twenty-four debugging patches.

Eighteen kernel boots.

That is software debugging.

The fact that AI could stay alongside someone like Linus Torvalds through that process and make the tedious parts easier is much more interesting to me than whether an AI can generate another application from a prompt.

This is where I think these tools become genuinely valuable.

Not replacing the engineer.

Helping a stubborn engineer keep digging.