What LLMs get wrong about embedded code

Our last post made a claim: language models hallucinate register names and phantom HAL APIs with the same confidence they use for real ones, and a clean build won't catch either.

We wrote that from experience, not from data. This post is the data.

We prompted two models, one big and one small, for hello-world firmware on three boards: STM32F4 Discovery, STM32L073 Nucleo, nRF52840 DK. No docs and no tools of any kind, not even a datasheet, and only one attempt each. Reply with main.rs, nothing else. The agent compiled every answer exactly as returned, byte for byte, and ran every binary that built on our simulated boards.

To be clear about what this is: no real agent works this way. Agents read the docs, they grep around in the crates they depend on, and they try again when something breaks. Cold-prompting measures what's actually in the weights, which is exactly what you're trusting when a model writes register code and nothing checks it.

Layer one: what the compiler catches

We asked both models for the same thing twice: once with raw register writes, once using stm32f4xx-hal 0.23. Both HAL attempts failed to compile. Not on typos. On API drift:

error[E0616]: field `cfgr` of struct `stm32f4xx_hal::pac::rcc::RegisterBlock` is private
error[E0061]: this method takes 1 argument but 0 arguments were supplied

Both models wrote the rcc.constrain() / cfgr.freeze() API that the crate dropped two major versions ago. This is the boring failure class, and it's fully mechanical. The model's knowledge of a moving API is frozen at training time, the compiler catches the drift, and an agent with the error output in front of it fixes the whole thing in one round. If your loop includes cargo build, this class is already handled.

The raw-register attempts are where it gets interesting, because there the compiler has no opinion. An address is an address. All six raw-register attempts, across both models, compiled clean on the first try.

Layer two: what the run catches

The small model's nRF52840 attempt is worth walking through, because it failed the way firmware actually fails: silently, one plausible decision at a time.

Its first attempt used the nRF's legacy byte-at-a-time UART interface, deprecated for a decade but heavily represented in old tutorials. The run produced nothing. Told that, it switched to the modern UARTE peripheral, which moves data with EasyDMA. Its second attempt died in the compiler, because Rust's rules forbid the static mut buffers it used.

Its third attempt is the one that matters. To satisfy the borrow rules it made the DMA buffer an immutable static, which the linker places in flash, then copied the message into it at runtime through a const-cast pointer:

static TX_BUFFER: [u8; 64] = [0; 64];
// ...
let tx_ptr = TX_BUFFER.as_ptr() as *mut u8;
ptr::write(tx_ptr.add(i), byte);
// ...
ptr::write_volatile(UARTE_TXD_PTR as *mut u32, TX_BUFFER.as_ptr() as u32);

This compiles without a single warning that matters, and it reviews fine if you're skimming. It also can never work: the writes into flash are silently dropped, so the DMA streams out whatever the buffer held at link time, which is zeros. The run said so immediately:

18:55:22.6729 [INFO] machine-0: Machine started.
18:55:27.6771 [INFO] machine-0: Machine paused.

Five seconds of virtual CPU time, zero bytes on the UART. No fault, no error, no output.

That is the exact bug class the target post described in the abstract: a DMA buffer the peripheral can't reach. Here it was produced organically, by a model under iteration pressure. Invisible to the compiler, invisible to casual review, and unambiguous the moment the firmware actually ran.

Layer three: the bench, once

Before trusting any of this, we probed what the simulation actually models. We took a known-good STM32F4 hello world and sabotaged it two ways on purpose: one variant with the USART clock enable removed, one with the baud divisor programmed for 9600 instead of 115200. Both printed Hello world! anyway.

The UART model in the simulator is byte-level. It hands over bytes when the firmware writes them, and it does not model clock gating, baud mismatch or pin multiplexing. Which means some of the small model's bugs sailed straight through. Its STM32F4 attempt divides the baud clock by 16 twice, with a comment proudly showing the math, and routes the TX pin's alternate function to the wrong pin's bits. It prints fine in simulation. On your desk it would produce garbage at 1.8 Mbaud on a dead pin.

So the bench isn't going anywhere, and we're not going to pretend otherwise. Analog and configuration detail below the byte level, meaning exact baud, pin routing and clock trees under load, gets its final word from real silicon.

But look at what's left for the bench to do. Every failure in this post except those two was caught earlier, in seconds, without a board: the stale APIs by the compiler, then the wrong peripheral mode and the unreachable DMA buffer by the run, none of it needing a human. The whole experiment, fourteen runs across three chips with a three-iteration debug loop, took a few minutes and zero hardware. Do that loop by flashing a physical board every iteration and you're queueing for dev kits and reading a logic debugger.

That's the actual division of labour. The cheap layers eat the iteration, and silicon does what only silicon can: final verification of a binary that is already structurally right.

The gap nobody talks about

The big model's register-level recall was flawless on every chip we tried: correct fractional baud encoding on the F4, the L0's renamed clock-enable register, and the UARTE's RAM-only DMA restriction, which it flagged unprompted, in a comment, while copying its buffer to the stack.

The small model produced something plausible-but-wrong on every single target. Never nonsense. It would be one register off, or it had misremembered an encoding, or it reached for a peripheral mode that was deprecated. That is the dangerous kind of wrong, because everything downstream of it (the build, the review, the vibe) says ship it.

What this adds up to

Three layers, each catching what the previous one can't, ordered by how cheap they are to run:

• The compiler catches frozen knowledge in seconds: stale APIs, missing dependencies, edition rules from after the training cutoff.
• The run catches structural lies in under a minute. A peripheral that was never enabled the right way, a buffer the DMA can't read, or code that sits there waiting for an event that will never fire.
• Silicon signs off at the end, once, on a binary the first two layers already debugged.

Everything above the sign-off runs without a single physical board on a desk. That was the whole argument of the target post, and after this experiment we believe it slightly more than when we wrote it.

• Repo & examples: github.com/veecle/chiplab
• Chiplab: veecle.ai/chiplab
• Discord: discord.com/invite/F6GwZJ6ktP