HIL won't scale to the AI era

The agent changed how the firmware boots. Again.

The code is ready to test. The test is waiting for the one HIL bench that has the right harness, a free USB port, the right firmware already flashed, and nobody else using it.

That's not a story about bad engineers. It's what the hardware-in-the-loop bench is for. Real controller. Real I/O. Real timing. Real electrical failure modes.

The problem is asking that bench to be the default feedback loop for every generated change.

AI doesn't make HIL obsolete.

It makes the work before HIL non-negotiable.

HIL isn't the enemy

Let's get this out of the way first.

HIL exists because some things need physical hardware: signal delays, electrical fault injection, bus timing, the actual controller running under the conditions it'll see in a product.

ISO 26262 names HIL as part of the verification toolchain.¹ For high-ASIL work, it recommends checking results across environments because they can disagree. That's not a bug in the standard — it's the point of running more than one layer.

HIL also has a lifecycle of its own. A 2012 paper on HIL testing under ISO 26262² lays it out plainly: build the rig, qualify it, use it, modify it, qualify it again. The bench is infrastructure. Infrastructure needs maintenance.

So this isn't an argument to throw away the bench.

It's an argument to stop using the most expensive, least parallel part of the stack to catch failures that a cheaper layer can reject first.

The test pyramid got a new input

The usual embedded testing stack already has the shape:

1. Build and static checks.
2. Software and virtual targets.
3. HIL.
4. Product or vehicle testing.

That's not a new idea. A 2024 paper on virtual ECUs³ makes the sensible case: move the requirements that can be tested in software off HIL, and keep the requirements that need physical hardware on HIL. It doesn't pretend the two are interchangeable.

AI changes the pressure on the top of that stack.

The code-quality story is messier than the AI discourse wants it to be. Google's DORA reports⁴ don't show a clean "AI makes teams faster" law. A randomized METR study⁵ found experienced open-source developers slower with the tools it tested. In safety-critical automotive C, a Scania/KTH study⁶ found LLM success varied wildly by module.

Fine. We don't need a universal AI productivity number to see the problem.

If an agent can propose ten firmware changes before lunch, a team needs a way to reject the obviously broken ones before those changes compete for a shared bench. The verification system has to handle candidate code at machine speed, even when the final answer still comes from real hardware.

That means the pyramid becomes a funnel.

Many changes enter at the top. Few should make it to the bottom.

Diagram 1: Firmware verification funnel. Build and static checks → virtual-board runs → HIL bench → product sign-off. Full designer brief: diagrams.md.

Give every layer one job

The compiler is ruthless and cheap.

It catches stale APIs, missing dependencies, broken types, and the language rules the model forgot existed. In our recent experiment, two models wrote an old STM32 HAL API; both died at compile time. Good — that's exactly where that failure belongs.

The virtual board is the next gate.

It runs the ELF against a model of the target board and gives the agent a receipt: expected output, a hang, a fault, or silence. That catches a different class of failure: wrong peripheral mode, a state machine that never starts, code waiting forever for an event, or firmware that simply never emits the expected behavior.

We saw that too: an agent's nRF52840 attempt compiled cleanly, then ran for five seconds and produced none of the expected output. Not a style nitpick, not a vibe from code review — a real, unambiguous "this firmware didn't do the thing" receipt. The full run and evidence are public.

Then comes HIL.

That's where you validate the physical stuff the virtual layer shouldn't claim to own: exact timing, pin routing, electrical behavior, real buses, sensors, actuators, and the ugly interactions that only show up when there are cables involved.

The layers don't compete. They protect each other.

Diagram 2: One generated change, two feedback loops. Bench-first versus funnel-first, converging on the same final HIL bench. Full designer brief: diagrams.md.

The bench is a scarce resource

HIL vendors don't dispute this. NI's own documentation⁷ notes that high-channel systems can require hundreds or thousands of connected signals, with wiring that's costly and time-consuming to maintain. The global chip shortage made the broader point even more brutal: when physical controllers and prototypes are scarce, every hardware-dependent test plan inherits that scarcity.

That doesn't mean every team is waiting 26 weeks⁸ for a board today. It means hardware availability is a delivery constraint that software-only feedback loops don't have.

And there is a less dramatic version of the same problem in every lab: one capable bench can only run one setup at a time. People book it. Someone changes a cable. Someone misconfigures the hardware. Someone is debugging a failure a virtual run could have rejected in seconds.

That's poor allocation of the most valuable test environment you own.

Be honest about the boundary

Simulation doesn't get a free pass here.

In our own probes, a byte-level simulation model accepted clock and configuration mistakes that real hardware wouldn't forgive. We wrote that down instead of editing it out, because the boundary matters.

A virtual target isn't a magic copy of reality. It's a test environment with a scope. So is HIL. So is a vehicle. The point is to know what each one can prove.

Failure or requirement	Build	Virtual board	HIL / silicon
Stale API or missing dependency	Strong	Not needed	Not needed
Firmware boots, progresses, and produces expected output	No	Strong	Final confirmation
Wrong peripheral mode or a hang	No	Strong	Can confirm
Exact timing, pin routing, electrical behavior	No	Not sufficient for sign-off	Final authority
Real sensor, actuator, and electrical fault behavior	No	Not sufficient	Final authority

The goal was never fewer layers. It's fewer bad binaries reaching the expensive one.

What changes now

Embedded teams used to accept a slow feedback loop because firmware changes were slow too. A human wrote the code, flashed the board, and watched the output — the whole loop was human-paced.

It's not anymore.

AI can generate code faster than a bench can be booked. That doesn't change what real hardware is for. It changes when we should ask it a question.

Run the cheap checks first. Run the virtual target next. Keep the bench for the tests that need a bench.

That's how HIL stays valuable in the AI era: not as the place every change begins, but as the place the surviving changes earn their final sign-off.

Sources

Footnotes

1. ISO 26262-4:2018 and ISO 26262-6:2018, "Road vehicles — Functional safety." https://www.iso.org/standard/68386.html / https://www.iso.org/standard/68388.html ↩

2. Himmler, Lamberg, and Beine, "Hardware-in-the-Loop Testing in the Context of ISO 26262," 2012. http://web1.see.asso.fr/erts2012/Site/0P2RUC89/TC-2.pdf ↩

3. Keil et al., "Evaluation of SiL Testing Potential—Shifting from HiL by Identifying Compatible Requirements with vECUs," Vehicles, 2024. https://doi.org/10.3390/vehicles6020044 ↩

4. Google, "State of AI-Assisted Software Development 2025." https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf ↩

5. Becker, Rush, Barnes, and Rein (METR), "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," 2025. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ ↩

6. Sevenhuijsen, Patil, Nyberg, and Ung, "Generating Safety-Critical Automotive C-programs using LLMs with Formal Verification," PMLR 284, 2025. https://proceedings.mlr.press/v284/sevenhuijsen25a.html ↩

7. NI, "HIL Test System Architectures." https://www.ni.com/en/solutions/transportation/hardware-in-the-loop/hardware-in-the-loop--hil--test-system-architectures.html ↩

8. KPMG, "Surviving the Silicon Storm," 2021. https://assets.kpmg.com/content/dam/kpmg/br/pdf/2021/06/automotive-semiconductor-shortage.pdf ↩