Simulation vs HIL: what belongs in CI and what belongs in the lab

You pushed a commit. Something should test it. The only real question is which something, and how long you have to wait for the answer.

That is the entire simulation vs HIL argument, and almost nobody writes about it from where you're standing.

Who the existing SIL vs HIL literature is actually for

I read the top results before writing this. They're good. They're just not written for someone with a branch open.

OPAL-RT sells real-time simulators and HIL hardware. Their guide still tells you to start with software-in-the-loop and move to the bench only when timing and I/O behavior must be proven — otherwise "the lab becomes a bottleneck."¹ A HIL vendor writing that down is worth more than me writing it down. TofuPilot, which sells hardware test- tracking software, publishes a MIL/SIL/HIL guide with a comparison matrix that has one row I'd put on a poster: CI-friendly — MIL yes, SIL yes, HIL "No (dedicated bench)."² Wirtek, an embedded engineering services shop, is the most honest of the three about limits: their HIL primer says a HIL test "is only as good as the model behind it, so poor model fidelity can give false confidence."³

All three are correct. All three are written for someone who owns a validation plan. The unit of decision is a project phase — early design, integration, release candidate. Nobody tells you where test_can_frame_parse.c goes.

The one place I've seen the decision actually encoded for developers is Zephyr's Twister — config, not philosophy. A board declares simulation: qemu or is typed native. A test declares fixture: when it depends on an external device like a sensor, so the runner only schedules it where that device exists. Physical runs need --device-testing plus a hardware map with one entry per connected board, serial port and all.⁴ That last detail is the whole embedded test strategy problem in one YAML file: simulated targets scale with your CI runners, physical targets scale with how many boards you literally own.

Simulation vs HIL, decided in three questions

Here's the procedure. It's three questions, and it takes about ten seconds per test.

1. Does this test need the target's instruction set, memory map, or peripheral registers? If no — it's parsing, arithmetic, a state machine, a protocol encoder — it's a host unit test. Compile it for your laptop. Milliseconds, thousands of cases, no target involved. Most firmware developers under-use this tier because "it's embedded," which is not a reason.

2. Can this test fail in a way that only shows up as a voltage, a nanosecond, or a degree Celsius? If no, it goes in simulation, in CI, on every commit. Peripheral register order, driver bring-up, boot progress, does the thing actually print — a virtual target answers those and answers them the same way every time.

3. If yes — is the physics something a rig can drive, or something only an instrument can measure? A rig with a real MCU and a simulated plant handles real bus peers, real sensor parts, and real interrupt latency. A current probe, a thermal chamber and an anechoic chamber handle the rest. Those are different rooms, not the same tier.

If you can't tell which, put the test in both CI and the nightly suite until one of them catches something — that's how you find where the boundary actually sits for your board.

The two properties are independent. How fast you need an answer is a workflow constraint. How much the answer depends on physics is a property of the test. Cross them and one of the four boxes is genuinely uncomfortable.

The bottom-left cell is the one teams get wrong. If a test is pure logic and you're not in a hurry, people still send it to the bench out of habit, because the bench is where tests happen. It's free in CI. Run it there anyway.

The top-right cell is the honest one. Sometimes you need a fast answer about something physical, and the truthful response is that you get an approximation now and a real answer later. Anyone selling you a way out of that box is selling something.

The table: what to test in CI, what to test in the lab

This is the artifact. Find your test, read the row.

Test category	Where it belongs	Why
Business logic, state machines	Host unit test	No silicon involved. Thousands of cases in under a second.
Protocol framing and parsing	Host unit test	Bytes in, bytes out. The target adds nothing to the answer.
Peripheral register sequences	Simulation in CI	Needs a real memory map; wrong-order writes show up as no output.
Driver init and bring-up path	Simulation in CI	Catches "clock never enabled, peripheral never started" with no board.
Boot progress and expected output	Simulation in CI	The cheapest possible "did my firmware do the thing" receipt.
Fault handling and recovery logic	Simulation in CI — if your simulator models faults	Many don't. Verify yours does before you trust a green check.
Interrupt priority and preemption order	Split: order in CI, latency on HIL	Ordering is logic. Latency is physics. Don't conflate them.
Timing budgets and deadlines	HIL in the lab	Instruction counts are not wall-clock nanoseconds.
Analog thresholds, ADC and comparator trips	HIL in the lab	A modeled ADC returns the number you told it to return.
Real bus peers and sensor parts	HIL in the lab	Real parts have startup delays, noise and errata your model doesn't.
Power draw and sleep current	Physical only	You need a current probe. There is no software version of this.
EMI and EMC	Physical only	Needs a chamber and a calibrated antenna.5
Thermal behavior	Physical only	Needs a chamber and hours of real time.
Long-run soak	Split: counters and leaks in CI, drift in the lab	Counter rollover and heap fragmentation are logic. Component drift isn't.

What simulation cannot answer, said plainly

I'm not going to argue simulation replaces hardware-in-the-loop testing. It doesn't, and the fastest way to make a team distrust simulation entirely is to oversell it once.

The failure mode is a green test, not a red one. A simulated peripheral can accept a configuration real silicon would reject and hand you a pass. A fault your board would trap can go unmodeled and simply never fire. That is the subject of what simulation can't catch, with the runs behind it. A model is a scope, not a copy of reality.

Physical tests have scopes too, which is the part the vendor material skips. There's work out of Clemson's automotive research center noting that the correlation between component-level EMC results and vehicle-level EMC performance "is generally poor."⁵ Even the chamber is answering a narrower question than you think it is. Nothing in this stack is truth. Everything is a measurement with a boundary, and your job is to know which boundary you just crossed.

Tier it: simulation every commit, HIL on a schedule

The tiering argument is the practical payoff, and the ordering is not arbitrary.

Simulation in CI on every commit. HIL nightly and pre-release, on a much smaller suite that only contains tests from the right-hand column above. Not the other way around.

The reason is throughput, not ideology. Say a HIL rig runs two hundred jobs a day. If most of those would have died at compile or at boot anyway, that rig is a queue with a cable harness, not a test system. Put the cheap filter first and the same rig spends its time on jobs that genuinely needed a rig, and people stop fighting over slots.

That scarcity is structural, not a symptom of a badly run lab. Linaro's LAVA describes itself as a way to share boards among teams — an entire piece of infrastructure whose job is rationing physical targets.⁶ Twister's hardware map is the same fact at smaller scale: one YAML entry per board you own.

The every-commit tier is genuinely off-the-shelf now. Antmicro ships a Renode GitHub Action that drops simulated-hardware tests into a normal CI pipeline,⁷ and Memfault's Interrupt has a full walkthrough of wiring Renode tests to GitHub Actions on pull requests.⁸ We build Chiplab for that same middle tier, and it does not answer anything in the right-hand column of my table.

Firmware CI without hardware still has a floor

"We have CI" and "we test firmware" are not the same sentence. Plenty of embedded pipelines stop at a successful link and call it green, which is why most firmware CI ends at the linker needed writing. Firmware CI without hardware only counts as testing if something in the pipeline actually executes the binary against a model of the target and checks what came out.

There's a timing pressure on all of this now that didn't exist five years ago. An agent can propose ten firmware changes before lunch, and a feedback loop that routes through a shared physical rig is a ticket queue, not a feedback loop. I've made that argument at length in why HIL won't scale to the AI era, so I won't relitigate it here — it's the same tiering, just under more pressure.

Can simulation replace hardware-in-the-loop testing?

No. It replaces the first hundred trips to the bench, not the last one.

If a test can be wrong in volts, nanoseconds or degrees, it needs the lab, and no amount of model fidelity changes that. What simulation buys you is that every binary reaching the lab already compiles, boots, configures its peripherals and produces output.

Go look at your CI config. If the last step is arm-none-eabi-gcc exiting zero, you don't have a test tier — you have a compiler.

Sources

Footnotes

1. OPAL-RT. "Hardware in the loop vs software in the loop guide for engineers," May 13, 2025. https://www.opal-rt.com/blog/hardware-in-the-loop-vs-software-in-the-loop ↩

2. Buteau, Julien. "HIL vs SIL vs MIL Testing: When to Use Each." TofuPilot Guides, March 14, 2026. https://www.tofupilot.com/guides/hil-vs-sil-vs-mil-testing-when-to-use-each-with-tofupilot ↩

3. Wirtek. "Hardware-in-the-loop testing for embedded systems," June 18, 2026. https://www.wirtek.com/blog/hardware-in-the-loop-testing-for-embedded-systems ↩

4. Zephyr Project. "Test Runner (Twister)," Zephyr Documentation. https://docs.zephyrproject.org/latest/develop/twister/index.html ↩

5. Hubing, Todd H. "Module-Level Characterization for Vehicle-Level Emissions Modeling," Clemson University International Center for Automotive Research, APEMC 2010. https://cecas.clemson.edu/cvel/pdf/APEMC2010-692.pdf ↩ ↩²

6. Linaro. "Linaro Automated Validation Architecture (LAVA)." https://www.lavasoftware.org ↩

7. Antmicro. "Renode GitHub Action for automated testing on simulated hardware," October 16, 2024. https://renode.io/news/renode-github-action-for-automated-testing-in-simulation ↩

8. "Firmware Testing with Renode and GitHub Actions." Interrupt by Memfault. https://interrupt.memfault.com/blog/test-automation-renode ↩