Automated Validation in Embedded IoT Development: A Review of Continuous Integration and Delivery across the Fidelity Ladder
TL;DR
The review weighs the empirical evidence on continuous practices in embedded organisations, identifies areas where evidence is thin—benchmarks, cross-level coverage reasoning, and pipeline-scale evaluation—and closes with directions for research and practice.
Abstract
Embedded software for the Internet of Things is developed under pressures that pull in opposite directions: the delivery cadence of connected products, sustained by over-the-air update channels, demands the automation of continuous integration and delivery, while the software's defining property—correct behaviour on constrained hardware interacting with physical signals and unreliable networks—is exactly what conventional automated pipelines exercise least. A substantial literature, scattered across software engineering, embedded systems, testing, and security venues, now addresses this tension, and this paper reviews it. The review organises automated validation for embedded IoT development as a fidelity ladder that pipelines climb: static analysis and compilation-time checking, whose industrial-scale evidence is strong and whose embedded-specific value lies in catching the defect classes hardware testing finds expensively; host-based unit and component testing with hardware abstraction, where adequacy measurement and property-based techniques transfer directly; simulation and software-in-the-loop validation, including digital-twin approaches that make plant models maintained assets; emulation and firmware re-hosting, the enabling layer for dynamic analysis and fuzzing of embedded binaries, whose distinctive failure modes the security literature has characterised; hardware-in-the-loop testing, the automotive-proven apex whose integration into continuous pipelines the empirical literature identifies as the field's persistent gap; and field-level validation through staged rollout and telemetry against fleets. Cross-cutting concerns are reviewed in depth: test selection and prioritisation under slow suites, flaky-test management under physical variability, environment-as-code for reproducible test infrastructure, model-based test generation, fault injection for dependability claims, and the validation of the update channel itself. The review weighs the empirical evidence on continuous practices in embedded organisations, identifies areas where evidence is thin—benchmarks, cross-level coverage reasoning, and pipeline-scale evaluation—and closes with directions for research and practice.
