Hardware Integration Tests
There is no guarantee that the software works as expected if all unit tests succeed. There can be errors in the interfaces between software modules or software and hardware. Especially on embedded systems, the configuration of hardware can be problematic. Hardware integration tests are run against the target hardware and validate the behavior of multiple modules together.
The kernel uses hardware integration tests because a real-time kernel contains critical architecture-dependent code (e.g., context switching or system calls) with assembly code that can only be tested on the correct architecture. Microcontrollers also have hardware to accelerate the code execution in the form of direct memory access (DMA) unit or caching, the effects of which can only be observed on hardware.
A binary for a microcontroller cannot be run on a desktop computer directly, but it can be emulated. QEMU [41] is an emulator suitable for microcontroller architectures. The cortex-m crate uses QEMU to run emulated tests on the Arm Cortex-M3 architecture. Unfortunately, memory protection unit (MPU) is not supported [59], which the kernel heavily relies on. Thus, hardware integration tests are run on actual hardware.
Without the standard library, there is no test framework. Luckily, defmt-test [35] provides a framework that is similar to the standard library but can run on a microcontroller. However, the framework only runs on the Arm Cortex-M architecture and communication only supports the detfmt crate. To overcome this issue, defmt-test was forked into bern-test and adapted to support any architecture and any serial communication interface. The API was changed to be even more similar to the standard library syntax.
bern_kernel/tests/arm_cm4/tests/mutex.rs
#![no_main]
#![no_std]
/*uses*/
#[bern_test::tests]1
mod tests {
/*uses*/
#[test_set_up]2
fn init_scheduler() {
bern_kernel::init();
time::set_tick_frequency(1.kHz(), 72.MHz());
}
#[test_tear_down]3
fn reset() {
cortex_m::peripheral::SCB::sys_reset();
}
#[tear_down]4
fn stop() {
cortex_m::asm::bkpt();
}
#[test]5
fn wait_for_lock(_board: &mut Board) {
let mutex = Arc::new(Mutex::new(MyStruct{ a: 42 }));
PROC.init(move |c| {
Thread::new(c)
.priority(Priority::new(1))
.stack(Stack::try_new_in(c, 1024).unwrap())
.spawn(move || {
match mutex.lock(1000) {
Ok(value) => { assert_eq!6 (value.a, 42); },
Err(_) => panic!7 ("Did not wait for mutex"),
}
});
// watchdog
Thread::new(c)
.priority(Priority::new(0))
.stack(Stack::try_new_in(c, 1024).unwrap())
.spawn(move || {
sleep(1000);
/* if the test does not fail within x time it succeeded */
bern_test::test_succeeded();8
__tear_down();
});
}).ok();
bern_kernel::start();
}
}
Listing: Hardware integration test of a semaphore.
In the listing a mutex is tested on hardware. A test module is first marked with the bern_test::tests macro
The test itself