We have hundreds of tests for Rust but we need to review each and every one to ensure they actually test the actual item in a way that ensures functionality rather than it being a test that always reaches a true result. We also need to review whether or not each tests are well founded or even needed at this point.
We have hundreds of tests for Rust but we need to review each and every one to ensure they actually test the actual item in a way that ensures functionality rather than it being a test that always reaches a true result. We also need to review whether or not each tests are well founded or even needed at this point.
Salastil
added this to the v1.0 milestone 2026-10-11 00:14:23 -04:00
Tests that could not fail now can, and mutation testing (cargo-mutants) was used to find the gaps in the daemon's tests; the gaps it found are covered and one real fault turned up (an unpinned Kick message never left the screen).
Scope, plainly: the mutation run covered 317 mutants (211 caught, 100 missed, 6 timed out) and the misses it pointed at were tested; the roughly 800 other tests were scanned for ones that assert nothing, not each read line by line.
Finished.
Tests that could not fail now can, and mutation testing (cargo-mutants) was used to find the gaps in the daemon's tests; the gaps it found are covered and one real fault turned up (an unpinned Kick message never left the screen).
- moho: https://git.salastil.com/Salastil/moho/commit/8a7e3e7 , https://git.salastil.com/Salastil/moho/commit/b0e6775 , https://git.salastil.com/Salastil/moho/commit/47653c5
- nobilis: https://git.salastil.com/Salastil/nobilis/commit/2477514 , https://git.salastil.com/Salastil/nobilis/commit/81a33a1 , https://git.salastil.com/Salastil/nobilis/commit/b522fe6
Scope, plainly: the mutation run covered 317 mutants (211 caught, 100 missed, 6 timed out) and the misses it pointed at were tested; the roughly 800 other tests were scanned for ones that assert nothing, not each read line by line.
Every one of the 993 tests across 119 files was read for whether it tests the code that runs. Nine tested a copy of the logic or a test-only helper and now test the real thing; one could not fail (a DCC offer nobody takes) and now can; the forget-account sweep test had a substring blind spot. Real faults found and fixed: IRC CAP LS multiline reading, DCC cancel ignored while waiting for a taker, six per-account sets never swept on account removal, and "1 viewers" in a raid line. Rewritten tests were checked by breaking the code under them. 964 pass, 27 ignored (live services/hardware), clippy -D warnings clean.
Three ignored diagnostic probes still assert nothing, by design.
Finished.
Every one of the 993 tests across 119 files was read for whether it tests the code that runs. Nine tested a copy of the logic or a test-only helper and now test the real thing; one could not fail (a DCC offer nobody takes) and now can; the forget-account sweep test had a substring blind spot. Real faults found and fixed: IRC CAP LS multiline reading, DCC cancel ignored while waiting for a taker, six per-account sets never swept on account removal, and "1 viewers" in a raid line. Rewritten tests were checked by breaking the code under them. 964 pass, 27 ignored (live services/hardware), clippy -D warnings clean.
- nobilis: https://git.salastil.com/Salastil/nobilis/commit/b3e92b5
- moho: https://git.salastil.com/Salastil/moho/commit/09c060d
Three ignored diagnostic probes still assert nothing, by design.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
We have hundreds of tests for Rust but we need to review each and every one to ensure they actually test the actual item in a way that ensures functionality rather than it being a test that always reaches a true result. We also need to review whether or not each tests are well founded or even needed at this point.
Finished.
Tests that could not fail now can, and mutation testing (cargo-mutants) was used to find the gaps in the daemon's tests; the gaps it found are covered and one real fault turned up (an unpinned Kick message never left the screen).
Scope, plainly: the mutation run covered 317 mutants (211 caught, 100 missed, 6 timed out) and the misses it pointed at were tested; the roughly 800 other tests were scanned for ones that assert nothing, not each read line by line.
Finished.
Every one of the 993 tests across 119 files was read for whether it tests the code that runs. Nine tested a copy of the logic or a test-only helper and now test the real thing; one could not fail (a DCC offer nobody takes) and now can; the forget-account sweep test had a substring blind spot. Real faults found and fixed: IRC CAP LS multiline reading, DCC cancel ignored while waiting for a taker, six per-account sets never swept on account removal, and "1 viewers" in a raid line. Rewritten tests were checked by breaking the code under them. 964 pass, 27 ignored (live services/hardware), clippy -D warnings clean.
Three ignored diagnostic probes still assert nothing, by design.