How we verify our claims
"Tested" is a word people use loosely, so here is exactly what it means for each product, and what it does not.
The method
- Clean environment. Tests run in fresh containers (Debian 12) so nothing depends on a developer's machine.
- Real services, not mocks, where it matters. Real Caddy, real PostgreSQL 15, real restic, a real sshd and ssh client, systemd as PID 1, a real firewall with traffic from a second container. Where a service could not be real (the Telegram API), a local fake was used and the page says so.
- Failure cases. A corrupt database, a wrong repository, a broken release, a release that crashes on start, a full disk, an out-of-memory kill, a stale or tampered or replayed signature.
- Numbers with their setup. Load, hardware and counting rules are stated next to every measurement. Results that vary between runs are reported as a range.
- Bugs are reported, not hidden. Several of our first drafts were wrong; the tests caught them, and each product's changelog and page say what was fixed.
What each product has and has not been checked against
| Product | Verified | Not verified |
|---|---|---|
| VPS Deploy & Backup Kit | 59 container checks: backups and restores with SQLite and Postgres, deploys and rollback under 20 requests per second through real Caddy, alerts against a fake API, restic over SFTP | Real Telegram API, S3 and rclone backends, newer Caddy versions, WebSockets and long uploads, Go builds |
| Go Webhook Receiver Starter | 41 tests with the race detector, signature known-answer vector, throughput on one laptop, real signed requests against the compiled binary | A live Polar delivery, the Docker image, Linux or cloud disks, long soak runs |
| Handbook | Every script and chapter in clean containers, a timed recovery drill, failure cases | Real certificate issuance, a real reboot with swap, Ubuntu specifics, remote backends, point-in-time recovery |
Limits of any single test
A container is not your server, one laptop is not your cloud VM, and a test that passes today does not guarantee behaviour on a version we have not run. Treat our measurements as evidence that something works and as a starting point for your own check, especially for anything on the "not verified" list.
Found something we got wrong? Tell us and we will fix the page and credit the correction in the changelog.