Field Validation Checklist

This checklist is the field half of the factory runbook. It is the test pass a production device must survive, and it doubles as the acceptance pass when the runbook's stages 2 and 4 run on the bench. Every item is executable by one person with the device, a phone, a WiFi network, and the dashboard.

Items marked pending-user need a human at the hardware; they cannot be verified from logs alone. Until a sacrificial device exists (see the runbook status), the whole list is pending-user by definition. Run it on the bench unit before any eFuse burn, and again on the burned unit after.

1. Audio playback over speaker

Step	Expected
Play a known SD file from the dashboard Library page	Sound from the speaker, correct surah and reciter shown
Step volume up and down on the dashboard and the encoder	Volume follows both, the two stay in sync
Play a second file, pause, resume	Seek and resume work without stutter
Watch the underrun counter during a full surah	Stays at zero
Power off mid-play, power on	Auto-resume offers the saved position if enabled

A nonzero underrun counter means the network or the pipeline could not keep up; see the audio troubleshooting rows if it appears.

2. Internet radio: http and https streams

Step	Expected
Play an http:// station (64 or 128 kbps MP3)	Audio plays, station name on the display
Play an https:// station	Audio plays; the TLS handshake uses the certificate bundle
Watch the stream title	ICY metadata appears in the dashboard when the station sends it
Drop WiFi mid-stream, restore it	Reconnection with backoff, playback resumes by itself

Stations die over time. Keep two known-good URLs per protocol in the test log, one Icecast and one Shoutcast style if possible.

3. NFC tap-to-play binding

Step	Expected
Register a card on the dashboard NFC page (scan or typed UID)	Card listed; UID fills itself when scanning works
Tap the registered card	Its binding plays (SD file or radio stream)
Tap an unregistered card	Brief bilingual toast: gold Arabic title بطاقة غير معروفة over English "Unknown card", nothing else
Change the card removal policy to Pause	Lifting the card pauses, re-tapping resumes
Delete the card	Tap no longer triggers it

4. OTA A/B update and rollback

This is the drill that proves the update model end to end. Run it on the bench before burning (unsigned images accepted), and again after burning (only signed images accepted; an unsigned image must be refused).

4.1 Build the OTA image

1. Build the release artifact exactly as the factory does: idf.py --preset production build.
2. Take build-prod/Safi.bin as the OTA payload. On a secure-booted device it must be signed with SAFI_SECURE_BOOT_SIGNING_KEY or the bootloader will refuse it. That refusal is itself the negative test below.
3. Record the running version first (GET /api/system or the dashboard Home page) so the rollback check has a baseline.

4.2 Upload and verify the slot switch

1. Upload via the dashboard Update page (POST /api/ota/upload), then repeat the whole drill once using a pull (POST /api/ota/pull with a https:// URL) so both transports are covered.
2. Watch progress reach 100 percent, then the reboot. GET /api/ota/status returns idle once the update is done.
3. Verify the switch: new version string on the splash screen, the dashboard, and GET /api/system. The boot report stays empty, and settings, cards, and favorites survive.

4.3 Negative test: unsigned or corrupt image

1. Attempt to upload a corrupt file (truncated or garbage). The update must fail with an error and leave the running slot untouched.
2. On a secure-booted device, attempt to upload an unsigned image. The bootloader must refuse to boot it. This is the visible proof of secure boot; the device comes back on the signed firmware by itself.

4.4 Force a bad image and verify rollback

1. Build a bad image: a build that boots but crashes before the app confirms itself (the firmware marks the slot valid only after its health checks pass, so any early crash qualifies). On a secure-booted device the bad image must still be signed to get past bootloader verification; the point of this test is the app-level confirmation, not the signature.
2. Upload the bad image and watch it boot and crash. Because the new slot never confirmed, the bootloader's next boot reverts to the previous slot.
3. Verify the baseline version is running again, and the device is healthy: boot report empty, playback works. A bad update is a no-op. That is the property this drill exists to prove.

5. SoftAP provisioning with the ESP SoftAP Provisioning app

Step	Expected
Start a unit with no saved credentials	The access point Safi-XXXXXX appears (MAC suffix)
Open the ESP SoftAP Provisioning app, refresh, tap the AP	App prompts for username and PoP
Enter a wrong PoP	Session refused; nothing is accepted
Enter username safi and the label PoP (16 chars, [A-Za-z0-9])	Session starts over security 2 (SRP6a plus AES-GCM)
Pick a 2.4 GHz network and enter its password	Credentials save, device connects, app confirms
Reboot	Device rejoins from saved credentials without provisioning

Verify the label PoP against the chip: read the base MAC with esptool.py read_mac and derive HMAC-SHA256(factory secret, MAC) with the runbook's tooling; the first 16 digest bytes mapped onto [A-Za-z0-9] must equal the label. The firmware derives the same value at boot, so a mismatch means the label and the device will never pair.

6. BLE provisioning pairing

BLE is a compile-time choice (SAFI_PROVISIONING_TRANSPORT); prebuilt firmware ships SoftAP, so this test needs a BLE build.

Step	Expected
Boot a BLE build with no saved credentials	Device advertises as PROV_SafiXXXXXX
App with BLE transport selected, scan	PROV_SafiXXXXXX appears (the PROV_ prefix is what the app filters on)
Pair with username safi and the same label PoP	Same security 2 setup over BLE, credentials save, device connects

7. Deep sleep and encoder wake

Step	Expected
Dashboard Settings, Sleep button (POST /api/system/sleep)	Device goes to deep sleep; current draw drops to microamps if you can measure it
Press the encoder knob	Device wakes into a normal boot
Set a schedule, sleep again	The RTC alarm wakes the device for the scheduled recitation
Wake with a charged battery after an audio interruption	Auto-resume (if enabled) continues the recitation

The wake source is the encoder push pin (GPIO15) plus the RTC alarm pin when a schedule is armed.

8. Battery low-voltage shutdown path

Step	Expected
Run on battery, drain toward ~3.05 V	No shutdown on a transient sag; three consecutive low reads trigger the orderly shutdown
Watch the shutdown	Resume position saved, a short on-screen notice, then deep sleep, not a hard cut
Press the knob on a still-empty battery	The latch fires again quickly and the device returns to sleep
Charge, wake, play	Auto-resume continues where the shutdown stopped
Bench check with no battery sense divider	Monitoring disables itself ("no battery sensed"), no false shutdown on USB power

This path is the reason the shutdown latch is debounced and unit-tested; the checklist is its acceptance test.

9. Dashboard token flow and WebSocket dot states

Step	Expected
Open the dashboard for the first time	Token prompt appears; entering the token saves it in the browser and the page stays put (no navigation, no ?token= in the address bar)
Protected action with the correct token	Works; the header dot is green once the socket is authenticated
Watch the dot through a reload	Gold while connecting, green when authenticated
Factory reset the device (POST /api/reset)	Every protected action is refused with the token-changed reply; the prompt reopens on its own; the dot turns red
Paste the new token and save	Dot returns to green; live updates resume without a page reload
Kill the WiFi, restore it	The socket reconnects with backoff; a manual refresh resynchronizes

The dot is the user-facing proof of the token and socket model: gold connecting, green authenticated, red unauthorized or closed.

Sign-off

One row per device, attached to the runbook's sign-off log:

Device serial	Firmware version	Checklist result	Tester	Date

A failed item blocks the next gate. Fixes go through the normal flow (issue, patch, rebuild) and the whole checklist reruns, because a validation pass is only as good as its weakest green box.

English-only page

The Arabic translation of this page is out of scope for now and tracked as todo in safi-docs/TRANSLATIONS.md.