Lewati ke isi

Acceptance dan Traceability — OS-v1.0

Last verified: 2026-09-12. Semua tes di bawah REQUIRED / NOT VERIFIED sampai evidence exact-candidate tersedia. Sumber requirement FR/NFR/CAM dari paket dipertahankan; approved deviations D01–D09 tercatat di baseline. Owner/dependency/status story di backlog; gap closure di ledger.

1. Acceptance cases

AC ID Requirement sumber Acceptance yang dapat diuji Story / gate
AC-01 FR-COM-001/002/006 Create/edit/clone/archive competition minimum; scheduled session/room scoped; frozen settings cannot mutate; audit actor/reason OS-021 / I2
AC-02 FR-COM-003/004/005 CSV dry-run row errors/duplicate; enrollment/room/token issuance scoped; incompatible overlapping assignments rejected OS-022 / I2
AC-03 FR-QST-001/002/003 PG single/numeric/essay only; sanitize HTML/media/LaTeX; changes produce version; multi-select rejected in v1 OS-023 / I2
AC-04 FR-QST-004/005/006/007 Persist exact form items/order/weights/rubric/key; freeze before publish; attempt snapshot stable across edit/refresh OS-024, RM-003 / I2–I3
AC-05 NFR-SEC-001/002/003/004/005/006/007 TLS/config fail-closed, privileged MFA/recovery, encryption-at-rest sesuai threat model/customer policy, no secret/PII logs, scans; production policy no unresolved retention OS-011/020, RM-002 / I1–I7
AC-06 FR-EXM-001, FR-SEC-005 Cross-event/tenant/enrollment/device rejected on every runner operation; revoked credential cannot reuse token RM-004, OS-030 / I3
AC-07 FR-EXM-003/004, NFR-REL-002 Identical retry returns original ACK; key collision different payload conflict; stale revision preserved; foreign question rejected RM-005, OS-031 / I3
AC-08 FR-EXM-002/006, NFR-REL-005 now >= deadline after lock rejects new answer; clock spoof has no effect; no 5s grace/extension; queue/restart cannot bypass expiry OS-003/033 / I3
AC-09 FR-EXM-005, NFR-USA-001/002 Offline shows pending; refresh/reconnect reads server revision and reconciles explicitly; no false saved label OS-032 / I3
AC-10 FR-EXM-007/008/009 Concurrent save/submit/auto-submit yields one receipt/transition; submitted edits denied; recovery reason/device revoke without extra time OS-033 / I3
AC-11 NFR-REL-001/003 ACK survives API/worker/Redis restart; report/evidence failure does not block answer acceptance OS-070 / I7
AC-12 FR-SEC-001/002/006 Actual device matrix readiness, unsupported/missing/busy/remediation paths and safe support code OS-040 / I4
AC-13 FR-SEC-003/004 Signed launcher install/update/uninstall; expired/replayed/wrong enrollment handoff denied; ordinary browser/forged context denied OS-041/042 / I4
AC-14 FR-MON-001/002/003 Scoped room statuses rebuild after reconnect/Redis loss; presence separate from lifecycle; no keys/answers leak OS-050 / I5
AC-15 FR-MON-004/006, FR-AUD-001/002 Observation/action/response/disposition timeline separates machine facts and human judgment; unauthorized recovery/penalty denied OS-051 / I5
AC-16 FR-MON-005 Zoom refs accessible only assigned participant/proctor; not public/logged; angle/failure/escalation rehearsed OS-052/066 / I5
CAM-01 CAM-01 Camera support/permission tested: allowed, denied, missing, busy in approved SEB OS-004/064 / I0/I5
CAM-02 CAM-02 Persistent local preview survives question navigation without restarting attempt OS-064 / I5
CAM-03 CAM-03 Pre-exam still links correct attempt/policy; only authorized evidence viewers can retrieve OS-064, RM-008 / I5
CAM-04 CAM-04 Server-controlled randomized capture within approved cadence/count; cannot derive full schedule from client alone OS-065 / I5
CAM-05 CAM-05 Only approved trigger requests still; upload and trigger linked by correlation; spam/rate limits tested OS-065 / I5
CAM-06 CAM-06 Observable track end/mute/permission/device change facts recorded; no automatic misconduct verdict OS-065 / I5
CAM-07 CAM-07 Storage failure/retry/load while saving answers: no changed deadline, no lost ACK, bounded buffer OS-065/070 / I5/I7
CAM-08 CAM-08 Cross-attempt/org/room evidence access/replay rejected; grader cannot de-anonymize via image/metadata RM-008, OS-071 / I5/I7
CAM-09 CAM-09 Approved expiry/deletion/hold tested including thumbnails/object versions/export/backup and post-restore tombstones RM-008, OS-072 / I7
CAM-10 CAM-10 Proctor records facts/action/response/disposition; consequential action authorized and auditable OS-051/066 / I5
AC-17 FR-SCR-001/002/003 Golden objective/essay scores; unknown/duplicate/missing/NaN/out-of-range criteria rejected; separate reviewer; no grading pre-submit RM-006, OS-060 / I6
AC-18 FR-SCR-004/005/006 Deterministic shared rank 1,1,3; no speed tie-break; immutable policy/input/ranking snapshots RM-009, OS-061 / I6
AC-19 FR-SCR-007/008 Ungraded/unmoderated blocks publication; dual control, frozen-only results; supersession preserves previous result; no live mode OS-061/062 / I6
AC-20 FR-AUD-003/004/005 Result/attendance/incident/audit exports scoped with version/requester/time; CSV formula safety; private expiry/delete OS-063 / I6
AC-21 NFR-MNT-001/002/003/004/005 Contract drift fails CI; generated client matches API; additive migration tested; disposable guard prevents wrong DB reset OS-010/012, RM-001/010 / I1
AC-22 NFR-PER-001/002/003/004/005 Real deployment: 500 active/120m/mixed questions + camera/proctor; start/submit bursts; actual denominator; twice full run and headroom OS-070 / I7
AC-23 NFR-REL-004 Isolated physical-base+WAL PITR or approved equivalent; content integrity and app recovery; measured RPO/RTO, not dump-only timing OS-072 / I7
AC-24 NFR-OPS-001/002/003/004/005 Health/readiness/metrics, alerts/runbooks/owners, immutable identity, rehearsal, rollback plan and support roster OS-011/072, RM-012 / I7
AC-25 NFR-USA-003/004 Keyboard/focus/contrast/screen-reader critical flows and understandable readiness/support instructions on supported Windows OS-073 / I7
AC-26 Local D04 Certificate public/participant/admin routes and UI unavailable in certified v1; historical data not deleted RM-011 / I6
AC-27 Local D09/DEC-12 Data-loss/force-majeure drill does not extend deadline automatically; incident authority controls restart/retest decision OS-072/073 / I7
AC-28 Local connector Scola outage/replay/duplicate/forged webhook/wrong tenant does not corrupt result or block independent exams; no automatic rapor mapping SC-001/002/003 / I9

2. Gate evidence

Unit untuk pure rules, integration terhadap real PostgreSQL/migrations, API contract dan generated client, browser E2E terhadap real API, Windows manual/automated evidence untuk launcher/camera, load di isolated staging, fault injection dan DR. Setiap requirement ID expanded pada implementasi/story; range di sumber bukan alasan melewatkan satu requirement.

MFA/tenant/secure context/answer/deadline/scoring/evidence/DR tests release-critical tidak boleh dianggap PASS jika skipped. Tes lama tests/validate_phase0.py atau simulate_500_concurrent.py tidak cukup untuk menutup gate. Baseline “G0 PASSED” historis tidak mengesahkan I0/I7.

Tidak mencantumkan npm/pytest command yang belum ada sebagai runnable. OS-010/RM-010 harus menyediakan command registry + isolated target guard; sebelum menulis tes baca panduan QA workspace dan inspeksi route/selector/API aktual.