From f699c27f32a55e6564fc32be8295e7aba798f043 Mon Sep 17 00:00:00 2001 From: Dwindi Ramadhana Date: Sat, 20 Jun 2026 13:31:10 +0700 Subject: [PATCH] Add snapshot workflow implementation plan --- Snapshot_Driven_Workflow_PLAN.md | 174 +++++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+) create mode 100644 Snapshot_Driven_Workflow_PLAN.md diff --git a/Snapshot_Driven_Workflow_PLAN.md b/Snapshot_Driven_Workflow_PLAN.md new file mode 100644 index 0000000..779a1d4 --- /dev/null +++ b/Snapshot_Driven_Workflow_PLAN.md @@ -0,0 +1,174 @@ +# Snapshot-Driven Live Question Workflow Plan + +## Summary + +Rework tryout import so the first imported snapshot creates the live exam automatically, while later imports become non-destructive snapshot candidates inside the existing tryout workspace. Admins then intentionally replace live questions slot-by-slot after reviewing differences. Imported JSON snapshots remain full historical records, but later snapshot accordions hide unchanged rows by default to keep review focused and prevent accidental no-op replacements. + +## Key Changes + +- Main `/admin/tryouts` import creates a new tryout/exam and auto-promotes all valid imported questions from the first snapshot into live `items`. +- Existing tryout workspace gets an **Import New Snapshot** action. Importing there creates a full queued snapshot record, but does not change the live exam. +- Replace the current single “Imported Snapshot Questions” section with collapsible snapshot sections: + - Snapshot 1: baseline source rows plus live/retired status. + - Snapshot 2+: review queue showing changed/new/removed/invalid rows by default. + - Add a **Show unchanged questions** toggle per snapshot; default hidden. +- Snapshot comparison for UI review must compare each snapshot row against the current live slot, not only against the immediately previous snapshot. +- If a newer snapshot supersedes an older pending candidate for the same slot, mark the older candidate as `Superseded by Snapshot N`. +- Snapshot comparison must use a canonical content hash so "same content" and "changed content" are deterministic across backend, UI, and tests. +- Rename the normal state away from “Promoted”. Use lifecycle language: + - `Live`: currently served for that slot. + - `Pending Snapshot`: imported but not live. + - `Changed`: differs from current live slot. + - `No Change`: same as current live slot; hidden by default. + - `Retired`: previously live, no longer served. + - `Stale Variant`: derived from replaced/retired content, excluded from exam delivery. +- Keep retired originals and stale variants as operational history, not moved between backend tables. Frontend may visually group retired originals and variants under their snapshot/slot accordion. +- Removed slots from later snapshots are warnings by default. They do not change live delivery unless an admin explicitly retires the live slot. +- Add **Simulate Exam Flow** to show what the runtime would serve by slot using current live items and selection mode. + +## Backend/API Changes + +- Store every imported JSON snapshot as a full immutable import record. +- Add clear state ownership: + - `Item` owns original question lifecycle delivery state: `live` or `retired`. + - Variants keep their existing review state (`draft`, `approved`, `rejected`) and gain/keep an explicit delivery exclusion state such as `stale`. + - Snapshot row review statuses (`Changed`, `No Change`, `New Slot`, `Removed`, `Invalid`, `Superseded`) are computed from snapshot row + current live slot unless persistence is required for audit/history. +- Add canonical content hashing for snapshot/live comparison: + - Hash normalized question stem, options, correct answer, explanation, media references, scoring metadata, and any fields that affect what students see or how the item is scored. + - Ignore import timestamps, snapshot IDs, database row IDs, admin notes, and formatting-only whitespace differences. + - Option order should count as content unless the runtime already randomizes options independently. +- Add slot identity rules: + - Slot number is the primary comparison key. + - Slot number must be present for a row to become live. + - Slot number must be unique inside one snapshot. + - Reordered JSON rows should not matter. + - Duplicate slot numbers should block snapshot commit or force an explicit admin resolution flow before commit. +- Extend import preview for existing tryout imports to compare full incoming snapshot rows against the current live item per slot: + - same content: `No Change`. + - changed content: replacement candidate. + - new slot: add candidate. + - removed slot: latest snapshot no longer contains a previously live slot. + - invalid row: missing options/answer or cannot become live. +- Add a tryout-scoped import endpoint for “new snapshot inside existing tryout”; it must reject JSON for a different source tryout unless admin confirms title/id/count warnings in the request. +- Add a slot replacement endpoint that accepts selected snapshot question IDs and explicit confirmations: + - replacing live original for changed slot. + - retiring existing variants as stale. + - accepting count/title/source mismatch if relevant. +- Replacement requests must include stale-preview guards: + - `expected_live_item_id`. + - `expected_live_content_hash`. + - Backend must run replacement in a transaction. + - If the current live slot changed after preview, reject and ask the admin to refresh/re-preview. +- Backend must reject selected replacements whose snapshot content is the same as the current live slot. +- On first snapshot only, auto-create live original `Item` rows for all valid imported questions. +- Invalid first-snapshot rows are still stored in the immutable snapshot record, never become live items, and should create an import health warning in the tryout workspace. +- On later snapshot replacement: + - Always create a new live original revision and mark the previous live original `retired`, even if the current live item has no answers. + - Mark variants under the retired original as `stale`. + - Exclude `retired` originals and `stale` variants from exam selection. +- Add an explicit retire-live-slot endpoint/action for removed slots. Importing a snapshot that omits a slot must never auto-retire the current live slot. +- Support restoring an older snapshot slot: + - Restored original becomes `Live`. + - Current live original becomes `Retired`. + - Variant review status remains unchanged while stale/unstale delivery state changes with the parent original. + - Previously approved variants under the restored original become servable again. + - Draft/rejected variants keep their review status. +- Define superseded rules: + - Only pending candidates for the same slot can be superseded. + - A newer snapshot candidate for the same slot supersedes older pending candidates. + - If a newer snapshot matches current live, older pending candidates for that slot become superseded by the newer no-change snapshot. + - Already-live or retired items are never marked superseded. +- Add migration/backfill for existing tryouts: + - Backfill existing imported/promoted questions into a Snapshot 1-style baseline where possible. + - Map current user-facing “promoted” live questions to `live`. + - Map replaced or older originals to `retired` where the relationship is known. + - Verify runtime filters exclude pending, retired, draft, rejected, and stale records after migration. +- Preview/replacement API should return a stable shape that the frontend can render without re-deriving lifecycle rules, for example: + +```ts +{ + snapshotId?: string; + slotNumber: number; + status: "changed" | "new_slot" | "removed" | "invalid" | "no_change" | "superseded"; + currentLiveItemId?: string; + snapshotQuestionId?: string; + currentLiveContentHash?: string; + snapshotContentHash?: string; + warnings: string[]; + canReplace: boolean; + canRetireLiveSlot: boolean; +} +``` + +Primary backend areas: `tryout_json_import.py`, `admin.py`, CAT/session selection filters. + +## Frontend Changes + +- In `/admin/tryouts/{id}/questions`, render snapshots as collapsible groups with slot rows. +- First snapshot should look like the live exam baseline, not a manual promotion queue. +- Later snapshots should hide `No Change` rows by default and show only actionable review rows. +- Add **Show unchanged questions** toggle inside each later snapshot accordion. +- Later snapshot row statuses: + - `Changed` + - `New Slot` + - `Removed From Latest Snapshot` + - `Invalid` + - `No Change` + - `Superseded` +- Removed-slot rows should show as warnings with an explicit `Retire Live Slot` action, not as automatic changes. +- Invalid rows should show the reason they cannot become live, including missing slot number, duplicate slot number, missing options, missing answer, or unsupported content. +- Import inside tryout opens a modal: + - select JSON. + - show preview before committing snapshot. + - require checkbox confirmations only for title/source/count mismatch. +- Import preview should block commit or require an explicit resolution path for duplicate slot numbers. +- Replacement modal requires explicit confirmations only when the action changes live exam behavior: + - “I understand this replaces the live question for slot X.” + - “I understand existing variants for this slot will become stale.” +- Replacement modal should submit the expected live item ID and content hash from the preview so stale-preview conflicts can be detected. +- Replace “Open Live Question” with clearer actions: + - `Open Live Item` + - `Preview Slot` + - `Restore This Version` for retired originals. +- Add **Simulate Exam Flow** view showing current live delivery order and any skipped/missing slots. +- Simulate Exam Flow should also surface import health warnings such as invalid first-snapshot rows, missing live slots, and removed-slot warnings that have not been acted on. + +Primary frontend area: `QuestionManagement.tsx`. + +## Test Plan + +- First import creates a tryout, full snapshot record, and live original items for all valid slots. +- First import with invalid rows stores invalid rows in the snapshot, creates live items only for valid rows, and shows import health warnings. +- Later import stores the full JSON snapshot but shows unchanged rows hidden by default. +- `Show unchanged questions` reveals `No Change` rows. +- Snapshot 3 comparison still marks slot #1 as changed if current live is Snapshot 1 content and Snapshot 3 matches Snapshot 2 content. +- Backend rejects replacing a live slot with identical snapshot content. +- Backend rejects replacement when `expected_live_item_id` or `expected_live_content_hash` no longer matches current live state. +- Newer snapshot marks older pending candidate for same changed slot as superseded. +- Newer no-change snapshot marks older pending candidates for the same slot as superseded. +- Snapshot 2 with changed slot #1 leaves current live item untouched until replacement. +- Replacing changed slot #1 retires prior original, marks its variants stale, and makes new snapshot item live. +- Restoring Snapshot 1 slot #1 makes that version live again and restores eligible variants. +- Restore reactivates only approved variants under the restored original; draft/rejected variants remain non-servable. +- Removed slot in a later snapshot does not auto-retire the live slot. +- Explicit retire-live-slot action removes that slot from runtime delivery and appears in simulation. +- Missing slot number makes a snapshot row invalid. +- Duplicate slot numbers block snapshot commit or require explicit admin resolution. +- Reordered JSON rows do not produce false changes. +- Canonical content hash ignores formatting-only whitespace differences. +- Option order changes are detected as content changes unless runtime option randomization makes order irrelevant. +- Same file re-import behavior is deterministic and does not create conflicting live state. +- Migration/backfill maps existing promoted/live questions and excludes non-live records from runtime delivery. +- Fixed exam simulation shows only current live servable items in slot order. +- Runtime session next-item endpoint never serves pending, retired, rejected, draft, or stale items. + +## Assumptions + +- First snapshot is trusted as the initial canonical exam and should auto-promote valid questions. +- Later snapshots are destructive only when admin promotes/replaces selected slots, not when imported. +- Imported JSON snapshots should remain complete historical records. +- Snapshot review UI should default to actionable differences only. +- Slot number is the primary comparison key for snapshot-to-live review. +- Variants belong to the live original version they were generated from. +- Historical answers/calibration must remain attached to the exact item version students saw. +- “Promoted” should disappear as a normal user-facing state after this workflow is implemented.