Snapshot-Driven Live Question Workflow Plan

Summary

Rework tryout import so the first imported snapshot creates the live exam automatically, while later imports become non-destructive snapshot candidates inside the existing tryout workspace. Admins then intentionally replace live questions slot-by-slot after reviewing differences. Imported JSON snapshots remain full historical records, but later snapshot accordions hide unchanged rows by default to keep review focused and prevent accidental no-op replacements.

Key Changes

Main /admin/tryouts import creates a new tryout/exam and auto-promotes all valid imported questions from the first snapshot into live items.
Existing tryout workspace gets an Import New Snapshot action. Importing there creates a full queued snapshot record, but does not change the live exam.
Replace the current single “Imported Snapshot Questions” section with collapsible snapshot sections:
- Snapshot 1: baseline source rows plus live/retired status.
- Snapshot 2+: review queue showing changed/new/removed/invalid rows by default.
- Add a Show unchanged questions toggle per snapshot; default hidden.
Snapshot comparison for UI review must compare each snapshot row against the current live slot, not only against the immediately previous snapshot.
If a newer snapshot supersedes an older pending candidate for the same slot, mark the older candidate as Superseded by Snapshot N.
Snapshot comparison must use a canonical content hash so "same content" and "changed content" are deterministic across backend, UI, and tests.
Rename the normal state away from “Promoted”. Use lifecycle language:
- Live: currently served for that slot.
- Pending Snapshot: imported but not live.
- Changed: differs from current live slot.
- No Change: same as current live slot; hidden by default.
- Retired: previously live, no longer served.
- Stale Variant: derived from replaced/retired content, excluded from exam delivery.
Keep retired originals and stale variants as operational history, not moved between backend tables. Frontend may visually group retired originals and variants under their snapshot/slot accordion.
Removed slots from later snapshots are warnings by default. They do not change live delivery unless an admin explicitly retires the live slot.
Add Simulate Exam Flow to show what the runtime would serve by slot using current live items and selection mode.

Backend/API Changes

Store every imported JSON snapshot as a full immutable import record.
Add clear state ownership:
- Item owns original question lifecycle delivery state: live or retired.
- Variants keep their existing review state (draft, approved, rejected) and gain/keep an explicit delivery exclusion state such as stale.
- Snapshot row review statuses (Changed, No Change, New Slot, Removed, Invalid, Superseded) are computed from snapshot row + current live slot unless persistence is required for audit/history.
Add canonical content hashing for snapshot/live comparison:
- Hash normalized question stem, options, correct answer, explanation, media references, scoring metadata, and any fields that affect what students see or how the item is scored.
- Ignore import timestamps, snapshot IDs, database row IDs, admin notes, and formatting-only whitespace differences.
- Option order should count as content unless the runtime already randomizes options independently.
Add slot identity rules:
- Slot number is the primary comparison key.
- Slot number must be present for a row to become live.
- Slot number must be unique inside one snapshot.
- Reordered JSON rows should not matter.
- Duplicate slot numbers should block snapshot commit or force an explicit admin resolution flow before commit.
Extend import preview for existing tryout imports to compare full incoming snapshot rows against the current live item per slot:
- same content: No Change.
- changed content: replacement candidate.
- new slot: add candidate.
- removed slot: latest snapshot no longer contains a previously live slot.
- invalid row: missing options/answer or cannot become live.
Add a tryout-scoped import endpoint for “new snapshot inside existing tryout”; it must reject JSON for a different source tryout unless admin confirms title/id/count warnings in the request.
Add a slot replacement endpoint that accepts selected snapshot question IDs and explicit confirmations:
- replacing live original for changed slot.
- retiring existing variants as stale.
- accepting count/title/source mismatch if relevant.
Replacement requests must include stale-preview guards:
- expected_live_item_id.
- expected_live_content_hash.
- Backend must run replacement in a transaction.
- If the current live slot changed after preview, reject and ask the admin to refresh/re-preview.
Backend must reject selected replacements whose snapshot content is the same as the current live slot.
On first snapshot only, auto-create live original Item rows for all valid imported questions.
Invalid first-snapshot rows are still stored in the immutable snapshot record, never become live items, and should create an import health warning in the tryout workspace.
On later snapshot replacement:
- Always create a new live original revision and mark the previous live original retired, even if the current live item has no answers.
- Mark variants under the retired original as stale.
- Exclude retired originals and stale variants from exam selection.
Add an explicit retire-live-slot endpoint/action for removed slots. Importing a snapshot that omits a slot must never auto-retire the current live slot.
Support restoring an older snapshot slot:
- Restored original becomes Live.
- Current live original becomes Retired.
- Variant review status remains unchanged while stale/unstale delivery state changes with the parent original.
- Previously approved variants under the restored original become servable again.
- Draft/rejected variants keep their review status.
Define superseded rules:
- Only pending candidates for the same slot can be superseded.
- A newer snapshot candidate for the same slot supersedes older pending candidates.
- If a newer snapshot matches current live, older pending candidates for that slot become superseded by the newer no-change snapshot.
- Already-live or retired items are never marked superseded.
Add migration/backfill for existing tryouts:
- Backfill existing imported/promoted questions into a Snapshot 1-style baseline where possible.
- Map current user-facing “promoted” live questions to live.
- Map replaced or older originals to retired where the relationship is known.
- Verify runtime filters exclude pending, retired, draft, rejected, and stale records after migration.
Preview/replacement API should return a stable shape that the frontend can render without re-deriving lifecycle rules, for example:

{
  snapshotId?: string;
  slotNumber: number;
  status: "changed" | "new_slot" | "removed" | "invalid" | "no_change" | "superseded";
  currentLiveItemId?: string;
  snapshotQuestionId?: string;
  currentLiveContentHash?: string;
  snapshotContentHash?: string;
  warnings: string[];
  canReplace: boolean;
  canRetireLiveSlot: boolean;
}

Primary backend areas: tryout_json_import.py, admin.py, CAT/session selection filters.

Frontend Changes

In /admin/tryouts/{id}/questions, render snapshots as collapsible groups with slot rows.
First snapshot should look like the live exam baseline, not a manual promotion queue.
Later snapshots should hide No Change rows by default and show only actionable review rows.
Add Show unchanged questions toggle inside each later snapshot accordion.
Later snapshot row statuses:
- Changed
- New Slot
- Removed From Latest Snapshot
- Invalid
- No Change
- Superseded
Removed-slot rows should show as warnings with an explicit Retire Live Slot action, not as automatic changes.
Invalid rows should show the reason they cannot become live, including missing slot number, duplicate slot number, missing options, missing answer, or unsupported content.
Import inside tryout opens a modal:
- select JSON.
- show preview before committing snapshot.
- require checkbox confirmations only for title/source/count mismatch.
Import preview should block commit or require an explicit resolution path for duplicate slot numbers.
Replacement modal requires explicit confirmations only when the action changes live exam behavior:
- “I understand this replaces the live question for slot X.”
- “I understand existing variants for this slot will become stale.”
Replacement modal should submit the expected live item ID and content hash from the preview so stale-preview conflicts can be detected.
Replace “Open Live Question” with clearer actions:
- Open Live Item
- Preview Slot
- Restore This Version for retired originals.
Add Simulate Exam Flow view showing current live delivery order and any skipped/missing slots.
Simulate Exam Flow should also surface import health warnings such as invalid first-snapshot rows, missing live slots, and removed-slot warnings that have not been acted on.

Primary frontend area: QuestionManagement.tsx.

Test Plan

First import creates a tryout, full snapshot record, and live original items for all valid slots.
First import with invalid rows stores invalid rows in the snapshot, creates live items only for valid rows, and shows import health warnings.
Later import stores the full JSON snapshot but shows unchanged rows hidden by default.
Show unchanged questions reveals No Change rows.
Snapshot 3 comparison still marks slot #1 as changed if current live is Snapshot 1 content and Snapshot 3 matches Snapshot 2 content.
Backend rejects replacing a live slot with identical snapshot content.
Backend rejects replacement when expected_live_item_id or expected_live_content_hash no longer matches current live state.
Newer snapshot marks older pending candidate for same changed slot as superseded.
Newer no-change snapshot marks older pending candidates for the same slot as superseded.
Snapshot 2 with changed slot #1 leaves current live item untouched until replacement.
Replacing changed slot #1 retires prior original, marks its variants stale, and makes new snapshot item live.
Restoring Snapshot 1 slot #1 makes that version live again and restores eligible variants.
Restore reactivates only approved variants under the restored original; draft/rejected variants remain non-servable.
Removed slot in a later snapshot does not auto-retire the live slot.
Explicit retire-live-slot action removes that slot from runtime delivery and appears in simulation.
Missing slot number makes a snapshot row invalid.
Duplicate slot numbers block snapshot commit or require explicit admin resolution.
Reordered JSON rows do not produce false changes.
Canonical content hash ignores formatting-only whitespace differences.
Option order changes are detected as content changes unless runtime option randomization makes order irrelevant.
Same file re-import behavior is deterministic and does not create conflicting live state.
Migration/backfill maps existing promoted/live questions and excludes non-live records from runtime delivery.
Fixed exam simulation shows only current live servable items in slot order.
Runtime session next-item endpoint never serves pending, retired, rejected, draft, or stale items.

Assumptions

First snapshot is trusted as the initial canonical exam and should auto-promote valid questions.
Later snapshots are destructive only when admin promotes/replaces selected slots, not when imported.
Imported JSON snapshots should remain complete historical records.
Snapshot review UI should default to actionable differences only.
Slot number is the primary comparison key for snapshot-to-live review.
Variants belong to the live original version they were generated from.
Historical answers/calibration must remain attached to the exact item version students saw.
“Promoted” should disappear as a normal user-facing state after this workflow is implemented.

12 KiB Raw Blame History