dwindown/yellow-bank-soal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dwindown:7adbc5fb9720721a12a993556ddb7522c33633b9
Branches Tags
dwindown:main
...
compare: dwindown:main
Branches Tags
dwindown:main

4 Commits
7adbc5fb97 ... main

Author SHA1 Message Date
Dwindi Ramadhana
f699c27f32 Add snapshot workflow implementation plan 2026-06-20 13:31:10 +07:00
Dwindi Ramadhana
b8e201b45f Checkpoint React frontend migration 2026-06-20 01:43:39 +07:00
Dwindi Ramadhana
ab86c254d1 docs: Add REACT_Migration_Plan.md with feasibility assessment 
- Fixed broken Markdown formatting (removed excessive backslashes)
- Added Section 11: Feasibility Assessment
  - Current state summary (8 items already in place)
  - Identified gaps (6 items)
  - Detailed gap analysis for session timer, monorepo, nested routes
  - Feasibility score: 7/10
  - Recommended execution order (Phase 0-4)
  - Summary with challenges and strengths
 2026-06-17 21:09:40 +07:00
Dwindi Ramadhana
792f9b7483 refactor: redesign admin permalinks to RESTful paths 
- Replace fragile /admin/ai-playground and /admin/ai-generation routes
  with item-scoped /admin/questions/{id}/generate endpoints
- Add /admin/tryouts/{id}/questions route using Tryout primary key
  instead of composite query params (?tryout_id=X&website_id=Y)
- Fix variant detail and review-bulk endpoints to use scoped paths
- Update all internal links (dashboard, hierarchy, exams) to new routes
- Remove obsolete ai_playground_view/submit/save functions
 2026-06-16 23:54:24 +07:00
178 changed files with 41729 additions and 3963 deletions
Expand all files Collapse all files

365
ADMIN_TRYOUT_RESTRUCTURE_PLAN.md Normal file
View File

@@ -0,0 +1,365 @@
# Admin UI Redesign - Implementation Plan
## Overview
This plan outlines the migration from the current scattered admin structure to a clean, hierarchy-driven navigation centered on **Tryouts**.
### Guiding Principles
1. **One main page per domain** - Features live under their parent, not as separate menu items
2. **URL reflects depth** - Path structure shows relationship (`/admin/tryout/{id}/questions`)
3. **Tree as map** - Hierarchy tree shows structure; drill-down shows details
4. **Consistent naming** - Use "Tryout" instead of "Exam" throughout
---
## 1. URL Structure
### New URL Scheme
| Old Route | New Route | Description |
|-----------|-----------|-------------|
| `/admin/exams` | `/admin/tryouts` | Hierarchy tree (main entry) |
| `/admin/student-attempts` | `/admin/tryout/{tryout_id}/attempts` | Attempts filtered by tryout |
| - | `/admin/tryout/{tryout_id}/questions` | Questions filtered by tryout |
| - | `/admin/tryout/{tryout_id}/questions/{question_id}/workspace` | Question workspace |
| - | `/admin/tryout/{tryout_id}/questions/{question_id}/workspace/{tab}` | Workspace tabs |
| - | `/admin/tryout/{tryout_id}/normalization` | Normalization settings for this tryout |
| `/admin/questions` | `/admin/questions` | Global question list (kept) |
| (none) | `/admin/import-tryout` | Import tryout modal/page |
> **Note:** Import is tryout-level, not question-level. Import button lives on `/admin/tryouts` page header.
### Hierarchy Depth Convention
```
/admin/tryouts → Level 0: Root
/admin/tryout/{tryout_id} → Level 1: Entity
/admin/tryout/{tryout_id}/attempts → Level 2: Related data
/admin/tryout/{tryout_id}/questions → Level 2: Related data
/admin/tryout/{tryout_id}/questions/{id} → Level 3: Specific item
/admin/tryout/{tryout_id}/questions/{id}/workspace → Level 4: Detail view
```
---
## 2. Navigation Structure
### Proposed Navigation
```
Questions
├── /admin/questions # Global question list
└── /admin/tryout/*/questions/*/workspace # Direct link from tree
Tryouts
├── /admin/tryouts # Tree: Website → Tryout → Stat → Actions + Import button
├── /admin/tryout/*/attempts # Filtered attempts
├── /admin/tryout/*/questions # Questions in this tryout
├── /admin/tryout/*/normalization # Normalization settings
└── /admin/import-tryout # Import modal/page
Reports
├── /admin/reports # Dashboard
├── /admin/item-statistics
└── /admin/calibration-status
Settings
├── /admin/settings
├── /admin/websites
└── /admin/password
```
### Navigation Item Definition
```python
ADMIN_NAV_ITEMS = (
("Dashboard", "/admin/dashboard", ("/admin/dashboard",)),
("Questions", "/admin/questions", (
"/admin/questions",
"/admin/tryout/*/questions/*/workspace", # Pattern for direct links
)),
("Tryouts", "/admin/tryouts", (
"/admin/tryouts",
"/admin/tryout/*/attempts",
"/admin/tryout/*/questions",
"/admin/tryout/*/normalization",
"/admin/import-tryout",
)),
("Reports", "/admin/reports", (
"/admin/reports",
"/admin/item-statistics",
"/admin/calibration-status",
)),
("Settings", "/admin/settings", (
"/admin/settings",
"/admin/websites",
"/admin/password",
)),
("Logout", "/admin/logout", ("/admin/logout",)),
)
```
---
## 3. Tryouts Tree Structure
### Visual Design
```
┌─ Tryouts ───────────────────────────────────────────────────────────────────┐
│ │
│ [+ Import Tryout] │
│ │
│ 🌐 Website A │
│ │ │
│ ├─ 📋 132380 - UTBK 2024 [●] │
│ │ └─ [Expanded on click] │
│ │ │
│ ├─ 📋 132381 - SIMAK UI [✓] │
│ │ │
│ └─ 📋 132382 - PAS Semester 1 [○] │
│ │
│ 🌐 Website B │
│ └─ ... │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
Expanded Tryout View:
┌─ 📋 132380 - UTBK 2024 ─────────────────────────────────────────────────────┐
│ │
│ ┌─ Stat Card ─────────────────────────────────────────────────────────┐ │
│ │ 👥 150 participants │ NM: 672 avg │ NN: 505 avg │ │
│ │ ✓ 98% completion │ 📐 Calibration: ████████░░ 85% (17/20) │ │
│ └────────────────────────────────────────────────────────────────────┘ │
│ │
│ [📝 Questions (20)] [👥 Attempts (150)] [📐 Normalization] [⚙ Settings]│
│ │
└──────────────────────────────────────────────────────────────────────────────┘
Legend:
[●] Partial (50-89% calibrated)
[✓] Ready (≥90% calibrated)
[○] Needs Data (<50% calibrated)
```
### Import Button Location
- **Location:** Header of `/admin/tryouts` page
- **Label:** "[+ Import Tryout]" or "[Import Tryout JSON]"
- **Behavior:** Opens import modal/page
- **Why:** Import is tryout-level operation (imports questions WITH tryout context)
### Stat Card Components
| Field | Source | Display |
|-------|--------|---------|
| Participants | `TryoutStats.participant_count` | 👥 {count} |
| Avg NM | `AVG(Session.NM)` where completed | 📊 {value} avg |
| Avg NN | `AVG(Session.NN)` where completed | 📈 {value} avg |
| Completion Rate | `completed / participants * 100` | ✓ {percentage}% |
| Calibration | `calibrated_items / total_items` | 📐 Progress bar + {count}/{total} |
### Action Buttons
| Action | Target URL | Icon |
|--------|------------|------|
| Questions | `/admin/tryout/{id}/questions` | 📝 |
| Attempts | `/admin/tryout/{id}/attempts` | 👥 |
| Normalization | `/admin/tryout/{id}/normalization` | 📐 |
| Settings | `/admin/tryout/{id}/settings` (or modal) | ⚙ |
---
## 4. Page Specifications
### 4.1 `/admin/tryouts` (Main Tree)
**Purpose:** Primary navigation entry, shows structure at a glance
**Default State:**
- Websites expanded
- Tryouts collapsed
- Shows calibration indicator dot next to each tryout
**Interactions:**
- Click tryout → expand/collapse
- Expanded tryout shows stat card + action buttons
- Actions navigate to filtered views
### 4.2 `/admin/tryout/{tryout_id}/questions`
**Purpose:** View all questions in a specific tryout
**Behavior:**
- Shows only original/imported questions (basis items)
- Pre-filtered by `tryout_id`
- Links to workspace for AI variant generation
**Table Columns:**
| Column | Description |
|--------|-------------|
| ID | Question internal ID |
| Stem Preview | First 100 chars of question text |
| Difficulty | Current difficulty level |
| Calibration | P-value or IRT-b indicator |
| Variants | Count of generated variants |
| Actions | [View] [Workspace] |
### 4.3 `/admin/tryout/{tryout_id}/questions/{question_id}/workspace`
**Purpose:** Generate and manage question variants
**Tabs:**
| Tab | Purpose |
|-----|---------|
| Generate | AI variant generation interface |
| Review | Review generated variants |
| Batch | Batch generation options |
**Access Pattern:**
- Opens from question list or tree direct link
- Context: knows parent tryout, parent question
### 4.4 `/admin/tryout/{tryout_id}/attempts`
**Purpose:** View student attempts for specific tryout
**Current Implementation:** Already exists at `/admin/student-attempts` → migrate URL
**Enhancements:**
- Pre-filtered by `tryout_id` (no dropdown needed on this page)
- Stat card from parent tryout shown at top
### 4.5 `/admin/tryout/{tryout_id}/normalization`
**Purpose:** Configure normalization settings for a specific tryout
**Settings (per-tryout):**
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| Mode | Select | Auto | Auto (calculate from data) or Manual (fixed values) |
| Rataan | Number | 500 | Target mean for normalization |
| SB | Number | 100 | Target standard deviation |
| Recalculate | Button | - | Re-run normalization on existing sessions |
**Formula:** `NN = 500 + 100 × ((NM - Rataan) / SB)`
**UI:**
- Simple form with current values
- "Recalculate" button triggers normalization job
- Shows last normalization timestamp
### 4.6 `/admin/import-tryout`
**Purpose:** Import tryout data (questions + metadata) from JSON
**Access:** Via "[+ Import Tryout]" button on `/admin/tryouts` page
**Behavior:**
- Opens modal or dedicated page
- Upload JSON file or paste JSON content
- Preview import before confirming
- Creates new tryout with questions
**URL Convention:** Not under specific tryout (it's creating a new one)
---
## 5. Deprecations
### Routes to Remove
| Route | Reason |
|-------|--------|
| `/admin/exams` | Renamed to `/admin/tryouts` |
| `/admin/student-attempts` | URL changed to `/admin/tryout/{id}/attempts` |
| `/admin/templates` | AI uses basis items directly |
| `/admin/basis-items` | Merge into question workspace |
| `/admin/hierarchy` | Tree IS the hierarchy |
| `/admin/question-quality` | Merged into tryout stat card |
### Legacy Redirects
```python
LEGACY_URL_MAP = {
"/admin/exams": "/admin/tryouts",
"/admin/student-attempts": "/admin/tryouts", # Or redirect to tryouts with guidance
"/admin/hierarchy": "/admin/tryouts",
"/admin/question-quality": "/admin/tryouts",
# Templates and basis-items: 404 (removed)
}
```
---
## 6. Implementation Phases
### Phase 1: Foundation
- [ ] Rename `/admin/exams` → `/admin/tryouts` (keep old route for now)
- [ ] Implement tree structure in `/admin/tryouts`
- [ ] Move `TryoutStats` info into tree stat cards
- [ ] Add calibration indicator dots
### Phase 2: URL Migration
- [ ] Create `/admin/tryout/{id}/attempts` (redirect from old route)
- [ ] Create `/admin/tryout/{id}/questions`
- [ ] Update navigation items
### Phase 3: Workspace Integration
- [ ] Create question workspace route
- [ ] Implement workspace tabs
- [ ] Connect workspace to tree and question list
### Phase 4: Cleanup
- [ ] Add legacy redirects
- [ ] Remove deprecated routes
- [ ] Update all hardcoded links in views
### Phase 5: Polish
- [ ] Review all pages for consistency
- [ ] Update documentation
- [ ] Test all navigation paths
---
## 7. Open Questions
1. ~~Normalization settings~~ - **RESOLVED**: Move under tryout context as `/admin/tryout/{id}/normalization`
2. ~~Import questions page~~ - **RESOLVED**: Import is tryout-level. Button on `/admin/tryouts` header, not a separate page.
3. **Tryout settings** - What settings are actually needed? (Scoring mode, time limits, selection criteria?)
4. **Global questions page** - Is `/admin/questions` (unfiltered) still useful, or should every question access go through tryout context?
5. **Templates deprecation** - Confirm that `/admin/templates` is truly unused and can be safely removed?
6. **Legacy routes for deleted pages** - Should `/admin/templates` and `/admin/basis-items` redirect somewhere or return 404?
---
## 8. Files to Modify
### Primary Changes
- `app/admin_web.py` - Major route restructuring
- Navigation definition in `admin_web.py`
- Legacy URL map
### Likely Additions
- Static assets for tree expansion/collapse (if not using existing)
### Documentation Updates
- `ADMIN_UI_REDESIGN_PLAN.md` - Update to reflect final structure
- `PROJECT_UNDERSTANDING.md` - Update route documentation
---
## 9. Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2026-06-17 | Initial draft based on discussion |
| 1.1 | 2026-06-17 | - Move normalization to `/admin/tryout/{id}/normalization`<br>- Move import button to `/admin/tryouts` header<br>- Add normalization page spec (4.5)<br>- Rename import page spec (4.6)<br>- Update navigation and action buttons |

700
ADMIN_UI_REDESIGN_PLAN.md Normal file
View File

@@ -0,0 +1,700 @@
# Admin UI Redesign Plan
> **Document Type:** UI/UX Improvement Plan
> **Current System:** IRT Bank Soal Admin
> **Date:** 2026-06-15
> **Status:** Draft for Review
---
## Executive Summary
The current admin interface is built from a **developer/system perspective** rather than a **human/admin perspective**. This plan outlines a complete redesign to make the admin dashboard intuitive, workflow-oriented, and human-readable.
### Current Problems
| Problem | Impact |
|---------|--------|
| Navigation uses technical terms | Admins don't understand menu labels |
| Multiple unrelated features in one view | Confusing, overwhelming |
| Data displayed in database terminology | Hard to interpret scores |
| No clear workflow guidance | Admin doesn't know what to do first |
| No contextual help | Unclear what each feature does |
| Mixed concern pages | AI + Questions + Calibration all on one page |
---
## Current State Analysis
### Current Navigation (System POV)
```
├── Dashboard (raw counts)
├── Websites (technical list)
├── Tryout Import (system term)
├── Data Hierarchy (developer term)
├── Basis Items (technical term)
├── Calibration Status (technical term)
├── Item Statistics (technical term)
├── Session Overview (technical term)
├── AI Playground (slang)
└── Password Info (unrelated)
```
### Current Issues
1. **Naming Problems:**
- "Basis Items" → Should be "Question Templates" or "Original Questions"
- "Data Hierarchy" → Should be "Data Overview" or "Website Structure"
- "Tryout Import" → Should be "Import Questions"
- "Calibration Status" → Should be "Question Quality" or "Difficulty Analysis"
- "AI Playground" → Should be "Generate AI Questions"
- "Session Overview" → Should be "Student Attempts"
2. **Dashboard Issues:**
- Shows raw database counts (Tryouts, Items, Sessions)
- No meaningful KPIs or actionable insights
- No visual indicators of system health
3. **Page Organization:**
- Too many technical terms on each page
- Tables show raw data without explanation
- No breadcrumbs or context
---
## Proposed Redesign
### New Navigation Structure (Human POV)
```
🎯 Dashboard (Home)
├── System Health Summary
├── Quick Actions
└── Recent Activity
📋 Questions Bank
├── All Questions (list + search)
├── Question Templates (basis items)
├── Import Questions (from Excel/JSON)
└── Question Quality (calibration status)
🤖 AI Generation
├── Generate New Questions
├── Review Generated Questions
└── Generation History
📊 Exams (Tryouts)
├── All Exams (list)
├── Exam Settings (scoring mode)
├── Student Attempts
└── Normalization Settings
📈 Reports
├── Student Performance Report
├── Item Analysis Report
├── Exam Comparison Report
└── Scheduled Reports
⚙️ Settings
├── Websites Management
├── Account Settings
└── System Info
```
### Navigation Mapping Table
| Current Menu | New Menu | Reason |
|-------------|----------|--------|
| Dashboard | 🎯 Dashboard | Home base |
| Websites | ⚙️ Settings > Websites | Configuration |
| Tryout Import | 📋 Questions > Import Questions | Workflow step |
| Data Hierarchy | ⚙️ Settings | Admin settings |
| Basis Items | 📋 Questions > Question Templates | Content management |
| Calibration Status | 📋 Questions > Question Quality | Quality assurance |
| Item Statistics | 📈 Reports > Item Analysis | Reporting |
| Session Overview | 📊 Exams > Student Attempts | Workflow |
| AI Playground | 🤖 AI Generation | Dedicated feature |
| Password Info | ⚙️ Settings > Account | Configuration |
---
## Detailed Page Redesigns
### 1. Dashboard (Home) — `GET /admin/dashboard`
**Current State:**
```python
# Shows raw counts
body = f"""
<p>Signed in as <strong>{admin}</strong>.</p>
<div class="grid">
<div class="stat">Tryouts<strong>{tryouts}</strong></div>
<div class="stat">Items<strong>{items}</strong></div>
<div class="stat">Sessions<strong>{sessions}</strong></div>
<div class="stat">Completed Sessions<strong>{completed}</strong></div>
</div>
<p><a href="/admin/ai-playground">Open AI Playground</a></p>
"""
```
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ Good Morning, Admin! 👋 │
│ Last login: Today at 9:00 AM │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 📊 System Overview │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ 5 Exams │ │ 450 │ │ 1,234 │ │ 89% │ │
│ │ Active │ │ Questions│ │ Students │ │ Avg Score│ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ ⚠️ Attention Needed │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ⚡ 23 questions need calibration (do this first!) │ │
│ │ 📝 5 AI-generated questions pending review │ │
│ │ 📥 2 exam exports ready for download │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ 🚀 Quick Actions │
│ [Import Questions] [Generate AI] [View Reports] [Add Exam] │
│ │
│ 📈 Recent Activity │
│ • 12 students completed "UTBK 2024" in last hour │
│ • 3 new questions generated via AI │
│ • Calibration completed for "SIMAK UI" (95% ready) │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Human-readable greeting with time
- Meaningful metrics (not raw counts)
- Actionable alerts with urgency indicators
- Quick action buttons with clear labels
- Recent activity feed
---
### 2. Questions Bank — Questions List (`/admin/questions`)
**Current State:** Table with raw database fields
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 📋 Question Bank │
├─────────────────────────────────────────────────────────────────┤
│ │
│ [Search: "matematika" ] [Filter ▼] [🔍] │
│ │
│ Showing 450 questions across 5 exams │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ☐ │ Q1 │ Berapakah hasil dari 2 + 2? │ │
│ │ │ │ ▸ Easy (p=0.85) | Used 234x | SIMAK UI │ │
│ ├────┼─────┼──────────────────────────────────────────────┤ │
│ │ ☐ │ Q5 │ Hitung integral dari x² dx... │ │
│ │ │ │ ▸ Medium (p=0.45) | Used 89x | UTBK 2024 │ │
│ ├────┼─────┼──────────────────────────────────────────────┤ │
│ │ ☐ │ Q12 │ Jelaskan teori evolusi... │ │
│ │ │ │ ▸ Hard (p=0.22) | Used 45x | ONM 2024 │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ [Delete Selected] [Export Selected] [Edit Selected] │
│ │
│ 📄 Page 1 of 23 [<] [1] [2] [3] ... [>] │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Question preview in list (not just ID)
- Human-readable difficulty (Easy/Medium/Hard)
- Usage count (how many times used)
- Which exam it belongs to
- Visual indicators for difficulty colors
---
### 3. Question Templates — (`/admin/templates`)
**Current State:** "Basis Items" - confusing technical term
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 📝 Question Templates │
│ (Original questions used to generate AI variants) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Templates are your "master questions" that AI uses to │
│ create different versions with varying difficulty levels. │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 📝 Template #45: "Berapakah hasil dari 2 + 2?" │ │
│ │ AI Generated Variants: 12 (3 easy, 6 medium, 3 hard) │ │
│ │ [View All Variants] [Generate More] [Edit] │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ 📝 Template #89: "Hitung integral dari x² dx..." │ │
│ │ AI Generated Variants: 8 (2 easy, 4 medium, 2 hard) │ │
│ │ [View All Variants] [Generate More] [Edit] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ [+ Create New Template] │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Clear explanation of what templates are
- Visual representation of variants
- Easy action buttons
- "Create New Template" prominent
---
### 4. AI Generation — (`/admin/ai-generation`)
**Current State:** "AI Playground" - informal, confusing tabs
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 🤖 AI Question Generator │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Generate new question variants using AI. │
│ Select a template question and specify difficulty level. │
│ │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ 📝 Select Template │ │ 🎯 Target Difficulty │ │
│ │ [Dropdown: Questions]│ │ ○ Easy (p > 0.70) │ │
│ └──────────────────────┘ │ ● Medium (p ≈ 0.50) │ │
│ │ ○ Hard (p < 0.30) │ │
│ ┌──────────────────────┐ └──────────────────────┘ │
│ │ 📝 How many variants?│ │
│ │ [1] [3] [5] [10] │ ┌──────────────────────┐ │
│ └──────────────────────┘ │ 💬 Additional Notes │ │
│ │ [Optional context...] │ │
│ └──────────────────────┘ │
│ │
│ [🚀 Generate Questions] │
│ │
├─────────────────────────────────────────────────────────────────┤
│ 📋 Generated Questions (Pending Review) │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 🔄 Generating... 2 of 5 questions completed │ │
│ │ [████████░░] 60% │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ ✅ Generated & Ready for Review: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ✓ Variant #123: "Berapakah hasil dari 3 + 4?" (Easy) │ │
│ │ [Preview] [Approve] [Regenerate] [Reject] │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ ✓ Variant #124: "Hitung hasil dari 5 + 6..." (Easy) │ │
│ │ [Preview] [Approve] [Regenerate] [Reject] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Clear, labeled sections
- Radio buttons for difficulty (not dropdown)
- Progress indicator during generation
- Clear action buttons (Approve/Reject/Regenerate)
- Explanation of what each option means
---
### 5. Question Quality (Calibration) — (`/admin/question-quality`)
**Current State:** "Calibration Status" - technical IRT terminology
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 📊 Question Quality Dashboard │
│ (Shows how well each question is "calibrated" for testing) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 📖 What is Question Quality? │
│ Questions become "calibrated" after many students answer them. │
│ Well-calibrated questions give accurate student scores. │
│ │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Overall Quality: ████████░░ 78% │
│ (78 out of 100 questions are ready for adaptive testing) │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 📋 By Exam │ │
│ │ │ │
│ │ UTBK 2024 ████████████ 95% ✓ Ready │ │
│ │ SIMAK UI █████████░░░ 72% ⚠️ Partial │ │
│ │ ONM 2024 ██████░░░░░░ 45% ❌ Needs more data│ │
│ │ PASIAD Selection ████████████ 100% ✓ Excellent │ │
│ │ │ │
│ │ [Run Calibration for All Exams] │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ Questions Needing Attention: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ⚠️ Q45 - "Hitung integral..." only answered 12 times │ │
│ │ Need at least 100 answers to calibrate properly. │ │
│ │ Current estimate: p=0.42 (might change) │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ ❌ Q78 - "Teori relativitas..." has conflicting answers │ │
│ │ Check if correct answer is correct in database. │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Clear explanation of what calibration means
- Progress bars for visual understanding
- Status indicators (✓ Ready, ⚠️ Partial, ❌ Needs data)
- Specific recommendations for action
- User-friendly difficulty explanation
---
### 6. Student Attempts — (`/admin/student-attempts`)
**Current State:** "Session Overview" - raw database table
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 📊 Student Attempts │
│ (See how students performed on each exam) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Filter: [Select Exam ▼] [Status ▼] [Date Range ▼] [Search] │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 📋 UTBK 2024 Results │ │
│ │ │ │
│ │ Participants: 1,234 students │ │
│ │ Average Score (NM): 672 / 1000 │ │
│ │ Average Score (NN): 505 / 1000 │ │
│ │ Completion Rate: 98% (1,209 completed) │ │
│ │ │ │
│ │ Score Distribution: │ │
│ │ ▁▂▃▇█▇▃▂▁ (bell curve centered around 500) │ │
│ │ 200 300 400 500 600 700 800 900 1000 │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ Recent Attempts: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 👤 John Doe (john@example.com) │ │
│ │ Exam: UTBK 2024 | Completed: Today, 2:30 PM │ │
│ │ Score: NM=720 (85th percentile) | NN=645 │ │
│ │ Correct: 28/30 | Time: 45 minutes │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ 👤 Jane Smith (jane@example.com) │ │
│ │ Exam: SIMAK UI | Completed: Today, 1:15 PM │ │
│ │ Score: NM=580 (45th percentile) | NN=485 │ │
│ │ Correct: 22/30 | Time: 52 minutes │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ [Export All Results] [View Detailed Report] [Schedule Report] │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Grouped by exam with summary stats
- Human-readable student info
- Percentile ranking
- Score distribution visualization
- Clear action buttons
---
### 7. Reports — (`/admin/reports`)
**Proposed State:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 📈 Reports │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Generate detailed analysis reports for exams and students. │
│ │
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ 📊 Student Performance │ │ 📋 Item Analysis │ │
│ │ │ │ │ │
│ │ See individual student │ │ Analyze question │ │
│ │ scores, rankings, and │ │ difficulty, validity, │ │
│ │ detailed breakdowns. │ │ and discrimination. │ │
│ │ │ │ │ │
│ │ [Generate Report] │ │ [Generate Report] │ │
│ └─────────────────────────┘ └─────────────────────────┘ │
│ │
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ 📈 Exam Comparison │ │ 📅 Scheduled Reports │ │
│ │ │ │ │ │
│ │ Compare scores across │ │ Set up automatic │ │
│ │ different exams or │ │ weekly/monthly reports │ │
│ │ time periods. │ │ delivery. │ │
│ │ │ │ │ │
│ │ [Generate Report] │ │ [Manage Schedules] │ │
│ └─────────────────────────┘ └─────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Key Changes:**
- Card-based layout with icons
- Clear description of each report type
- Visual cards instead of dropdowns
- Scheduled reports as first-class feature
---
## Implementation Phases
### Phase 1: Navigation Redesign (Foundation)
**Files to modify:**
- `app/admin_web.py` - Update `ADMIN_NAV_ITEMS`
- Create new route handlers
**Steps:**
1. Rename navigation items with human labels
2. Create new route structure
3. Implement breadcrumb system
4. Add help tooltips
**New Navigation Structure:**
```python
ADMIN_NAV_ITEMS = (
("Dashboard", "/admin/dashboard", ("/admin/dashboard",)),
("Questions", "/admin/questions", ("/admin/questions", "/admin/templates")),
("AI Generator", "/admin/ai-generation", ("/admin/ai-generation",)),
("Exams", "/admin/exams", ("/admin/exams", "/admin/student-attempts")),
("Reports", "/admin/reports", ("/admin/reports",)),
("Settings", "/admin/settings", ("/admin/settings",)),
)
```
---
### Phase 2: Dashboard Overhaul
**New Dashboard Components:**
1. Greeting with user name and time
2. System health cards (with meaningful metrics)
3. Action alerts section
4. Quick action buttons
5. Recent activity feed
**Files to modify:**
- `dashboard_view()` function
- `_render_admin_page()` for dashboard-specific layout
---
### Phase 3: Questions Section
**New Pages:**
1. `/admin/questions` - List all questions with search/filter
2. `/admin/questions/{id}` - Question detail view
3. `/admin/templates` - Question templates (formerly basis items)
4. `/admin/questions/import` - Import wizard
**Key UI Components:**
- Question preview cards
- Difficulty badges (Easy/Medium/Hard)
- Color-coded indicators
- Inline search
---
### Phase 4: AI Generation Section
**New Pages:**
1. `/admin/ai-generation` - Main generation interface
2. `/admin/ai-generation/review` - Review pending variants
3. `/admin/ai-generation/history` - Generation history
**Key UI Components:**
- Template selector with preview
- Difficulty radio buttons
- Generation progress bar
- Batch approve/reject actions
---
### Phase 5: Exams Section
**New Pages:**
1. `/admin/exams` - List all exams
2. `/admin/exams/{id}/settings` - Exam configuration
3. `/admin/student-attempts` - Student attempts list
4. `/admin/normalization` - Normalization settings
**Key UI Components:**
- Exam cards with status indicators
- Student attempt cards
- Score distribution visualization
---
### Phase 6: Reports Section
**New Pages:**
1. `/admin/reports` - Report dashboard
2. `/admin/reports/student-performance` - Student report
3. `/admin/reports/item-analysis` - Item report
4. `/admin/reports/exam-comparison` - Comparison report
5. `/admin/reports/scheduled` - Scheduled reports
**Key UI Components:**
- Report type cards
- Export format options
- Schedule configuration
---
## Technical Implementation Notes
### CSS Class Naming Convention
```css
/* Old: System POV */
.stat { }
.grid { }
.table-wrap { }
/* New: Human POV */
.dashboard-hero { }
.metric-card { }
.question-list { }
.difficulty-badge { }
.difficulty-easy { background: #dcfce7; }
.difficulty-medium { background: #fef3c7; }
.difficulty-hard { background: #fee2e2; }
```
### Helper Functions to Create
```python
# In admin_web.py
def _render_question_card(item: Item) -> str:
"""Render a human-readable question card."""
difficulty = _human_difficulty(item.ctt_p)
difficulty_color = _difficulty_color(item.ctt_p)
return f"""
<div class="question-card">
<div class="difficulty-badge {difficulty_color}">{difficulty}</div>
<div class="question-stem">{escape(item.stem[:100])}...</div>
<div class="question-meta">
Used {item.calibration_sample_size}x |
{item.tryout_id}
</div>
</div>
"""
def _human_difficulty(p_value: float | None) -> str:
"""Convert p-value to human-readable difficulty."""
if p_value is None:
return "Unknown"
if p_value > 0.70:
return "Easy"
elif p_value >= 0.30:
return "Medium"
else:
return "Hard"
def _difficulty_color(p_value: float | None) -> str:
"""Get color class for difficulty badge."""
if p_value is None:
return "difficulty-unknown"
if p_value > 0.70:
return "difficulty-easy"
elif p_value >= 0.30:
return "difficulty-medium"
else:
return "difficulty-hard"
```
### Responsive Design
```css
/* Mobile-friendly layout */
@media (max-width: 768px) {
.admin-layout {
flex-direction: column;
}
.sidebar-nav {
display: flex;
overflow-x: auto;
padding: 8px;
}
.metric-cards {
grid-template-columns: repeat(2, 1fr);
}
}
```
---
## Success Metrics
| Metric | Target |
|--------|--------|
| Time to complete common task | Reduce by 50% |
| Admin confusion score | < 2/5 |
| Support tickets about UI | Reduce by 80% |
| Feature discovery rate | > 90% can find features |
---
## Appendix: Terminology Mapping
| System Term | Human Term |
|------------|------------|
| Tryout | Exam / Test |
| Item | Question |
| Basis Item | Question Template / Original Question |
| Session | Student Attempt |
| Calibration | Question Quality / Difficulty Analysis |
| IRT | Adaptive Scoring |
| CTT | Standard Scoring |
| Bobot | Weight / Point Value |
| NM | Raw Score |
| NN | Normalized Score |
| p-value | Difficulty Score |
| Theta | Student Ability Score |
---
## Files to Modify
| File | Changes |
|------|---------|
| `app/admin_web.py` | Complete UI rewrite |
| `app/admin.py` | May need minor updates |
| `requirements.txt` | Add any new frontend deps (if needed) |
---
## Next Steps
1. [ ] Review and approve this plan
2. [ ] Prioritize phases (suggest starting with Phase 1 & 2)
3. [ ] Create mockups/wireframes for key pages
4. [ ] Implement Phase 1: Navigation & Dashboard
5. [ ] User testing with admin users
6. [ ] Iterate based on feedback

615
FRONTEND_MIGRATION_AUDIT_REPORT.md Normal file
View File

@@ -0,0 +1,615 @@
# Frontend Migration Audit Report
Date: 2026-06-19
Project: Yellow Bank Soal / IRT Bank Soal
Scope: Migration from root-level Python/FastAPI admin UI to `backend/` plus new React `frontend/`
Auditor: Codex
## 1. Executive Summary
The React frontend is scaffolded and builds successfully, but the migration is not yet feature-complete or integration-safe. The biggest risks are API address drift, tenant/website context bugs, missing parity with the legacy Python admin workflows, and placeholder React pages that appear functional but do not call real backend APIs.
Current readiness assessment: **not production-ready as the primary replacement for the Python admin UI**.
Top findings:
| Priority | Finding | Impact |
|---|---|---|
| P0 | Local frontend API base URL omits `/api/v1` | Most API calls 404 in `npm run dev` and any environment using `frontend/.env`. |
| P0 | System admin website scope starts as `website_id=0` and React Query keys ignore website selection | First dashboard loads empty or wrong scoped data; switching websites can show stale data. |
| P0 | Several React API calls target nonexistent or renamed backend endpoints | Reports, normalization, and Excel import workflows are broken. |
| P1 | Student tryout portal from the migration plan is absent | Core learner flow is not migrated to React. |
| P1 | AI generation UI has incomplete save/review/batch behavior | Operators can generate previews, but core review and batch workflow parity is missing. |
| P1 | Unsafe `dangerouslySetInnerHTML` use without sanitization | Imported or AI-generated HTML can become an admin XSS risk. |
| P2 | Multiple legacy admin features are missing or placeholders | Hierarchy, question quality, question details, password update, exports, and settings are incomplete. |
The build result is positive: `npm run build` completed successfully. This means the current issues are mainly behavioral and integration defects, not TypeScript compilation blockers.
## 2. Audit Scope And Methodology
Reviewed:
- Repository restructure from root `app/` to `backend/app/` and new `frontend/`.
- Current React routes, pages, state store, API client, and Docker/Nginx configuration.
- Current FastAPI router definitions and generated OpenAPI paths.
- Last committed Python admin surface via `git show HEAD:app/admin_web.py`.
- Existing planning documents: `REACT_Migration_Plan.md`, `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md`, and `UX_AUDIT_ADMIN_FLOW.md`.
Verification performed:
- `npm run build` inside `frontend/`: passed.
- FastAPI OpenAPI generation from `backend/app/main.py`: produced 55 paths.
- Static endpoint comparison between React `api.get/post/put/delete` calls and backend route definitions.
Not performed:
- Full browser E2E against a running backend/database.
- Live authentication, import, AI generation, or report generation.
- Full backend test suite run.
## 3. Current Architecture Snapshot
The current repository is in an uncommitted migration state. Git sees the old root-level Python files as deleted and `backend/` plus `frontend/` as new untracked folders.
React frontend:
- Admin-only route shell currently lives in `frontend/src/App.tsx`.
- API helper is `frontend/src/lib/api.ts`.
- Global website and auth token state is persisted in `frontend/src/store/useAppStore.ts`.
- The admin UI has pages for Dashboard, Questions, Tryouts, Reports, Settings, AI Generation, Import, and nested Tryout workspaces.
Backend:
- Main FastAPI app lives in `backend/app/main.py`.
- JSON APIs are generally under `/api/v1`.
- Legacy Python admin HTML router is still mounted at `/admin` when `ENABLE_ADMIN=true`.
- Import/export router hardcodes `/api/v1/import-export` inside the router prefix rather than relying on `settings.API_V1_STR`.
## 4. Verification Results
| Check | Result | Notes |
|---|---|---|
| React build | Passed | `tsc -b && vite build` completed. Vite warned that the main JS chunk is larger than 500 kB. |
| FastAPI OpenAPI paths | Passed | OpenAPI generated 55 paths. |
| API route parity | Failed | Multiple frontend calls do not map to backend paths or methods. |
| Feature parity with legacy Python admin | Partial | Several legacy workflows are absent, placeholders, or only implemented as HTML admin routes. |
| Local development readiness | Failed | `frontend/.env` and backend CORS settings do not match the default Vite dev setup. |
## 5. Backend API Paths Observed
The current OpenAPI schema exposes these relevant JSON paths:
```text
/api/v1/auth/admin-login
/api/v1/websites
/api/v1/admin/dashboard/stats
/api/v1/admin/questions
/api/v1/admin/templates
/api/v1/admin/tryouts/{tryout_id}/questions
/api/v1/admin/tryouts/{tryout_id}/attempts
/api/v1/admin/ai/models
/api/v1/admin/ai/generate-preview
/api/v1/admin/ai/generate-save
/api/v1/admin/ai/pending-reviews
/api/v1/admin/ai/review/{item_id}
/api/v1/import-export/preview
/api/v1/import-export/questions
/api/v1/import-export/export/questions
/api/v1/import-export/tryout-json/preview
/api/v1/import-export/tryout-json
/api/v1/tryout/
/api/v1/tryout/{tryout_id}/config
/api/v1/tryout/{tryout_id}/normalization
/api/v1/tryout/{tryout_id}/calibration-status
/api/v1/reports/student/performance
/api/v1/reports/items/analysis
/api/v1/reports/calibration/status
```
Legacy HTML-only admin paths still exist under `/admin`, including:
```text
/admin/hierarchy
/admin/question-quality
/admin/calibration-status
/admin/item-statistics
/admin/session-overview
/admin/snapshot-questions
/admin/snapshot-questions/promote-bulk
/admin/basis-items
/admin/basis-items/{basis_item_id}
/admin/basis-items/{basis_item_id}/generate
/admin/basis-items/{basis_item_id}/review-bulk
/admin/questions/{item_id}
/admin/questions/{item_id}/generate
/admin/questions/{item_id}/generate/review-bulk
```
Those HTML paths are not a substitute for React JSON API parity unless the React app intentionally navigates users back into the legacy admin UI.
## 6. Endpoint Compatibility Matrix
This matrix assumes the frontend base URL is configured as `http://localhost:8000/api/v1`, as Docker currently does. If the base URL is `http://localhost:8000`, most rows fail one level earlier because `/api/v1` is missing.
| React call | Backend status | Migration status |
|---|---|---|
| `/auth/admin-login` | Exists as `/api/v1/auth/admin-login` | OK when base URL includes `/api/v1`. |
| `/websites` | Exists as `/api/v1/websites` | OK when base URL includes `/api/v1`. |
| `/admin/dashboard/stats` | Exists as `/api/v1/admin/dashboard/stats` | Path OK, but website scoping can return empty/stale data. |
| `/tryout/` | Exists as `/api/v1/tryout/` | OK when base URL includes `/api/v1`. |
| `/admin/questions` | Exists as `/api/v1/admin/questions` | OK when base URL includes `/api/v1`. |
| `/admin/templates` | Exists as `/api/v1/admin/templates` | Path OK; verify runtime lazy relationship behavior. |
| `/admin/tryouts/{id}/questions` | Exists as `/api/v1/admin/tryouts/{tryout_id}/questions` | OK when base URL includes `/api/v1`. |
| `/admin/tryouts/{id}/attempts` | Exists as `/api/v1/admin/tryouts/{tryout_id}/attempts` | OK when base URL includes `/api/v1`. |
| `/admin/ai/models` | Exists as `/api/v1/admin/ai/models` | OK when base URL includes `/api/v1`. |
| `/admin/ai/generate-preview` | Exists as `/api/v1/admin/ai/generate-preview` | Path OK; payload includes unsupported `operator_notes` in one page but Pydantic ignores extras by default. |
| `/admin/ai/generate-save` | Exists as `/api/v1/admin/ai/generate-save` | Path OK; React passes placeholder slot and can cause duplicate/conflict behavior. |
| `/import-export/tryout-json/preview` | Exists as `/api/v1/import-export/tryout-json/preview` | OK when base URL includes `/api/v1`. |
| `/import-export/tryout-json` | Exists as `/api/v1/import-export/tryout-json` | OK when base URL includes `/api/v1`. |
| `/reports/calibration-status` | Backend has `/api/v1/reports/calibration/status?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/reports/item-analysis` | Backend has `/api/v1/reports/items/analysis?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/reports/student-performance` | Backend has `/api/v1/reports/student/performance?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/tryouts/{id}/config` | Backend has `/api/v1/tryout/{id}/config` | Broken. Uses plural `tryouts`. |
| `POST /tryouts/{id}/normalization` | Backend has `PUT /api/v1/tryout/{id}/normalization` | Broken. Wrong path, method, and payload schema. |
| `/tryouts/{id}/normalization/recalculate` | No JSON API found | Broken. |
| `/import-export/tryout-import` | No JSON API found | Broken. Should likely use `/import-export/preview` or `/import-export/questions`. |
| `/import-export/snapshot-questions/promote-bulk` | No JSON API found | Broken. Legacy equivalent is HTML form POST `/admin/snapshot-questions/promote-bulk`. |
## 7. Findings
### P0-01: Local API base URL omits `/api/v1`
Severity: P0
Category: API routing / environment configuration
Evidence:
- `frontend/src/lib/api.ts:5` defaults to `http://localhost:8000`.
- `frontend/.env:1` sets `VITE_API_URL=http://localhost:8000`.
- Backend JSON admin APIs are exposed under `/api/v1/...`.
- Docker build uses `VITE_API_BASE_URL: "http://localhost:8000/api/v1"` in `docker-compose.yml:62`, so Docker and local dev behave differently.
Impact:
- Running `npm run dev` or using the checked-in `frontend/.env` makes calls such as `/auth/admin-login`, `/websites`, and `/admin/dashboard/stats` hit the wrong backend URLs.
- Developers can see a compiling app but get login/API failures at runtime.
- Bugs may be masked in Docker while reappearing in local development or other deployments.
Recommendation:
- Standardize one env var name, preferably `VITE_API_URL`, and set it to the full API root: `http://localhost:8000/api/v1`.
- Add `frontend/.env.example` with the same value.
- Add a startup assertion or dev console warning if `VITE_API_URL` does not end in `/api/v1`.
- Consider making the Axios helper append `/api/v1` itself so pages never depend on a base URL convention.
### P0-02: System-admin website scope and React Query cache can show empty or stale tenant data
Severity: P0
Category: Multi-tenant data isolation / state management
Evidence:
- Login issues a system-admin token with `website_id=0` in `backend/app/routers/auth.py:50-55`.
- The comment says this placeholder should produce global access, but `require_website_auth` returns `auth.website_id` whenever it is not `None` in `backend/app/core/auth.py:147-150`.
- `WebsiteSelector` auto-selects the first website asynchronously in `frontend/src/components/WebsiteSelector.tsx:25-29`.
- Dashboard query key is `['dashboard-stats']` and does not include `websiteId` in `frontend/src/pages/admin/Dashboard.tsx:45-50`.
- Other query keys also omit `websiteId`, including `['tryouts']`, `['admin-questions']`, and `['ai-pending-reviews']`.
Impact:
- First-load dashboard requests can run before the selector sets `X-Website-ID`; backend may interpret the request as website `0` and return empty data.
- Switching websites can leave cached data from the prior website because React Query keys do not include the website id.
- Multi-tenant admin data can appear wrong even when the API endpoint is otherwise correct.
Recommendation:
- Fix backend system-admin semantics: use `website_id=None` for global system admin or make `website_id=0` explicitly mean global access.
- In React, gate website-scoped queries until `websiteId` is set, except for the websites list itself.
- Include `websiteId` in every website-scoped React Query key, for example `['dashboard-stats', websiteId]`.
- Invalidate website-scoped queries when `WebsiteSelector` changes.
### P0-03: Reports page calls nonexistent backend paths and omits required filters
Severity: P0
Category: API contract / reporting
Evidence:
- React calls `/reports/calibration-status`, `/reports/item-analysis`, and `/reports/student-performance` in `frontend/src/pages/admin/reports/index.tsx:14`, `:58`, and `:88`.
- Backend exposes `/reports/calibration/status`, `/reports/items/analysis`, and `/reports/student/performance` in `backend/app/routers/reports.py:68-80`, `:172-184`, and `:231-241`.
- Each backend report endpoint requires `tryout_id`.
- React report export buttons have no handlers in `frontend/src/pages/admin/reports/index.tsx:29-31`, `:73-75`, and `:103-105`.
Impact:
- All three report tabs fail at runtime.
- Even after path correction, the page needs tryout selection or route context because backend requires `tryout_id`.
- Export buttons are misleading because they do not call the export APIs.
Recommendation:
- Use the backend paths:
- `/reports/calibration/status?tryout_id={id}`
- `/reports/items/analysis?tryout_id={id}`
- `/reports/student/performance?tryout_id={id}`
- Add tryout selector/context to the Reports page.
- Wire export buttons to `/reports/.../export/{format}` endpoints.
- Render real report tables from response fields instead of placeholder text.
### P0-04: Tryout normalization page uses wrong paths, method, payload, and silent fallback
Severity: P0
Category: API contract / scoring configuration
Evidence:
- React fetches `/tryouts/{id}/config` in `frontend/src/pages/admin/tryouts/Normalization.tsx:22`.
- Backend route is `/tryout/{tryout_id}/config` in `backend/app/routers/tryouts.py:34-44`.
- React posts `/tryouts/{id}/normalization` with `{ rataan, sb, mode }` in `frontend/src/pages/admin/tryouts/Normalization.tsx:39-45`.
- Backend expects `PUT /tryout/{tryout_id}/normalization` with fields `normalization_mode`, `static_rataan`, and `static_sb` in `backend/app/routers/tryouts.py:109-120`.
- React calls `/tryouts/{id}/normalization/recalculate` in `frontend/src/pages/admin/tryouts/Normalization.tsx:53-56`, but no matching JSON API was found.
- The page catches config load failures and silently displays defaults in `frontend/src/pages/admin/tryouts/Normalization.tsx:21-26`.
Impact:
- Operators can believe they changed normalization settings when the requests actually failed or hit nonexistent endpoints.
- Silent defaults can overwrite user trust in scoring configuration by hiding missing data.
- Normalization is core to NM/NN scoring, so this is a production blocker.
Recommendation:
- Change GET to `/tryout/{id}/config`.
- Change save to `PUT /tryout/{id}/normalization`.
- Send backend schema names: `normalization_mode`, `static_rataan`, `static_sb`.
- Remove silent fallback for API failures; show an error state.
- Either add a backend recalculation endpoint or remove the button until the API exists.
### P0-05: Excel import page is wired to nonexistent endpoints
Severity: P0
Category: Import workflow / API contract
Evidence:
- React posts preview/upload to `/import-export/tryout-import` in `frontend/src/pages/admin/import/index.tsx:17-23`.
- React posts confirmation to `/import-export/snapshot-questions/promote-bulk` in `frontend/src/pages/admin/import/index.tsx:31-35`.
- Backend Excel import APIs are `/api/v1/import-export/preview` and `/api/v1/import-export/questions` in `backend/app/routers/import_export.py:53-62` and `:150-160`.
- Snapshot promotion currently exists only in the legacy HTML admin as `/admin/snapshot-questions/promote-bulk`.
Impact:
- The standalone Excel Import page cannot complete its workflow.
- Users have two import surfaces: a working JSON import modal under Tryouts and a broken Excel import page under `/admin/import`.
- The comments in the React file explicitly show uncertainty about endpoint names.
Recommendation:
- Decide whether Excel import remains in the React admin.
- If yes, wire preview to `/import-export/preview` and confirm to `/import-export/questions` with required `tryout_id`.
- If snapshot promotion is required in React, add a JSON API for selected snapshot question IDs and update the UI accordingly.
- Hide `/admin/import` until the contract is implemented.
### P1-01: Student tryout portal is missing from React
Severity: P1
Category: Feature parity / core business flow
Evidence:
- `REACT_Migration_Plan.md:73-85` describes Phase 3 Student Portal construction: tryout listing, exam session, async answer submission, state recovery, server timer, and result page.
- Current `frontend/src/App.tsx:38-66` only defines `/login` and `/admin/*` routes.
- No student session routes or pages were found under `frontend/src/pages`.
Impact:
- The React migration does not yet cover the learner-facing tryout experience.
- If the goal is full Python frontend replacement, core user-facing functionality remains unmigrated.
Recommendation:
- Add student routes for tryout listing, active session, answer submission, completion, and result summary.
- Use existing backend session APIs under `/api/v1/session`.
- Add E2E coverage for refresh recovery and server-synced timer behavior.
### P1-02: AI generation workflow is incomplete and can save invalid variants
Severity: P1
Category: AI generation / operator workflow
Evidence:
- Global AI page uses manual basis item id and comments that a real template selector is missing in `frontend/src/pages/admin/ai/Workspace.tsx:26-27`.
- Global AI page has "Discard" and "Save & Queue Review" buttons with no handlers in `frontend/src/pages/admin/ai/Workspace.tsx:138-140`.
- Tryout AI workspace saves generated questions with `slot: basisItem ? 1 : 1` in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:64-76`.
- Tryout AI workspace "Review Variants" and "Batch Generation" tabs are placeholder text in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:233-253`.
- Legacy Python admin supported batch count, operator notes, note inclusion, run history, filters, review-bulk, and variant detail pages.
Impact:
- Operators cannot reliably save variants with correct slot linkage.
- Batch generation and review parity are missing.
- Duplicate slot conflicts are likely because saved AI variants always use slot `1`.
Recommendation:
- Use the selected basis item's real `slot`, `tryout_id`, `website_id`, and source snapshot metadata.
- Add JSON APIs if needed for batch generation, run history, review filtering, and bulk review.
- Disable save buttons until all required fields are present.
- Remove or implement the global AI workspace to avoid two partial AI workflows.
### P1-03: Imported and generated HTML is rendered without sanitization
Severity: P1
Category: Security / XSS
Evidence:
- React renders question HTML with `dangerouslySetInnerHTML` in `frontend/src/pages/admin/tryouts/QuestionManagement.tsx`.
- React renders AI preview stem/options/explanation with `dangerouslySetInnerHTML` in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:180-200`.
- The migration plan explicitly calls out HTML sanitization as a security checklist item in `REACT_Migration_Plan.md:204-208`.
Impact:
- Imported Sejoli payloads or AI-generated content could inject scripts or unsafe markup into admin pages.
- Admin XSS is high impact because admins hold cross-website operational access.
Recommendation:
- Add a sanitizer such as DOMPurify.
- Create a single `SafeHtml` component and forbid direct `dangerouslySetInnerHTML` in pages.
- Sanitize on render and consider backend-side validation for stored HTML.
### P1-04: CORS config does not include the default Vite dev origin
Severity: P1
Category: Local development / environment configuration
Evidence:
- Backend `.env` allows `http://localhost:3000` and `http://localhost:8000` in `backend/.env:15`.
- Vite dev normally serves at `http://localhost:5173`.
- `REACT_Migration_Plan.md:53` explicitly calls out adding frontend dev origins.
Impact:
- Local `npm run dev` can fail with CORS errors even after the API base URL is corrected.
- Developers may mistakenly debug auth/API code when the root cause is CORS.
Recommendation:
- Add `http://localhost:5173` to `ALLOWED_ORIGINS`.
- Keep Docker/static origin and Vite dev origin both represented in `.env.example`.
### P2-01: Question quality page is a static placeholder
Severity: P2
Category: Missing feature / reporting parity
Evidence:
- `frontend/src/pages/admin/questions/QuestionQuality.tsx:14-47` displays `...` for all metrics.
- `frontend/src/pages/admin/questions/QuestionQuality.tsx:61-65` says diagnostic charts are coming soon.
- Legacy Python admin had a real `/admin/question-quality` view that computed calibrated totals and per-tryout readiness.
Impact:
- Operators lose the prior calibration diagnostics workflow.
- The page appears present but does not provide operational data.
Recommendation:
- Either wire this page to `/reports/calibration/status` per selected tryout or add a dashboard-level quality summary API.
- Replace placeholder cards with real metrics and loading/error states.
### P2-02: Tryout settings and general/security settings are placeholders
Severity: P2
Category: Missing feature / admin configuration
Evidence:
- Tryout settings page contains only placeholder text in `frontend/src/pages/admin/tryouts/TryoutSettings.tsx:14-16`.
- Security settings form has inputs and button but no mutation in `frontend/src/pages/admin/settings/index.tsx:151-176`.
- General settings tab is placeholder text in `frontend/src/pages/admin/settings/index.tsx:203-211`.
- Legacy Python admin had `/admin/password` and website management.
Impact:
- Operators cannot update tryout scoring/selection/AI settings from React.
- Password update looks available but does nothing.
Recommendation:
- Implement tryout settings using `/tryout/{id}/config` plus update endpoints for scoring mode, selection mode, AI generation, and calibration thresholds.
- Add or expose a JSON password-change endpoint, or hide Security until implemented.
- Replace "General" with concrete settings or remove the tab.
### P2-03: Hierarchy/data overview was not migrated
Severity: P2
Category: Missing feature / operator orientation
Evidence:
- Legacy Python admin exposed `/admin/hierarchy`.
- `UX_AUDIT_ADMIN_FLOW.md` and `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md` identified hierarchy visibility as important.
- Current React sidebar has Dashboard, Questions, Tryouts, Reports, Settings only in `frontend/src/layouts/AdminLayout.tsx:10-16`.
- No React hierarchy page exists.
Impact:
- Operators lose the data relationship map for Website -> Tryout -> Snapshot -> Basis Item -> AI Run -> Variant.
- This was specifically identified as important for reducing confusion after import and AI generation.
Recommendation:
- Add a React Data Overview/Hierarchy page.
- Expose a JSON hierarchy API instead of relying on legacy HTML.
- Link it from Dashboard or Tryouts, not only Settings.
### P2-04: Route structure deviates from the planned tryout-centric URL model
Severity: P2
Category: Navigation / route consistency
Evidence:
- `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md:19-28` planned singular `/admin/tryout/{tryout_id}/...` route depth.
- Current React uses plural `/admin/tryouts/:id/...` in `frontend/src/App.tsx:55-60`.
- Planned question workspace route includes question id, but current route is `/admin/tryouts/:id/questions/ai-workspace` without question id.
- TryoutLayout tabs omit Normalization from the visible tab list even though the route exists.
Impact:
- URL semantics differ from the planned hierarchy.
- AI workspace lacks clear parent question context.
- Users navigating to Normalization see a page that is not represented in the tab state.
Recommendation:
- Decide on singular or plural route convention and align docs, React routes, and links.
- Include `questionId` in AI workspace routes.
- Add a visible Normalization tab or move normalization under Settings consistently.
### P2-05: Legacy Python admin remains mounted, creating deployment ambiguity
Severity: P2
Category: Deployment / migration completeness
Evidence:
- `backend/app/main.py` still includes `admin_web_router` when admin is enabled.
- Docker serves React at port `3000` and backend at port `8000`.
- Backend still owns `/admin/*` on port `8000`; React owns `/admin/*` on port `3000`.
Impact:
- If production routing later places frontend and backend behind one host, `/admin` routing can easily point to the wrong application.
- Operators may accidentally use two different admin UIs with different feature coverage.
Recommendation:
- Define production routing explicitly:
- Frontend owns `/admin/*`.
- Backend owns `/api/v1/*`, `/docs`, `/health`, and possibly legacy admin only behind a temporary fallback path.
- Add a migration flag to disable legacy admin once React parity is reached.
### P2-06: Query invalidation and cache keys are not website-aware
Severity: P2
Category: State management / data freshness
Evidence:
- Examples: `['dashboard-stats']`, `['tryouts']`, `['admin-questions']`, `['ai-pending-reviews']`.
- The API interceptor changes `X-Website-ID` based on Zustand state, but React Query cache keys do not reflect that state.
Impact:
- After switching websites, React Query can return prior website data without refetching.
- The visible WebsiteSelector can imply a different scope than the data currently shown.
Recommendation:
- Include `websiteId` in all website-scoped query keys.
- Add a small helper for scoped keys to avoid drift.
- Consider clearing scoped query cache on logout and website switch.
### P3-01: Several buttons look actionable but do nothing
Severity: P3
Category: UX polish / trust
Evidence:
- Report export buttons have no click handlers.
- Excel "Download Template" button has no click handler.
- Global AI "Discard" and "Save & Queue Review" buttons have no click handlers.
- Settings "Update Password" button has no click handler.
Impact:
- The UI feels more complete than it is, which can mislead testers and operators.
Recommendation:
- Remove disabled/nonfunctional controls or wire them to real mutations/downloads.
- Prefer disabled buttons with explanatory tooltip only when the missing backend is intentional.
## 8. Feature Parity Checklist
| Area | Legacy Python admin | React status | Notes |
|---|---|---|---|
| Login/logout | Present | Partial | JWT login works by path only if base URL includes `/api/v1`; no remember-me equivalent. |
| Dashboard | Present | Partial | React has KPI cards, but first-load website scoping and query key issues affect data. |
| Website management | Present | Partial | React CRUD exists; confirm delete cascade semantics and query invalidation. |
| Tryout import JSON | Present | Mostly present | Modal maps to real JSON endpoints when base URL includes `/api/v1`. |
| Excel import | Present via API | Broken | React page calls nonexistent endpoints. |
| Snapshot question promotion | Present as legacy HTML | Missing JSON/React | React calls nonexistent API. |
| Global question list | Present with filters/detail | Partial | React list exists, but filters and detail page are missing. |
| Question detail | Present | Missing | No React route/page. |
| Question quality | Present | Placeholder | Static cards only. |
| Tryout list/tree | Present/planned | Partial | Accordion exists; average NM/NN and some plan details missing. |
| Tryout attempts | Present | Present basic | Filtered table exists. |
| Normalization | Present | Broken | Wrong API contract. |
| Tryout settings | Present via backend fields | Placeholder | No real form. |
| AI basis workspace | Present | Partial | Preview and single save partially exist; batch/review/run history missing. |
| AI pending review | Present | Partial | List and approve/reject exist; preview/detail missing. |
| Variant detail | Present | Missing | No React page. |
| Bulk variant review | Present | Missing | No React workflow. |
| Hierarchy/data overview | Present | Missing | Important operator context lost. |
| Reports dashboard | Present | Broken/placeholder | Wrong endpoints and no tryout filters. |
| Report exports | Present in backend | Missing in React | Buttons not wired. |
| Password update | Present in legacy HTML | Placeholder | No API/mutation. |
| Student tryout portal | Planned | Missing | No React student/session routes. |
## 9. Recommended Remediation Plan
### Phase 0: Stop the bleeding
1. Fix `VITE_API_URL` to include `/api/v1` in `frontend/.env`, Docker build args, and `.env.example`.
2. Add `http://localhost:5173` to backend CORS for Vite dev.
3. Fix system-admin website scoping so no-header system admin is global or explicitly blocked until a website is selected.
4. Gate website-scoped React queries until `websiteId` is available.
5. Add `websiteId` to all scoped query keys.
### Phase 1: Repair broken API contracts
1. Fix Reports paths and require a selected tryout.
2. Fix Normalization GET/PUT paths and payload schema.
3. Remove or fix the broken Excel import page.
4. Decide whether snapshot promotion needs a JSON API and add it if React owns the workflow.
5. Add API contract tests that compare frontend endpoint constants against OpenAPI paths.
### Phase 2: Recover feature parity
1. Implement real Tryout Settings.
2. Implement Question Detail and Variant Detail pages.
3. Implement AI run history, review filters, batch generation, and bulk review.
4. Implement Question Quality with real metrics.
5. Implement Data Overview/Hierarchy in React.
6. Wire report export buttons.
### Phase 3: Student portal
1. Add learner tryout listing.
2. Add active session page using `/session/{id}/next_item` and `/submit_answer`.
3. Add server-synced timer from `expires_at`.
4. Persist session recovery state.
5. Add completion and result pages.
### Phase 4: Migration hardening
1. Decide the final production routing split between frontend `/admin/*` and backend `/api/v1/*`.
2. Disable or move legacy Python admin once React parity is complete.
3. Add Playwright smoke tests for login, website switch, import preview, tryout drilldown, AI preview, normalization save, and reports.
4. Add a route/API smoke test that verifies every visible navigation target and button either works or is intentionally disabled.
## 10. Suggested Test Plan
Minimum tests before considering the React migration complete:
| Test | Expected result |
|---|---|
| `npm run build` | Passes with no TypeScript errors. |
| Login with local env | Hits `/api/v1/auth/admin-login`, stores token, lands on dashboard. |
| First dashboard load | Waits for or uses a real selected website, never website `0`. |
| Website switch | Dashboard, tryouts, questions, reports, and AI pending reviews refetch for the selected website. |
| Tryout JSON preview/import | Calls `/api/v1/import-export/tryout-json/preview` and `/tryout-json`; new tryouts appear. |
| Excel import | Calls real preview and import endpoints or the page is hidden. |
| Normalization save | GET `/api/v1/tryout/{id}/config`, PUT `/api/v1/tryout/{id}/normalization`, visible success/error state. |
| Reports | Requires tryout context and loads real calibration/item/student data. |
| AI preview/save | Saves with correct basis slot and displays generated variant in review queue. |
| AI review/bulk | Approve/reject/archive works and status updates are visible. |
| XSS smoke | Imported HTML and AI HTML are sanitized before rendering. |
| Student session | Start/resume/answer/complete/result works with server timer. |
## 11. Final Assessment
The migration has a good foundation: React, routing, TanStack Query, Zustand, shadcn-style components, Docker/Nginx serving, and a number of admin pages are already present. The main risk is that the UI currently looks further along than its backend integration really is.
The highest leverage next move is to stabilize the API boundary: fix the `/api/v1` base URL, align endpoint paths and methods, make website scoping deterministic, and add website-aware query keys. Once that is done, the team can safely fill the larger parity gaps without chasing confusing 404s, empty dashboards, or stale tenant data.

31
FRONTEND_MIGRATION_CUTOVER.md Normal file
View File

@@ -0,0 +1,31 @@
# Frontend Migration Cutover Notes
## Route Ownership
- React owns browser-facing admin routes under `/admin/*` and student routes under `/student/*`.
- FastAPI owns JSON APIs under `/api/v1/*`.
- The legacy Python admin remains available as fallback until React parity smoke tests are accepted.
## Local Development
- React Vite dev server: `http://127.0.0.1:5173`
- Backend API root: `http://localhost:8000/api/v1`
- Frontend API config should keep `VITE_API_URL` pointed at the FastAPI v1 root.
- System-admin tokens may be global with `website_id: null`; React sends `X-Website-ID` only when the website selector has an explicit website.
## Cutover Guardrails
- Do not disable the legacy admin until React covers import, snapshot promotion, question detail, AI review, reports, normalization, settings, and student session smoke tests.
- Avoid adding new frontend calls to legacy or nonexistent API paths. New React API calls should map to OpenAPI paths.
- Website-scoped React Query keys must include the selected website ID and should be gated until a website is selected.
- Any page rendering question HTML must use the shared `SafeHtml` component.
## Smoke Coverage Used During Migration Fix
- Admin dashboard
- Global questions list and question detail
- Data overview hierarchy
- AI review, variants, and run history
- Excel import
- Tryout questions, snapshot promotion, settings, normalization, and AI workspace
- Student tryout list, session start, next item, answer submission, completion, and result summary

612
PROJECT_UNDERSTANDING.md Normal file
View File

@@ -0,0 +1,612 @@
# Project Understanding: IRT-Powered Adaptive Question Bank System
> **Project Name:** IRT Bank Soal
> **Version:** 1.0.0
> **Last Updated:** 2026-06-15
> **Repository:** https://git.backoffice.biz.id/dwindown/yellow-bank-soal
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Project Purpose](#project-purpose)
3. [Tech Stack](#tech-stack)
4. [Project Structure](#project-structure)
5. [Core Concepts](#core-concepts)
6. [Data Models](#data-models)
7. [API Endpoints](#api-endpoints)
8. [Key Services](#key-services)
9. [Scoring Formulas](#scoring-formulas)
10. [Configuration](#configuration)
11. [Workflows](#workflows)
12. [Deployment](#deployment)
---
## Executive Summary
This is a **FastAPI-based backend system** for managing adaptive assessment/tryout exams with sophisticated scoring capabilities. The system supports both **Classical Test Theory (CTT)** and **Item Response Theory (IRT)** scoring methods, with multi-website support for WordPress integration.
### Key Features
| Feature | Description |
|---------|-------------|
| **CTT Scoring** | Classical Test Theory with exact Excel formula compatibility |
| **IRT Support** | Item Response Theory (1PL Rasch model) for adaptive testing |
| **Multi-Site** | Single backend serving multiple WordPress sites |
| **AI Generation** | Automatic question variant generation via OpenRouter |
| **Excel Import/Export** | Bulk import/export questions from Excel files |
| **Adaptive Testing** | Computer Adaptive Testing (CAT) with theta estimation |
| **Normalization** | Static, dynamic, or hybrid score normalization |
---
## Project Purpose
The system replaces traditional fixed-difficulty exams with an **adaptive question bank** that:
1. **Measures student ability accurately** using IRT theta estimation
2. **Provides comparable scores** across different exam sessions via normalization
3. **Generates new questions** using AI when needed
4. **Integrates with WordPress** LMS platforms for student access
5. **Reduces exam fraud** by delivering different question variants to each student
---
## Tech Stack
### Core Technologies
```
Framework: FastAPI >= 0.104.1
Server: Uvicorn >= 0.24.0
Database: PostgreSQL + SQLAlchemy 2.0 (async)
ORM: SQLAlchemy >= 2.0.23
Driver: asyncpg >= 0.29.0
Migrations: Alembic >= 1.13.0
Validation: Pydantic >= 2.5.0
```
### Data Processing
```
Excel: openpyxl >= 3.1.2, pandas >= 2.1.4
Math/Science: numpy >= 1.26.2, scipy >= 1.11.4
```
### External Integrations
```
AI: OpenAI >= 1.6.1 (OpenRouter API)
Task Queue: Celery >= 5.3.6, Redis >= 5.0.1
Admin Panel: FastAPI-Admin >= 1.0.0
```
---
## Project Structure
```
yellow-bank-soal/
├── app/
│ ├── __init__.py
│ ├── main.py # FastAPI app entry point
│ ├── admin.py # FastAPI Admin configuration
│ ├── admin_web.py # Admin web interface
│ ├── database.py # Database configuration & session
│ │
│ ├── api/
│ │ └── v1/
│ │ ├── __init__.py
│ │ └── session.py # Adaptive session endpoints
│ │
│ ├── core/
│ │ ├── __init__.py
│ │ ├── auth.py # Authentication & authorization
│ │ ├── config.py # Settings from environment
│ │ └── rate_limit.py # Rate limiting
│ │
│ ├── models/
│ │ ├── __init__.py
│ │ ├── ai_generation_run.py
│ │ ├── item.py # Question items
│ │ ├── report_schedule.py
│ │ ├── session.py # Student tryout sessions
│ │ ├── tryout.py # Tryout configurations
│ │ ├── tryout_import_snapshot.py
│ │ ├── tryout_snapshot_question.py
│ │ ├── tryout_stats.py # Normalization statistics
│ │ ├── user.py
│ │ ├── user_answer.py # Student responses
│ │ └── website.py
│ │
│ ├── routers/
│ │ ├── __init__.py
│ │ ├── admin.py # Admin-only endpoints
│ │ ├── ai.py # AI generation endpoints
│ │ ├── import_export.py # Excel import/export
│ │ ├── reports.py # Report generation
│ │ ├── sessions.py # Session management
│ │ ├── tryouts.py # Tryout configuration
│ │ └── wordpress.py # WordPress integration
│ │
│ ├── schemas/ # Pydantic request/response models
│ │ ├── __init__.py
│ │ ├── ai.py
│ │ ├── report.py
│ │ ├── session.py
│ │ ├── tryout.py
│ │ └── wordpress.py
│ │
│ └── services/
│ ├── __init__.py
│ ├── ai_generation.py # OpenRouter integration
│ ├── cat_selection.py # Computer Adaptive Testing
│ ├── config_management.py
│ ├── ctt_scoring.py # CTT scoring engine
│ ├── excel_import.py # Excel parsing
│ ├── irt_calibration.py # IRT calibration
│ ├── normalization.py
│ ├── reporting.py
│ ├── tryout_json_import.py
│ └── wordpress_auth.py
│
├── alembic/ # Database migrations
│ ├── env.py
│ ├── script.py.mako
│ └── versions/
│
├── tests/ # Unit & integration tests
│ ├── test_auth_scope.py
│ ├── test_auth_tokens.py
│ ├── test_model_mappings.py
│ ├── test_normalization.py
│ ├── test_operational_hardening.py
│ ├── test_route_wiring.py
│ ├── test_security_regressions.py
│ └── test_tryout_json_import.py
│
├── requirements.txt
├── alembic.ini
├── irt_1pl_mle.py # Standalone IRT MLE script
├── PRD.md # Product Requirements Document
├── project-brief.md # Technical specification
└── handoff.md # Project handoff context
```
---
## Core Concepts
### 1. Tryout (Exam)
A **Tryout** represents a complete exam/test with configurable behavior:
```python
scoring_mode: "ctt" | "irt" | "hybrid"
selection_mode: "fixed" | "adaptive" | "hybrid"
normalization_mode: "static" | "dynamic" | "hybrid"
```
### 2. Item (Question)
An **Item** represents a single question with:
- **Content**: stem (question text), options (A/B/C/D), correct_answer
- **CTT Parameters**: p-value (difficulty), bobot (weight)
- **IRT Parameters**: b (difficulty), se (standard error)
- **Metadata**: slot position, difficulty level, AI generation info
### 3. Session (Student Attempt)
A **Session** tracks a student's attempt:
- Links student (`wp_user_id`) to a Tryout
- Records all answers via `UserAnswer` records
- Stores computed scores: NM, NN, theta
### 4. Website (Multi-Tenant)
The system supports **multiple WordPress websites** from a single backend:
- Each website has isolated data
- Authenticated via `X-Website-ID` header
- WordPress JWT tokens for authentication
---
## Data Models
### Entity Relationship Diagram
```mermaid
erDiagram
Website ||--o{ Tryout : "hosts"
Website ||--o{ User : "contains"
Website ||--o{ Session : "serves"
Website ||--o{ Item : "contains"
Tryout ||--o{ Item : "contains"
Tryout ||--o{ Session : "has"
Tryout ||--o{ TryoutStats : "tracks"
Session ||--o{ UserAnswer : "contains"
Session ||--o{ User : "belongs to"
Item ||--o{ UserAnswer : "answered by"
Item ||--o{ Item : "has variants"
AIGenerationRun ||--o{ Item : "generates"
```
### Model Summary
| Model | Purpose | Key Fields |
|-------|---------|------------|
| `Website` | Multi-tenant isolation | domain, wordpress_url |
| `User` | WordPress user mapping | wp_user_id, website_id |
| `Tryout` | Exam configuration | scoring_mode, selection_mode, normalization_mode |
| `Item` | Question | stem, options, ctt_p, ctt_bobot, irt_b, irt_se |
| `Session` | Student attempt | session_id, NM, NN, theta |
| `UserAnswer` | Single response | response, is_correct, bobot_earned |
| `TryoutStats` | Normalization data | participant_count, rataan, sb |
| `AIGenerationRun` | AI generation batch | model, status, items_generated |
---
## API Endpoints
### Public API (via `/api/v1`)
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/tryout/{tryout_id}/config` | Get tryout configuration |
| `PUT` | `/tryout/{tryout_id}/normalization` | Update normalization settings |
| `GET` | `/tryout/` | List tryouts for website |
| `GET` | `/tryout/{tryout_id}/calibration-status` | Get IRT calibration status |
| `POST` | `/tryout/{tryout_id}/calibrate` | Trigger IRT calibration |
| `POST` | `/session/` | Create new session |
| `GET` | `/session/{session_id}` | Get session details |
| `POST` | `/session/{session_id}/complete` | Submit answers, calculate scores |
### Admin API (requires admin role)
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/ai/generate` | Generate AI questions |
| `POST` | `/import/excel` | Import questions from Excel |
| `GET` | `/export/excel/{tryout_id}` | Export questions to Excel |
| `GET` | `/reports/*` | Generate various reports |
### Adaptive Session API (via `/api/v1/session`)
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/adaptive/start` | Start adaptive session |
| `POST` | `/adaptive/respond` | Submit answer, get next item |
| `POST` | `/adaptive/complete` | Complete adaptive session |
---
## Key Services
### 1. CTT Scoring Engine (`ctt_scoring.py`)
Implements Classical Test Theory scoring with exact Excel formulas.
**Key Functions:**
- `calculate_ctt_p()` - Difficulty: p = Σ Benar / Total Peserta
- `calculate_ctt_bobot()` - Weight: Bobot = 1 - p
- `calculate_ctt_nm()` - Raw Score: NM = (Total_Bobot / Total_Bobot_Max) × 1000
- `calculate_ctt_nn()` - Normalized: NN = 500 + 100 × ((NM - Rataan) / SB)
- `categorize_difficulty()` - Categorize by p-value
- `update_tryout_stats()` - Incrementally update normalization stats
### 2. IRT Calibration (`irt_calibration.py`)
Implements Item Response Theory (1PL Rasch model) for adaptive testing.
**Key Functions:**
- `estimate_theta_mle()` - MLE theta estimation for students
- `estimate_b()` - IRT difficulty calibration for items
- `calibrate_item()` - Calibrate single item from response data
- `calibrate_all()` - Batch calibrate all items in tryout
- `calculate_fisher_information()` - Fisher information for item selection
**Parameters:**
- θ (theta): Student ability [-3, +3]
- b: Item difficulty [-3, +3]
- Probability: P(θ) = 1 / (1 + exp(-(θ - b)))
### 3. AI Generation (`ai_generation.py`)
Generates question variants using OpenRouter API.
**Key Functions:**
- `generate_question()` - Generate single question via OpenRouter
- `generate_questions_batch()` - Generate multiple questions
- `save_ai_question()` - Save generated question to database
- `check_cache_reuse()` - Check for reusable similar questions
**Models Supported:**
- Qwen 2.5 32B (balanced)
- Mistral Small (low cost)
- Llama 3.3 70B (premium)
### 4. Excel Import/Export (`excel_import.py`)
Bulk import/export questions from Excel files.
**Key Functions:**
- `parse_excel_import()` - Parse Excel file to items
- `bulk_insert_items()` - Insert parsed items to database
- `export_questions_to_excel()` - Export tryout to Excel
### 5. CAT Selection (`cat_selection.py`)
Computer Adaptive Testing item selection algorithm.
**Key Functions:**
- `select_next_item()` - Select next item based on theta estimate
- `calculate_theta_update()` - Update theta after response
- `check_termination()` - Check if test should end
---
## Scoring Formulas
### CTT (Classical Test Theory)
Based on exact client Excel formulas:
```python
# STEP 1: Tingkat Kesukaran (p-value)
p = Σ Benar / Total Peserta
# STEP 2: Bobot (Weight)
Bobot = 1 - p
# STEP 3: Total Benar per Siswa
Total_Benar = count of correct answers
# STEP 4: Total Bobot Earned per Siswa
Total_Bobot_Siswa = Σ Bobot for each correct answer
# STEP 5: Nilai Mentah (Raw Score)
NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000
# STEP 6: Nilai Nasional (Normalized Score)
NN = 500 + 100 × ((NM - Rataan) / SB)
```
### IRT (Item Response Theory)
1PL Rasch Model:
```python
# Probability of correct response
P(θ, b) = 1 / (1 + exp(-(θ - b)))
# Log-likelihood for MLE
LL = Σ [u_i × log(P) + (1-u_i) × log(1-P)]
# Theta estimation via MLE
θ_mle = argmax_θ LL(θ)
```
### Difficulty Categories (CTT Standard)
| p-value | Category | Description |
|---------|----------|-------------|
| p < 0.30 | Sulit | Difficult |
| 0.30 ≤ p ≤ 0.70 | Sedang | Medium |
| p > 0.70 | Mudah | Easy |
---
## Configuration
### Environment Variables
```bash
# Database
DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/irt_bank_soal
# FastAPI
SECRET_KEY=your-secret-key-here
ENVIRONMENT=development # development, staging, production
ENABLE_ADMIN=true
ADMIN_USERNAME=admin
ADMIN_PASSWORD=your-password
# OpenRouter (AI)
OPENROUTER_API_KEY=sk-or-v1-xxx
OPENROUTER_MODEL_QWEN=qwen/qwen2.5-32b-instruct
OPENROUTER_MODEL_CHEAP=mistralai/mistral-small-2603
OPENROUTER_MODEL_LLAMA=meta-llama/llama-3.3-70b-instruct
# Redis/Celery
REDIS_URL=redis://localhost:6379/0
CELERY_BROKER_URL=redis://localhost:6379/0
# CORS
ALLOWED_ORIGINS=http://localhost:3000,https://yourdomain.com
```
### Tryout Configuration Options
```python
# Scoring Mode
scoring_mode = "ctt" # Classical Test Theory
scoring_mode = "irt" # Item Response Theory
scoring_mode = "hybrid" # Both (IRT for calibration, CTT for scoring)
# Selection Mode
selection_mode = "fixed" # Fixed order questions
selection_mode = "adaptive" # Computer Adaptive Testing
selection_mode = "hybrid" # Start fixed, switch to adaptive
# Normalization Mode
normalization_mode = "static" # Use hardcoded rataan/sb
normalization_mode = "dynamic" # Calculate from participant data
normalization_mode = "hybrid" # Dynamic when sufficient data
```
---
## Workflows
### 1. Student Taking a Tryout
```mermaid
sequenceDiagram
participant S as Student
participant API as FastAPI
participant WP as WordPress
S->>API: POST /session/ (start session)
API-->>S: session_id
loop For each question
S->>API: GET /session/{id}/next-item
API-->>S: Question data
S->>API: POST /session/{id}/answer
API-->>S: Next question or completion
end
S->>API: POST /session/{id}/complete
API-->>S: NM, NN scores
```
### 2. Admin Importing Questions
```mermaid
flowchart TD
A[Upload Excel File] --> B[Parse Excel]
B --> C{Validate Structure}
C -->|Invalid| D[Return Error]
C -->|Valid| E[Calculate CTT p & bobot]
E --> F[Bulk Insert Items]
F --> G[Commit to Database]
G --> H[Return Import Summary]
```
### 3. AI Question Generation
```mermaid
flowchart TD
A[Request Generation] --> B{Check Cache}
B -->|Found similar| C[Return Cached]
B -->|Not found| D[Call OpenRouter API]
D --> E{Parse Response}
E -->|Parse Error| F[Return Error]
E -->|Success| G[Save to Database]
G --> H[Return Generated Item]
```
### 4. IRT Calibration
```mermaid
flowchart TD
A[Collect Responses] --> B{Enough Data?}
B -->|No| C[Wait for more]
B -->|Yes| D[For each Item]
D --> E[Get Response Matrix]
E --> F[Estimate b via MLE]
F --> G[Calculate Standard Error]
G --> H[Update Item]
H --> D
D --> I[Mark Items Calibrated]
```
---
## Deployment
### Requirements
- Python 3.10+
- PostgreSQL 14+
- Redis 6+ (for Celery)
- Nginx (reverse proxy)
- aaPanel with Python Manager (recommended)
### Running the Application
```bash
# Install dependencies
pip install -r requirements.txt
# Run migrations
alembic upgrade head
# Start server
uvicorn app.main:app --host 0.0.0.0 --port 8000
# Or with reload (development)
uvicorn app.main:app --reload
```
### Running Tests
```bash
pytest tests/ -v
```
### API Documentation
- Swagger UI: `http://localhost:8000/docs`
- ReDoc: `http://localhost:8000/redoc`
- OpenAPI JSON: `http://localhost:8000/openapi.json`
---
## Security Considerations
### Authentication
- WordPress JWT tokens for user authentication
- `X-Website-ID` header for multi-tenant isolation
- Admin routes protected by admin role check
### Production Hardening
1. **SECRET_KEY** must be set to a strong, unique value
2. **ADMIN_PASSWORD** must not be the default
3. **CORS** origins should be explicitly configured
4. **Database** connections should use SSL in production
5. **Rate limiting** enabled for AI generation endpoints
---
## Glossary
| Term | Definition |
|------|------------|
| **Tryout** | An exam/test assessment |
| **Item** | A single question in a tryout |
| **Session** | A student's attempt at a tryout |
| **CTT** | Classical Test Theory - traditional scoring |
| **IRT** | Item Response Theory - modern adaptive scoring |
| **NM** | Nilai Mentah - raw score [0-1000] |
| **NN** | Nilai Nasional - normalized score [0-1000] |
| **θ (theta)** | IRT ability estimate [-3 to +3] |
| **b** | IRT item difficulty [-3 to +3] |
| **p-value** | CTT proportion correct [0 to 1] |
| **Bobot** | CTT weight (1 - p) |
| **Rataan** | Mean (Indonesian) |
| **SB** | Simpangan Baku - Standard Deviation |
| **CAT** | Computer Adaptive Testing |
| **MLE** | Maximum Likelihood Estimation |
---
## References
- [PRD.md](./PRD.md) - Complete Product Requirements Document
- [project-brief.md](./project-brief.md) - Original technical specification
- [FastAPI Documentation](https://fastapi.tiangolo.com/)
- [SQLAlchemy 2.0](https://docs.sqlalchemy.org/en/20/)
- [Item Response Theory](https://en.wikipedia.org/wiki/Item_response_theory)

393
REACT_Migration_Plan.md Normal file
View File

@@ -0,0 +1,393 @@
# **Frontend Migration Plan: React + Tailwind + shadcn/ui**
**Project:** IRT Bank Soal
**Target Architecture:** Decoupled (FastAPI Backend + React SPA Frontend)
**Date Prepared:** 2026-06-17
## **1. Executive Summary**
This document outlines the strategic plan to migrate the user interface of the **IRT Bank Soal** project from a server-rendered application (FastAPI-Admin & Jinja/Web) to a modern Single Page Application (SPA) architecture using **React, Tailwind CSS, and shadcn/ui components**. This migration aims to significantly enhance UI responsiveness—especially for the Computer Adaptive Testing (CAT) feature—and streamline the development of future interactive features.
## **2. Target Frontend Tech Stack**
| Category | Primary Technology | Rationale |
| :---- | :---- | :---- |
| **Framework** | Vite + React (TypeScript) | Fast build times, responsive Hot Module Replacement (HMR), industry standard for SPAs. |
| **Styling** | Tailwind CSS | Utility-first styling approach, accelerating the UI slicing process. |
| **UI Components** | shadcn/ui | Accessible headless components (Radix UI) that are copy-pasteable and fully customizable via Tailwind. |
| **Data Fetching** | TanStack Query (React Query) | Server-state management, caching, automatic retry logic for unstable connections, and elegant loading/error state handling. |
| **State Management** | Zustand / React Context | Lightweight client-state storage (e.g., WordPress JWT tokens, X-Website-ID, UI themes). Employs persist middleware for crash/reload recovery. |
| **Routing** | React Router DOM | Seamless navigation using **BrowserRouter** for clean URLs. Enables robust **Nested Routing** required for the new hierarchical admin structure. |
| **Form Handling** | React Hook Form + Zod | Strict input validation and type safety (aligning perfectly with Pydantic models in the backend). |
## **3. Repository Restructuring (Monorepo Approach)**
It is highly recommended to use a single repository (monorepo) structure with separate folders for the frontend and backend to simplify version control.
```
yellow-bank-soal/
├── backend/ # Existing FastAPI application folder (app/, alembic/, etc.)
│ ├── Dockerfile # Patched Backend Dockerfile
│ ├── app/
│ ├── requirements.txt
│ └── ...
├── frontend/ # New React application
│ ├── Dockerfile # New Frontend Dockerfile (Multi-stage Nginx)
│ ├── .env.example # VITE_API_BASE_URL references
│ ├── src/
│ ├── package.json
│ └── ...
└── docker-compose.yml # Orchestrates ALL services (API, UI, Redis, Celery)
```
## **4. Migration Phases**
The migration will be divided into 4 sequential phases to minimize disruption to the existing system.
### **Phase 1: Backend Preparation (API Readiness)**
Focus on preparing FastAPI to securely communicate with an external React application.
1. **CORS Configuration:** Update `ALLOWED_ORIGINS` in `app/core/config.py` to permit frontend origins (e.g., `http://localhost:5173` for development and the target production domain).
2. **Endpoint Audit & Restructuring:** Ensure all administrative functionalities are exposed via RESTful API endpoints. **Crucially, restructure the API endpoints to match the new UI hierarchy** (e.g., `GET /api/v1/admin/tryouts/{id}/questions` instead of a standalone `/questions` endpoint).
3. **Data Scrubbing:** Strictly ensure that endpoints like `/api/v1/session/{id}/next-item` omit sensitive fields such as `correct_answer`, `ctt_p`, and `irt_b` from the response payload to prevent client-side cheating.
### **Phase 2: Frontend Scaffolding & Design System**
Focus on project initialization and establishing the UI foundation.
1. **Vite Initialization:** Run `npm create vite@latest frontend -- --template react-ts`.
2. **Environment Variables Setup:** Define `VITE_API_BASE_URL` in frontend `.env` so Axios knows exactly where to route the API requests across different environments.
3. **Tailwind & shadcn Setup:**
* Install Tailwind CSS and configure `tailwind.config.js`.
* Run `npx shadcn-ui@latest init` to set up base CSS variables and utility functions.
4. **Core Components (shadcn):** Install frequently used foundational components:
`npx shadcn-ui@latest add button card input label dialog alert table tabs progress radio-group toast collapsible accordion badge`
5. **API Client Setup:** Configure an Axios instance to universally append:
* The Authorization Bearer Token (from WordPress).
* The `X-Website-ID` header for multi-tenant isolation.
### **Phase 3: Student Portal Construction (Core Business Flow)**
Focus on the primary user interaction: executing the adaptive tryout.
1. **Tryout Listing:** A view displaying available tryouts for the user based on their X-Website-ID.
2. **Exam Dashboard (Session) & Asynchronous Forms:**
* **UI:** Utilize shadcn's Card for the question area, RadioGroup for options, and a Progress bar for tracking exam status.
* **No-Reload Submissions (AJAX):** Completely replace legacy PHP `$_POST` form actions. Forms will use `e.preventDefault()` to stop browser reloads. Submissions to `/adaptive/respond` will be handled asynchronously in the background via TanStack Query.
* **Instant Feedback (Toast):** Upon submitting an answer, the UI will instantly display a non-intrusive shadcn Toast notification and smoothly render the next question.
3. **State Recovery & Timer Security (Crucial):**
* **Anti-Refresh:** Use Zustand's persist middleware to save the current session ID and active question ID into localStorage. If the user accidentally hits F5 or closes the tab, the React app can instantly resume the exam state.
* **Server-Synced Timer:** Do not rely on the client's `Date.now()`. Fetch the exam's exact server-side end time (`expires_at`) from the FastAPI backend and calculate the countdown on the frontend based on that fixed timestamp.
4. **Result Page:** A summary view to display the Raw Score (NM) and Normalized Score (NN) upon exam completion.
### **Phase 4: Admin Panel Construction (Hierarchy-Driven Redesign)**
Based on the `ADMIN_TRYOUT_RESTRUCTURE_PLAN`, the admin UI will shift from a scattered menu to a deeply nested, Tryout-centric navigation leveraging React Router DOM.
1. **Tree-Based Root Navigation (`/admin/tryouts`):**
* Replace standard data tables with an interactive Tree/Collapsible layout grouped by Websites.
* Implement **Stat Cards** inline for each Tryout (showing NM, NN averages, and Calibration Progress).
* Add a global `[+ Import Tryout]` button/modal directly in the header of this tree view.
2. **Nested Tryout Workspaces:**
* Utilize React Router's nested routing to build drill-down pages maintaining the parent Tryout context:
* `/admin/tryout/:id/attempts` (filtered DataTable of sessions)
* `/admin/tryout/:id/normalization` (settings form to update NM/NN targets)
* `/admin/tryout/:id/questions` (filtered DataTable of basis questions)
3. **Question AI Workspace (`/admin/tryout/:id/questions/:questionId/workspace`):**
* Build a dedicated tabbed interface using shadcn Tabs (Generate, Review, Batch).
* Provide seamless integration with the OpenRouter AI generation API endpoints.
## **5. UI/UX Design Guidelines (Human-Centric Approach)**
To resolve the "developer-centric" nature of the legacy system, the frontend must adopt a "Human POV" ensuring the dashboard is intuitive, workflow-oriented, and actionable.
### **5.1. Dashboard Layout Re-imagination**
Shift away from displaying raw database counts. The new Home Dashboard (`/admin/dashboard`) must include:
* **Personalized Greeting:** E.g., "Good Morning, Admin! Last login..."
* **Actionable System Overview:** Display meaningful KPIs (Active Tryouts, Average Scores, Completion Rates).
* **Attention Needed (Alerts):** A dedicated section highlighting urgent tasks (e.g., "23 questions need calibration", "5 AI questions pending review").
* **Quick Actions:** Prominent buttons for daily workflows (`[Import Tryout]`, `[Generate AI]`).
### **5.2. Visual Indicators & Color Coding**
Use Tailwind CSS classes to create consistent, semantic color coding across the application.
* **Difficulty Badges:**
* **Easy (p > 0.70):** Green (`bg-green-100 text-green-800`)
* **Medium (0.30 ≤ p ≤ 0.70):** Yellow (`bg-yellow-100 text-yellow-800`)
* **Hard (p < 0.30):** Red (`bg-red-100 text-red-800`)
* **Calibration Status:**
* **Ready (≥90%):** Green Checkmark icon / Progress Bar
* **Partial (50-89%):** Yellow Warning icon / Progress Bar
* **Needs Data (<50%):** Red Cross icon / Progress Bar
### **5.3. Terminology Mapping (System to UI)**
While the FastAPI backend retains technical database names, the React frontend must translate these terms into human-readable labels:
| System Term (Backend) | UI Label (Frontend) | Context |
| :---- | :---- | :---- |
| Session | Student Attempt | Table headers, Navigation |
| Calibration | Question Quality | Dashboards, Menus |
| IRT | Adaptive Scoring | Tryout Settings |
| CTT | Standard Scoring | Tryout Settings |
| NM (Nilai Mentah) | Raw Score | Reports, Attempt Lists |
| NN (Nilai Nasional) | Normalized Score | Reports, Attempt Lists |
| p-value | Difficulty Score | Question Data Grids |
## **6. Key shadcn/ui Component Mapping**
| Feature Requirement | Recommended shadcn/ui Component | Usage / Context |
| :---- | :---- | :---- |
| Tryouts Hierarchy Map | Collapsible, Accordion | Creates the nested tree structure for Websites -> Tryouts list. |
| Tryout Stat Cards | Card, Badge, Progress | Displays quick metrics (Participants, NM avg, Calibration progress) in the tree. |
| Visual Indicators | Badge | Colored badges for difficulty levels and calibration status. |
| Nested Navigation | Tabs | Switches between "Generate", "Review", and "Batch" in the Question Workspace. |
| Question & Options View | Card, RadioGroup | Wraps the question stem and manages A/B/C/D selections. |
| Form Feedback & Notices | Toast | Displays non-blocking success/error messages asynchronously without page reloads. |
| Alerts / Errors | Alert | Displays prominent API error messages (e.g., lost internet connection). |
| Submit Confirmation | AlertDialog | Prompts the user before calling `/session/complete` to prevent accidental submissions. |
| Item/Student Roster | DataTable (Table) | Renders data grids for Attempts and Questions with server-side sort/filter capabilities. |
| Tryout Settings | Switch, Form | Configures CTT/IRT parameters and Normalization targets. |
## **7. Security Checklist**
* [ ] **Client-Side Authorization:** Implement Route Guards via React Router to prevent unauthorized access to active exam sessions and admin pages.
* [ ] **Sensitive Data Protection:** Ensure the client-state manager (Zustand) never stores data the student shouldn't see (e.g., b values, p-values, or correct answers).
* [ ] **HTML Sanitization:** If the question text (stem) contains rich HTML (from an editor), process it through a library like dompurify before rendering it via `dangerouslySetInnerHTML` to prevent XSS attacks.
## **8. References**
* [Vite Documentation](https://vitejs.dev/)
* [Tailwind CSS Documentation](https://tailwindcss.com/)
* [shadcn/ui Documentation](https://ui.shadcn.com/)
* [TanStack Query Documentation](https://tanstack.com/query/latest)
* [React Router Documentation](https://reactrouter.com/)
## **9. Deployment & Routing Strategy**
Since the React application operates as a standalone frontend and relies on WordPress exclusively via API for data integration, the URL routing must be handled cleanly without interference from WordPress.
### **Chosen Strategy: BrowserRouter (HTML5 History API)**
* **URL Format:** `https://app.domain.com/session/123` (Clean, SEO-friendly URLs)
* **Rationale:** As a standalone deployment, there are no conflicts with WordPress's internal rewrite rules or `.htaccess`. BrowserRouter provides the standard React routing experience.
* **Server Requirement (Crucial):** To prevent 404 Not Found errors when users manually refresh a page (e.g., hitting F5 on `/session/123`), the web server hosting the built React files must be configured to redirect all missing paths back to `index.html`.
## **10. Docker & Containerization Strategy**
To support the decoupled architecture and the existing AI features, the deployment process will utilize Docker Compose to orchestrate the backend, frontend, Redis, and Celery workers.
### **10.1. Backend Dockerfile Patch (`backend/Dockerfile`)**
The existing Dockerfile is well-structured but needs a minor patch for production. The `--reload` flag must be removed as it consumes excessive resources.
```dockerfile
# ... existing setup ...
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# PATCH: Removed the '--reload' flag for production readiness
CMD ["sh", "-c", "alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000"]
```
### **10.2. Frontend Dockerfile (`frontend/Dockerfile`)**
A new multi-stage Dockerfile is required for the React application. Stage 1 compiles the application using Node.js, and Stage 2 serves the static files using a lightweight Nginx server configured for BrowserRouter.
```dockerfile
# Stage 1: Build the React application
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
# Using ARG to inject API URL during the build phase
ARG VITE_API_BASE_URL
ENV VITE_API_BASE_URL=$VITE_API_BASE_URL
RUN npm run build
# Stage 2: Serve with Nginx
FROM nginx:alpine
# Copy the built assets from Stage 1
COPY --from=builder /app/dist /usr/share/nginx/html
# Add a custom Nginx configuration to support BrowserRouter (catch-all rule)
RUN echo 'server { \
listen 80; \
location / { \
root /usr/share/nginx/html; \
index index.html index.htm; \
try_files $uri $uri/ /index.html; \
} \
}' > /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
### **10.3. Full Orchestration (`docker-compose.yml`)**
Place this at the root of the monorepo to run the complete ecosystem. **This includes Redis and Celery which are required for the OpenRouter AI generation feature outlined in the PRD.**
```yaml
version: '3.8'
services:
# 1. FastAPI Backend
backend:
build:
context: ./backend
ports:
- "8000:8000"
env_file:
- ./backend/.env
depends_on:
- redis
restart: unless-stopped
# 2. Redis Message Broker (Required by Celery)
redis:
image: redis:7-alpine
ports:
- "6379:6379"
restart: unless-stopped
# 3. Celery Worker (For AI Generation Background Tasks)
celery_worker:
build:
context: ./backend
command: ["celery", "-A", "app.core.celery_app", "worker", "--loglevel=info"]
env_file:
- ./backend/.env
depends_on:
- backend
- redis
restart: unless-stopped
# 4. React Frontend SPA
frontend:
build:
context: ./frontend
args:
# Inject the backend API URL into the React build
VITE_API_BASE_URL: "https://api.yourdomain.com/api/v1"
ports:
- "80:80"
depends_on:
- backend
restart: unless-stopped
```
## **11. Feasibility Assessment**
*Assessed: 2026-06-17*
### **11.1 Current State Summary**
| Item | Status | Notes |
| :---- | :---- | :---- |
| FastAPI Backend | ✅ Complete | Well-structured with routers, models, services |
| CORS Configuration | ✅ Configured | `ALLOWED_ORIGINS` in `app/core/config.py` - just needs to add Vite dev server port (`localhost:5173`) |
| Session API Endpoints | ✅ Complete | `GET /api/v1/session/{id}/next_item`, `POST /session/{id}/submit_answer`, `POST /session/{id}/complete` |
| Tryout API Endpoints | ✅ Complete | `GET /tryout`, `GET /tryout/{id}/config`, `PUT /tryout/{id}/normalization` |
| Admin Endpoints | ✅ Complete | Calibration, AI toggle, normalization reset |
| Redis/Celery Setup | ✅ In docker-compose.dev.yml | Used for AI generation |
| WordPress Auth Integration | ✅ In place | `X-Website-ID` header support via `app/core/auth.py` |
| Data Scrubbing (Security) | ✅ Done | Session endpoints already omit sensitive fields (`correct_answer`, `ctt_p`, `irt_b`) |
### **11.2 Identified Gaps**
| Item | Status | Action Required |
| :---- | :---- | :---- |
| No `frontend/` folder | ❌ Missing | Create React app from scratch |
| No root `docker-compose.yml` | ❌ Missing | Currently only `docker-compose.dev.yml` exists |
| Backend Dockerfile location | ⚠️ Inconsistent | Current `Dockerfile` is at root, needs to be moved to `backend/` |
| `expires_at` in session | ⚠️ Not found | Session model may not have server-side end time for timer sync |
| Nested admin routes | ⚠️ Flat structure | Current admin routes are flat, need hierarchical restructuring |
| Monorepo structure | ⚠️ Not set up | Root currently IS the backend, needs folder restructuring |
### **11.3 Detailed Gap Analysis**
#### **Gap 1: Session Timer Implementation**
The plan mentions fetching `expires_at` from the backend for server-synced timers. However, the Session model (`app/models/session.py`) does not have an explicit `expires_at` field. Only `start_time` exists.
**Action needed:** Add `expires_at` field to Session model and update session creation endpoint.
#### **Gap 2: Monorepo Structure**
Current repository layout:
```
yellow-bank-soal/ # Root = backend
├── app/ # FastAPI app
├── Dockerfile # Backend Dockerfile at root
├── docker-compose.dev.yml # Dev setup
```
Plan requires:
```
yellow-bank-soal/ # Root = monorepo
├── backend/ # Move current root to backend/
├── frontend/ # New React app
└── docker-compose.yml # New orchestration
```
**Action needed:** Significant file reorganization - move existing files to `backend/` subfolder.
#### **Gap 3: Nested Admin Routes**
Current admin endpoints are flat:
- `/api/v1/admin/{tryout_id}/calibrate` ✅
- `/api/v1/tryout/{tryout_id}/config` ✅
Plan requires nested structure:
- `/api/v1/admin/tryouts/{id}/questions` ❌ (doesn't exist)
- `/api/v1/admin/tryout/{id}/attempts` ❌ (doesn't exist)
**Action needed:** Create new nested router structure in `app/routers/admin/`.
### **11.4 Feasibility Score: 7/10**
| Category | Score | Notes |
| :---- | :---- | :---- |
| Backend API Readiness | 8/10 | Core endpoints exist, minor gaps in session expiration |
| Infrastructure | 6/10 | Needs restructuring for monorepo |
| Auth Integration | 9/10 | WordPress JWT + X-Website-ID already in place |
| Docker Setup | 5/10 | Need new docker-compose.yml + frontend Dockerfile |
| Data Security | 9/10 | Already scrubbing sensitive fields |
### **11.5 Recommended Execution Order**
1. **Phase 0: Repository Restructuring** (High Impact)
- Move current root contents → `backend/`
- Create `frontend/` with Vite scaffold
- Update Dockerfile references
- Create root `docker-compose.yml`
2. **Phase 1: Backend Additions**
- Add `expires_at` to Session model
- Create nested admin endpoints (`/admin/tryout/:id/questions`, etc.)
- Update CORS for `localhost:5173`
3. **Phase 2-4: Frontend Build** (Follow original plan)
### **11.6 Summary**
The migration plan is **doable** but requires significant upfront work on repository restructuring and a few backend additions. The core FastAPI infrastructure is solid, and the auth/scoring logic is already well-implemented.
**Main challenges:**
1. **Monorepo migration** - moving existing code to `backend/` subfolder
2. **Session expiration tracking** - adding server-side timer (`expires_at`)
3. **Nested admin routes** - restructuring some API endpoints
**Strengths:**
- Complete session/tryout API already exists
- Data scrubbing already implemented
- WordPress integration already in place
- Redis/Celery for AI already configured

174
Snapshot_Driven_Workflow_PLAN.md Normal file
View File

@@ -0,0 +1,174 @@
# Snapshot-Driven Live Question Workflow Plan
## Summary
Rework tryout import so the first imported snapshot creates the live exam automatically, while later imports become non-destructive snapshot candidates inside the existing tryout workspace. Admins then intentionally replace live questions slot-by-slot after reviewing differences. Imported JSON snapshots remain full historical records, but later snapshot accordions hide unchanged rows by default to keep review focused and prevent accidental no-op replacements.
## Key Changes
- Main `/admin/tryouts` import creates a new tryout/exam and auto-promotes all valid imported questions from the first snapshot into live `items`.
- Existing tryout workspace gets an **Import New Snapshot** action. Importing there creates a full queued snapshot record, but does not change the live exam.
- Replace the current single “Imported Snapshot Questions” section with collapsible snapshot sections:
- Snapshot 1: baseline source rows plus live/retired status.
- Snapshot 2+: review queue showing changed/new/removed/invalid rows by default.
- Add a **Show unchanged questions** toggle per snapshot; default hidden.
- Snapshot comparison for UI review must compare each snapshot row against the current live slot, not only against the immediately previous snapshot.
- If a newer snapshot supersedes an older pending candidate for the same slot, mark the older candidate as `Superseded by Snapshot N`.
- Snapshot comparison must use a canonical content hash so "same content" and "changed content" are deterministic across backend, UI, and tests.
- Rename the normal state away from “Promoted”. Use lifecycle language:
- `Live`: currently served for that slot.
- `Pending Snapshot`: imported but not live.
- `Changed`: differs from current live slot.
- `No Change`: same as current live slot; hidden by default.
- `Retired`: previously live, no longer served.
- `Stale Variant`: derived from replaced/retired content, excluded from exam delivery.
- Keep retired originals and stale variants as operational history, not moved between backend tables. Frontend may visually group retired originals and variants under their snapshot/slot accordion.
- Removed slots from later snapshots are warnings by default. They do not change live delivery unless an admin explicitly retires the live slot.
- Add **Simulate Exam Flow** to show what the runtime would serve by slot using current live items and selection mode.
## Backend/API Changes
- Store every imported JSON snapshot as a full immutable import record.
- Add clear state ownership:
- `Item` owns original question lifecycle delivery state: `live` or `retired`.
- Variants keep their existing review state (`draft`, `approved`, `rejected`) and gain/keep an explicit delivery exclusion state such as `stale`.
- Snapshot row review statuses (`Changed`, `No Change`, `New Slot`, `Removed`, `Invalid`, `Superseded`) are computed from snapshot row + current live slot unless persistence is required for audit/history.
- Add canonical content hashing for snapshot/live comparison:
- Hash normalized question stem, options, correct answer, explanation, media references, scoring metadata, and any fields that affect what students see or how the item is scored.
- Ignore import timestamps, snapshot IDs, database row IDs, admin notes, and formatting-only whitespace differences.
- Option order should count as content unless the runtime already randomizes options independently.
- Add slot identity rules:
- Slot number is the primary comparison key.
- Slot number must be present for a row to become live.
- Slot number must be unique inside one snapshot.
- Reordered JSON rows should not matter.
- Duplicate slot numbers should block snapshot commit or force an explicit admin resolution flow before commit.
- Extend import preview for existing tryout imports to compare full incoming snapshot rows against the current live item per slot:
- same content: `No Change`.
- changed content: replacement candidate.
- new slot: add candidate.
- removed slot: latest snapshot no longer contains a previously live slot.
- invalid row: missing options/answer or cannot become live.
- Add a tryout-scoped import endpoint for “new snapshot inside existing tryout”; it must reject JSON for a different source tryout unless admin confirms title/id/count warnings in the request.
- Add a slot replacement endpoint that accepts selected snapshot question IDs and explicit confirmations:
- replacing live original for changed slot.
- retiring existing variants as stale.
- accepting count/title/source mismatch if relevant.
- Replacement requests must include stale-preview guards:
- `expected_live_item_id`.
- `expected_live_content_hash`.
- Backend must run replacement in a transaction.
- If the current live slot changed after preview, reject and ask the admin to refresh/re-preview.
- Backend must reject selected replacements whose snapshot content is the same as the current live slot.
- On first snapshot only, auto-create live original `Item` rows for all valid imported questions.
- Invalid first-snapshot rows are still stored in the immutable snapshot record, never become live items, and should create an import health warning in the tryout workspace.
- On later snapshot replacement:
- Always create a new live original revision and mark the previous live original `retired`, even if the current live item has no answers.
- Mark variants under the retired original as `stale`.
- Exclude `retired` originals and `stale` variants from exam selection.
- Add an explicit retire-live-slot endpoint/action for removed slots. Importing a snapshot that omits a slot must never auto-retire the current live slot.
- Support restoring an older snapshot slot:
- Restored original becomes `Live`.
- Current live original becomes `Retired`.
- Variant review status remains unchanged while stale/unstale delivery state changes with the parent original.
- Previously approved variants under the restored original become servable again.
- Draft/rejected variants keep their review status.
- Define superseded rules:
- Only pending candidates for the same slot can be superseded.
- A newer snapshot candidate for the same slot supersedes older pending candidates.
- If a newer snapshot matches current live, older pending candidates for that slot become superseded by the newer no-change snapshot.
- Already-live or retired items are never marked superseded.
- Add migration/backfill for existing tryouts:
- Backfill existing imported/promoted questions into a Snapshot 1-style baseline where possible.
- Map current user-facing “promoted” live questions to `live`.
- Map replaced or older originals to `retired` where the relationship is known.
- Verify runtime filters exclude pending, retired, draft, rejected, and stale records after migration.
- Preview/replacement API should return a stable shape that the frontend can render without re-deriving lifecycle rules, for example:
```ts
{
snapshotId?: string;
slotNumber: number;
status: "changed" | "new_slot" | "removed" | "invalid" | "no_change" | "superseded";
currentLiveItemId?: string;
snapshotQuestionId?: string;
currentLiveContentHash?: string;
snapshotContentHash?: string;
warnings: string[];
canReplace: boolean;
canRetireLiveSlot: boolean;
}
```
Primary backend areas: `tryout_json_import.py`, `admin.py`, CAT/session selection filters.
## Frontend Changes
- In `/admin/tryouts/{id}/questions`, render snapshots as collapsible groups with slot rows.
- First snapshot should look like the live exam baseline, not a manual promotion queue.
- Later snapshots should hide `No Change` rows by default and show only actionable review rows.
- Add **Show unchanged questions** toggle inside each later snapshot accordion.
- Later snapshot row statuses:
- `Changed`
- `New Slot`
- `Removed From Latest Snapshot`
- `Invalid`
- `No Change`
- `Superseded`
- Removed-slot rows should show as warnings with an explicit `Retire Live Slot` action, not as automatic changes.
- Invalid rows should show the reason they cannot become live, including missing slot number, duplicate slot number, missing options, missing answer, or unsupported content.
- Import inside tryout opens a modal:
- select JSON.
- show preview before committing snapshot.
- require checkbox confirmations only for title/source/count mismatch.
- Import preview should block commit or require an explicit resolution path for duplicate slot numbers.
- Replacement modal requires explicit confirmations only when the action changes live exam behavior:
- “I understand this replaces the live question for slot X.”
- “I understand existing variants for this slot will become stale.”
- Replacement modal should submit the expected live item ID and content hash from the preview so stale-preview conflicts can be detected.
- Replace “Open Live Question” with clearer actions:
- `Open Live Item`
- `Preview Slot`
- `Restore This Version` for retired originals.
- Add **Simulate Exam Flow** view showing current live delivery order and any skipped/missing slots.
- Simulate Exam Flow should also surface import health warnings such as invalid first-snapshot rows, missing live slots, and removed-slot warnings that have not been acted on.
Primary frontend area: `QuestionManagement.tsx`.
## Test Plan
- First import creates a tryout, full snapshot record, and live original items for all valid slots.
- First import with invalid rows stores invalid rows in the snapshot, creates live items only for valid rows, and shows import health warnings.
- Later import stores the full JSON snapshot but shows unchanged rows hidden by default.
- `Show unchanged questions` reveals `No Change` rows.
- Snapshot 3 comparison still marks slot #1 as changed if current live is Snapshot 1 content and Snapshot 3 matches Snapshot 2 content.
- Backend rejects replacing a live slot with identical snapshot content.
- Backend rejects replacement when `expected_live_item_id` or `expected_live_content_hash` no longer matches current live state.
- Newer snapshot marks older pending candidate for same changed slot as superseded.
- Newer no-change snapshot marks older pending candidates for the same slot as superseded.
- Snapshot 2 with changed slot #1 leaves current live item untouched until replacement.
- Replacing changed slot #1 retires prior original, marks its variants stale, and makes new snapshot item live.
- Restoring Snapshot 1 slot #1 makes that version live again and restores eligible variants.
- Restore reactivates only approved variants under the restored original; draft/rejected variants remain non-servable.
- Removed slot in a later snapshot does not auto-retire the live slot.
- Explicit retire-live-slot action removes that slot from runtime delivery and appears in simulation.
- Missing slot number makes a snapshot row invalid.
- Duplicate slot numbers block snapshot commit or require explicit admin resolution.
- Reordered JSON rows do not produce false changes.
- Canonical content hash ignores formatting-only whitespace differences.
- Option order changes are detected as content changes unless runtime option randomization makes order irrelevant.
- Same file re-import behavior is deterministic and does not create conflicting live state.
- Migration/backfill maps existing promoted/live questions and excludes non-live records from runtime delivery.
- Fixed exam simulation shows only current live servable items in slot order.
- Runtime session next-item endpoint never serves pending, retired, rejected, draft, or stale items.
## Assumptions
- First snapshot is trusted as the initial canonical exam and should auto-promote valid questions.
- Later snapshots are destructive only when admin promotes/replaces selected slots, not when imported.
- Imported JSON snapshots should remain complete historical records.
- Snapshot review UI should default to actionable differences only.
- Slot number is the primary comparison key for snapshot-to-live review.
- Variants belong to the live original version they were generated from.
- Historical answers/calibration must remain attached to the exact item version students saw.
- “Promoted” should disappear as a normal user-facing state after this workflow is implemented.

439
UX_AUDIT_ADMIN_FLOW.md Normal file
View File

@@ -0,0 +1,439 @@
# UX Audit: Admin Flow - IRT Bank Soal
> **Audit Date:** 2026-06-17
> **Auditor:** Dev Agent
> **Focus:** Login → First-time experience → Navigation discoverability → Hierarchy visibility
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Login Flow Analysis](#login-flow-analysis)
3. [Post-Login Experience](#post-login-experience)
4. [Navigation & Discoverability](#navigation--discoverability)
5. [Hierarchy Visibility](#hierarchy-visibility)
6. [Issue Summary & Priority Matrix](#issue-summary--priority-matrix)
7. [Recommended Improvements](#recommended-improvements)
8. [Appendix: Current vs Proposed Flow](#appendix-current-vs-proposed-flow)
---
## Executive Summary
The current admin flow has significant UX gaps that make it difficult for new administrators to orient themselves and complete tasks efficiently. The main issues are:
| Category | Severity | Count |
|----------|----------|-------|
| Critical (blocks usage) | 🔴 High | 4 |
| Medium (confuses users) | 🟡 Medium | 6 |
| Low (minor friction) | 🟢 Low | 5 |
### Key Findings
1. **No onboarding guidance** after login - users land on Dashboard with no context
2. **Hierarchy is hidden** in Settings submenu - should be prominently visible
3. **Navigation labels are inconsistent** - mixed technical and human terms
4. **Login page lacks branding** - no visual connection to the product
5. **No breadcrumb navigation** - users get lost in deep pages
---
## Login Flow Analysis
### Current State
The login page (`/admin/login`) presents:
- Simple username/password form
- "Remember me" checkbox
- Minimal error messaging
- Help button (bottom-right corner)
```python
# Current login form elements
- Username field
- Password field
- Remember me checkbox
- Sign in button
```
### Issues Found
| # | Issue | Impact | Severity |
|---|-------|--------|----------|
| 1.1 | **No product branding/logo** | Users don't know what system they're logging into | 🟡 Medium |
| 1.2 | **No error state distinction** | Failed login looks same as rate limiting | 🟡 Medium |
| 1.3 | **"Remember me" is unclear** | Doesn't explain session duration or implications | 🟢 Low |
| 1.4 | **No "forgot password" path** | No recovery mechanism exists | 🟡 Medium |
| 1.5 | **Help button is discoverable** | Good: floating help exists but underutilized | 🟢 Positive |
### Login → Dashboard Redirect
**Current behavior:** After successful login → `/admin/dashboard`
**What users see:**
```
┌─────────────────────────────────────────┐
│ Good Morning, admin! 👋 │
│ Here's what's happening today. │
│ │
│ ⚠️ 25 questions need calibration │
│ 📝 3 AI-generated questions pending │
│ 💡 Tip: Start by importing questions... │
│ │
│ 📊 System Overview │
│ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │
│ │ 5 │ │ 150 │ │ 890 │ │ 2 │ │
│ │Exams │ │Quest │ │Tests │ │Sites │ │
│ └──────┘ └──────┘ └──────┘ └──────┘ │
└─────────────────────────────────────────┘
```
### Problems After Login
| # | Issue | Why It's a Problem |
|---|-------|-------------------|
| 2.1 | **No welcome message explaining the system** | First-time users don't know what IRT Bank Soal does |
| 2.2 | **"5 Exams" is meaningless without context** | Users don't know what an Exam/Tryout means |
| 2.3 | **Alerts are action-oriented but not instructive** | "Import questions" - but where? How? |
| 2.4 | **Quick Actions use technical language** | "Generate AI Questions" doesn't explain what happens |
| 2.5 | **No first-time setup wizard** | Empty state users have no guidance |
---
## Navigation & Discoverability
### Current Navigation Structure
```
Sidebar Navigation (collapsed view):
┌─────────────────────────┐
│ IRT Bank Soal Admin │
├─────────────────────────┤
│ 📊 Dashboard │ ← Always first
│ 📝 Questions │ ← What is this?
│ 📥 Import Questions │ ← Separate from Questions?
│ 🤖 AI Generator │ ← Is this part of Questions?
│ 📋 Exams │ ← Tryout = Exam?
│ 📈 Reports │
│ ⚙️ Settings │ ← Hierarchy buried here
│ ─────────────────────── │
│ 🚪 Logout │
└─────────────────────────┘
```
### Label Analysis
| Current Label | User Interpretation | Issue |
|---------------|---------------------|-------|
| Questions | "Where I view questions?" | ✅ Clear |
| Import Questions | "Is this separate from Questions?" | ⚠️ Unclear relationship |
| AI Generator | "What does AI Generate?" | ⚠️ Vague |
| Exams | "Same as Tryout?" | ⚠️ Mismatch with backend term |
| Reports | "Student scores?" | ✅ Clear |
| Settings → Hierarchy | "What is hierarchy?" | 🔴 Wrong place + wrong term |
### Missing Navigation Features
| # | Missing Feature | Impact |
|---|-----------------|--------|
| 3.1 | **No breadcrumbs** | Users can't trace their path back |
| 3.2 | **No "back to parent" links** | Deep pages have no escape route |
| 3.3 | **No search/global nav** | Can't jump to specific pages |
| 3.4 | **No recent pages** | Can't quickly return to work in progress |
| 3.5 | **Settings is a catch-all** | Mixes Website management, Hierarchy, Password |
---
## Hierarchy Visibility
### Current Hierarchy Location
Hierarchy is located at: **Settings → Data Structure** (`/admin/hierarchy`)
### Problems with Current Hierarchy Placement
| # | Issue | Why It Matters |
|---|-------|----------------|
| 4.1 | **Buried 2 levels deep** | First-time users never find it |
| 4.2 | **Label is technical** | "Data Structure" vs "How data connects" |
| 4.3 | **No explanation of the hierarchy concept** | Users don't know Website → Tryout → Questions → Variants |
| 4.4 | **No visual flowchart on Dashboard** | Users should see the big picture immediately |
### Expected Mental Model
```
┌─────────────────────────────────────────────────────────────┐
│ USER'S EXPECTED FLOW │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Website (where exams are hosted) │
│ │ │
│ ▼ │
│ 2. Tryout/Exam (the test itself) │
│ │ │
│ ▼ │
│ 3. Questions (individual items in the test) │
│ │ │
│ ├── Original/Basis Question ──────────────────────┐ │
│ │ │ │ │
│ │ ▼ │ │
│ │ AI Variant (different version) │ │
│ │ │ │
│ └── (repeated for each question slot) │ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Where Users Expect Hierarchy Info
| Location | User Expectation |
|----------|------------------|
| **Dashboard** | "Show me the big picture" - visual overview |
| **First-time tooltip** | "Here's how things connect" |
| **Help/Docs** | "Explain the data model" |
| **Settings sidebar** | ❌ Too late - user already lost |
---
## Issue Summary & Priority Matrix
### Priority Matrix
```
│ High Value │ Low Value │
────────────────────┼──────────────┼──────────────┤
High Effort │ [A] Refactor │ [B] Nice to │
│ Navigation │ have │
────────────────────┼──────────────┼──────────────┤
Low Effort │ [C] Quick │ [D] Ignore │
│ Wins │ │
────────────────────┼──────────────┼──────────────┤
```
### Cell [A] - High Value, High Effort (Do First)
| Issue ID | Description | Notes |
|----------|-------------|-------|
| P1 | **Add Dashboard onboarding section** | Explain the system + show hierarchy flow |
| P2 | **Move Hierarchy to prominent location** | Dashboard or separate nav item |
| P3 | **Redesign navigation labels** | Human-friendly, consistent terminology |
| P4 | **Add breadcrumbs** | Across all pages |
### Cell [C] - High Value, Low Effort (Quick Wins)
| Issue ID | Description | Effort |
|----------|-------------|--------|
| Q1 | Add product logo to login page | 15 min |
| Q2 | Improve dashboard welcome message | 10 min |
| Q3 | Add "How it works" section to Dashboard | 30 min |
| Q4 | Rename "Data Structure" → "Data Overview" in Settings | 5 min |
| Q5 | Add contextual tooltips to Quick Actions | 20 min |
### Cell [B] - Low Value, High Effort (Consider Later)
| Issue ID | Description |
|----------|-------------|
| L1 | Global search across all pages |
| L2 | Recent pages sidebar widget |
| L3 | Full first-time setup wizard |
### Cell [D] - Low Value, Low Effort (Ignore)
| Issue ID | Description |
|----------|-------------|
| N1 | Custom "Remember me" tooltip |
| N2 | Login page background gradient (cosmetic only) |
---
## Recommended Improvements
### Phase 1: Critical Fixes (Same Session)
#### 1. Login Page Enhancement
```html
<!-- Add to login page -->
<div class="login-header">
<img src="/static/logo.png" alt="IRT Bank Soal" class="login-logo">
<h1>IRT Bank Soal</h1>
<p>Adaptive Question Bank System</p>
</div>
```
#### 2. Dashboard - Add "How It Works" Section
Add this block to dashboard after greeting:
```html
<div class="onboarding-flow">
<h3>How Your Exam System Works</h3>
<div class="flow-steps">
<div class="step">
<span class="step-num">1</span>
<span class="step-title">Add Website</span>
<span class="step-desc">Connect your WordPress site</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">2</span>
<span class="step-title">Import Questions</span>
<span class="step-desc">Upload your exam questions</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">3</span>
<span class="step-title">Generate Variants</span>
<span class="step-desc">AI creates different versions</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">4</span>
<span class="step-title">Students Take Tests</span>
<span class="step-desc">Adaptive difficulty adjusts</span>
</div>
</div>
<a href="/admin/hierarchy" class="flow-link">View full data structure →</a>
</div>
```
#### 3. Dashboard - Add "Get Started" for Empty State
When `tryouts_count == 0`:
```html
<div class="getting-started">
<h2>🚀 Welcome to IRT Bank Soal!</h2>
<p>Get started in 3 simple steps:</p>
<div class="steps">
<div class="step-card">
<span class="num">1</span>
<h3>Connect a Website</h3>
<p>Add your WordPress site to the system</p>
<a href="/admin/websites" class="btn">Add Website →</a>
</div>
<div class="step-card">
<span class="num">2</span>
<h3>Import Questions</h3>
<p>Upload questions from Excel or JSON</p>
<a href="/admin/tryout-import" class="btn">Import Questions →</a>
</div>
<div class="step-card">
<span class="num">3</span>
<h3>Generate Variants</h3>
<p>Use AI to create question variations</p>
<a href="/admin/basis-items" class="btn">Generate Variants →</a>
</div>
</div>
</div>
```
### Phase 2: Navigation Improvement (Next Sprint)
#### 4. Rename Navigation Items
| Current | Proposed | Reason |
|---------|----------|--------|
| Import Questions | Import from Excel | More specific |
| AI Generator | Generate AI Questions | Action-oriented |
| Settings → Hierarchy | (move to Dashboard) | Too hidden |
| Questions | Question Bank | Clarify scope |
#### 5. Add Breadcrumbs Component
```html
<nav class="breadcrumbs">
<a href="/admin/dashboard">Dashboard</a>
<span class="sep">›</span>
<a href="/admin/questions">Questions</a>
<span class="sep">›</span>
<span class="current">Question #123</span>
</nav>
```
### Phase 3: Advanced Features (Future)
#### 6. First-Time Setup Wizard
Modal that walks new admins through:
1. Website configuration
2. First import
3. Basic settings review
#### 7. Interactive Hierarchy Diagram
Replace static hierarchy view with interactive visualization:
```mermaid
graph LR
A[Website] --> B[Tryout]
B --> C[Questions]
C --> D[Variants]
C --> E[Student Answers]
D --> F[AI Generation]
```
---
## Appendix: Current vs Proposed Flow
### Current Flow (Confusing)
```
Login
↓
Dashboard (counts, no context)
↓ (guess where to go)
Settings? Questions? Import? (trial & error)
↓
Get lost → Leave → Ask for help
```
### Proposed Flow (Guided)
```
Login
↓
Dashboard
├─ "Here's how it works" (visual flow)
├─ Quick Stats (with explanations)
├─ Alerts (with direct action buttons)
└─ Recent Activity
↓
Follow guided steps OR jump to specific task
↓
Complete task → Return to Dashboard
↓
See updated progress
```
---
## Files to Modify
| File | Changes Needed |
|------|---------------|
| `app/admin_web.py` | Dashboard content, navigation labels, breadcrumbs |
| `app/admin_web_icons.py` | (No changes needed) |
| `app/templates/` | (Add if using templates) |
---
## Test Checklist
After implementing changes, verify:
- [ ] Login page shows product branding
- [ ] Dashboard explains the system for first-time users
- [ ] Empty state shows guided setup
- [ ] Navigation labels are consistent and clear
- [ ] Hierarchy is accessible from Dashboard
- [ ] Breadcrumbs appear on all sub-pages
- [ ] Quick Actions have explanatory tooltips
- [ ] User can complete first import without help
---
*End of Audit Report*

3446
app/admin_web.py
View File

File diff suppressed because it is too large Load Diff

227
app/routers/admin.py
View File

@@ -1,227 +0,0 @@
"""
Admin API router for custom admin actions.
Provides admin-specific endpoints for triggering calibration,
toggling AI generation, and resetting normalization.
"""
from typing import Any, Dict
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.auth import AuthContext, get_auth_context, require_website_auth
from app.core.config import get_settings
from app.database import get_db
from app.models import Tryout, TryoutStats
from app.services.irt_calibration import (
calibrate_all,
CALIBRATION_SAMPLE_THRESHOLD,
)
router = APIRouter(prefix="/admin", tags=["admin"])
settings = get_settings()
@router.post(
"/{tryout_id}/calibrate",
summary="Trigger IRT calibration",
description="Trigger IRT calibration for all items in this tryout with sufficient response data.",
)
async def admin_trigger_calibration(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Trigger IRT calibration for all items in a tryout.
Runs calibration for items with >= min_calibration_sample responses.
Updates item.irt_b, item.irt_se, and item.calibrated status.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Calibration results summary
Raises:
HTTPException: If tryout not found or calibration fails
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Run calibration
result = await calibrate_all(
tryout_id=tryout_id,
website_id=website_id,
db=db,
min_sample_size=tryout.min_calibration_sample or CALIBRATION_SAMPLE_THRESHOLD,
)
return {
"tryout_id": tryout_id,
"total_items": result.total_items,
"calibrated_items": result.calibrated_items,
"failed_items": result.failed_items,
"calibration_percentage": round(result.calibration_percentage * 100, 2),
"ready_for_irt": result.ready_for_irt,
"message": f"Calibration complete: {result.calibrated_items}/{result.total_items} items calibrated",
}
@router.post(
"/{tryout_id}/toggle-ai-generation",
summary="Toggle AI generation",
description="Toggle AI question generation for a tryout.",
)
async def admin_toggle_ai_generation(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Toggle AI generation for a tryout.
Updates Tryout.AI_generation_enabled field.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Updated AI generation status
Raises:
HTTPException: If tryout not found
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Toggle AI generation
tryout.ai_generation_enabled = not tryout.ai_generation_enabled
await db.commit()
await db.refresh(tryout)
status = "enabled" if tryout.ai_generation_enabled else "disabled"
return {
"tryout_id": tryout_id,
"ai_generation_enabled": tryout.ai_generation_enabled,
"message": f"AI generation {status} for tryout {tryout_id}",
}
@router.post(
"/{tryout_id}/reset-normalization",
summary="Reset normalization",
description="Reset normalization to static values and clear incremental stats.",
)
async def admin_reset_normalization(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Reset normalization for a tryout.
Resets rataan, sb to static values and clears incremental stats.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Reset statistics
Raises:
HTTPException: If tryout or stats not found
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"TryoutStats for {tryout_id} not found for website {website_id}",
)
# Get tryout for static values
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout:
# Reset to static values
stats.rataan = tryout.static_rataan
stats.sb = tryout.static_sb
else:
# Reset to default values
stats.rataan = 500.0
stats.sb = 100.0
# Clear incremental stats
old_participant_count = stats.participant_count
stats.participant_count = 0
stats.total_nm_sum = 0.0
stats.total_nm_sq_sum = 0.0
stats.min_nm = None
stats.max_nm = None
stats.last_calculated = None
await db.commit()
await db.refresh(stats)
return {
"tryout_id": tryout_id,
"rataan": stats.rataan,
"sb": stats.sb,
"cleared_stats": {
"previous_participant_count": old_participant_count,
},
"message": f"Normalization reset to static values (rataan={stats.rataan}, sb={stats.sb}). Incremental stats cleared.",
}

102
app/schemas/ai.py
View File

@@ -1,102 +0,0 @@
"""
Pydantic schemas for AI generation endpoints.
Request/response models for admin AI generation playground.
"""
from typing import Dict, Literal, Optional
from pydantic import BaseModel, Field, field_validator
class AIGeneratePreviewRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated question"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
class AIGeneratePreviewResponse(BaseModel):
success: bool = Field(..., description="Whether generation was successful")
stem: Optional[str] = None
options: Optional[Dict[str, str]] = None
correct: Optional[str] = None
explanation: Optional[str] = None
ai_model: Optional[str] = None
basis_item_id: Optional[int] = None
target_level: Optional[str] = None
error: Optional[str] = None
cached: bool = False
class AISaveRequest(BaseModel):
stem: str = Field(..., description="Question stem")
options: Dict[str, str] = Field(
..., description="Answer options (A, B, C, D)"
)
correct: str = Field(..., description="Correct answer (A/B/C/D)")
explanation: Optional[str] = None
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
basis_item_id: int = Field(..., description="Basis item ID")
slot: int = Field(..., description="Question slot position")
level: Literal["mudah", "sedang", "sulit"] = Field(
..., description="Difficulty level"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model used for generation",
)
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()
@field_validator("options")
@classmethod
def validate_options(cls, v: Dict[str, str]) -> Dict[str, str]:
required_keys = {"A", "B", "C", "D"}
if not required_keys.issubset(set(v.keys())):
raise ValueError("Options must contain keys A, B, C, D")
return v
class AISaveResponse(BaseModel):
success: bool = Field(..., description="Whether save was successful")
item_id: Optional[int] = None
error: Optional[str] = None
class AIStatsResponse(BaseModel):
total_ai_items: int = Field(..., description="Total AI-generated items")
items_by_model: Dict[str, int] = Field(
default_factory=dict, description="Items count by AI model"
)
cache_hit_rate: float = Field(
default=0.0, description="Cache hit rate (0.0 to 1.0)"
)
total_cache_hits: int = Field(default=0, description="Total cache hits")
total_requests: int = Field(default=0, description="Total generation requests")
class GeneratedQuestion(BaseModel):
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()

0
.env.example → backend/.env.example
View File

18
backend/Dockerfile Normal file
View File

@@ -0,0 +1,18 @@
FROM python:3.11-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
postgresql-client \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements and install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Run migrations and start the app
CMD ["sh", "-c", "alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000"]

2
alembic.ini → backend/alembic.ini
View File

@@ -84,7 +84,7 @@ path_separator = os
# database URL. This is consumed by the user-maintained env.py script only.
# other means of configuring database URLs may be customized within the env.py
# file.
sqlalchemy.url = postgresql+asyncpg://postgres:postgres@localhost:5432/irt_bank_soal
sqlalchemy.url = driver://user:pass@localhost/dbname
[post_write_hooks]

0
alembic/README → backend/alembic/README
View File

0
alembic/env.py → backend/alembic/env.py
View File

0
alembic/script.py.mako → backend/alembic/script.py.mako
View File

0
alembic/versions/20260331_000001_initial_schema.py → backend/alembic/versions/20260331_000001_initial_schema.py
View File

0
alembic/versions/20260402_000002_tryout_json_snapshots.py → backend/alembic/versions/20260402_000002_tryout_json_snapshots.py
View File

0
alembic/versions/20260404_000003_ai_runs_and_variant_status.py → backend/alembic/versions/20260404_000003_ai_runs_and_variant_status.py
View File

0
alembic/versions/20260405_000004_report_schedules.py → backend/alembic/versions/20260405_000004_report_schedules.py
View File

26
backend/alembic/versions/20260617_000005_session_expires_at.py Normal file
View File

@@ -0,0 +1,26 @@
"""add session expires at
Revision ID: 20260617_000005
Revises: 20260405_000004
Create Date: 2026-06-17 15:00:00
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260617_000005"
down_revision: Union[str, None] = "20260405_000004"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("sessions", sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True))
def downgrade() -> None:
op.drop_column("sessions", "expires_at")

0
app/__init__.py → backend/app/__init__.py
View File

0
app/admin.py → backend/app/admin.py
View File

6456
backend/app/admin_web.py Normal file
View File

File diff suppressed because it is too large Load Diff

110
backend/app/admin_web_icons.py Normal file
View File

@@ -0,0 +1,110 @@
"""
Icon constants using inline SVG (Heroicons style).
These replace emoji usage in the admin UI for consistent, professional icons.
"""
# Navigation icons
ICON_DASHBOARD = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3.75 6A2.25 2.25 0 0 1 6 3.75h2.25A2.25 2.25 0 0 1 10.5 6v2.25a2.25 2.25 0 0 1-2.25 2.25H6a2.25 2.25 0 0 1-2.25-2.25V6ZM3.75 15.75A2.25 2.25 0 0 1 6 13.5h2.25a2.25 2.25 0 0 1 2.25 2.25V18a2.25 2.25 0 0 1-2.25 2.25H6A2.25 2.25 0 0 1 3.75 18v-2.25ZM13.5 6a2.25 2.25 0 0 1 2.25-2.25H18A2.25 2.25 0 0 1 20.25 6v2.25A2.25 2.25 0 0 1 18 10.5h-2.25a2.25 2.25 0 0 1-2.25-2.25V6ZM13.5 15.75a2.25 2.25 0 0 1 2.25-2.25H18a2.25 2.25 0 0 1 2.25 2.25V18A2.25 2.25 0 0 1 18 20.25h-2.25A2.25 2.25 0 0 1 13.5 18v-2.25Z" /></svg>"""
ICON_QUESTIONS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M9.879 7.519c1.171-1.025 3.071-1.025 4.242 0 1.172 1.025 1.172 2.687 0 3.712-.203.179-.43.326-.67.442-.745.361-1.45.999-1.45 1.827v.75M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9 5.25h.008v.008H12v-.008Z" /></svg>"""
ICON_IMPORT = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5m-13.5-9L12 3m0 0 4.5 4.5M12 3v13.5" /></svg>"""
ICON_AI = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M9.813 15.904 9 18.75l-.813-2.846a4.5 4.5 0 0 0-3.09-3.09L2.25 12l2.846-.813a4.5 4.5 0 0 0 3.09-3.09L9 5.25l.813 2.846a4.5 4.5 0 0 0 3.09 3.09L15.75 12l-2.846.813a4.5 4.5 0 0 0-3.09 3.09ZM18.259 8.715 18 9.75l-.259-1.035a3.375 3.375 0 0 0-2.455-2.456L14.25 6l1.036-.259a3.375 3.375 0 0 0 2.455-2.456L18 2.25l.259 1.035a3.375 3.375 0 0 0 2.456 2.456L21.75 6l-1.035.259a3.375 3.375 0 0 0-2.456 2.456ZM16.894 20.567 16.5 21.75l-.394-1.183a2.25 2.25 0 0 0-1.423-1.423L13.5 18.75l1.183-.394a2.25 2.25 0 0 0 1.423-1.423l.394-1.183.394 1.183a2.25 2.25 0 0 0 1.423 1.423l1.183.394-1.183.394a2.25 2.25 0 0 0-1.423 1.423Z" /></svg>"""
ICON_EXAMS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" /></svg>"""
ICON_REPORTS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3 13.125C3 12.504 3.504 12 4.125 12h2.25c.621 0 1.125.504 1.125 1.125v6.75C7.5 20.496 6.996 21 6.375 21h-2.25A1.125 1.125 0 0 1 3 19.875v-6.75ZM9.75 8.625c0-.621.504-1.125 1.125-1.125h2.25c.621 0 1.125.504 1.125 1.125v11.25c0 .621-.504 1.125-1.125 1.125h-2.25a1.125 1.125 0 0 1-1.125-1.125V8.625ZM16.5 4.125c0-.621.504-1.125 1.125-1.125h2.25C20.496 3 21 3.504 21 4.125v15.75c0 .621-.504 1.125-1.125 1.125h-2.25a1.125 1.125 0 0 1-1.125-1.125V4.125Z" /></svg>"""
ICON_SETTINGS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M9.594 3.94c.09-.542.56-.94 1.11-.94h2.593c.55 0 1.02.398 1.11.94l.213 1.281c.063.374.313.686.645.87.074.04.147.083.22.127.325.196.72.257 1.075.124l1.217-.456a1.125 1.125 0 0 1 1.37.49l1.296 2.247a1.125 1.125 0 0 1-.26 1.431l-1.003.827c-.293.241-.438.613-.43.992a7.723 7.723 0 0 1 0 .255c-.008.378.137.75.43.991l1.004.827c.424.35.534.955.26 1.43l-1.298 2.247a1.125 1.125 0 0 1-1.369.491l-1.217-.456c-.355-.133-.75-.072-1.076.124a6.47 6.47 0 0 1-.22.128c-.331.183-.581.495-.644.869l-.213 1.281c-.09.543-.56.94-1.11.94h-2.594c-.55 0-1.019-.398-1.11-.94l-.213-1.281c-.062-.374-.312-.686-.644-.87a6.52 6.52 0 0 1-.22-.127c-.325-.196-.72-.257-1.076-.124l-1.217.456a1.125 1.125 0 0 1-1.369-.49l-1.297-2.247a1.125 1.125 0 0 1 .26-1.431l1.004-.827c.292-.24.437-.613.43-.991a6.932 6.932 0 0 1 0-.255c.007-.38-.138-.751-.43-.992l-1.004-.827a1.125 1.125 0 0 1-.26-1.43l1.297-2.247a1.125 1.125 0 0 1 1.37-.491l1.216.456c.356.133.751.072 1.076-.124.072-.044.146-.086.22-.128.332-.183.582-.495.644-.869l.214-1.28Z" /><path stroke-linecap="round" stroke-linejoin="round" d="M15 12a3 3 0 1 1-6 0 3 3 0 0 1 6 0Z" /></svg>"""
ICON_LOGOUT = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M15.75 9V5.25A2.25 2.25 0 0 0 13.5 3h-6a2.25 2.25 0 0 0-2.25 2.25v13.5A2.25 2.25 0 0 0 7.5 21h6a2.25 2.25 0 0 0 2.25-2.25V15M12 9l-3 3m0 0 3 3m-3-3h12.75" /></svg>"""
# Page icons
ICON_TARGET = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 18v-5.25m0 0a6.01 6.01 0 0 0 1.5-.189m-1.5.189a6.01 6.01 0 0 1-1.5-.189m3.75 7.478a12.06 12.06 0 0 1-4.5 0m3.75 2.383a14.406 14.406 0 0 1-3 0M14.25 18v-.192c0-.983.658-1.727 1.5-1.727s1.5.744 1.5 1.727V18m-4.5 0h.008v.008H14.25v-.008Z" /></svg>"""
ICON_USERS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M15 19.128a9.38 9.38 0 0 0 2.625.372 9.337 9.337 0 0 0 4.121-.952 4.125 4.125 0 0 0-7.533-2.493M15 19.128v-.003c0-1.113-.285-2.16-.786-3.07M15 19.128v.106A12.318 12.318 0 0 1 8.624 21c-2.331 0-4.512-.645-6.374-1.766l-.001-.109a6.375 6.375 0 0 1 11.964-3.07M12 6.375a3.375 3.375 0 1 1-6.75 0 3.375 3.375 0 0 1 6.75 0Zm8.25 2.25a2.625 2.625 0 1 1-5.25 0 2.625 2.625 0 0 1 5.25 0Z" /></svg>"""
ICON_CALIBRATION = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M10.5 6a7.5 7.5 0 1 0 0 15 7.5 7.5 0 0 0 0-15ZM21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Z" /><path stroke-linecap="round" stroke-linejoin="round" d="M12 6v6h6" /></svg>"""
ICON_STUDENTS = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M4.26 10.147a60.438 60.438 0 0 0-.491 6.347A48.62 48.62 0 0 1 12 20.904a48.62 48.62 0 0 1 8.232-4.41 60.46 60.46 0 0 0-.491-6.347m-15.482 0a50.636 50.636 0 0 0-2.658-.813A59.906 59.906 0 0 1 12 3.493a59.903 59.903 0 0 1 10.399 5.84c-.896.248-1.783.52-2.658.814m-15.482 0A50.717 50.717 0 0 1 12 13.489a50.702 50.702 0 0 1 7.74-3.342M6.75 15a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5Zm0 0v-3.675A55.378 55.378 0 0 1 12 8.443m-7.007 11.55A5.981 5.981 0 0 0 6.75 15.75v-1.5" /></svg>"""
ICON_DOWNLOAD = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" /></svg>"""
ICON_UPLOAD = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5m-13.5-9L12 3m0 0 4.5 4.5M12 3v13.5" /></svg>"""
ICON_SEARCH = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="m21 21-5.197-5.197m0 0A7.5 7.5 0 1 0 5.196 5.196a7.5 7.5 0 0 0 10.607 10.607Z" /></svg>"""
ICON_CHECK = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="m4.5 12.75 6 6 9-13.5" /></svg>"""
ICON_WARNING = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0L2.697 16.126ZM12 15.75h.007v.008H12v-.008Z" /></svg>"""
ICON_INFO = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z" /></svg>"""
ICON_LIGHTBULB = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 18v-5.25m0 0a6.01 6.01 0 0 0 1.5-.189m-1.5.189a6.01 6.01 0 0 1-1.5-.189m3.75 7.478a12.06 12.06 0 0 1-4.5 0m3.75 2.383a14.406 14.406 0 0 1-3 0M14.25 18v-.192c0-.983.658-1.727 1.5-1.727s1.5.744 1.5 1.727V18m-4.5 0h.008v.008H14.25v-.008Z" /></svg>"""
ICON_TREND_UP = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M2.25 18 9 11.25l4.306 4.306a11.95 11.95 0 0 1 5.814-5.518l2.74-1.22m0 0-5.94-2.281m5.94 2.28-2.28 5.941" /></svg>"""
ICON_TREND_DOWN = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="page-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M2.25 6 9 12.75l4.286-4.286a11.948 11.948 0 0 1 4.306 6.43l.776 2.898m0 0 3.182-5.511m-3.182 5.51-5.511-3.181" /></svg>"""
# Huge icons for replacing emojis (24x24 with larger visual weight)
ICON_HUGE_TARGET = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 18v-5.25m0 0a8.01 8.01 0 0 0 1.5-.189m-1.5.189a8.01 8.01 0 0 1-1.5-.189m3.75 7.478a10.56 10.56 0 0 1-4.5 0m3.75 2.383a13.406 13.406 0 0 1-3 0M14.25 18v-.192c0-.983.658-1.727 1.5-1.727s1.5.744 1.5 1.727V18m-4.5 0h.008v.008H14.25v-.008Z" /></svg>"""
ICON_HUGE_USER = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M15.75 6a3.75 3.75 0 1 1-7.5 0 3.75 3.75 0 0 1 7.5 0ZM4.501 20.118a7.5 7.5 0 0 1 14.998 0A17.933 17.933 0 0 1 12 21.75c-2.676 0-5.216-.584-7.499-1.632Z" /></svg>"""
ICON_HUGE_CHECK = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="m4.5 12.75 6 6 9-13.5" /></svg>"""
ICON_HUGE_CLOCK = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 6v6h4.5m4.5 0a9 9 0 1 1-18 0 9 9 0 0 1 18 0Z" /></svg>"""
ICON_HUGE_ROCKET = """<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24"><path d="M0 0h24v24H0z" fill="none" /><g fill="none" stroke="currentColor" stroke-width="1.5"><path stroke-linejoin="round" d="m11.801 6.49l1.486-1.486c1.673-1.673 3.862-2.367 6.18-2.48c.902-.044 1.352-.066 1.714.295c.361.362.34.812.295 1.714c-.113 2.318-.807 4.507-2.48 6.18L17.511 12.2c-1.224 1.223-1.572 1.571-1.315 2.898c.254 1.014.499 1.995-.238 2.732c-.894.895-1.71.895-2.604 0l-7.183-7.183c-.895-.894-.895-1.71 0-2.604c.737-.737 1.718-.492 2.732-.238c1.327.257 1.675-.091 2.898-1.315Z" /><path stroke-linecap="round" d="m2.5 21.5l5-5m1 5l2-2m-8-4l2-2" /><path stroke-linecap="round" stroke-linejoin="round" d="M17.125 7H17m.25 0a.25.25 0 1 1-.5 0a.25.25 0 0 1 .5 0" /></g></svg>"""
ICON_HUGE_CHART = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M3 13.125C3 12.504 3.504 12 4.125 12h2.25c.621 0 1.125.504 1.125 1.125v6.75C7.5 20.496 6.996 21 6.375 21h-2.25A1.125 1.125 0 0 1 3 19.875v-6.75ZM9.75 8.625c0-.621.504-1.125 1.125-1.125h2.25c.621 0 1.125.504 1.125 1.125v11.25c0 .621-.504 1.125-1.125 1.125h-2.25a1.125 1.125 0 0 1-1.125-1.125V8.625ZM16.5 4.125c0-.621.504-1.125 1.125-1.125h2.25C20.496 3 21 3.504 21 4.125v15.75c0 .621-.504 1.125-1.125 1.125h-2.25a1.125 1.125 0 0 1-1.125-1.125V4.125Z" /></svg>"""
# Emoji to SVG icon mapping for replacement
EMOJI_TO_ICON = {
# Navigation & main icons
"🏠": ICON_DASHBOARD,
"📝": ICON_QUESTIONS,
"📥": ICON_IMPORT,
"🤖": ICON_AI,
"📋": ICON_EXAMS,
"📊": ICON_REPORTS,
"⚙️": ICON_SETTINGS,
"🚪": ICON_LOGOUT,
"🎯": ICON_HUGE_TARGET,
"👤": ICON_HUGE_USER,
"👥": ICON_USERS,
"⚠️": ICON_WARNING,
"ℹ️": ICON_INFO,
"🚀": ICON_HUGE_ROCKET,
"✅": ICON_HUGE_CHECK,
"❌": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12" /></svg>""",
"⏳": ICON_HUGE_CLOCK,
"📈": ICON_TREND_UP,
"📉": ICON_TREND_DOWN,
"💡": ICON_LIGHTBULB,
"👋": '<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" style="display:inline;width:28px;height:28px;margin-bottom:-4px;"><path stroke-linecap="round" stroke-linejoin="round" d="M15.042 21.672 13.684 16.6m0 0-2.51 2.225.569-9.47 5.227 7.917-3.286-.672ZM12 2.25V4.5m5.834.166-1.591 1.591M20.25 10.5H18M7.757 14.743l-1.59 1.59M6 10.5H3.75m4.007-4.243-1.59-1.59" /></svg>',
"📊": ICON_REPORTS,
"🚀": ICON_HUGE_ROCKET,
"📈": ICON_TREND_UP,
# Additional icons from UI
"🌐": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M12 21a9.004 9.004 0 0 0 8.716-6.747M12 21a9.004 9.004 0 0 1-8.716-6.747M12 21c2.485 0 4.5-4.03 4.5-9S14.485 3 12 3m0 18c-2.485 0-4.5-4.03-4.5-9S9.515 3 12 3m0 0a8.997 8.997 0 0 1 7.843 4.582M12 3a8.997 8.997 0 0 0-7.843 4.582m15.686 0A11.953 11.953 0 0 1 12 10.5c-2.998 0-5.74-1.1-7.843-2.918m15.686 0A8.959 8.959 0 0 1 21 12c0 .778-.099 1.533-.284 2.253m0 0A17.919 17.919 0 0 1 12 16.5c-3.162 0-6.133-.815-8.716-2.247m0 0A9.015 9.015 0 0 1 3 12c0-1.605.42-3.113 1.157-4.418" /></svg>""",
"🔍": ICON_SEARCH,
"📁": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M2.25 12.75V12A2.25 2.25 0 0 1 4.5 9.75h15A2.25 2.25 0 0 1 21.75 12v.75m-8.69-6.44-2.12-2.12a1.5 1.5 0 0 0-1.061-.44H4.5A2.25 2.25 0 0 0 2.25 6v12a2.25 2.25 0 0 0 2.25 2.25h15A2.25 2.25 0 0 0 21.75 18V9a2.25 2.25 0 0 0-2.25-2.25h-5.379a1.5 1.5 0 0 1-1.06-.44Z" /></svg>""",
"🔐": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 10.5V6.75a4.5 4.5 0 1 0-9 0v3.75m-.75 11.25h10.5a2.25 2.25 0 0 0 2.25-2.25v-6.75a2.25 2.25 0 0 0-2.25-2.25H6.75a2.25 2.25 0 0 0-2.25 2.25v6.75a2.25 2.25 0 0 0 2.25 2.25Z" /></svg>""",
"⚡": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="m3.75 13.5 10.5-11.25L12 10.5h8.25L9.75 21.75 12 13.5H3.75Z" /></svg>""",
"💾": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M20.25 6.375c0 2.278-3.694 4.125-8.25 4.125S3.75 8.653 3.75 6.375m16.5 0c0-2.278-3.694-4.125-8.25-4.125S3.75 4.097 3.75 6.375m16.5 0v11.25c0 2.278-3.694 4.125-8.25 4.125s-8.25-1.847-8.25-4.125V6.375m16.5 0v3.75m-16.5-3.75v3.75m16.5 0v3.75C20.25 16.153 16.556 18 12 18s-8.25-1.847-8.25-4.125v-3.75m16.5 0v3.75C20.25 20.653 16.556 22.5 12 22.5s-8.25-1.847-8.25-4.125v-3.75m-16.5 0v3.75" /></svg>""",
"🔄": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="huge-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0 3.181 3.183a8.25 8.25 0 0 0 13.803-3.7M4.031 9.865a8.25 8.25 0 0 1 13.803-3.7l3.181 3.182m0-4.991v4.99" /></svg>""",
"🔘": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M8.25 6.75h12M8.25 12h12m-12 5.25h12M3.75 6.75h.007v.008H3.75V6.75Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0ZM3.75 12h.007v.008H3.75V12Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Zm-.375 5.25h.007v.008H3.75v-.008Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Z" /></svg>""",
"📍": """<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="nav-icon"><path stroke-linecap="round" stroke-linejoin="round" d="M15 10.5a3 3 0 1 1-6 0 3 3 0 0 1 6 0Z" /><path stroke-linecap="round" stroke-linejoin="round" d="M19.5 10.5c0 7.142-7.5 11.25-7.5 11.25S4.5 17.642 4.5 10.5a7.5 7.5 0 1 1 15 0Z" /></svg>""",
}
# Navigation icon mapping
NAV_ICONS_SVG = {
"Dashboard": ICON_DASHBOARD,
"Import": ICON_IMPORT,
"Exams": ICON_EXAMS,
"Reports": ICON_REPORTS,
"Settings": ICON_SETTINGS,
"Logout": ICON_LOGOUT,
}

0
app/api/__init__.py → backend/app/api/__init__.py
View File

0
app/api/v1/__init__.py → backend/app/api/v1/__init__.py
View File

8
app/api/v1/session.py → backend/app/api/v1/session.py
View File

@@ -50,6 +50,9 @@ class NextItemResponse(BaseModel):
options: Optional[dict] = None
slot: Optional[int] = None
level: Optional[str] = None
display_level: Optional[str] = None
generated_by: Optional[str] = None
source_snapshot_question_id: Optional[int] = None
selection_method: Optional[str] = None
reason: Optional[str] = None
current_theta: Optional[float] = None
@@ -212,6 +215,11 @@ async def get_next_item_endpoint(
options=item.options,
slot=item.slot,
level=item.level,
display_level="Original"
if item.generated_by != "ai" and item.source_snapshot_question_id is not None
else item.level,
generated_by=item.generated_by,
source_snapshot_question_id=item.source_snapshot_question_id,
selection_method=result.selection_method,
reason=result.reason,
current_theta=session.theta,

0
app/core/__init__.py → backend/app/core/__init__.py
View File

42
app/core/auth.py → backend/app/core/auth.py
View File

@@ -21,7 +21,7 @@ settings = get_settings()
@dataclass
class AuthContext:
website_id: int
website_id: Optional[int]
role: str
wp_user_id: Optional[str] = None
@@ -36,13 +36,13 @@ def _b64url_decode(raw: str) -> bytes:
def issue_access_token(
website_id: int,
website_id: int | None,
role: str = "student",
wp_user_id: str | None = None,
expires_in_seconds: int = 3600,
) -> str:
payload = {
"website_id": int(website_id),
"website_id": int(website_id) if website_id is not None else None,
"role": role,
"wp_user_id": wp_user_id,
"exp": int(time.time()) + int(expires_in_seconds),
@@ -91,14 +91,19 @@ def decode_access_token(token: str) -> AuthContext:
website_id = payload.get("website_id")
role = payload.get("role")
if website_id is None or not role:
if not role:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token missing required claims",
)
if website_id is None and role != "system_admin":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token missing website scope",
)
return AuthContext(
website_id=int(website_id),
website_id=int(website_id) if website_id is not None else None,
role=str(role),
wp_user_id=payload.get("wp_user_id"),
)
@@ -106,6 +111,7 @@ def decode_access_token(token: str) -> AuthContext:
def get_auth_context(
authorization: str | None = Header(None, alias="Authorization"),
x_website_id: str | None = Header(None, alias="X-Website-ID"),
) -> AuthContext:
if authorization is None:
raise HTTPException(
@@ -118,25 +124,45 @@ def get_auth_context(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid Authorization header format. Use: Bearer {token}",
)
return decode_access_token(parts[1])
context = decode_access_token(parts[1])
# If system_admin explicitly sets a website context via header, use it
if context.role == "system_admin" and x_website_id and x_website_id.isdigit():
context.website_id = int(x_website_id)
return context
def require_website_auth(
auth: AuthContext,
allowed_roles: set[str] | None = None,
) -> int:
) -> Optional[int]:
"""
Check if the authenticated user has required roles.
Returns the website_id if scoped to a specific website.
Returns None if the user is a system_admin with global access and no specific website context.
"""
if allowed_roles is not None and auth.role not in allowed_roles:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Insufficient permissions for this endpoint",
)
if auth.role == "system_admin":
if auth.website_id is not None:
return auth.website_id
return None
return auth.website_id
def ensure_website_scope_matches(
auth_website_id: int,
auth_website_id: int | None,
payload_website_id: int,
) -> None:
if auth_website_id is None:
return
if int(auth_website_id) != int(payload_website_id):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,

8
app/core/config.py → backend/app/core/config.py
View File

@@ -4,10 +4,10 @@ Application configuration using Pydantic Settings.
Loads configuration from environment variables with validation.
"""
from typing import Literal, List, Union
from typing import Annotated, Literal, List, Union
from pydantic import Field, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
class Settings(BaseSettings):
@@ -98,8 +98,8 @@ class Settings(BaseSettings):
)
# CORS - stored as list, accepts comma-separated string from env
ALLOWED_ORIGINS: List[str] = Field(
default=["http://localhost:3000"],
ALLOWED_ORIGINS: Annotated[List[str], NoDecode] = Field(
default=["http://localhost:3000", "http://localhost:5173", "http://127.0.0.1:5173"],
description="List of allowed CORS origins",
)

0
app/core/rate_limit.py → backend/app/core/rate_limit.py
View File

0
app/database.py → backend/app/database.py
View File

10
app/main.py → backend/app/main.py
View File

@@ -31,11 +31,13 @@ from app.database import close_db, init_db
from app.routers import (
admin_router,
ai_router,
auth_router,
import_export_router,
reports_router,
sessions_router,
tryouts_router,
wordpress_router,
websites_router,
)
settings = get_settings()
@@ -190,6 +192,10 @@ async def health_check():
# Include API routers with version prefix
app.include_router(
auth_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
import_export_router,
)
@@ -213,6 +219,10 @@ app.include_router(
reports_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
websites_router,
prefix=f"{settings.API_V1_STR}",
)
if settings.ENABLE_ADMIN:
app.include_router(

0
app/models/__init__.py → backend/app/models/__init__.py
View File

0
app/models/ai_generation_run.py → backend/app/models/ai_generation_run.py
View File

0
app/models/item.py → backend/app/models/item.py
View File

0
app/models/report_schedule.py → backend/app/models/report_schedule.py
View File

3
app/models/session.py → backend/app/models/session.py
View File

@@ -89,6 +89,9 @@ class Session(Base):
end_time: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True), nullable=True, comment="Session end timestamp"
)
expires_at: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True), nullable=True, comment="Session expiration timestamp"
)
is_completed: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, comment="Completion status"
)

0
app/models/tryout.py → backend/app/models/tryout.py
View File

0
app/models/tryout_import_snapshot.py → backend/app/models/tryout_import_snapshot.py
View File

0
app/models/tryout_snapshot_question.py → backend/app/models/tryout_snapshot_question.py
View File

0
app/models/tryout_stats.py → backend/app/models/tryout_stats.py
View File

0
app/models/user.py → backend/app/models/user.py
View File

0
app/models/user_answer.py → backend/app/models/user_answer.py
View File

0
app/models/website.py → backend/app/models/website.py
View File

4
app/routers/__init__.py → backend/app/routers/__init__.py
View File

@@ -4,18 +4,22 @@ API routers package.
from app.routers.admin import router as admin_router
from app.routers.ai import router as ai_router
from app.routers.auth import router as auth_router
from app.routers.import_export import router as import_export_router
from app.routers.reports import router as reports_router
from app.routers.sessions import router as sessions_router
from app.routers.tryouts import router as tryouts_router
from app.routers.wordpress import router as wordpress_router
from app.routers.websites import router as websites_router
__all__ = [
"admin_router",
"ai_router",
"auth_router",
"import_export_router",
"reports_router",
"sessions_router",
"tryouts_router",
"wordpress_router",
"websites_router",
]

1077
backend/app/routers/admin.py Normal file
View File

File diff suppressed because it is too large Load Diff

260
app/routers/ai.py → backend/app/routers/ai.py
View File

@@ -8,7 +8,7 @@ import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request, status
from sqlalchemy import and_, select
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
@@ -22,6 +22,9 @@ from app.core.rate_limit import enforce_rate_limit
from app.database import get_db
from app.models.item import Item
from app.schemas.ai import (
AIBatchGeneratedItem,
AIGenerateBatchRequest,
AIGenerateBatchResponse,
AIGeneratePreviewRequest,
AIGeneratePreviewResponse,
AISaveRequest,
@@ -30,8 +33,13 @@ from app.schemas.ai import (
)
from app.services.ai_generation import (
SUPPORTED_MODELS,
combine_usage,
create_generation_run,
generate_question,
generate_questions_batch,
generated_matches_basis_options,
get_ai_stats,
get_model_pricing,
save_ai_question,
validate_ai_model,
)
@@ -42,6 +50,19 @@ settings = get_settings()
router = APIRouter(prefix="/admin/ai", tags=["admin", "ai-generation"])
def _validate_original_basis_item(basis_item: Item) -> None:
if basis_item.level != "sedang":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Basis item must be 'sedang' level, got: {basis_item.level}",
)
if basis_item.generated_by == "ai":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Basis item must be an original question, not an AI-generated variant.",
)
@router.post(
"/generate-preview",
response_model=AIGeneratePreviewResponse,
@@ -107,12 +128,7 @@ async def generate_preview(
)
ensure_website_scope_matches(website_id, basis_item.website_id)
# Validate basis item is sedang level
if basis_item.level != "sedang":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Basis item must be 'sedang' level, got: {basis_item.level}",
)
_validate_original_basis_item(basis_item)
# Generate question
try:
@@ -137,6 +153,7 @@ async def generate_preview(
options=generated.options,
correct=generated.correct,
explanation=generated.explanation,
usage=generated.usage,
ai_model=request.ai_model,
basis_item_id=request.basis_item_id,
target_level=request.target_level,
@@ -171,7 +188,6 @@ async def generate_preview(
200: {"description": "Question saved successfully"},
400: {"description": "Invalid request data"},
404: {"description": "Basis item or tryout not found"},
409: {"description": "Item already exists at this slot/level"},
500: {"description": "Database save failed"},
},
)
@@ -185,8 +201,8 @@ async def generate_save(
Save AI-generated question to database.
- **stem**: Question text
- **options**: Dict with A, B, C, D options
- **correct**: Correct answer (A/B/C/D)
- **options**: Dict with the same option labels as the basis item
- **correct**: Correct answer label from the generated options
- **explanation**: Answer explanation (optional)
- **tryout_id**: Tryout identifier
- **website_id**: Website identifier
@@ -216,26 +232,7 @@ async def generate_save(
detail=f"Basis item not found: {request.basis_item_id}",
)
ensure_website_scope_matches(website_id, basis_item.website_id)
# Check for duplicate (same tryout, website, slot, level)
existing_result = await db.execute(
select(Item).where(
and_(
Item.tryout_id == request.tryout_id,
Item.website_id == request.website_id,
Item.slot == request.slot,
Item.level == request.level,
)
)
)
existing = existing_result.scalar_one_or_none()
if existing:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"Item already exists at slot={request.slot}, level={request.level} "
f"for tryout={request.tryout_id}",
)
_validate_original_basis_item(basis_item)
# Create GeneratedQuestion from request
from app.schemas.ai import GeneratedQuestion
@@ -246,6 +243,21 @@ async def generate_save(
correct=request.correct,
explanation=request.explanation,
)
if not generated_matches_basis_options(generated_data, basis_item):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Generated options must match the basis question option labels exactly.",
)
run_id = await create_generation_run(
basis_item_id=basis_item.id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
target_level=request.level,
requested_count=1,
model=request.ai_model,
created_by=auth.wp_user_id or auth.role,
db=db,
)
# Save to database
item_id = await save_ai_question(
@@ -256,6 +268,9 @@ async def generate_save(
slot=request.slot,
level=request.level,
ai_model=request.ai_model,
generation_run_id=run_id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
variant_status=request.variant_status,
db=db,
)
@@ -268,6 +283,111 @@ async def generate_save(
return AISaveResponse(
success=True,
item_id=item_id,
run_id=run_id,
)
@router.post(
"/generate-batch",
response_model=AIGenerateBatchResponse,
summary="Generate and save AI question batch",
description="Generate multiple trusted active variants from one medium-level basis question and track the run.",
)
async def generate_batch(
request_http: Request,
request: AIGenerateBatchRequest,
db: Annotated[AsyncSession, Depends(get_db)],
auth: AuthContext = Depends(get_auth_context),
) -> AIGenerateBatchResponse:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
await enforce_rate_limit(
request_http,
scope="ai.generate_batch",
max_requests=10,
window_seconds=300,
)
if not validate_ai_model(request.ai_model):
supported = ", ".join(SUPPORTED_MODELS.keys())
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Unsupported AI model: {request.ai_model}. Supported models: {supported}",
)
result = await db.execute(select(Item).where(Item.id == request.basis_item_id))
basis_item = result.scalar_one_or_none()
if not basis_item:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Basis item not found: {request.basis_item_id}",
)
ensure_website_scope_matches(website_id, basis_item.website_id)
_validate_original_basis_item(basis_item)
run_id = await create_generation_run(
basis_item_id=basis_item.id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
target_level=request.target_level,
requested_count=request.count,
model=request.ai_model,
created_by=auth.wp_user_id or auth.role,
operator_notes=request.operator_notes,
db=db,
)
generated_questions = await generate_questions_batch(
basis_item=basis_item,
target_level=request.target_level,
ai_model=request.ai_model,
count=request.count,
operator_notes=request.operator_notes,
)
item_ids: list[int] = []
response_items: list[AIBatchGeneratedItem] = []
for generated in generated_questions:
item_id = await save_ai_question(
generated_data=generated,
tryout_id=basis_item.tryout_id,
website_id=basis_item.website_id,
basis_item_id=basis_item.id,
slot=basis_item.slot,
level=request.target_level,
ai_model=request.ai_model,
db=db,
generation_run_id=run_id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
variant_status="active",
)
if item_id is not None:
item_ids.append(item_id)
response_items.append(
AIBatchGeneratedItem(
item_id=item_id,
stem=generated.stem,
options=generated.options,
correct=generated.correct,
explanation=generated.explanation,
level=request.target_level,
variant_status="active",
usage=generated.usage,
)
)
if not item_ids:
return AIGenerateBatchResponse(
success=False,
run_id=run_id,
generated_count=0,
error="AI generation failed. No variants were saved.",
)
return AIGenerateBatchResponse(
success=True,
run_id=run_id,
item_ids=item_ids,
items=response_items,
generated_count=len(item_ids),
usage=combine_usage([item.usage for item in response_items]),
)
@@ -313,8 +433,7 @@ async def list_models(auth: AuthContext = Depends(get_auth_context)) -> dict:
List supported AI models.
"""
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
return {
"models": [
configured_models = [
{
"id": settings.OPENROUTER_MODEL_CHEAP,
"name": "Mistral Small 4",
@@ -331,4 +450,81 @@ async def list_models(auth: AuthContext = Depends(get_auth_context)) -> dict:
"description": "Premium fallback when you want better quality over cost",
},
]
models = []
for model in configured_models:
pricing = await get_model_pricing(model["id"])
models.append({**model, "pricing": pricing})
return {"models": models}
@router.get(
"/pending-reviews",
summary="Get pending AI generated questions",
description="Retrieve all AI generated questions that are pending review (variant_status='draft').",
)
async def admin_get_pending_reviews(
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> dict:
"""Retrieve pending reviews."""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
query = (
select(Item)
.where(Item.generated_by == "ai", Item.variant_status == "draft")
.order_by(Item.created_at.desc())
.limit(200)
)
if website_id is not None:
query = query.where(Item.website_id == website_id)
result = await db.execute(query)
items = result.scalars().all()
return {
"items": [
{
"id": i.id,
"tryout_id": i.tryout_id,
"level": i.level,
"stem_text": i.stem_text if hasattr(i, 'stem_text') else i.stem[:100],
"ai_model": i.ai_model,
"basis_item_id": i.basis_item_id,
"created_at": i.created_at,
"status": i.variant_status,
}
for i in items
]
}
@router.post(
"/review/{item_id}",
summary="Approve or reject AI generated question",
description="Update the variant_status of an AI generated question.",
)
async def admin_review_ai_question(
item_id: int,
status: str, # "active", "rejected"
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> dict:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
result = await db.execute(select(Item).where(Item.id == item_id))
item = result.scalar_one_or_none()
if not item:
raise HTTPException(status_code=404, detail="Item not found")
if website_id is not None and item.website_id != website_id:
raise HTTPException(status_code=403, detail="Not authorized for this website")
if status not in ["active", "rejected"]:
raise HTTPException(status_code=400, detail="Status must be active or rejected")
item.variant_status = status
await db.commit()
return {"success": True, "item_id": item_id, "status": status}

60
backend/app/routers/auth.py Normal file
View File

@@ -0,0 +1,60 @@
"""
Authentication endpoints.
"""
from typing import Any, Dict
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel
from app.core.auth import issue_access_token
from app.core.config import get_settings
router = APIRouter(prefix="/auth", tags=["auth"])
settings = get_settings()
class LoginRequest(BaseModel):
username: str
password: str
@router.post(
"/admin-login",
summary="Admin Login",
description="Login for standalone app administration.",
)
async def admin_login(request: LoginRequest) -> Dict[str, Any]:
"""Authenticate an app admin and issue a JWT token."""
if not settings.ENABLE_ADMIN:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin functionality is disabled.",
)
if not settings.ADMIN_USERNAME or not settings.ADMIN_PASSWORD:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Admin credentials not configured.",
)
if (
request.username != settings.ADMIN_USERNAME
or request.password != settings.ADMIN_PASSWORD
):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid credentials",
)
token = issue_access_token(
website_id=None,
role="system_admin",
expires_in_seconds=86400 * 7, # 7 days
)
return {
"access_token": token,
"token_type": "bearer",
"role": "system_admin",
}

11
app/routers/import_export.py → backend/app/routers/import_export.py
View File

@@ -292,12 +292,6 @@ async def export_questions(
"""
Export questions to Excel file.
Creates Excel file with standardized format:
- Row 2: KUNCI (answer key)
- Row 4: TK (p-values)
- Row 5: BOBOT (weights)
- Rows 6+: Question data
Args:
tryout_id: Tryout identifier
website_id: Website ID from header
@@ -394,6 +388,11 @@ async def import_tryout_json(
db: AsyncSession = Depends(get_db),
) -> dict:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
if website_id is None:
x_website_id = request.headers.get("x-website-id")
if not x_website_id or not x_website_id.isdigit():
raise HTTPException(status_code=400, detail="X-Website-ID header is required for system_admin")
website_id = int(x_website_id)
await enforce_rate_limit(
request,
scope="import.tryout_json",

0
app/routers/normalization.py → backend/app/routers/normalization.py
View File

0
app/routers/reports.py → backend/app/routers/reports.py
View File

81
app/routers/sessions.py → backend/app/routers/sessions.py
View File

@@ -7,7 +7,7 @@ Endpoints:
- POST /session: Create new session
"""
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
@@ -25,6 +25,7 @@ from app.models.item import Item
from app.models.session import Session
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.user import User
from app.models.user_answer import UserAnswer
from app.schemas.session import (
SessionCompleteRequest,
@@ -83,14 +84,15 @@ async def complete_session(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get session with tryout relationship
result = await db.execute(
session_query = (
select(Session)
.options(selectinload(Session.tryout))
.where(
Session.session_id == session_id,
Session.website_id == website_id,
)
.where(Session.session_id == session_id)
)
if website_id is not None:
session_query = session_query.where(Session.website_id == website_id)
result = await db.execute(session_query)
session = result.scalar_one_or_none()
if session is None:
@@ -110,18 +112,25 @@ async def complete_session(
detail="Session does not belong to this authenticated user",
)
effective_website_id = session.website_id
# Get tryout configuration
tryout = session.tryout
# Get all items for this tryout to calculate bobot
items_result = await db.execute(
select(Item).where(
Item.website_id == website_id,
Item.website_id == effective_website_id,
Item.tryout_id == session.tryout_id,
)
)
items = {item.id: item for item in items_result.scalars().all()}
existing_answers_full_result = await db.execute(
select(UserAnswer).where(UserAnswer.session_id == session.session_id)
)
existing_answer_records = list(existing_answers_full_result.scalars().all())
# Process each answer
submitted_item_ids = [answer.item_id for answer in request.user_answers]
if len(submitted_item_ids) != len(set(submitted_item_ids)):
@@ -130,10 +139,7 @@ async def complete_session(
detail="Duplicate item answers are not allowed in a session completion",
)
existing_answers_result = await db.execute(
select(UserAnswer.item_id).where(UserAnswer.session_id == session.session_id)
)
existing_answered_item_ids = {row[0] for row in existing_answers_result.all()}
existing_answered_item_ids = {answer.item_id for answer in existing_answer_records}
duplicate_existing_ids = sorted(set(submitted_item_ids) & existing_answered_item_ids)
if duplicate_existing_ids:
raise HTTPException(
@@ -148,7 +154,15 @@ async def complete_session(
total_bobot_earned = 0.0
user_answer_records = []
for answer_input in request.user_answers:
if request.user_answers:
answers_to_score = request.user_answers
else:
answers_to_score = []
user_answer_records = existing_answer_records
total_benar = sum(1 for answer in existing_answer_records if answer.is_correct)
total_bobot_earned = sum(answer.bobot_earned or 0.0 for answer in existing_answer_records)
for answer_input in answers_to_score:
item = items.get(answer_input.item_id)
if item is None:
@@ -172,7 +186,7 @@ async def complete_session(
user_answer = UserAnswer(
session_id=session.session_id,
wp_user_id=session.wp_user_id,
website_id=website_id,
website_id=effective_website_id,
tryout_id=session.tryout_id,
item_id=item.id,
response=answer_input.response.upper(),
@@ -187,7 +201,7 @@ async def complete_session(
# Calculate total_bobot_max for NM calculation
try:
total_bobot_max = await get_total_bobot_max(
db, website_id, session.tryout_id, level="sedang"
db, effective_website_id, session.tryout_id, level="sedang"
)
except ValueError:
# Fallback: calculate from items we have
@@ -209,7 +223,7 @@ async def complete_session(
# Get current stats for dynamic normalization
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.website_id == effective_website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
@@ -226,7 +240,7 @@ async def complete_session(
# Hybrid: use dynamic if enough data, otherwise static
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.website_id == effective_website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
@@ -253,7 +267,7 @@ async def complete_session(
session.sb_used = sb
# Update tryout stats incrementally
await update_tryout_stats(db, website_id, session.tryout_id, nm)
await update_tryout_stats(db, effective_website_id, session.tryout_id, nm)
# Commit all changes
try:
@@ -276,6 +290,7 @@ async def complete_session(
tryout_id=session.tryout_id,
start_time=session.start_time,
end_time=session.end_time,
expires_at=session.expires_at,
is_completed=session.is_completed,
scoring_mode_used=session.scoring_mode_used,
total_benar=session.total_benar,
@@ -325,12 +340,11 @@ async def get_session(
"""
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
result = await db.execute(
select(Session).where(
Session.session_id == session_id,
Session.website_id == website_id,
)
)
session_query = select(Session).where(Session.session_id == session_id)
if website_id is not None:
session_query = session_query.where(Session.website_id == website_id)
result = await db.execute(session_query)
session = result.scalar_one_or_none()
if session is None:
@@ -375,6 +389,7 @@ async def create_session(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
ensure_website_scope_matches(website_id, request.website_id)
effective_website_id = website_id if website_id is not None else request.website_id
if auth.role == "student" and request.wp_user_id != auth.wp_user_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
@@ -384,7 +399,7 @@ async def create_session(
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.website_id == effective_website_id,
Tryout.tryout_id == request.tryout_id,
)
)
@@ -393,7 +408,7 @@ async def create_session(
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {request.tryout_id} not found for website {website_id}",
detail=f"Tryout {request.tryout_id} not found for website {effective_website_id}",
)
# Check if session already exists
@@ -408,14 +423,26 @@ async def create_session(
detail=f"Session {request.session_id} already exists",
)
user_result = await db.execute(
select(User).where(
User.wp_user_id == request.wp_user_id,
User.website_id == effective_website_id,
)
)
if user_result.scalar_one_or_none() is None:
db.add(User(wp_user_id=request.wp_user_id, website_id=effective_website_id))
started_at = datetime.now(timezone.utc)
# Create new session
session = Session(
session_id=request.session_id,
wp_user_id=request.wp_user_id,
website_id=website_id,
website_id=effective_website_id,
tryout_id=request.tryout_id,
scoring_mode_used=request.scoring_mode,
start_time=datetime.now(timezone.utc),
start_time=started_at,
expires_at=started_at + timedelta(hours=2),
is_completed=False,
total_benar=0,
total_bobot_earned=0.0,

190
app/routers/tryouts.py → backend/app/routers/tryouts.py
View File

@@ -19,11 +19,13 @@ from app.core.auth import AuthContext, get_auth_context, require_website_auth
from app.models.item import Item
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.tryout_snapshot_question import TryoutSnapshotQuestion
from app.schemas.tryout import (
NormalizationUpdateRequest,
NormalizationUpdateResponse,
TryoutConfigBrief,
TryoutConfigResponse,
TryoutConfigUpdateRequest,
TryoutStatsResponse,
)
@@ -53,14 +55,15 @@ async def get_tryout_config(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get tryout with stats
result = await db.execute(
query = (
select(Tryout)
.options(selectinload(Tryout.stats))
.where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
.where(Tryout.tryout_id == tryout_id)
)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
@@ -104,6 +107,73 @@ async def get_tryout_config(
)
@router.put(
"/{tryout_id}/config",
response_model=TryoutConfigResponse,
summary="Update tryout configuration",
description="Update editable tryout configuration fields.",
)
async def update_tryout_config(
tryout_id: str,
request: TryoutConfigUpdateRequest,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> TryoutConfigResponse:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
query = select(Tryout).options(selectinload(Tryout.stats)).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
update_data = request.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(tryout, field, value)
await db.commit()
await db.refresh(tryout)
current_stats = None
if tryout.stats:
current_stats = TryoutStatsResponse(
participant_count=tryout.stats.participant_count,
rataan=tryout.stats.rataan,
sb=tryout.stats.sb,
min_nm=tryout.stats.min_nm,
max_nm=tryout.stats.max_nm,
last_calculated=tryout.stats.last_calculated,
)
return TryoutConfigResponse(
id=tryout.id,
website_id=tryout.website_id,
tryout_id=tryout.tryout_id,
name=tryout.name,
description=tryout.description,
scoring_mode=tryout.scoring_mode,
selection_mode=tryout.selection_mode,
normalization_mode=tryout.normalization_mode,
min_sample_for_dynamic=tryout.min_sample_for_dynamic,
static_rataan=tryout.static_rataan,
static_sb=tryout.static_sb,
ai_generation_enabled=tryout.ai_generation_enabled,
hybrid_transition_slot=tryout.hybrid_transition_slot,
min_calibration_sample=tryout.min_calibration_sample,
theta_estimation_method=tryout.theta_estimation_method,
fallback_to_ctt_on_error=tryout.fallback_to_ctt_on_error,
current_stats=current_stats,
created_at=tryout.created_at,
updated_at=tryout.updated_at,
)
@router.put(
"/{tryout_id}/normalization",
response_model=NormalizationUpdateResponse,
@@ -134,12 +204,11 @@ async def update_normalization(
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
@@ -160,12 +229,11 @@ async def update_normalization(
tryout.static_sb = request.static_sb
# Get current stats for participant count
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats_query = select(TryoutStats).where(TryoutStats.tryout_id == tryout_id)
if website_id is not None:
stats_query = stats_query.where(TryoutStats.website_id == website_id)
stats_result = await db.execute(stats_query)
stats = stats_result.scalar_one_or_none()
current_participant_count = stats.participant_count if stats else 0
@@ -204,22 +272,42 @@ async def list_tryouts(
"""
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get tryouts with stats
result = await db.execute(
select(Tryout)
.options(selectinload(Tryout.stats))
.where(Tryout.website_id == website_id)
)
# Get tryouts with stats and items
query = select(Tryout).options(selectinload(Tryout.stats), selectinload(Tryout.items))
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryouts = result.scalars().all()
# Get snapshot counts for tryouts to show accurate item_count for JSON imports
snapshot_counts = {}
if tryouts:
tryout_ids = [t.tryout_id for t in tryouts]
count_query = (
select(TryoutSnapshotQuestion.source_tryout_id, func.count(TryoutSnapshotQuestion.id))
.where(TryoutSnapshotQuestion.source_tryout_id.in_(tryout_ids))
)
if website_id is not None:
count_query = count_query.where(TryoutSnapshotQuestion.website_id == website_id)
count_query = count_query.group_by(TryoutSnapshotQuestion.source_tryout_id)
count_result = await db.execute(count_query)
snapshot_counts = dict(count_result.all())
return [
TryoutConfigBrief(
website_id=t.website_id,
tryout_id=t.tryout_id,
name=t.name,
scoring_mode=t.scoring_mode,
selection_mode=t.selection_mode,
normalization_mode=t.normalization_mode,
participant_count=t.stats.participant_count if t.stats else 0,
rataan=t.stats.rataan if t.stats else None,
sb=t.stats.sb if t.stats else None,
item_count=len(t.items) or snapshot_counts.get(t.tryout_id, 0),
calibrated_item_count=sum(1 for i in t.items if i.calibrated),
)
for t in tryouts
]
@@ -254,12 +342,11 @@ async def get_calibration_status(
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -269,16 +356,16 @@ async def get_calibration_status(
)
# Get calibration statistics
stats_result = await db.execute(
select(
stats_query = select(
func.count().label("total_items"),
func.sum(cast(Item.calibrated, Integer)).label("calibrated_items"),
func.avg(Item.calibration_sample_size).label("avg_sample_size"),
).where(
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
).where(Item.tryout_id == tryout_id)
if website_id is not None:
stats_query = stats_query.where(Item.website_id == website_id)
stats_result = await db.execute(stats_query)
stats = stats_result.first()
total_items = stats.total_items or 0
@@ -331,12 +418,11 @@ async def trigger_calibration(
)
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -395,12 +481,11 @@ async def trigger_item_calibration(
from app.services.irt_calibration import calibrate_item, CALIBRATION_SAMPLE_THRESHOLD
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -410,13 +495,14 @@ async def trigger_item_calibration(
)
# Verify item belongs to this tryout
item_result = await db.execute(
select(Item).where(
item_query = select(Item).where(
Item.id == item_id,
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
if website_id is not None:
item_query = item_query.where(Item.website_id == website_id)
item_result = await db.execute(item_query)
item = item_result.scalar_one_or_none()
if item is None:

84
backend/app/routers/websites.py Normal file
View File

@@ -0,0 +1,84 @@
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.future import select
from pydantic import BaseModel
from app.database import get_db
from app.models import Website
from app.core.auth import AuthContext, get_auth_context, require_website_auth
router = APIRouter(tags=["websites"])
class WebsiteBase(BaseModel):
name: str
domain: str
class WebsiteResponse(WebsiteBase):
id: int
class Config:
from_attributes = True
@router.get("/websites", response_model=List[WebsiteResponse])
async def get_websites(
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
result = await db.execute(select(Website).order_by(Website.id.asc()))
websites = result.scalars().all()
# Map old columns (site_name, site_url) to new response format
return [
WebsiteResponse(
id=w.id,
name=w.site_name,
domain=w.site_url
) for w in websites
]
@router.post("/websites", response_model=WebsiteResponse)
async def create_website(
payload: WebsiteBase,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = Website(site_name=payload.name, site_url=payload.domain)
db.add(website)
await db.commit()
await db.refresh(website)
return WebsiteResponse(id=website.id, name=website.site_name, domain=website.site_url)
@router.put("/websites/{website_id}", response_model=WebsiteResponse)
async def update_website(
website_id: int,
payload: WebsiteBase,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = await db.get(Website, website_id)
if not website:
raise HTTPException(status_code=404, detail="Website not found")
website.site_name = payload.name
website.site_url = payload.domain
await db.commit()
await db.refresh(website)
return WebsiteResponse(id=website.id, name=website.site_name, domain=website.site_url)
@router.delete("/websites/{website_id}")
async def delete_website(
website_id: int,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = await db.get(Website, website_id)
if not website:
raise HTTPException(status_code=404, detail="Website not found")
await db.delete(website)
await db.commit()
return {"status": "success", "message": "Website deleted"}

0
app/routers/wordpress.py → backend/app/routers/wordpress.py
View File

0
app/schemas/__init__.py → backend/app/schemas/__init__.py
View File

180
backend/app/schemas/ai.py Normal file
View File

@@ -0,0 +1,180 @@
"""
Pydantic schemas for AI generation endpoints.
Request/response models for admin AI generation playground.
"""
from typing import Dict, Literal, Optional
from pydantic import BaseModel, Field, field_validator
OPTION_LABELS = tuple("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
class AIGeneratePreviewRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated question"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
class AIModelPricing(BaseModel):
prompt: Optional[float] = Field(
default=None, description="Input token price in USD per token"
)
completion: Optional[float] = Field(
default=None, description="Output token price in USD per token"
)
prompt_per_million: Optional[float] = Field(
default=None, description="Input token price in USD per 1M tokens"
)
completion_per_million: Optional[float] = Field(
default=None, description="Output token price in USD per 1M tokens"
)
currency: str = "USD"
source: str = "openrouter"
class AIUsageInfo(BaseModel):
prompt_tokens: Optional[int] = None
completion_tokens: Optional[int] = None
total_tokens: Optional[int] = None
cost_usd: Optional[float] = None
class AIGeneratePreviewResponse(BaseModel):
success: bool = Field(..., description="Whether generation was successful")
stem: Optional[str] = None
options: Optional[Dict[str, str]] = None
correct: Optional[str] = None
explanation: Optional[str] = None
ai_model: Optional[str] = None
basis_item_id: Optional[int] = None
target_level: Optional[str] = None
usage: Optional[AIUsageInfo] = None
error: Optional[str] = None
cached: bool = False
class AISaveRequest(BaseModel):
stem: str = Field(..., description="Question stem")
options: Dict[str, str] = Field(
..., description="Answer options. Labels must match the basis item exactly."
)
correct: str = Field(..., description="Correct answer option label")
explanation: Optional[str] = None
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
basis_item_id: int = Field(..., description="Basis item ID")
slot: int = Field(..., description="Question slot position")
level: Literal["mudah", "sedang", "sulit"] = Field(
..., description="Difficulty level"
)
variant_status: Literal["active", "draft"] = Field(
default="active",
description="Lifecycle status for the saved variant. Workspace approvals save active variants.",
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model used for generation",
)
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
label = v.upper()
if label not in OPTION_LABELS:
raise ValueError("Correct answer must be an option label A-Z")
return label
@field_validator("options")
@classmethod
def validate_options(cls, v: Dict[str, str]) -> Dict[str, str]:
normalized = {
str(key).strip().upper(): str(value).strip()
for key, value in v.items()
if str(key).strip() and str(value).strip()
}
if len(normalized) < 2:
raise ValueError("Options must contain at least two non-empty choices")
invalid_keys = sorted(set(normalized) - set(OPTION_LABELS))
if invalid_keys:
raise ValueError(f"Options contain invalid labels: {', '.join(invalid_keys)}")
return normalized
class AISaveResponse(BaseModel):
success: bool = Field(..., description="Whether save was successful")
item_id: Optional[int] = None
run_id: Optional[int] = None
error: Optional[str] = None
class AIGenerateBatchRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated questions"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
count: int = Field(default=3, ge=1, le=10, description="Number of variants to generate")
operator_notes: Optional[str] = None
class AIBatchGeneratedItem(BaseModel):
item_id: int
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
level: str
variant_status: str
usage: Optional[AIUsageInfo] = None
class AIGenerateBatchResponse(BaseModel):
success: bool
run_id: Optional[int] = None
item_ids: list[int] = Field(default_factory=list)
items: list[AIBatchGeneratedItem] = Field(default_factory=list)
generated_count: int = 0
usage: Optional[AIUsageInfo] = None
error: Optional[str] = None
class AIStatsResponse(BaseModel):
total_ai_items: int = Field(..., description="Total AI-generated items")
items_by_model: Dict[str, int] = Field(
default_factory=dict, description="Items count by AI model"
)
cache_hit_rate: float = Field(
default=0.0, description="Cache hit rate (0.0 to 1.0)"
)
total_cache_hits: int = Field(default=0, description="Total cache hits")
total_requests: int = Field(default=0, description="Total generation requests")
class GeneratedQuestion(BaseModel):
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
usage: Optional[AIUsageInfo] = None
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
label = v.upper()
if label not in OPTION_LABELS:
raise ValueError("Correct answer must be an option label A-Z")
return label

0
app/schemas/report.py → backend/app/schemas/report.py
View File

2
app/schemas/session.py → backend/app/schemas/session.py
View File

@@ -52,6 +52,7 @@ class SessionCompleteResponse(BaseModel):
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
expires_at: Optional[datetime] = None
is_completed: bool
scoring_mode_used: str
@@ -99,6 +100,7 @@ class SessionResponse(BaseModel):
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
expires_at: Optional[datetime] = None
is_completed: bool
scoring_mode_used: str

23
app/schemas/tryout.py → backend/app/schemas/tryout.py
View File

@@ -64,16 +64,39 @@ class TryoutStatsResponse(BaseModel):
class TryoutConfigBrief(BaseModel):
"""Brief tryout config for list responses."""
website_id: int
tryout_id: str
name: str
scoring_mode: str
selection_mode: str
normalization_mode: str
participant_count: Optional[int] = None
rataan: Optional[float] = None
sb: Optional[float] = None
item_count: int = 0
calibrated_item_count: int = 0
model_config = {"from_attributes": True}
class TryoutConfigUpdateRequest(BaseModel):
"""Request schema for updating editable tryout configuration."""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
scoring_mode: Optional[Literal["ctt", "irt", "hybrid"]] = None
selection_mode: Optional[Literal["fixed", "adaptive", "hybrid"]] = None
normalization_mode: Optional[Literal["static", "dynamic", "hybrid"]] = None
min_sample_for_dynamic: Optional[int] = Field(None, ge=1)
static_rataan: Optional[float] = Field(None, ge=0)
static_sb: Optional[float] = Field(None, gt=0)
ai_generation_enabled: Optional[bool] = None
hybrid_transition_slot: Optional[int] = Field(None, ge=1)
min_calibration_sample: Optional[int] = Field(None, ge=1)
theta_estimation_method: Optional[Literal["mle", "map", "eap"]] = None
fallback_to_ctt_on_error: Optional[bool] = None
class NormalizationUpdateRequest(BaseModel):
"""Request schema for updating normalization settings."""

0
app/schemas/wordpress.py → backend/app/schemas/wordpress.py
View File

0
app/services/__init__.py → backend/app/services/__init__.py
View File

250
app/services/ai_generation.py → backend/app/services/ai_generation.py
View File

@@ -9,6 +9,8 @@ import json
import logging
import re
import ast
import time
from dataclasses import dataclass
from typing import Any, Dict, Literal, Optional, Union
import httpx
@@ -20,13 +22,14 @@ from app.models.item import Item
from app.models.ai_generation_run import AIGenerationRun
from app.models.tryout import Tryout
from app.models.user_answer import UserAnswer
from app.schemas.ai import GeneratedQuestion
from app.schemas.ai import AIModelPricing, AIUsageInfo, GeneratedQuestion
logger = logging.getLogger(__name__)
settings = get_settings()
# OpenRouter API configuration
OPENROUTER_API_URL = "https://openrouter.ai/api/v1/chat/completions"
OPENROUTER_MODELS_URL = "https://openrouter.ai/api/v1/models"
# Supported AI models
SUPPORTED_MODELS = {
@@ -42,6 +45,159 @@ LEVEL_DESCRIPTIONS = {
"sulit": "harder (more complex concepts, multi-step reasoning)",
}
OPTION_LABELS = tuple("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
MODEL_PRICING_CACHE_TTL_SECONDS = 60 * 30
_model_pricing_cache: dict[str, tuple[float, AIModelPricing | None]] = {}
@dataclass
class OpenRouterCallResult:
content: str
usage: AIUsageInfo | None = None
def get_option_labels(options: Dict[str, str] | None) -> list[str]:
labels = {
str(key).strip().upper()
for key, value in (options or {}).items()
if str(key).strip() and str(value).strip()
}
return [label for label in OPTION_LABELS if label in labels]
def _parse_openrouter_price(value: Any) -> float | None:
if value is None:
return None
try:
price = float(value)
except (TypeError, ValueError):
return None
return price if price >= 0 else None
def _build_pricing(raw_pricing: dict[str, Any] | None) -> AIModelPricing | None:
if not raw_pricing:
return None
prompt = _parse_openrouter_price(raw_pricing.get("prompt"))
completion = _parse_openrouter_price(raw_pricing.get("completion"))
if prompt is None and completion is None:
return None
return AIModelPricing(
prompt=prompt,
completion=completion,
prompt_per_million=prompt * 1_000_000 if prompt is not None else None,
completion_per_million=completion * 1_000_000 if completion is not None else None,
)
async def get_model_pricing(model_id: str) -> AIModelPricing | None:
cached = _model_pricing_cache.get(model_id)
now = time.monotonic()
if cached and now - cached[0] < MODEL_PRICING_CACHE_TTL_SECONDS:
return cached[1]
headers = {"Content-Type": "application/json"}
if settings.OPENROUTER_API_KEY:
headers["Authorization"] = f"Bearer {settings.OPENROUTER_API_KEY}"
try:
timeout = httpx.Timeout(min(settings.OPENROUTER_TIMEOUT, 5))
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.get(OPENROUTER_MODELS_URL, headers=headers)
if response.status_code != 200:
logger.warning(
"OpenRouter models pricing request failed: %s - %s",
response.status_code,
response.text[:240],
)
_model_pricing_cache[model_id] = (now, None)
return None
for model in response.json().get("data", []):
if model.get("id") == model_id:
pricing = _build_pricing(model.get("pricing"))
_model_pricing_cache[model_id] = (now, pricing)
return pricing
except Exception as exc:
logger.warning("OpenRouter model pricing lookup failed for %s: %s", model_id, exc)
_model_pricing_cache[model_id] = (now, None)
return None
def _calculate_usage_cost(
prompt_tokens: int | None,
completion_tokens: int | None,
pricing: AIModelPricing | None,
provider_cost: Any = None,
) -> float | None:
provider_cost_value = _parse_openrouter_price(provider_cost)
if provider_cost_value is not None:
return provider_cost_value
if pricing is None:
return None
cost = 0.0
has_cost_component = False
if prompt_tokens is not None and pricing.prompt is not None:
cost += prompt_tokens * pricing.prompt
has_cost_component = True
if completion_tokens is not None and pricing.completion is not None:
cost += completion_tokens * pricing.completion
has_cost_component = True
return cost if has_cost_component else None
async def build_usage_info(raw_usage: dict[str, Any] | None, model_id: str) -> AIUsageInfo | None:
if not raw_usage:
return None
def token_count(key: str) -> int | None:
value = raw_usage.get(key)
if value is None:
return None
try:
return int(value)
except (TypeError, ValueError):
return None
prompt_tokens = token_count("prompt_tokens")
completion_tokens = token_count("completion_tokens")
total_tokens = token_count("total_tokens")
if total_tokens is None and (prompt_tokens is not None or completion_tokens is not None):
total_tokens = (prompt_tokens or 0) + (completion_tokens or 0)
pricing = await get_model_pricing(model_id)
cost_usd = _calculate_usage_cost(
prompt_tokens,
completion_tokens,
pricing,
provider_cost=raw_usage.get("cost"),
)
return AIUsageInfo(
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
total_tokens=total_tokens,
cost_usd=cost_usd,
)
def combine_usage(usages: list[AIUsageInfo | None]) -> AIUsageInfo | None:
filtered = [usage for usage in usages if usage is not None]
if not filtered:
return None
def summed(field: str) -> int | float | None:
values = [getattr(usage, field) for usage in filtered]
present = [value for value in values if value is not None]
return sum(present) if present else None
return AIUsageInfo(
prompt_tokens=summed("prompt_tokens"),
completion_tokens=summed("completion_tokens"),
total_tokens=summed("total_tokens"),
cost_usd=summed("cost_usd"),
)
def get_prompt_template(
basis_stem: str,
@@ -65,6 +221,10 @@ def get_prompt_template(
Formatted prompt string
"""
level_desc = LEVEL_DESCRIPTIONS.get(target_level, target_level)
option_labels = get_option_labels(basis_options) or ["A", "B", "C", "D"]
option_count = len(option_labels)
option_label_text = ", ".join(option_labels)
example_options = {label: f"Option {label} text" for label in option_labels}
options_text = "\n".join(
[f" {key}: {value}" for key, value in basis_options.items()]
@@ -103,16 +263,19 @@ Generate 1 new question that is {level_desc} than the basis question above.
REQUIREMENTS:
1. Keep the SAME topic/subject matter as the basis question
2. Use similar context and terminology
3. Create exactly 4 answer options (A, B, C, D)
4. Only ONE correct answer
5. Include a clear explanation of why the correct answer is correct
6. Make the question noticeably {level_desc} - not just a minor variation
3. Create exactly {option_count} answer options with labels exactly: {option_label_text}
4. Preserve the basis option count and option labels. Do not omit, add, rename, or merge answer options.
5. Only ONE correct answer, and it must be one of: {option_label_text}
6. Include a clear explanation of why the correct answer is correct
7. Make the question noticeably {level_desc} - not just a minor variation
8. Follow and preserve the basis question's inline HTML style. Keep structural and inline tags such as <p>, <br>, <strong>, <b>, <em>, <i>, <u>, <sub>, <sup>, and simple inline attributes such as text alignment when the basis uses them.
9. Do not escape HTML tags as text. Return HTML markup in the JSON string values exactly as markup.
OUTPUT FORMAT:
Return ONLY a valid JSON object with this exact structure (no markdown, no code blocks):
{{"stem": "Your question text here", "options": {{"A": "Option A text", "B": "Option B text", "C": "Option C text", "D": "Option D text"}}, "correct": "A", "explanation": "Explanation text here"}}
{{"stem": "Your question text here", "options": {json.dumps(example_options, ensure_ascii=False)}, "correct": "{option_labels[0]}", "explanation": "Explanation text here"}}
Remember: The correct field must be exactly "A", "B", "C", or "D"."""
Remember: The correct field must be exactly one of: {option_label_text}."""
return prompt
@@ -163,18 +326,13 @@ def validate_and_create_question(data: Dict[str, Any]) -> Optional[GeneratedQues
options = _normalize_options(data.get("options"))
if not options:
logger.warning("Options cannot be normalized to A/B/C/D map")
return None
required_options = {"A", "B", "C", "D"}
if not required_options.issubset(set(options.keys())):
logger.warning(f"Missing required options: {required_options - set(options.keys())}")
logger.warning("Options cannot be normalized to a labeled option map")
return None
correct = _normalize_correct_answer(
data.get("correct") or data.get("correct_answer") or data.get("answer")
)
if correct not in required_options:
if correct not in set(options.keys()):
logger.warning(f"Invalid correct answer: {correct}")
return None
@@ -257,7 +415,7 @@ def _try_parse_json_like(candidate: str) -> Any:
def _normalize_options(raw_options: Any) -> dict[str, str]:
if isinstance(raw_options, dict):
normalized = {str(k).strip().upper(): str(v).strip() for k, v in raw_options.items()}
return {k: normalized.get(k, "") for k in ["A", "B", "C", "D"] if normalized.get(k, "")}
return {k: normalized[k] for k in OPTION_LABELS if normalized.get(k, "")}
if isinstance(raw_options, list):
mapped: dict[str, str] = {}
@@ -268,9 +426,9 @@ def _normalize_options(raw_options: Any) -> dict[str, str]:
else:
key = ""
text = str(opt).strip()
if not key and idx < 4:
key = ["A", "B", "C", "D"][idx]
if key in {"A", "B", "C", "D"} and text:
if not key and idx < len(OPTION_LABELS):
key = OPTION_LABELS[idx]
if key in OPTION_LABELS and text:
mapped[key] = text
return mapped
@@ -281,24 +439,44 @@ def _normalize_correct_answer(raw_correct: Any) -> str:
if raw_correct is None:
return ""
raw_text = str(raw_correct).strip().upper()
if raw_text in {"A", "B", "C", "D"}:
if raw_text in OPTION_LABELS:
return raw_text
if raw_text.isdigit():
idx = int(raw_text)
if 1 <= idx <= 4:
return ["A", "B", "C", "D"][idx - 1]
if 0 <= idx <= 3:
return ["A", "B", "C", "D"][idx]
if raw_text in {"OPTION A", "OPTION B", "OPTION C", "OPTION D"}:
if 1 <= idx <= len(OPTION_LABELS):
return OPTION_LABELS[idx - 1]
if 0 <= idx < len(OPTION_LABELS):
return OPTION_LABELS[idx]
if raw_text.startswith("OPTION ") and raw_text[-1:] in OPTION_LABELS:
return raw_text[-1]
return raw_text[:1]
def generated_matches_basis_options(generated: GeneratedQuestion, basis_item: Item) -> bool:
basis_labels = get_option_labels(basis_item.options)
generated_labels = get_option_labels(generated.options)
if basis_labels != generated_labels:
logger.warning(
"Generated option labels do not match basis: basis=%s generated=%s",
basis_labels,
generated_labels,
)
return False
if generated.correct not in set(basis_labels):
logger.warning(
"Generated correct answer %s is outside basis labels %s",
generated.correct,
basis_labels,
)
return False
return True
async def call_openrouter_api(
prompt: str,
model: str,
max_retries: int = 3,
) -> Optional[str]:
) -> Optional[OpenRouterCallResult]:
"""
Call OpenRouter API to generate question.
@@ -308,7 +486,7 @@ async def call_openrouter_api(
max_retries: Maximum retry attempts
Returns:
API response text or None if failed
OpenRouterCallResult with response text and usage, or None if failed
"""
if not settings.OPENROUTER_API_KEY:
logger.error("OPENROUTER_API_KEY not configured")
@@ -361,7 +539,12 @@ async def call_openrouter_api(
choices = data.get("choices", [])
if choices:
message = choices[0].get("message", {})
return message.get("content")
content = message.get("content")
if not content:
logger.warning("OpenRouter response had no message content")
return None
usage = await build_usage_info(data.get("usage"), model)
return OpenRouterCallResult(content=content, usage=usage)
logger.warning("No choices in OpenRouter response")
return None
@@ -422,19 +605,20 @@ async def generate_question(
operator_notes=operator_notes,
)
max_generation_attempts = 2
max_generation_attempts = 3
for attempt in range(1, max_generation_attempts + 1):
response_text = await call_openrouter_api(prompt, ai_model)
if not response_text:
api_result = await call_openrouter_api(prompt, ai_model)
if not api_result:
logger.error("No response from OpenRouter API")
continue
generated = parse_ai_response(response_text)
if generated:
generated = parse_ai_response(api_result.content)
if generated and generated_matches_basis_options(generated, basis_item):
generated = generated.model_copy(update={"usage": api_result.usage})
return generated
logger.warning(
"Failed to parse AI response (attempt %s/%s), retrying",
"Failed to parse or validate AI response (attempt %s/%s), retrying",
attempt,
max_generation_attempts,
)

56
app/services/cat_selection.py → backend/app/services/cat_selection.py
View File

@@ -53,6 +53,30 @@ class TerminationCheck:
DEFAULT_SE_THRESHOLD = 0.5
# Default max items if not configured
DEFAULT_MAX_ITEMS = 50
SERVABLE_VARIANT_STATUSES = ("active", "approved")
def _servable_item_filter():
return Item.variant_status.in_(SERVABLE_VARIANT_STATUSES)
async def _get_user_answered_slot_levels(
db: AsyncSession,
wp_user_id: str,
website_id: int,
tryout_id: str,
) -> set[tuple[int, str]]:
"""Return slot/level pairs this user has already seen for this tryout."""
result = await db.execute(
select(Item.slot, Item.level)
.join(UserAnswer, UserAnswer.item_id == Item.id)
.where(
UserAnswer.wp_user_id == wp_user_id,
UserAnswer.website_id == website_id,
UserAnswer.tryout_id == tryout_id,
)
)
return {(int(slot), str(level)) for slot, level in result.all()}
async def get_next_item_fixed(
@@ -99,7 +123,8 @@ async def get_next_item_fixed(
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
Item.website_id == website_id,
_servable_item_filter(),
)
.order_by(Item.slot, Item.level)
)
@@ -113,7 +138,16 @@ async def get_next_item_fixed(
query = query.where(not_(Item.id.in_(answered_item_ids)))
result = await db.execute(query)
items = result.scalars().all()
items = list(result.scalars().all())
user_answered_slot_levels = await _get_user_answered_slot_levels(
db, session.wp_user_id, website_id, tryout_id
)
if user_answered_slot_levels:
items = [
item
for item in items
if (item.slot, item.level) not in user_answered_slot_levels
]
if not items:
return NextItemResult(
@@ -187,6 +221,7 @@ async def get_next_item_adaptive(
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
_servable_item_filter(),
Item.calibrated == True # Only calibrated items for IRT
)
)
@@ -204,7 +239,16 @@ async def get_next_item_adaptive(
query = query.where(Item.generated_by == 'manual')
result = await db.execute(query)
items = result.scalars().all()
items = list(result.scalars().all())
user_answered_slot_levels = await _get_user_answered_slot_levels(
db, session.wp_user_id, website_id, tryout_id
)
if user_answered_slot_levels:
items = [
item
for item in items
if (item.slot, item.level) not in user_answered_slot_levels
]
if not items:
return NextItemResult(
@@ -553,7 +597,8 @@ async def get_available_levels_for_slot(
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.slot == slot
Item.slot == slot,
_servable_item_filter(),
)
.distinct()
)
@@ -599,7 +644,8 @@ async def simulate_cat_selection(
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
Item.website_id == website_id,
_servable_item_filter(),
)
.order_by(Item.slot)
)

0
app/services/config_management.py → backend/app/services/config_management.py
View File

0
app/services/ctt_scoring.py → backend/app/services/ctt_scoring.py
View File

0
app/services/excel_import.py → backend/app/services/excel_import.py
View File

0
app/services/irt_calibration.py → backend/app/services/irt_calibration.py
View File

0
app/services/normalization.py → backend/app/services/normalization.py
View File

0
app/services/reporting.py → backend/app/services/reporting.py
View File

24
app/services/tryout_json_import.py → backend/app/services/tryout_json_import.py
View File

@@ -17,7 +17,7 @@ from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models import Item, TryoutImportSnapshot, TryoutSnapshotQuestion, Website
from app.models import Item, Tryout, TryoutImportSnapshot, TryoutSnapshotQuestion, Website
SOURCE_FORMAT = "sejoli_json"
DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
@@ -248,6 +248,28 @@ async def import_tryout_json_snapshot(payload: dict[str, Any], website_id: int,
db.add(snapshot)
await db.flush()
# Ensure operational tryout exists
result_tryout = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == source_tryout_id,
)
)
tryout = result_tryout.scalar_one_or_none()
if not tryout:
tryout = Tryout(
website_id=website_id,
tryout_id=source_tryout_id,
name=title,
description=f"Operational tryout basis created from imported snapshot #{snapshot.id}.",
scoring_mode="ctt",
selection_mode="fixed",
normalization_mode="static",
ai_generation_enabled=True,
)
db.add(tryout)
await db.flush()
existing_result = await db.execute(
select(TryoutSnapshotQuestion).where(
TryoutSnapshotQuestion.website_id == website_id,

0
app/services/wordpress_auth.py → backend/app/services/wordpress_auth.py
View File

37
backend/docker-compose.dev.yml Normal file
View File

@@ -0,0 +1,37 @@
services:
postgres:
image: postgres:15-alpine
environment:
POSTGRES_USER: irt_user
POSTGRES_PASSWORD: dev_password
POSTGRES_DB: irt_bank_soal
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
user: "70:70" # postgres user
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
app:
build: .
ports:
- "8080:8000"
environment:
DATABASE_URL: postgresql+asyncpg://irt_user:dev_password@postgres:5432/irt_bank_soal
REDIS_URL: redis://redis:6379
depends_on:
- postgres
- redis
volumes:
- .:/app
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
volumes:
postgres_data:
redis_data:

626
backend/docs/ALUR-APLIKASI-DAN-IRT.md Normal file
View File

@@ -0,0 +1,626 @@
# Alur Aplikasi IRT-Powered Question Bank
Dokumen ini menjelaskan alur lengkap aplikasi dari input data hingga menghasilkan next-question berbasis IRT.
---
## 1. Arsitektur Sistem
### 1.1 Teknologi Stack
```
Framework: FastAPI >= 0.104.1
Database: PostgreSQL + SQLAlchemy 2.0 (async)
AI: OpenAI (OpenRouter API)
Admin Panel: FastAPI-Admin
Math: numpy, scipy
Excel: openpyxl, pandas
```
### 1.2 Entity Relationship
```mermaid
erDiagram
Website ||--o{ Tryout : "hosts"
Website ||--o{ User : "contains"
Website ||--o{ Session : "serves"
Website ||--o{ Item : "contains"
Tryout ||--o{ Item : "contains"
Tryout ||--o{ Session : "has"
Session ||--o{ UserAnswer : "contains"
Item ||--o{ Item : "has variants"
Item ||--o{ UserAnswer : "answered by"
AIGenerationRun ||--o{ Item : "generates"
```
---
## 2. Konsep Inti
### 2.1 Tryout (Exam)
**Tryout** merepresentasikan 1 ujian lengkap dengan konfigurasi:
| Field | Opsi | Default | Deskripsi |
|-------|------|---------|-----------|
| `scoring_mode` | `ctt`, `irt`, `hybrid` | `ctt` | Metode kalkulasi score |
| `selection_mode` | `fixed`, `adaptive`, `hybrid` | `fixed` | Strategi pemilihan soal |
| `normalization_mode` | `static`, `dynamic`, `hybrid` | `static` | Metode normalisasi |
### 2.2 Item (Soal)
**Item** merepresentasikan 1 soal dengan parameter:
| Field | Deskripsi |
|-------|-----------|
| `stem` | Teks pertanyaan |
| `options` | Pilihan jawaban (A/B/C/D/E) |
| `correct_answer` | Kunci jawaban |
| `slot` | Posisi nomor soal (1, 2, 3...) |
| `level` | Kategori kesulitan (mudah/sedang/sulit) |
| `parent_item_id` | ID soal original (jika ini variant) |
| `calibrated` | Status IRT calibration |
| `irt_b` | Item difficulty parameter |
| `irt_se` | Standard error |
| `ctt_p` | P-value (tingkat kesukaran CTT) |
| `ctt_bobot` | Bobot soal = 1 - p |
### 2.3 Session (Percobaan Siswa)
**Session** melacak aktivitas siswa:
| Field | Deskripsi |
|-------|-----------|
| `session_id` | Identifier unik |
| `wp_user_id` | ID user dari WordPress |
| `tryout_id` | Tryout yang diambil |
| `theta` | Kemampuan estimasi IRT |
| `theta_se` | Standard error theta |
| `NM` | Nilai Mentah (raw score) |
| `NN` | Nilai Nasional (normalized) |
| `is_completed` | Status selesai |
### 2.4 Website (Multi-Tenant)
Sistem mendukung multiple WordPress websites dari 1 backend:
- Isolasi data per website
- Auth via `X-Website-ID` header
- WordPress JWT tokens
---
## 3. Alur Input Data
### 3.1 Sumber Data Masuk
| Sumber | Format | Endpoint | Fungsi |
|--------|--------|----------|--------|
| Admin Import | Excel (.xlsx) | `POST /import/excel` | Bulk import dari file Excel |
| JSON Import | JSON | `tryout_json_import.py` | Import dari JSON (LMS external) |
| AI Generation | API Request | `POST /ai/generate` | Generate variant soal baru |
### 3.2 Flow Import JSON
```
┌─────────────────────────────────────────────────────────────┐
│ ADMIN: Import Tryout JSON │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Upload JSON file │
│ └─> File berisi 1 tryout lengkap (misal: "TO 2024") │
│ └─> Terdiri dari N soal (slot 1, 2, 3, ...) │
│ │
│ 2. Parse JSON │
│ └─> Extract setiap soal → Item record │
│ └─> Generate unique item_id │
│ │
│ 3. Simpan ke Database │
│ └─> Item.calibrated = False (belum ada IRT params) │
│ └─> Item.ctt_p = NULL (belum ada response data) │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 3.3 Flow AI Generate Variants
```
┌─────────────────────────────────────────────────────────────┐
│ ADMIN: Generate AI Variants │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Pilih Item Original │
│ └─> Ambil 1 soal dari imported tryout │
│ │
│ 2. Request ke OpenRouter API │
│ └─> Kirim prompt dengan soal original │
│ └─> Minta generate variant dengan level berbeda │
│ │
│ 3. Simpan Variant │
│ └─> variant.item_id = unique_id │
│ └─> variant.parent_item_id = original.id │
│ └─> variant.slot = original.slot (nomor sama) │
│ │
│ 4. Result │
│ └─> Slot 1: 1 original + 1 variant = 2 soal │
│ └─> Slot 2: 1 original + 1 variant = 2 soal │
│ └─> Total: 2N soal (N slot × 2 variant) │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 3.4 Contoh Struktur Data Setelah Import + Generate
```
Tryout: "TO-2024"
├── Slot 1
│ ├── Item #1 (original, calibrated=True, irt_b=0.5)
│ └── Item #2 (variant, calibrated=True, irt_b=-0.3)
├── Slot 2
│ ├── Item #3 (original, calibrated=True, irt_b=0.8)
│ └── Item #4 (variant, calibrated=True, irt_b=0.2)
└── ...
```
---
## 4. Pemrosesan Scoring
### 4.1 CTT (Classical Test Theory)
#### Step-by-Step Formula:
```python
# STEP 1: Tingkat Kesukaran (p-value)
p = Σ Benar / Total Peserta
# Contoh: 70 siswa menjawab benar dari 100 siswa → p = 0.70
# STEP 2: Bobot (Weight)
bobot = 1 - p
# Contoh: bobot = 1 - 0.70 = 0.30
# STEP 3: Total Benar per Siswa
total_benar = count(correct answers)
# STEP 4: Total Bobot Earned per Siswa
total_bobot_siswa = Σ bobot for each correct answer
# Contoh: Jawab benar 3 soal dengan bobot [0.3, 0.5, 0.2] = 1.0
# STEP 5: Nilai Mentah (Raw Score)
NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000
# Contoh: NM = (1.0 / 2.5) × 1000 = 400
# STEP 6: Nilai Nasional (Normalized Score)
NN = 500 + 100 × ((NM - Rataan) / SB)
# Contoh: NN = 500 + 100 × ((400 - 450) / 80) = 437.5
```
#### Kategori Kesulitan (CTT Standard):
| p-value | Kategori | Arti |
|---------|----------|------|
| p < 0.30 | Sulit | Hanya <30% siswa menjawab benar |
| 0.30 ≤ p ≤ 0.70 | Sedang | 30-70% siswa menjawab benar |
| p > 0.70 | Mudah | >70% siswa menjawab benar |
### 4.2 IRT (Item Response Theory) - 1PL Rasch Model
#### Formula Inti:
```python
# Probability of correct response
P(θ, b) = 1 / (1 + exp(-(θ - b)))
# Di mana:
# - θ (theta) = kemampuan siswa [-3, +3]
# - b = difficulty soal [-3, +3]
# Contoh:
# - Siswa dengan θ = 0.5 menghadapi soal dengan b = 0.5
# - P(0.5, 0.5) = 1 / (1 + exp(0)) = 0.5 (50% kemungkinan benar)
```
#### Interpretasi Theta:
| Theta | Kemampuan | Persentase Benar (jika b=0) |
|-------|-----------|------------------------------|
| -3.0 | Sangat Lemah | ~5% |
| -1.5 | Lemah | ~18% |
| 0.0 | Rata-rata | ~50% |
| +1.5 | Cerdas | ~82% |
| +3.0 | Sangat Cerdas | ~95% |
#### Theta Estimation via MLE:
```python
# Log-likelihood
LL = Σ [u_i × log(P) + (1-u_i) × log(1-P)]
# u_i = 1 jika benar, 0 jika salah
# Theta estimation = maximize LL
θ_mle = argmax_θ LL(θ)
```
### 4.3 Kombinasi Scoring Mode
| Konfigurasi | Arti |
|-------------|------|
| `scoring_mode="ctt"` | Score akhir = NM, NN |
| `scoring_mode="irt"` | Score akhir = theta × 200 + 500 |
| `scoring_mode="hybrid"` | CTT score + IRT theta keduanya di-track |
---
## 5. IRT Calibration
### 5.1 Apa Itu Calibration?
**IRT Calibration** adalah proses mengestimasi parameter `b` (difficulty) untuk setiap soal berdasarkan response data dari siswa.
### 5.2 Kapan Item Became Calibrated?
```
┌─────────────────────────────────────────────────────────────┐
│ SYARAT ITEM CALIBRATED │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Minimum Response Sample │
│ └─> Ada cukup response data (default: 100 siswa) │
│ │
│ 2. IRT b Parameter │
│ └─> Sudah diestimasi via MLE │
│ │
│ 3. IRT SE (Standard Error) │
│ └─> Sudah dihitung │
│ │
│ 4. Item.calibrated = True │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 5.3 Flow IRT Calibration
```mermaid
flowchart TD
A[Collect Response Data] --> B{Have Min Sample?}
B -->|No| C[Wait for more students]
C --> A
B -->|Yes| D[For each Item]
D --> E[Build Response Matrix]
E --> F[Estimate b via MLE]
F --> G[Calculate Standard Error]
G --> H[Update Item.irt_b]
H --> I[Item.calibrated = True]
I --> D
D --> J[Calibration Complete]
```
### 5.4 Trigger Calibration
Calibration bisa dipicu via:
1. **API Endpoint:**
```
POST /tryout/{tryout_id}/calibrate
```
2. **Admin Panel:**
- Buka `/admin` → Tryouts → Pilih tryout → Trigger calibration
3. **Background Job (jika configured):**
- Setelah enough responses terkumpul
---
## 6. Item Selection Modes
### 6.1 Fixed Selection
**Fixed** = Soal disajikan berurutan berdasarkan slot.
```python
# Flow:
1. Siswa mulai session
2. Ambil item dengan slot=1 (urutan terendah)
3. Setelah dijawab, ambil slot=2
4. Lanjutkan sampai selesai
```
**Karakteristik:**
- Predictable, urutan soal tetap
- Tidak butuh IRT calibration
- Semua siswa dapat soal sama di posisi sama
### 6.2 Adaptive Selection (CAT)
**Adaptive** = Soal dipilih berdasarkan kemampuan siswa saat ini (theta).
```python
# Flow:
1. Siswa mulai session (θ = 0.0, default)
2. Pilih item dengan b ≈ θ
3. Siswa jawab → update θ
4. Pilih item baru dengan b ≈ θ baru
5. Ulangi sampai terminate condition
```
**Karakteristik:**
- Personalized, setiap siswa beda soal
- Butuh item calibrated
- Item selection pakai Fisher Information
#### Fisher Information Formula:
```python
# Information at current theta
I(θ) = P(θ) × (1 - P(θ))
# Di mana P(θ) = 1 / (1 + exp(-(θ - b)))
# Item dengan MAX information dipilih
# Maximum information = item paling informatif untuk theta saat ini
```
### 6.3 Hybrid Selection
**Hybrid** = Gabungan fixed + adaptive.
```python
# Flow:
1. Slot 1-N: Fixed selection (sequential)
2. Setelah slot N: Switch ke adaptive selection
3. Theta sudah ter-update dari fixed portion
4. Adaptive portion pakai theta untuk pilih soal
```
**Parameter:**
- `hybrid_transition_slot` = Slot dimana switch ke adaptive
### 6.4 Perbandingan Selection Modes
| Mode | Butuh Calibration | Personalisasi | Predictable |
|------|-------------------|---------------|-------------|
| Fixed | Tidak | Tidak | Ya |
| Adaptive | Ya | Ya | Tidak |
| Hybrid | Parsial | Parsial | Parsial |
---
## 7. Student Session Flow
### 7.1 Full Student Flow
```mermaid
sequenceDiagram
participant S as Student
participant API as FastAPI
participant DB as Database
S->>API: POST /session/ (start session)
API->>DB: Create session, θ=0.0
DB-->>API: session_id
API-->>S: session_id
loop For each question (adaptive/fixed/hybrid)
S->>API: GET /session/{id}/next-item
API->>DB: Query next item based on selection_mode
DB-->>API: Item data
API-->>S: Question
S->>API: POST /session/{id}/answer
API->>API: Update θ (if adaptive)
API->>DB: Save UserAnswer
DB-->>API: Saved
API-->>S: Ack + next question
end
S->>API: POST /session/{id}/complete
API->>API: Calculate NM, NN, final theta
API->>DB: Update session
DB-->>API: Updated
API-->>S: Final scores
```
### 7.2 Next-Item Selection Berdasarkan Mode
```
┌─────────────────────────────────────────────────────────────┐
│ SELECTION MODE = FIXED │
├─────────────────────────────────────────────────────────────┤
│ │
│ SELECT * FROM items │
│ WHERE tryout_id = ? │
│ AND item.id NOT IN (answered_items) │
│ ORDER BY slot ASC │
│ LIMIT 1 │
│ │
│ Result: Item dengan slot terkecil yang belum dijawab │
│ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ SELECTION MODE = ADAPTIVE │
├─────────────────────────────────────────────────────────────┤
│ │
│ current_theta = session.theta -- e.g., 0.5 │
│ │
│ SELECT * FROM items │
│ WHERE tryout_id = ? │
│ AND calibrated = TRUE │
│ AND item.id NOT IN (answered_items) │
│ ORDER BY ABS(irt_b - current_theta) ASC -- terdekat │
│ LIMIT 1 │
│ │
│ Result: Item dengan b ≈ θ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## 8. Konfigurasi Tryout
### 8.1 Semua Opsi Konfigurasi
```python
# Scoring
scoring_mode = "ctt" # ctt, irt, hybrid
scoring_mode = "irt" #
scoring_mode = "hybrid" #
# Selection
selection_mode = "fixed" # Sequential
selection_mode = "adaptive" # CAT based on theta
selection_mode = "hybrid" # Fixed until transition slot
# Normalization
normalization_mode = "static" # Use static_rataan, static_sb
normalization_mode = "dynamic" # Calculate from participant data
normalization_mode = "hybrid" # Dynamic when min_sample reached
# IRT Settings
min_calibration_sample = 100 # Min responses for calibration
theta_estimation_method = "mle" # mle, map, eap
fallback_to_ctt_on_error = True # Fallback if IRT fails
# Hybrid Settings
hybrid_transition_slot = 10 # Switch to adaptive at slot 10
# AI Settings
ai_generation_enabled = True # Allow AI generated items
```
### 8.2 Cara Mengubah Konfigurasi
#### Via Database:
```sql
UPDATE tryouts
SET
scoring_mode = 'hybrid',
selection_mode = 'adaptive',
normalization_mode = 'dynamic'
WHERE tryout_id = 'your-tryout-id';
```
#### Via Admin Panel:
1. Buka `/admin`
2. Pilih menu **Tryouts**
3. Edit tryout yang diinginkan
4. Ubah field-field sesuai kebutuhan
5. Save
---
## 9. Ringkasan Alur End-to-End
### 9.1 Admin Flow (Sekali / Periodik)
```
┌─────────────────────────────────────────────────────────────┐
│ 1. IMPORT TRYOUT JSON │
│ Input: File JSON (1 tryout = 1 exam) │
│ Output: N items dalam database │
│ │
│ 2. AI GENERATE VARIANTS │
│ Input: Item original │
│ Output: Item variant (same slot, different content) │
│ Result: 2N items (N slot × 2 variant) │
│ │
│ 3. COLLECT RESPONSE DATA │
│ Input: Student answers │
│ Output: UserAnswer records │
│ │
│ 4. IRT CALIBRATION │
│ Input: Response data (min 100 students) │
│ Output: Item.irt_b, Item.irt_se, Item.calibrated=True │
│ │
│ 5. CONFIGURE TRYOUT │
│ Input: Set selection_mode = 'adaptive' │
│ Output: Tryout siap untuk adaptive testing │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 9.2 Student Flow (Setiap Ujian)
```
┌─────────────────────────────────────────────────────────────┐
│ 1. START SESSION │
│ Input: tryout_id │
│ Output: session_id, theta=0.0 │
│ │
│ 2. ANSWER LOOP │
│ For each question: │
│ - Get next item (based on selection_mode) │
│ - Submit answer │
│ - If adaptive: update theta │
│ │
│ 3. COMPLETE SESSION │
│ Input: All answers │
│ Output: NM, NN, theta, completion status │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 9.3 Konsep Kunci
| Konsep | Penjelasan |
|--------|------------|
| **Tryout** | 1 exam yang di-import dari JSON |
| **Item** | 1 soal (original atau variant) |
| **Slot** | Posisi nomor soal (1, 2, 3...) |
| **Variant** | Soal berbeda di slot yang sama |
| **Calibrated** | Item sudah punya irt_b (siap untuk adaptive) |
| **Theta** | Estimasi kemampuan siswa dalam IRT scale |
---
## 10. FAQ
### Q: Kenapa default scoring_mode = "ctt"?
A: CTT lebih simpel, tidak butuh IRT calibration. Cocok untuk awal sebelum cukup data.
### Q: Kenapa default selection_mode = "fixed"?
A: Fixed selection tidak butuh item calibrated. Bisa jalan langsung setelah import.
### Q: Bagaimana switch ke adaptive?
A:
1. Pastikan item sudah calibrated (`calibrated = True`)
2. Ubah `selection_mode = 'adaptive'` di tryout
3. Student baru akan dapat adaptive selection
### Q: Adaptive butuh berapa banyak data?
A: Default `min_calibration_sample = 100`. Artinya minimal 100 siswa harus sudah menjawab sebelum calibration bisa jalan.
### Q: CTT dan Fixed itu sama?
A: Tidak. Mereka orthogonal:
- **scoring_mode** = bagaimana menghitung score akhir
- **selection_mode** = bagaimana memilih soal berikutnya
### Q: Aplikasi ini membuat exam?
A: Tidak. Aplikasi ini adalah **question bank**. Exam sudah di-import dari JSON. Aplikasi "mengembangbiakkan" soal dengan membuat variants.
---
## 11. Referensi Code
| File | Fungsi |
|------|--------|
| `app/services/ctt_scoring.py` | CTT scoring calculations |
| `app/services/irt_calibration.py` | IRT calibration, theta estimation |
| `app/services/cat_selection.py` | Item selection (fixed/adaptive/hybrid) |
| `app/services/ai_generation.py` | OpenRouter AI integration |
| `app/services/excel_import.py` | Excel import/export |
| `app/routers/sessions.py` | Session management API |
| `app/models/tryout.py` | Tryout model definition |
| `app/models/item.py` | Item model definition |
| `app/models/session.py` | Session model definition |
---
*Document version: 1.0*
*Last updated: 2026-06-15*

0
backend/error.html Normal file
View File

0
irt_1pl_mle.py → backend/irt_1pl_mle.py
View File

19
backend/patch_css.py Normal file
View File

@@ -0,0 +1,19 @@
import re
with open("app/admin_web.py", "r") as f:
content = f.read()
# Fix activity feed CSS
content = content.replace(
".activity-feed li:last-child {{ border-bottom: none; }}",
".activity-feed li:last-child {{ border-bottom: none; }}\n .activity-feed li svg, .activity-feed li svg.nav-icon, .activity-feed li svg.huge-icon {{ width: 20px; height: 20px; flex-shrink: 0; }}"
)
# Fix alert CSS
content = content.replace(
".alert-warning {{ background: #fef3c7; border: 1px solid #f59e0b; color: #92400e; }}",
".alert svg, .alert svg.huge-icon, .alert svg.page-icon {{ width: 24px; height: 24px; flex-shrink: 0; margin-right: 4px; margin-bottom: -4px; }}\n .alert-warning {{ background: #fef3c7; border: 1px solid #f59e0b; color: #92400e; }}"
)
with open("app/admin_web.py", "w") as f:
f.write(content)

17
backend/patch_icons.py Normal file
View File

@@ -0,0 +1,17 @@
import re
with open("app/admin_web_icons.py", "r") as f:
content = f.read()
new_mappings = """ "📈": ICON_TREND_UP,
"📉": ICON_TREND_DOWN,
"💡": ICON_LIGHTBULB,
"👋": '<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" style="display:inline;width:28px;height:28px;margin-bottom:-4px;"><path stroke-linecap="round" stroke-linejoin="round" d="M15.042 21.672 13.684 16.6m0 0-2.51 2.225.569-9.47 5.227 7.917-3.286-.672ZM12 2.25V4.5m5.834.166-1.591 1.591M20.25 10.5H18M7.757 14.743l-1.59 1.59M6 10.5H3.75m4.007-4.243-1.59-1.59" /></svg>',
"📊": ICON_REPORTS,
"🚀": ICON_HUGE_ROCKET,
"📈": ICON_TREND_UP,"""
content = content.replace(' "📈": ICON_TREND_UP,\n "📉": ICON_TREND_DOWN,', new_mappings)
with open("app/admin_web_icons.py", "w") as f:
f.write(content)

3
requirements.txt → backend/requirements.txt
View File

@@ -38,3 +38,6 @@ fastapi-admin>=1.0.0
# Utilities
python-dotenv>=1.0.0
# Async support
greenlet>=2.0.0

66
backend/run_local.sh Executable file
View File

@@ -0,0 +1,66 @@
#!/bin/bash
# Run local development server
set -e
echo "🚀 Starting IRT Bank Soal Local Dev Server"
echo "=========================================="
# Check if Docker is available
if ! command -v docker &> /dev/null; then
echo "❌ Docker not found. Please install Docker first."
exit 1
fi
# Check if docker-compose is available
if ! command -v docker-compose &> /dev/null && ! docker compose version &> /dev/null; then
echo "❌ Docker Compose not found. Please install Docker Compose first."
exit 1
fi
# Use docker compose command (Docker Desktop includes it as a plugin)
DOCKER_COMPOSE="docker compose"
# Start databases
echo "📦 Starting PostgreSQL and Redis..."
$DOCKER_COMPOSE -f docker-compose.dev.yml up -d postgres redis
# Wait for PostgreSQL to be ready
echo "⏳ Waiting for PostgreSQL..."
for i in {1..60}; do
if docker exec yellow-bank-soal-postgres-1 pg_isready -U irt_user -d irt_bank_soal &> /dev/null 2>&1; then
echo "✅ PostgreSQL is ready!"
break
fi
if [ $i -eq 60 ]; then
echo "❌ PostgreSQL failed to start"
docker logs yellow-bank-soal-postgres-1
exit 1
fi
sleep 1
done
# Check if venv exists, create if not
if [ ! -d "venv" ]; then
echo "📦 Creating Python virtual environment..."
python3 -m venv venv
fi
# Activate venv and install dependencies
echo "📦 Installing dependencies..."
source venv/bin/activate
pip install -r requirements.txt -q
# Run migrations
echo "🔄 Running database migrations..."
alembic upgrade head
# Start the dev server
echo ""
echo "🎉 Starting FastAPI dev server..."
echo " Admin UI: http://localhost:8000/admin"
echo " API Docs: http://localhost:8000/docs"
echo " Login: admin / admin123"
echo ""
echo "=========================================="
exec uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

275
backend/test_all_post_endpoints.py Normal file
View File

@@ -0,0 +1,275 @@
#!/usr/bin/env python3
"""
Comprehensive test of all form POST endpoints with proper authentication.
"""
import re
import sys
import httpx
BASE_URL = "http://localhost:8000"
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
response = client.get("/admin/login")
if response.status_code != 200:
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
return False
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
return response.status_code == 200 and "/admin/dashboard" in str(response.url)
def get_csrf_token(client: httpx.Client, page_url: str) -> str:
"""Extract CSRF token from a page."""
response = client.get(page_url)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1)
return ""
def test_endpoint(client: httpx.Client, name: str, url: str, data: dict) -> dict:
"""Test a single POST endpoint."""
csrf_token = get_csrf_token(client, url)
# Get the base URL (strip query params) for CSRF token extraction
base_url = url.split("?")[0] if "?" in url else url
# If we're on a different page, get CSRF token from there
if not csrf_token:
# Try to get CSRF from dashboard if it's a subpage
csrf_token = get_csrf_token(client, "/admin/dashboard")
if not csrf_token:
return {
"name": name,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Add CSRF token to data
test_data = data.copy()
test_data["csrf_token"] = csrf_token
response = client.post(
url,
data=test_data,
follow_redirects=True,
)
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
idx = response.text.find("Traceback")
traceback_text = response.text[idx : idx + 2000]
print(f"\n ⚠️ TRACEBACK on {name}:")
print(f" {traceback_text[:500]}...")
return {
"name": name,
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"error": None,
"response_preview": response.text[:500],
}
def main():
print("=" * 80)
print("Testing All Form POST Endpoints for Internal Server Errors")
print("=" * 80)
results = []
with httpx.Client(base_url=BASE_URL, timeout=60.0) as client:
print("\nStep 1: Logging in...")
if not login(client):
print("❌ Login failed")
return 1
print("✅ Login successful")
# Test 1: Variant approval (with item ID 4)
print("\nStep 2: Testing variant approval...")
result = test_endpoint(
client,
"Variant approval (/admin/questions/4/generate/review-bulk)",
"/admin/questions/4/generate?tab=review",
{"item_ids": "4", "action": "approved", "tab": "review"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 2: Basis item review
print("\nStep 3: Testing basis item review...")
result = test_endpoint(
client,
"Basis item review (/admin/basis-items/4/review-bulk)",
"/admin/basis-items/4",
{"item_ids": "4", "action": "approved"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 3: Generate variants for question
print("\nStep 4: Testing generate variants...")
result = test_endpoint(
client,
"Generate variants (/admin/questions/4/generate)",
"/admin/questions/4/generate?tab=generate",
{
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"include_note_for_admin": "on",
"include_note_in_prompt": "",
},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 5: Website creation
print("\nStep 5: Testing website creation...")
result = test_endpoint(
client,
"Website creation (/admin/websites)",
"/admin/websites",
{"site_name": "Test Site API", "site_url": "https://test-api.example.com"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 6: Website deletion (with test ID)
print("\nStep 6: Testing website deletion...")
# First create a website
result_create = test_endpoint(
client,
"Create test website",
"/admin/websites",
{
"site_name": "Delete Test Site",
"site_url": "https://delete-test.example.com",
},
)
# Now delete it (using website ID 2 if exists)
result = test_endpoint(
client,
"Website deletion (/admin/websites/2/delete)",
"/admin/websites/2/delete",
{},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 7: Tryout import preview (without file - should get validation error not server error)
print("\nStep 7: Testing tryout import preview...")
result = test_endpoint(
client,
"Tryout import preview (/admin/tryout-import/preview)",
"/admin/tryout-import",
{"website_id": "1"},
)
results.append(result)
print(f" Status: {result['status_code']} (validation error expected: 422)")
# Test 8: Snapshot promote bulk
print("\nStep 8: Testing snapshot promote bulk...")
result = test_endpoint(
client,
"Snapshot promote (/admin/snapshot-questions/promote-bulk)",
"/admin/snapshot-questions",
{"snapshot_id": "1", "snapshot_question_ids": ""},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 9: AI generation basis item
print("\nStep 9: Testing AI generation for basis item...")
result = test_endpoint(
client,
"Basis item generate (/admin/basis-items/4/generate)",
"/admin/basis-items/4",
{
"target_level": "mudah",
"ai_model": "",
"generation_count": "1",
"operator_notes": "",
},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Summary
print("\n" + "=" * 80)
print("RESULTS SUMMARY")
print("=" * 80)
errors = []
for result in results:
if result.get("has_traceback"):
errors.append(f"❌ {result['name']}: TRACEBACK")
print(f"❌ {result['name']}: TRACEBACK")
elif result.get("has_ise"):
errors.append(f"❌ {result['name']}: INTERNAL SERVER ERROR")
print(f"❌ {result['name']}: INTERNAL SERVER ERROR")
elif result.get("error"):
print(f"⚠️ {result['name']}: {result['error']}")
elif result["status_code"] in [200, 303]:
print(f"✅ {result['name']}: OK ({result['status_code']})")
elif result["status_code"] == 422:
print(f"✅ {result['name']}: Validation Error (expected)")
else:
print(f"⚠️ {result['name']}: Status {result['status_code']}")
print()
if errors:
print("❌ Some endpoints have INTERNAL SERVER ERRORS:")
for error in errors:
print(f" {error}")
return 1
else:
print("✅ All form POST endpoints tested successfully!")
print(" No Internal Server Errors detected.")
return 0
if __name__ == "__main__":
sys.exit(main())

346
backend/test_all_routes.py Normal file
View File

@@ -0,0 +1,346 @@
#!/usr/bin/env python3
"""
Test all routes in the IRT Bank Soal application.
Tests each endpoint and checks for Internal Server Errors.
"""
import json
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from urllib.parse import urlparse
import httpx
BASE_URL = "http://localhost:8000"
# All routes from OpenAPI spec
API_ROUTES = [
# Root endpoints
("GET", "/"),
("GET", "/health"),
# Session endpoints
("POST", "/api/v1/session/"),
("GET", "/api/v1/session/{session_id}"),
("POST", "/api/v1/session/{session_id}/complete"),
("GET", "/api/v1/session/{session_id}/next_item"),
("POST", "/api/v1/session/{session_id}/submit_answer"),
# Tryout endpoints
("GET", "/api/v1/tryout/"),
("GET", "/api/v1/tryout/{tryout_id}/config"),
("PUT", "/api/v1/tryout/{tryout_id}/normalization"),
("GET", "/api/v1/tryout/{tryout_id}/calibration-status"),
("POST", "/api/v1/tryout/{tryout_id}/calibrate"),
("POST", "/api/v1/tryout/{tryout_id}/calibrate/{item_id}"),
# WordPress endpoints
("POST", "/api/v1/wordpress/sync_users"),
("POST", "/api/v1/wordpress/verify_session"),
("GET", "/api/v1/wordpress/website/{website_id}/users"),
("GET", "/api/v1/wordpress/website/{website_id}/user/{wp_user_id}"),
# Reports endpoints
("POST", "/api/v1/reports/schedule"),
("GET", "/api/v1/reports/schedule/{schedule_id}"),
("GET", "/api/v1/reports/schedules"),
("DELETE", "/api/v1/reports/schedule/{schedule_id}"),
("POST", "/api/v1/reports/schedule/{schedule_id}/export"),
("GET", "/api/v1/reports/student/performance"),
("GET", "/api/v1/reports/student/performance/export/{format}"),
("GET", "/api/v1/reports/items/analysis"),
("GET", "/api/v1/reports/items/analysis/export/{format}"),
("GET", "/api/v1/reports/calibration/status"),
("GET", "/api/v1/reports/calibration/status/export/{format}"),
("GET", "/api/v1/reports/tryout/comparison"),
("GET", "/api/v1/reports/tryout/comparison/export/{format}"),
("GET", "/api/v1/reports/export/{schedule_id}/{format}"),
# Import/Export endpoints
("POST", "/api/v1/import-export/preview"),
("POST", "/api/v1/import-export/questions"),
("GET", "/api/v1/import-export/export/questions"),
("POST", "/api/v1/import-export/tryout-json/preview"),
("POST", "/api/v1/import-export/tryout-json"),
# Admin AI endpoints
("POST", "/api/v1/admin/ai/generate-preview"),
("POST", "/api/v1/admin/ai/generate-save"),
("GET", "/api/v1/admin/ai/stats"),
("GET", "/api/v1/admin/ai/models"),
# Admin endpoints
("POST", "/api/v1/admin/{tryout_id}/calibrate"),
("POST", "/api/v1/admin/{tryout_id}/toggle-ai-generation"),
("POST", "/api/v1/admin/{tryout_id}/reset-normalization"),
# Admin CAT endpoints
("POST", "/api/v1/admin/cat/test"),
("GET", "/api/v1/admin/session/{session_id}/status"),
# Admin web routes (HTML pages)
("GET", "/admin"),
("GET", "/admin/login"),
("POST", "/admin/login"),
("POST", "/admin/logout"),
("GET", "/admin/password"),
("POST", "/admin/password"),
("GET", "/admin/dashboard"),
("GET", "/admin/questions"),
("GET", "/admin/questions/{item_id}"),
("GET", "/admin/questions/{item_id}/quality"),
("GET", "/admin/exams"),
("GET", "/admin/exams/{tryout_id}"),
("GET", "/admin/reports"),
("GET", "/admin/settings"),
("GET", "/admin/hierarchy"),
("GET", "/admin/websites"),
("POST", "/admin/websites"),
("GET", "/admin/websites/new"),
("GET", "/admin/websites/{website_id}"),
("POST", "/admin/websites/{website_id}"),
("POST", "/admin/websites/{website_id}/delete"),
("GET", "/admin/tryout-import"),
("GET", "/admin/tryout-import/preview"),
("POST", "/admin/tryout-import"),
("GET", "/admin/snapshot-questions"),
("POST", "/admin/snapshot-questions/promote-bulk"),
("GET", "/admin/calibration-status"),
("GET", "/admin/item-statistics"),
("GET", "/admin/sessions"),
("GET", "/admin/basis-items"),
("GET", "/admin/basis-items/{item_id}"),
("POST", "/admin/basis-items/{item_id}/generate"),
("POST", "/admin/basis-items/{item_id}/generate/review-bulk"),
("GET", "/admin/basis-items/{item_id}/generate/variants/{variant_id}"),
]
# Placeholder values for path parameters
PLACEHOLDERS = {
"{session_id}": "test-session-123",
"{tryout_id}": "test-tryout-123",
"{item_id}": "1",
"{website_id}": "1",
"{wp_user_id}": "123",
"{schedule_id}": "test-schedule-123",
"{format}": "xlsx",
"{variant_id}": "test-variant-123",
}
# Minimal request bodies for POST endpoints
REQUEST_BODIES = {
"/api/v1/session/": {
"session_id": "test",
"tryout_id": "test",
"wp_user_id": "123",
"website_id": 1,
"scoring_mode": "ctt",
},
"/api/v1/session/{session_id}/complete": {
"end_time": "2024-01-01T00:00:00Z",
"user_answers": [],
},
"/api/v1/session/{session_id}/submit_answer": {
"item_id": 1,
"response": "A",
"time_spent": 10,
},
"/api/v1/tryout/{tryout_id}/normalization": {
"normalization_mode": "static",
"static_rataan": 500,
"static_sb": 100,
},
"/api/v1/wordpress/sync_users": {}, # Requires proper auth header
"/api/v1/wordpress/verify_session": {
"website_id": 1,
"wp_user_id": "123",
"token": "test",
},
"/api/v1/reports/schedule": {
"tryout_id": "test",
"report_type": "student_performance",
},
"/api/v1/admin/ai/generate-preview": {
"basis_item_id": 1,
"target_level": "sulit",
"ai_model": "qwen/qwen2.5-32b-instruct",
},
"/api/v1/admin/ai/generate-save": {
"stem": "Test?",
"options": {"A": "a", "B": "b", "C": "c", "D": "d"},
"correct": "A",
"tryout_id": "test",
"website_id": 1,
"basis_item_id": 1,
"slot": 1,
"level": "sulit",
"ai_model": "qwen/qwen2.5-32b-instruct",
},
"/api/v1/admin/cat/test": {"tryout_id": "test", "website_id": 1},
"/api/v1/admin/{tryout_id}/calibrate": {},
"/api/v1/admin/{tryout_id}/toggle-ai-generation": {},
"/api/v1/admin/{tryout_id}/reset-normalization": {},
"/api/v1/import-export/preview": None, # Requires file upload
"/api/v1/import-export/questions": None, # Requires file upload
"/api/v1/import-export/tryout-json/preview": None, # Requires file upload
"/api/v1/import-export/tryout-json": None, # Requires file upload
}
def expand_route(method: str, route: str) -> list:
"""Expand route with placeholders."""
expanded = []
test_route = route
for placeholder, value in PLACEHOLDERS.items():
if placeholder in test_route:
test_route = test_route.replace(placeholder, value)
expanded.append((method, test_route))
return expanded
def test_route(client: httpx.Client, method: str, route: str) -> dict:
"""Test a single route."""
# Expand placeholders
expanded = expand_route(method, route)
if not expanded:
return {
"route": route,
"method": method,
"error": "Could not expand route",
"status_code": None,
}
method, test_route = expanded[0]
# Determine request body
body = None
request_body = REQUEST_BODIES.get(route, REQUEST_BODIES.get(test_route, {}))
if request_body is not None:
body = request_body
# Determine query params
params = {}
if "export/questions" in route:
params = {"tryout_id": "test"}
headers = {"X-Website-ID": "1"}
try:
response = client.request(
method=method,
url=BASE_URL + test_route,
json=body if body and method in ["POST", "PUT", "PATCH"] else None,
params=params,
headers=headers,
timeout=10.0,
follow_redirects=True,
)
is_500 = response.status_code == 500
is_ise = "Internal Server Error" in response.text
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": response.status_code,
"has_500": is_500,
"has_ise": is_ise,
"response_preview": response.text[:200] if response.text else "",
"error": None,
}
except httpx.TimeoutException:
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": None,
"has_500": False,
"has_ise": False,
"response_preview": "",
"error": "Timeout",
}
except Exception as e:
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": None,
"has_500": False,
"has_ise": False,
"response_preview": "",
"error": str(e),
}
def main():
print("=" * 80)
print("Testing all IRT Bank Soal routes for Internal Server Errors")
print("=" * 80)
print()
results = []
has_errors = False
with httpx.Client(timeout=30.0) as client:
for method, route in API_ROUTES:
result = test_route(client, method, route)
results.append(result)
status = result["status_code"]
error_marker = ""
if result["error"]:
error_marker = f" [ERROR: {result['error']}]"
has_errors = True
elif status and status >= 500:
error_marker = f" [INTERNAL SERVER ERROR!]"
has_errors = True
elif status and status == 500:
error_marker = f" [500 - INTERNAL SERVER ERROR!]"
has_errors = True
elif "Internal Server Error" in str(result.get("response_preview", "")):
error_marker = " [500 - INTERNAL SERVER ERROR!]"
has_errors = True
status_str = str(status) if status else "N/A"
print(f"{method:6} {route:<60} -> {status_str}{error_marker}")
print()
print("=" * 80)
print("SUMMARY")
print("=" * 80)
total = len(results)
successful = sum(1 for r in results if r["status_code"] and r["status_code"] < 500)
client_errors = sum(
1 for r in results if r["status_code"] and 400 <= r["status_code"] < 500
)
server_errors = sum(
1 for r in results if r["status_code"] and r["status_code"] >= 500
)
timeouts = sum(1 for r in results if r["error"] == "Timeout")
exceptions = sum(1 for r in results if r["error"] and r["error"] != "Timeout")
ise_errors = sum(1 for r in results if r.get("has_ise") or r.get("has_500"))
print(f"Total routes tested: {total}")
print(f"Successful (2xx): {successful}")
print(f"Client errors (4xx): {client_errors}")
print(f"Server errors (5xx): {server_errors}")
print(f"Timeouts: {timeouts}")
print(f"Exceptions: {exceptions}")
print(f"Internal Server Errors: {ise_errors}")
print()
if has_errors:
print("Routes with issues:")
for r in results:
if r["status_code"] and r["status_code"] >= 500:
print(f" - {r['method']} {r['route']} -> {r['status_code']}")
elif r["error"]:
print(f" - {r['method']} {r['route']} -> ERROR: {r['error']}")
elif r.get("has_ise"):
print(f" - {r['method']} {r['route']} -> Internal Server Error")
print()
if ise_errors == 0 and exceptions == 0:
print("✅ All routes passed! No Internal Server Errors detected.")
return 0
else:
print("❌ Some routes have issues. Please review the output above.")
return 1
if __name__ == "__main__":
sys.exit(main())

50
backend/test_debug_login.py Normal file
View File

@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""
Debug login issue.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging login issue...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Get login page
response = client.get("/admin/login")
print(f"Login page status: {response.status_code}")
# Extract CSRF token
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
print(f"CSRF token: {csrf_token[:30]}...")
# Look for any error messages in the page
if "error" in response.text.lower():
print("\n=== Error messages in login page ===")
# Extract error div content
error_match = re.search(
r'<div class="error">(.*?)</div>', response.text, re.DOTALL
)
if error_match:
print(error_match.group(1))
else:
# Print a portion of the page around "error"
idx = response.text.lower().find("error")
print(response.text[max(0, idx - 50) : idx + 200])
# Try to check if Redis is accessible via the health endpoint
health = client.get("/health")
print(f"\nHealth check: {health.text}")
# Print login page content for inspection
print("\n=== Login page content (first 2000 chars) ===")
print(response.text[:2000])
if __name__ == "__main__":
main()

73
backend/test_debug_login2.py Normal file
View File

@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""
Debug login issue - check Redis.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging login issue - detailed...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Get login page
response = client.get("/admin/login")
print(f"Login page status: {response.status_code}")
# Extract CSRF token
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
print(f"CSRF token: {csrf_token}")
# Print ALL cookies
print(f"\nCookies before login: {dict(client.cookies)}")
# Submit login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=False, # Don't follow redirect to see the response
)
print(f"\nLogin response status: {response.status_code}")
print(f"Login response headers: {dict(response.headers)}")
print(f"Cookies after login: {dict(client.cookies)}")
# Check if response has any content
print(f"\nLogin response content (first 1000 chars):")
print(response.text[:1000])
# Now try with a redirect follow
print("\n\n=== Trying with redirect follow ===")
client2 = httpx.Client(base_url=BASE_URL, timeout=30.0)
response = client2.get("/admin/login")
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
response = client2.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f"Final status after redirect: {response.status_code}")
print(f"Final URL: {response.url}")
print(f"Final cookies: {dict(client2.cookies)}")
print(f"Final content (first 500 chars): {response.text[:500]}")
if __name__ == "__main__":
main()

142
backend/test_debug_traceback.py Normal file
View File

@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""
Debug the 500 Internal Server Error on variant approval - fixed CSRF.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
response = client.get("/admin/login")
if response.status_code != 200:
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
return False
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
return response.status_code == 200 and "/admin/dashboard" in str(response.url)
def get_csrf_from_page(client: httpx.Client, page_url: str) -> tuple:
"""Get CSRF token from a specific page and return both token and response."""
response = client.get(page_url, follow_redirects=True)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1), response
return "", response
def main():
print("=" * 80)
print("Debugging 500 Internal Server Error on Variant Approval")
print("=" * 80)
with httpx.Client(base_url=BASE_URL, timeout=60.0) as client:
print("\n1. Logging in...")
if not login(client):
print(" ❌ Login failed")
return
print(" ✅ Login successful")
# Test 1: Variant approval - get CSRF from the actual review page
print("\n2. Testing variant approval...")
# First access the review page to get the CSRF token
csrf_token, page_response = get_csrf_from_page(
client, "/admin/questions/4/generate?tab=review"
)
print(f" Page URL: {page_response.url}")
print(f" Page status: {page_response.status_code}")
print(f" CSRF token: {csrf_token[:30] if csrf_token else 'None'}...")
# If we got redirected, we can't test this endpoint
if "/generate" not in str(page_response.url):
print(
" ⚠️ Redirected away from AI playground - item may not exist or not be AI-generated"
)
print(" Skipping this test...")
else:
# Submit the form
response = client.post(
"/admin/questions/4/generate/review-bulk",
data={
"item_ids": "4",
"action": "approved",
"tab": "review",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
# Extract and print the full traceback
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print("\n" + "=" * 80)
print("FULL TRACEBACK:")
print("=" * 80)
print(response.text[idx:])
print("=" * 80)
elif response.status_code == 500:
print("\n ⚠️ Got 500 error but no traceback in response")
print(f" Response preview: {response.text[:500]}")
else:
print(f" Response preview: {response.text[:500]}")
# Test 2: Generate variants
print("\n3. Testing generate variants...")
csrf_token, page_response = get_csrf_from_page(
client, "/admin/questions/4/generate?tab=generate"
)
print(f" Page URL: {page_response.url}")
print(f" Page status: {page_response.status_code}")
if "/generate" not in str(page_response.url):
print(" ⚠️ Redirected away from AI playground")
else:
response = client.post(
"/admin/questions/4/generate",
data={
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print("\n" + "=" * 80)
print("FULL TRACEBACK:")
print("=" * 80)
print(response.text[idx:])
print("=" * 80)
if __name__ == "__main__":
main()

8
backend/test_error.py Normal file
View File

@@ -0,0 +1,8 @@
import asyncio
from fastapi.testclient import TestClient
from app.main import app
client = TestClient(app)
response = client.get("/admin/hierarchy")
print(response.status_code)
print(response.text)

9
backend/test_fetch.py Normal file
View File

@@ -0,0 +1,9 @@
import asyncio
from fastapi.testclient import TestClient
from app.main import app
client = TestClient(app)
client.post("/admin/login", data={"username": "admin", "password": "password"})
response = client.get("/admin/hierarchy")
print(response.status_code)
print(response.text)

404
backend/test_form_posts.py Normal file
View File

@@ -0,0 +1,404 @@
#!/usr/bin/env python3
"""
Test all form POST endpoints for Internal Server Errors.
"""
import json
import sys
import httpx
BASE_URL = "http://localhost:8000"
# All form POST endpoints from admin_web.py
FORM_POST_ENDPOINTS = [
# (endpoint, method, form_data, description)
(
"/admin/login",
"POST",
{"username": "admin", "password": "admin123"},
"Admin login",
),
(
"/admin/password",
"POST",
{
"old_password": "admin123",
"new_password": "admin123",
"re_new_password": "admin123",
},
"Change password",
),
(
"/admin/websites",
"POST",
{
"site_name": "Test Site",
"site_url": "https://test.example.com",
},
"Create website",
),
(
"/admin/websites/1/edit",
"POST",
{
"site_name": "Updated Test Site",
"site_url": "https://updated.example.com",
},
"Edit website",
),
("/admin/websites/1/delete", "POST", {}, "Delete website"),
(
"/admin/tryout-import/preview",
"POST",
{
"website_id": "1",
},
"Tryout import preview (no file)",
),
(
"/admin/tryout-import",
"POST",
{
"website_id": "1",
"preview_token": "invalid-token",
},
"Tryout import submit",
),
(
"/admin/snapshot-questions/promote-bulk",
"POST",
{
"snapshot_id": "1",
"snapshot_question_ids": [],
},
"Promote snapshot questions bulk",
),
(
"/admin/basis-items/1/generate",
"POST",
{
"target_level": "mudah",
"ai_model": "",
"generation_count": "1",
"operator_notes": "",
},
"Generate variants for basis item",
),
(
"/admin/basis-items/1/review-bulk",
"POST",
{
"item_ids": ["1"],
"action": "approved",
},
"Review bulk variants",
),
(
"/admin/questions/1/generate",
"POST",
{
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"include_note_for_admin": True,
"include_note_in_prompt": False,
},
"Generate question variants",
),
(
"/admin/questions/1/generate/review-bulk",
"POST",
{
"item_ids": ["1"],
"action": "approved",
"tab": "review",
},
"Review question variants bulk",
),
]
# API POST endpoints
API_POST_ENDPOINTS = [
(
"/api/v1/session/",
{
"session_id": "test-session-123",
"tryout_id": "test",
"wp_user_id": "123",
"website_id": 1,
"scoring_mode": "ctt",
},
"Create session",
),
(
"/api/v1/session/test-session-123/complete",
{
"end_time": "2024-01-01T00:00:00Z",
"user_answers": [],
},
"Complete session",
),
(
"/api/v1/session/test-session-123/submit_answer",
{
"item_id": 1,
"response": "A",
"time_spent": 10,
},
"Submit answer",
),
(
"/api/v1/wordpress/verify_session",
{
"website_id": 1,
"wp_user_id": "123",
"token": "test",
},
"Verify WordPress session",
),
(
"/api/v1/reports/schedule",
{
"tryout_id": "test",
"report_type": "student_performance",
},
"Schedule report",
),
(
"/api/v1/admin/cat/test",
{
"tryout_id": "test",
"website_id": 1,
},
"Test CAT algorithm",
),
("/api/v1/admin/1/calibrate", {}, "Calibrate tryout"),
("/api/v1/admin/1/toggle-ai-generation", {}, "Toggle AI generation"),
("/api/v1/admin/1/reset-normalization", {}, "Reset normalization"),
]
def get_admin_session():
"""Login and get session cookies for admin access."""
with httpx.Client(base_url=BASE_URL, follow_redirects=True, timeout=30.0) as client:
# Try to login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
},
)
print(f"Login response: {response.status_code}")
# Check if we have admin access
response = client.get("/admin")
print(f"Admin page response: {response.status_code}")
# Return cookies
return client.cookies
def test_endpoint(
client: httpx.Client, endpoint: str, method: str, data: dict, cookies: dict = None
) -> dict:
"""Test a single endpoint."""
headers = {"X-Website-ID": "1"}
try:
if method == "POST":
# Check if this looks like form data or JSON
if isinstance(data, dict) and all(
isinstance(v, str) or v is None for v in data.values()
):
# Form data
response = client.post(
endpoint,
data=data,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
else:
# JSON data
response = client.post(
endpoint,
json=data,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
else:
response = client.request(
method,
endpoint,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
# Check for internal server error
has_ise = (
response.status_code == 500
or "Internal Server Error" in response.text
or "500 Internal Server Error" in response.text
)
# Check for traceback
has_traceback = "Traceback" in response.text
return {
"endpoint": endpoint,
"method": method,
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:500] if response.text else "",
"redirect_location": response.headers.get("location", ""),
}
except httpx.TimeoutException:
return {
"endpoint": endpoint,
"method": method,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"response_preview": "",
"error": "Timeout",
}
except Exception as e:
return {
"endpoint": endpoint,
"method": method,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"response_preview": "",
"error": str(e),
}
def main():
print("=" * 80)
print("Testing all Form POST endpoints for Internal Server Errors")
print("=" * 80)
print()
# Get admin session
print("Getting admin session...")
cookies = get_admin_session()
print()
results = []
has_errors = False
with httpx.Client(base_url=BASE_URL, follow_redirects=True, timeout=30.0) as client:
# Test admin form POST endpoints
print("-" * 80)
print("ADMIN FORM POST ENDPOINTS")
print("-" * 80)
for endpoint, method, data, description in FORM_POST_ENDPOINTS:
print(f"\nTesting: {description}")
print(f" Endpoint: {endpoint}")
result = test_endpoint(client, endpoint, method, data, cookies)
results.append((description, result))
status = result["status_code"]
error_details = ""
if result.get("error"):
error_details = f" [ERROR: {result['error']}]"
has_errors = True
elif result.get("has_traceback"):
error_details = f" [TRACEBACK!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
elif result.get("has_ise"):
error_details = f" [INTERNAL SERVER ERROR!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
status_str = str(status) if status else "N/A"
print(f" Status: {status_str}{error_details}")
if result.get("redirect_location"):
print(f" Redirect: {result['redirect_location']}")
# Test API POST endpoints
print()
print("-" * 80)
print("API POST ENDPOINTS")
print("-" * 80)
for endpoint, data, description in API_POST_ENDPOINTS:
print(f"\nTesting: {description}")
print(f" Endpoint: {endpoint}")
result = test_endpoint(client, endpoint, "POST", data, cookies)
results.append((description, result))
status = result["status_code"]
error_details = ""
if result.get("error"):
error_details = f" [ERROR: {result['error']}]"
has_errors = True
elif result.get("has_traceback"):
error_details = f" [TRACEBACK!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
elif result.get("has_ise"):
error_details = f" [INTERNAL SERVER ERROR!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
status_str = str(status) if status else "N/A"
print(f" Status: {status_str}{error_details}")
# Summary
print()
print("=" * 80)
print("SUMMARY")
print("=" * 80)
total = len(results)
ise_errors = sum(1 for _, r in results if r.get("has_ise"))
tracebacks = sum(1 for _, r in results if r.get("has_traceback"))
timeouts = sum(1 for _, r in results if r.get("error") == "Timeout")
exceptions = sum(
1 for _, r in results if r.get("error") and r.get("error") != "Timeout"
)
print(f"Total endpoints tested: {total}")
print(f"Internal Server Errors: {ise_errors}")
print(f"Tracebacks: {tracebacks}")
print(f"Timeouts: {timeouts}")
print(f"Exceptions: {exceptions}")
print()
if ise_errors > 0 or tracebacks > 0:
print("Endpoints with issues:")
for desc, r in results:
if r.get("has_ise") or r.get("has_traceback"):
print(f" - {desc}: {r['endpoint']} -> {r['status_code']}")
if r.get("has_traceback"):
print(f" Traceback detected in response")
print()
if has_errors:
print("❌ Some endpoints have issues. Please review the output above.")
return 1
else:
print("✅ All endpoints passed! No Internal Server Errors detected.")
return 0
if __name__ == "__main__":
sys.exit(main())

68
backend/test_session_debug.py Normal file
View File

@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""
Debug redirect on AI playground page.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging redirect on AI playground page...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Login first
response = client.get("/admin/login")
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f"Logged in, URL: {response.url}")
# Get AI playground page without following redirects
print("\nGetting AI playground page without following redirects...")
response = client.get(
"/admin/questions/1/generate?tab=review", follow_redirects=False
)
print(f"Status: {response.status_code}")
print(f"Location header: {response.headers.get('location', 'None')}")
# Follow the redirect
if response.headers.get("location"):
redirect_url = response.headers["location"]
print(f"\nFollowing redirect to: {redirect_url}")
response = client.get(redirect_url, follow_redirects=True)
print(f"Final status: {response.status_code}")
print(f"Final URL: {response.url}")
# Check for forms
post_forms = re.findall(
r'<form[^>]*method="post"[^>]*>', response.text, re.IGNORECASE
)
print(f"\nFound {len(post_forms)} POST forms")
# Look for CSRF token
csrf_inputs = re.findall(
r'<input[^>]*name="csrf_token"[^>]*>', response.text, re.IGNORECASE
)
if csrf_inputs:
print(f"Found {len(csrf_inputs)} CSRF token inputs:")
for inp in csrf_inputs[:3]:
print(f" {inp}")
else:
print("No CSRF token inputs found")
if __name__ == "__main__":
main()

374
backend/test_variant_approval.py Normal file
View File

@@ -0,0 +1,374 @@
#!/usr/bin/env python3
"""
Test variant approval endpoints with proper session handling.
"""
import re
import sys
import httpx
BASE_URL = "http://localhost:8000"
def get_csrf_token(client: httpx.Client, page_url: str) -> str:
"""Extract CSRF token from a page."""
try:
response = client.get(page_url)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1)
except Exception as e:
print(f" Error getting CSRF token from {page_url}: {e}")
return ""
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
# Get login page
response = client.get("/admin/login")
if response.status_code != 200:
print(f" Failed to get login page: {response.status_code}")
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
print(" Failed to get CSRF token")
return False
# Submit login - follow redirects to complete login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
if response.status_code == 200 and "/admin/dashboard" in str(response.url):
print(" ✅ Successfully logged in!")
return True
print(f" Login failed: {response.status_code}, URL: {response.url}")
return False
def test_variant_approval(client: httpx.Client) -> dict:
"""Test the variant approval endpoint."""
# Get CSRF token from the review page
csrf_token = get_csrf_token(client, "/admin/questions/1/generate?tab=review")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit variant approval
response = client.post(
"/admin/questions/1/generate/review-bulk",
data={
"item_ids": "1",
"action": "approved",
"tab": "review",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
print(f" Final URL: {response.url}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
# Extract just the traceback part
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_basis_item_review(client: httpx.Client) -> dict:
"""Test the basis item review bulk endpoint."""
# Get CSRF token from the basis item page
csrf_token = get_csrf_token(client, "/admin/basis-items/1")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit basis item review
response = client.post(
"/admin/basis-items/1/review-bulk",
data={
"item_ids": "1",
"action": "approved",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
print(f" Final URL: {response.url}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_snapshot_promote(client: httpx.Client) -> dict:
"""Test the snapshot questions promote bulk endpoint."""
# Get CSRF token from the hierarchy page
csrf_token = get_csrf_token(client, "/admin/hierarchy")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit snapshot promote (with empty list)
response = client.post(
"/admin/snapshot-questions/promote-bulk",
data={
"snapshot_id": "1",
"snapshot_question_ids": "",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_tryout_import_preview(client: httpx.Client) -> dict:
"""Test the tryout import preview endpoint."""
csrf_token = get_csrf_token(client, "/admin/tryout-import")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Submit tryout import preview (without file)
response = client.post(
"/admin/tryout-import/preview",
data={
"website_id": "1",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_website_crud(client: httpx.Client) -> dict:
"""Test website creation endpoint."""
csrf_token = get_csrf_token(client, "/admin/websites")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Submit website creation
response = client.post(
"/admin/websites",
data={
"site_name": "Test Site",
"site_url": "https://test.example.com",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def main():
print("=" * 80)
print("Testing Form POST Endpoints for Internal Server Errors")
print("=" * 80)
print()
results = []
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Login
print("Step 1: Logging in...")
if not login(client):
print("❌ Login failed")
return 1
print()
# Test 1: Variant approval
print(
"Step 2: Testing variant approval (/admin/questions/1/generate/review-bulk)..."
)
result1 = test_variant_approval(client)
results.append(("Variant approval", result1))
print()
# Test 2: Basis item review
print("Step 3: Testing basis item review (/admin/basis-items/1/review-bulk)...")
result2 = test_basis_item_review(client)
results.append(("Basis item review", result2))
print()
# Test 3: Snapshot promote
print(
"Step 4: Testing snapshot promote (/admin/snapshot-questions/promote-bulk)..."
)
result3 = test_snapshot_promote(client)
results.append(("Snapshot promote", result3))
print()
# Test 4: Tryout import preview
print("Step 5: Testing tryout import preview (/admin/tryout-import/preview)...")
result4 = test_tryout_import_preview(client)
results.append(("Tryout import preview", result4))
print()
# Test 5: Website creation
print("Step 6: Testing website creation (/admin/websites)...")
result5 = test_website_crud(client)
results.append(("Website creation", result5))
print()
# Summary
print("=" * 80)
print("RESULTS SUMMARY")
print("=" * 80)
all_good = True
for name, result in results:
if result.get("has_ise") or result.get("has_traceback"):
print(f"❌ {name}: INTERNAL SERVER ERROR!")
print(f" Status: {result['status_code']}")
print(f" Preview: {result['response_preview'][:200]}...")
all_good = False
elif result.get("error"):
print(f"⚠️ {name}: {result['error']}")
elif result["status_code"] in [200, 303]:
print(f"✅ {name}: OK ({result['status_code']})")
else:
print(f"⚠️ {name}: Unexpected status {result['status_code']}")
print()
if all_good:
print("✅ All form POST endpoints passed! No Internal Server Errors detected.")
return 0
else:
print("❌ Some endpoints have issues. Please review the output above.")
return 1
if __name__ == "__main__":
sys.exit(main())

10
tests/test_auth_scope.py → backend/tests/test_auth_scope.py
View File

@@ -19,6 +19,12 @@ def test_require_website_auth_returns_scoped_website_for_allowed_role():
assert website_id == 5
def test_require_website_auth_allows_global_system_admin_scope():
auth = AuthContext(website_id=None, role="system_admin", wp_user_id=None)
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
assert website_id is None
def test_require_website_auth_rejects_disallowed_role():
auth = AuthContext(website_id=5, role="student", wp_user_id="u1")
with pytest.raises(HTTPException) as exc_info:
@@ -30,3 +36,7 @@ def test_cross_website_payload_mismatch_is_blocked():
with pytest.raises(HTTPException) as exc_info:
ensure_website_scope_matches(auth_website_id=10, payload_website_id=11)
assert exc_info.value.status_code == 403
def test_global_system_admin_scope_can_write_any_payload_website():
ensure_website_scope_matches(auth_website_id=None, payload_website_id=11)

24
tests/test_auth_tokens.py → backend/tests/test_auth_tokens.py
View File

@@ -23,6 +23,30 @@ def test_issue_and_decode_access_token_round_trip():
assert auth.wp_user_id == "wp-1001"
def test_system_admin_token_can_be_global_without_website_scope():
token = issue_access_token(
website_id=None,
role="system_admin",
wp_user_id=None,
expires_in_seconds=3600,
)
auth = decode_access_token(token)
assert auth.website_id is None
assert auth.role == "system_admin"
def test_non_system_admin_token_requires_website_scope():
token = issue_access_token(
website_id=None,
role="admin",
wp_user_id=None,
expires_in_seconds=3600,
)
with pytest.raises(HTTPException) as exc_info:
decode_access_token(token)
assert exc_info.value.status_code == 401
def test_decode_access_token_rejects_tampered_signature():
token = issue_access_token(
website_id=7,

0
tests/test_model_mappings.py → backend/tests/test_model_mappings.py
View File

0
tests/test_normalization.py → backend/tests/test_normalization.py
View File

59
tests/test_operational_hardening.py → backend/tests/test_operational_hardening.py
View File

@@ -8,12 +8,14 @@ from app.core import rate_limit
from app.core.config import Settings
from app.models.report_schedule import ReportScheduleModel
from app.services import ai_generation
from app.services import cat_selection
from app.services.reporting import (
cancel_scheduled_report,
get_scheduled_report,
list_scheduled_reports,
schedule_report,
)
from app.schemas.ai import GeneratedQuestion
class DummyRequest:
@@ -101,6 +103,63 @@ def test_ai_stats_accepts_website_scope(monkeypatch):
assert all("items.website_id" in query for query in captured_queries)
def test_ai_prompt_preserves_basis_option_labels():
prompt = ai_generation.get_prompt_template(
basis_stem="<p>Basis question?</p>",
basis_options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
"E": "Option E",
},
basis_correct="A",
basis_explanation="<p>Because A.</p>",
target_level="mudah",
)
assert "Create exactly 5 answer options with labels exactly: A, B, C, D, E" in prompt
assert '"E": "Option E text"' in prompt
assert "The correct field must be exactly one of: A, B, C, D, E" in prompt
def test_generated_question_must_match_basis_option_labels():
basis_item = SimpleNamespace(
options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
"E": "Option E",
}
)
generated = GeneratedQuestion(
stem="Generated",
options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
},
correct="A",
)
assert not ai_generation.generated_matches_basis_options(generated, basis_item)
def test_cat_selection_only_serves_active_or_approved_variants():
compiled = str(
cat_selection._servable_item_filter().compile(
compile_kwargs={"literal_binds": True}
)
)
assert "active" in compiled
assert "approved" in compiled
assert "draft" not in compiled
assert "rejected" not in compiled
def test_production_init_db_skips_create_all(monkeypatch):
import app.database as database

2
tests/test_route_wiring.py → backend/tests/test_route_wiring.py
View File

@@ -7,5 +7,5 @@ from app.main import app
def test_next_item_route_is_registered():
paths = {route.path for route in app.routes}
paths = set(app.openapi()["paths"])
assert "/api/v1/session/{session_id}/next_item" in paths

0
tests/test_security_regressions.py → backend/tests/test_security_regressions.py
View File

Some files were not shown because too many files have changed in this diff Show More