dwindown/yellow-bank-soal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Checkpoint React frontend migration

Browse Source
This commit is contained in:
Dwindi Ramadhana
2026-06-20 01:43:39 +07:00
parent ab86c254d1
commit b8e201b45f
173 changed files with 34116 additions and 782 deletions
Download Patch File Download Diff File Expand all files Collapse all files

365
ADMIN_TRYOUT_RESTRUCTURE_PLAN.md Normal file
View File

@@ -0,0 +1,365 @@
# Admin UI Redesign - Implementation Plan
## Overview
This plan outlines the migration from the current scattered admin structure to a clean, hierarchy-driven navigation centered on **Tryouts**.
### Guiding Principles
1. **One main page per domain** - Features live under their parent, not as separate menu items
2. **URL reflects depth** - Path structure shows relationship (`/admin/tryout/{id}/questions`)
3. **Tree as map** - Hierarchy tree shows structure; drill-down shows details
4. **Consistent naming** - Use "Tryout" instead of "Exam" throughout
---
## 1. URL Structure
### New URL Scheme
| Old Route | New Route | Description |
|-----------|-----------|-------------|
| `/admin/exams` | `/admin/tryouts` | Hierarchy tree (main entry) |
| `/admin/student-attempts` | `/admin/tryout/{tryout_id}/attempts` | Attempts filtered by tryout |
| - | `/admin/tryout/{tryout_id}/questions` | Questions filtered by tryout |
| - | `/admin/tryout/{tryout_id}/questions/{question_id}/workspace` | Question workspace |
| - | `/admin/tryout/{tryout_id}/questions/{question_id}/workspace/{tab}` | Workspace tabs |
| - | `/admin/tryout/{tryout_id}/normalization` | Normalization settings for this tryout |
| `/admin/questions` | `/admin/questions` | Global question list (kept) |
| (none) | `/admin/import-tryout` | Import tryout modal/page |
> **Note:** Import is tryout-level, not question-level. Import button lives on `/admin/tryouts` page header.
### Hierarchy Depth Convention
```
/admin/tryouts → Level 0: Root
/admin/tryout/{tryout_id} → Level 1: Entity
/admin/tryout/{tryout_id}/attempts → Level 2: Related data
/admin/tryout/{tryout_id}/questions → Level 2: Related data
/admin/tryout/{tryout_id}/questions/{id} → Level 3: Specific item
/admin/tryout/{tryout_id}/questions/{id}/workspace → Level 4: Detail view
```
---
## 2. Navigation Structure
### Proposed Navigation
```
Questions
├── /admin/questions # Global question list
└── /admin/tryout/*/questions/*/workspace # Direct link from tree
Tryouts
├── /admin/tryouts # Tree: Website → Tryout → Stat → Actions + Import button
├── /admin/tryout/*/attempts # Filtered attempts
├── /admin/tryout/*/questions # Questions in this tryout
├── /admin/tryout/*/normalization # Normalization settings
└── /admin/import-tryout # Import modal/page
Reports
├── /admin/reports # Dashboard
├── /admin/item-statistics
└── /admin/calibration-status
Settings
├── /admin/settings
├── /admin/websites
└── /admin/password
```
### Navigation Item Definition
```python
ADMIN_NAV_ITEMS = (
("Dashboard", "/admin/dashboard", ("/admin/dashboard",)),
("Questions", "/admin/questions", (
"/admin/questions",
"/admin/tryout/*/questions/*/workspace", # Pattern for direct links
)),
("Tryouts", "/admin/tryouts", (
"/admin/tryouts",
"/admin/tryout/*/attempts",
"/admin/tryout/*/questions",
"/admin/tryout/*/normalization",
"/admin/import-tryout",
)),
("Reports", "/admin/reports", (
"/admin/reports",
"/admin/item-statistics",
"/admin/calibration-status",
)),
("Settings", "/admin/settings", (
"/admin/settings",
"/admin/websites",
"/admin/password",
)),
("Logout", "/admin/logout", ("/admin/logout",)),
)
```
---
## 3. Tryouts Tree Structure
### Visual Design
```
┌─ Tryouts ───────────────────────────────────────────────────────────────────┐
│ │
│ [+ Import Tryout] │
│ │
│ 🌐 Website A │
│ │ │
│ ├─ 📋 132380 - UTBK 2024 [●] │
│ │ └─ [Expanded on click] │
│ │ │
│ ├─ 📋 132381 - SIMAK UI [✓] │
│ │ │
│ └─ 📋 132382 - PAS Semester 1 [○] │
│ │
│ 🌐 Website B │
│ └─ ... │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
Expanded Tryout View:
┌─ 📋 132380 - UTBK 2024 ─────────────────────────────────────────────────────┐
│ │
│ ┌─ Stat Card ─────────────────────────────────────────────────────────┐ │
│ │ 👥 150 participants │ NM: 672 avg │ NN: 505 avg │ │
│ │ ✓ 98% completion │ 📐 Calibration: ████████░░ 85% (17/20) │ │
│ └────────────────────────────────────────────────────────────────────┘ │
│ │
│ [📝 Questions (20)] [👥 Attempts (150)] [📐 Normalization] [⚙ Settings]│
│ │
└──────────────────────────────────────────────────────────────────────────────┘
Legend:
[●] Partial (50-89% calibrated)
[✓] Ready (≥90% calibrated)
[○] Needs Data (<50% calibrated)
```
### Import Button Location
- **Location:** Header of `/admin/tryouts` page
- **Label:** "[+ Import Tryout]" or "[Import Tryout JSON]"
- **Behavior:** Opens import modal/page
- **Why:** Import is tryout-level operation (imports questions WITH tryout context)
### Stat Card Components
| Field | Source | Display |
|-------|--------|---------|
| Participants | `TryoutStats.participant_count` | 👥 {count} |
| Avg NM | `AVG(Session.NM)` where completed | 📊 {value} avg |
| Avg NN | `AVG(Session.NN)` where completed | 📈 {value} avg |
| Completion Rate | `completed / participants * 100` | ✓ {percentage}% |
| Calibration | `calibrated_items / total_items` | 📐 Progress bar + {count}/{total} |
### Action Buttons
| Action | Target URL | Icon |
|--------|------------|------|
| Questions | `/admin/tryout/{id}/questions` | 📝 |
| Attempts | `/admin/tryout/{id}/attempts` | 👥 |
| Normalization | `/admin/tryout/{id}/normalization` | 📐 |
| Settings | `/admin/tryout/{id}/settings` (or modal) | ⚙ |
---
## 4. Page Specifications
### 4.1 `/admin/tryouts` (Main Tree)
**Purpose:** Primary navigation entry, shows structure at a glance
**Default State:**
- Websites expanded
- Tryouts collapsed
- Shows calibration indicator dot next to each tryout
**Interactions:**
- Click tryout → expand/collapse
- Expanded tryout shows stat card + action buttons
- Actions navigate to filtered views
### 4.2 `/admin/tryout/{tryout_id}/questions`
**Purpose:** View all questions in a specific tryout
**Behavior:**
- Shows only original/imported questions (basis items)
- Pre-filtered by `tryout_id`
- Links to workspace for AI variant generation
**Table Columns:**
| Column | Description |
|--------|-------------|
| ID | Question internal ID |
| Stem Preview | First 100 chars of question text |
| Difficulty | Current difficulty level |
| Calibration | P-value or IRT-b indicator |
| Variants | Count of generated variants |
| Actions | [View] [Workspace] |
### 4.3 `/admin/tryout/{tryout_id}/questions/{question_id}/workspace`
**Purpose:** Generate and manage question variants
**Tabs:**
| Tab | Purpose |
|-----|---------|
| Generate | AI variant generation interface |
| Review | Review generated variants |
| Batch | Batch generation options |
**Access Pattern:**
- Opens from question list or tree direct link
- Context: knows parent tryout, parent question
### 4.4 `/admin/tryout/{tryout_id}/attempts`
**Purpose:** View student attempts for specific tryout
**Current Implementation:** Already exists at `/admin/student-attempts` → migrate URL
**Enhancements:**
- Pre-filtered by `tryout_id` (no dropdown needed on this page)
- Stat card from parent tryout shown at top
### 4.5 `/admin/tryout/{tryout_id}/normalization`
**Purpose:** Configure normalization settings for a specific tryout
**Settings (per-tryout):**
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| Mode | Select | Auto | Auto (calculate from data) or Manual (fixed values) |
| Rataan | Number | 500 | Target mean for normalization |
| SB | Number | 100 | Target standard deviation |
| Recalculate | Button | - | Re-run normalization on existing sessions |
**Formula:** `NN = 500 + 100 × ((NM - Rataan) / SB)`
**UI:**
- Simple form with current values
- "Recalculate" button triggers normalization job
- Shows last normalization timestamp
### 4.6 `/admin/import-tryout`
**Purpose:** Import tryout data (questions + metadata) from JSON
**Access:** Via "[+ Import Tryout]" button on `/admin/tryouts` page
**Behavior:**
- Opens modal or dedicated page
- Upload JSON file or paste JSON content
- Preview import before confirming
- Creates new tryout with questions
**URL Convention:** Not under specific tryout (it's creating a new one)
---
## 5. Deprecations
### Routes to Remove
| Route | Reason |
|-------|--------|
| `/admin/exams` | Renamed to `/admin/tryouts` |
| `/admin/student-attempts` | URL changed to `/admin/tryout/{id}/attempts` |
| `/admin/templates` | AI uses basis items directly |
| `/admin/basis-items` | Merge into question workspace |
| `/admin/hierarchy` | Tree IS the hierarchy |
| `/admin/question-quality` | Merged into tryout stat card |
### Legacy Redirects
```python
LEGACY_URL_MAP = {
"/admin/exams": "/admin/tryouts",
"/admin/student-attempts": "/admin/tryouts", # Or redirect to tryouts with guidance
"/admin/hierarchy": "/admin/tryouts",
"/admin/question-quality": "/admin/tryouts",
# Templates and basis-items: 404 (removed)
}
```
---
## 6. Implementation Phases
### Phase 1: Foundation
- [ ] Rename `/admin/exams` → `/admin/tryouts` (keep old route for now)
- [ ] Implement tree structure in `/admin/tryouts`
- [ ] Move `TryoutStats` info into tree stat cards
- [ ] Add calibration indicator dots
### Phase 2: URL Migration
- [ ] Create `/admin/tryout/{id}/attempts` (redirect from old route)
- [ ] Create `/admin/tryout/{id}/questions`
- [ ] Update navigation items
### Phase 3: Workspace Integration
- [ ] Create question workspace route
- [ ] Implement workspace tabs
- [ ] Connect workspace to tree and question list
### Phase 4: Cleanup
- [ ] Add legacy redirects
- [ ] Remove deprecated routes
- [ ] Update all hardcoded links in views
### Phase 5: Polish
- [ ] Review all pages for consistency
- [ ] Update documentation
- [ ] Test all navigation paths
---
## 7. Open Questions
1. ~~Normalization settings~~ - **RESOLVED**: Move under tryout context as `/admin/tryout/{id}/normalization`
2. ~~Import questions page~~ - **RESOLVED**: Import is tryout-level. Button on `/admin/tryouts` header, not a separate page.
3. **Tryout settings** - What settings are actually needed? (Scoring mode, time limits, selection criteria?)
4. **Global questions page** - Is `/admin/questions` (unfiltered) still useful, or should every question access go through tryout context?
5. **Templates deprecation** - Confirm that `/admin/templates` is truly unused and can be safely removed?
6. **Legacy routes for deleted pages** - Should `/admin/templates` and `/admin/basis-items` redirect somewhere or return 404?
---
## 8. Files to Modify
### Primary Changes
- `app/admin_web.py` - Major route restructuring
- Navigation definition in `admin_web.py`
- Legacy URL map
### Likely Additions
- Static assets for tree expansion/collapse (if not using existing)
### Documentation Updates
- `ADMIN_UI_REDESIGN_PLAN.md` - Update to reflect final structure
- `PROJECT_UNDERSTANDING.md` - Update route documentation
---
## 9. Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2026-06-17 | Initial draft based on discussion |
| 1.1 | 2026-06-17 | - Move normalization to `/admin/tryout/{id}/normalization`<br>- Move import button to `/admin/tryouts` header<br>- Add normalization page spec (4.5)<br>- Rename import page spec (4.6)<br>- Update navigation and action buttons |

615
FRONTEND_MIGRATION_AUDIT_REPORT.md Normal file
View File

@@ -0,0 +1,615 @@
# Frontend Migration Audit Report
Date: 2026-06-19
Project: Yellow Bank Soal / IRT Bank Soal
Scope: Migration from root-level Python/FastAPI admin UI to `backend/` plus new React `frontend/`
Auditor: Codex
## 1. Executive Summary
The React frontend is scaffolded and builds successfully, but the migration is not yet feature-complete or integration-safe. The biggest risks are API address drift, tenant/website context bugs, missing parity with the legacy Python admin workflows, and placeholder React pages that appear functional but do not call real backend APIs.
Current readiness assessment: **not production-ready as the primary replacement for the Python admin UI**.
Top findings:
| Priority | Finding | Impact |
|---|---|---|
| P0 | Local frontend API base URL omits `/api/v1` | Most API calls 404 in `npm run dev` and any environment using `frontend/.env`. |
| P0 | System admin website scope starts as `website_id=0` and React Query keys ignore website selection | First dashboard loads empty or wrong scoped data; switching websites can show stale data. |
| P0 | Several React API calls target nonexistent or renamed backend endpoints | Reports, normalization, and Excel import workflows are broken. |
| P1 | Student tryout portal from the migration plan is absent | Core learner flow is not migrated to React. |
| P1 | AI generation UI has incomplete save/review/batch behavior | Operators can generate previews, but core review and batch workflow parity is missing. |
| P1 | Unsafe `dangerouslySetInnerHTML` use without sanitization | Imported or AI-generated HTML can become an admin XSS risk. |
| P2 | Multiple legacy admin features are missing or placeholders | Hierarchy, question quality, question details, password update, exports, and settings are incomplete. |
The build result is positive: `npm run build` completed successfully. This means the current issues are mainly behavioral and integration defects, not TypeScript compilation blockers.
## 2. Audit Scope And Methodology
Reviewed:
- Repository restructure from root `app/` to `backend/app/` and new `frontend/`.
- Current React routes, pages, state store, API client, and Docker/Nginx configuration.
- Current FastAPI router definitions and generated OpenAPI paths.
- Last committed Python admin surface via `git show HEAD:app/admin_web.py`.
- Existing planning documents: `REACT_Migration_Plan.md`, `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md`, and `UX_AUDIT_ADMIN_FLOW.md`.
Verification performed:
- `npm run build` inside `frontend/`: passed.
- FastAPI OpenAPI generation from `backend/app/main.py`: produced 55 paths.
- Static endpoint comparison between React `api.get/post/put/delete` calls and backend route definitions.
Not performed:
- Full browser E2E against a running backend/database.
- Live authentication, import, AI generation, or report generation.
- Full backend test suite run.
## 3. Current Architecture Snapshot
The current repository is in an uncommitted migration state. Git sees the old root-level Python files as deleted and `backend/` plus `frontend/` as new untracked folders.
React frontend:
- Admin-only route shell currently lives in `frontend/src/App.tsx`.
- API helper is `frontend/src/lib/api.ts`.
- Global website and auth token state is persisted in `frontend/src/store/useAppStore.ts`.
- The admin UI has pages for Dashboard, Questions, Tryouts, Reports, Settings, AI Generation, Import, and nested Tryout workspaces.
Backend:
- Main FastAPI app lives in `backend/app/main.py`.
- JSON APIs are generally under `/api/v1`.
- Legacy Python admin HTML router is still mounted at `/admin` when `ENABLE_ADMIN=true`.
- Import/export router hardcodes `/api/v1/import-export` inside the router prefix rather than relying on `settings.API_V1_STR`.
## 4. Verification Results
| Check | Result | Notes |
|---|---|---|
| React build | Passed | `tsc -b && vite build` completed. Vite warned that the main JS chunk is larger than 500 kB. |
| FastAPI OpenAPI paths | Passed | OpenAPI generated 55 paths. |
| API route parity | Failed | Multiple frontend calls do not map to backend paths or methods. |
| Feature parity with legacy Python admin | Partial | Several legacy workflows are absent, placeholders, or only implemented as HTML admin routes. |
| Local development readiness | Failed | `frontend/.env` and backend CORS settings do not match the default Vite dev setup. |
## 5. Backend API Paths Observed
The current OpenAPI schema exposes these relevant JSON paths:
```text
/api/v1/auth/admin-login
/api/v1/websites
/api/v1/admin/dashboard/stats
/api/v1/admin/questions
/api/v1/admin/templates
/api/v1/admin/tryouts/{tryout_id}/questions
/api/v1/admin/tryouts/{tryout_id}/attempts
/api/v1/admin/ai/models
/api/v1/admin/ai/generate-preview
/api/v1/admin/ai/generate-save
/api/v1/admin/ai/pending-reviews
/api/v1/admin/ai/review/{item_id}
/api/v1/import-export/preview
/api/v1/import-export/questions
/api/v1/import-export/export/questions
/api/v1/import-export/tryout-json/preview
/api/v1/import-export/tryout-json
/api/v1/tryout/
/api/v1/tryout/{tryout_id}/config
/api/v1/tryout/{tryout_id}/normalization
/api/v1/tryout/{tryout_id}/calibration-status
/api/v1/reports/student/performance
/api/v1/reports/items/analysis
/api/v1/reports/calibration/status
```
Legacy HTML-only admin paths still exist under `/admin`, including:
```text
/admin/hierarchy
/admin/question-quality
/admin/calibration-status
/admin/item-statistics
/admin/session-overview
/admin/snapshot-questions
/admin/snapshot-questions/promote-bulk
/admin/basis-items
/admin/basis-items/{basis_item_id}
/admin/basis-items/{basis_item_id}/generate
/admin/basis-items/{basis_item_id}/review-bulk
/admin/questions/{item_id}
/admin/questions/{item_id}/generate
/admin/questions/{item_id}/generate/review-bulk
```
Those HTML paths are not a substitute for React JSON API parity unless the React app intentionally navigates users back into the legacy admin UI.
## 6. Endpoint Compatibility Matrix
This matrix assumes the frontend base URL is configured as `http://localhost:8000/api/v1`, as Docker currently does. If the base URL is `http://localhost:8000`, most rows fail one level earlier because `/api/v1` is missing.
| React call | Backend status | Migration status |
|---|---|---|
| `/auth/admin-login` | Exists as `/api/v1/auth/admin-login` | OK when base URL includes `/api/v1`. |
| `/websites` | Exists as `/api/v1/websites` | OK when base URL includes `/api/v1`. |
| `/admin/dashboard/stats` | Exists as `/api/v1/admin/dashboard/stats` | Path OK, but website scoping can return empty/stale data. |
| `/tryout/` | Exists as `/api/v1/tryout/` | OK when base URL includes `/api/v1`. |
| `/admin/questions` | Exists as `/api/v1/admin/questions` | OK when base URL includes `/api/v1`. |
| `/admin/templates` | Exists as `/api/v1/admin/templates` | Path OK; verify runtime lazy relationship behavior. |
| `/admin/tryouts/{id}/questions` | Exists as `/api/v1/admin/tryouts/{tryout_id}/questions` | OK when base URL includes `/api/v1`. |
| `/admin/tryouts/{id}/attempts` | Exists as `/api/v1/admin/tryouts/{tryout_id}/attempts` | OK when base URL includes `/api/v1`. |
| `/admin/ai/models` | Exists as `/api/v1/admin/ai/models` | OK when base URL includes `/api/v1`. |
| `/admin/ai/generate-preview` | Exists as `/api/v1/admin/ai/generate-preview` | Path OK; payload includes unsupported `operator_notes` in one page but Pydantic ignores extras by default. |
| `/admin/ai/generate-save` | Exists as `/api/v1/admin/ai/generate-save` | Path OK; React passes placeholder slot and can cause duplicate/conflict behavior. |
| `/import-export/tryout-json/preview` | Exists as `/api/v1/import-export/tryout-json/preview` | OK when base URL includes `/api/v1`. |
| `/import-export/tryout-json` | Exists as `/api/v1/import-export/tryout-json` | OK when base URL includes `/api/v1`. |
| `/reports/calibration-status` | Backend has `/api/v1/reports/calibration/status?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/reports/item-analysis` | Backend has `/api/v1/reports/items/analysis?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/reports/student-performance` | Backend has `/api/v1/reports/student/performance?tryout_id=...` | Broken. Wrong path and missing required `tryout_id`. |
| `/tryouts/{id}/config` | Backend has `/api/v1/tryout/{id}/config` | Broken. Uses plural `tryouts`. |
| `POST /tryouts/{id}/normalization` | Backend has `PUT /api/v1/tryout/{id}/normalization` | Broken. Wrong path, method, and payload schema. |
| `/tryouts/{id}/normalization/recalculate` | No JSON API found | Broken. |
| `/import-export/tryout-import` | No JSON API found | Broken. Should likely use `/import-export/preview` or `/import-export/questions`. |
| `/import-export/snapshot-questions/promote-bulk` | No JSON API found | Broken. Legacy equivalent is HTML form POST `/admin/snapshot-questions/promote-bulk`. |
## 7. Findings
### P0-01: Local API base URL omits `/api/v1`
Severity: P0
Category: API routing / environment configuration
Evidence:
- `frontend/src/lib/api.ts:5` defaults to `http://localhost:8000`.
- `frontend/.env:1` sets `VITE_API_URL=http://localhost:8000`.
- Backend JSON admin APIs are exposed under `/api/v1/...`.
- Docker build uses `VITE_API_BASE_URL: "http://localhost:8000/api/v1"` in `docker-compose.yml:62`, so Docker and local dev behave differently.
Impact:
- Running `npm run dev` or using the checked-in `frontend/.env` makes calls such as `/auth/admin-login`, `/websites`, and `/admin/dashboard/stats` hit the wrong backend URLs.
- Developers can see a compiling app but get login/API failures at runtime.
- Bugs may be masked in Docker while reappearing in local development or other deployments.
Recommendation:
- Standardize one env var name, preferably `VITE_API_URL`, and set it to the full API root: `http://localhost:8000/api/v1`.
- Add `frontend/.env.example` with the same value.
- Add a startup assertion or dev console warning if `VITE_API_URL` does not end in `/api/v1`.
- Consider making the Axios helper append `/api/v1` itself so pages never depend on a base URL convention.
### P0-02: System-admin website scope and React Query cache can show empty or stale tenant data
Severity: P0
Category: Multi-tenant data isolation / state management
Evidence:
- Login issues a system-admin token with `website_id=0` in `backend/app/routers/auth.py:50-55`.
- The comment says this placeholder should produce global access, but `require_website_auth` returns `auth.website_id` whenever it is not `None` in `backend/app/core/auth.py:147-150`.
- `WebsiteSelector` auto-selects the first website asynchronously in `frontend/src/components/WebsiteSelector.tsx:25-29`.
- Dashboard query key is `['dashboard-stats']` and does not include `websiteId` in `frontend/src/pages/admin/Dashboard.tsx:45-50`.
- Other query keys also omit `websiteId`, including `['tryouts']`, `['admin-questions']`, and `['ai-pending-reviews']`.
Impact:
- First-load dashboard requests can run before the selector sets `X-Website-ID`; backend may interpret the request as website `0` and return empty data.
- Switching websites can leave cached data from the prior website because React Query keys do not include the website id.
- Multi-tenant admin data can appear wrong even when the API endpoint is otherwise correct.
Recommendation:
- Fix backend system-admin semantics: use `website_id=None` for global system admin or make `website_id=0` explicitly mean global access.
- In React, gate website-scoped queries until `websiteId` is set, except for the websites list itself.
- Include `websiteId` in every website-scoped React Query key, for example `['dashboard-stats', websiteId]`.
- Invalidate website-scoped queries when `WebsiteSelector` changes.
### P0-03: Reports page calls nonexistent backend paths and omits required filters
Severity: P0
Category: API contract / reporting
Evidence:
- React calls `/reports/calibration-status`, `/reports/item-analysis`, and `/reports/student-performance` in `frontend/src/pages/admin/reports/index.tsx:14`, `:58`, and `:88`.
- Backend exposes `/reports/calibration/status`, `/reports/items/analysis`, and `/reports/student/performance` in `backend/app/routers/reports.py:68-80`, `:172-184`, and `:231-241`.
- Each backend report endpoint requires `tryout_id`.
- React report export buttons have no handlers in `frontend/src/pages/admin/reports/index.tsx:29-31`, `:73-75`, and `:103-105`.
Impact:
- All three report tabs fail at runtime.
- Even after path correction, the page needs tryout selection or route context because backend requires `tryout_id`.
- Export buttons are misleading because they do not call the export APIs.
Recommendation:
- Use the backend paths:
- `/reports/calibration/status?tryout_id={id}`
- `/reports/items/analysis?tryout_id={id}`
- `/reports/student/performance?tryout_id={id}`
- Add tryout selector/context to the Reports page.
- Wire export buttons to `/reports/.../export/{format}` endpoints.
- Render real report tables from response fields instead of placeholder text.
### P0-04: Tryout normalization page uses wrong paths, method, payload, and silent fallback
Severity: P0
Category: API contract / scoring configuration
Evidence:
- React fetches `/tryouts/{id}/config` in `frontend/src/pages/admin/tryouts/Normalization.tsx:22`.
- Backend route is `/tryout/{tryout_id}/config` in `backend/app/routers/tryouts.py:34-44`.
- React posts `/tryouts/{id}/normalization` with `{ rataan, sb, mode }` in `frontend/src/pages/admin/tryouts/Normalization.tsx:39-45`.
- Backend expects `PUT /tryout/{tryout_id}/normalization` with fields `normalization_mode`, `static_rataan`, and `static_sb` in `backend/app/routers/tryouts.py:109-120`.
- React calls `/tryouts/{id}/normalization/recalculate` in `frontend/src/pages/admin/tryouts/Normalization.tsx:53-56`, but no matching JSON API was found.
- The page catches config load failures and silently displays defaults in `frontend/src/pages/admin/tryouts/Normalization.tsx:21-26`.
Impact:
- Operators can believe they changed normalization settings when the requests actually failed or hit nonexistent endpoints.
- Silent defaults can overwrite user trust in scoring configuration by hiding missing data.
- Normalization is core to NM/NN scoring, so this is a production blocker.
Recommendation:
- Change GET to `/tryout/{id}/config`.
- Change save to `PUT /tryout/{id}/normalization`.
- Send backend schema names: `normalization_mode`, `static_rataan`, `static_sb`.
- Remove silent fallback for API failures; show an error state.
- Either add a backend recalculation endpoint or remove the button until the API exists.
### P0-05: Excel import page is wired to nonexistent endpoints
Severity: P0
Category: Import workflow / API contract
Evidence:
- React posts preview/upload to `/import-export/tryout-import` in `frontend/src/pages/admin/import/index.tsx:17-23`.
- React posts confirmation to `/import-export/snapshot-questions/promote-bulk` in `frontend/src/pages/admin/import/index.tsx:31-35`.
- Backend Excel import APIs are `/api/v1/import-export/preview` and `/api/v1/import-export/questions` in `backend/app/routers/import_export.py:53-62` and `:150-160`.
- Snapshot promotion currently exists only in the legacy HTML admin as `/admin/snapshot-questions/promote-bulk`.
Impact:
- The standalone Excel Import page cannot complete its workflow.
- Users have two import surfaces: a working JSON import modal under Tryouts and a broken Excel import page under `/admin/import`.
- The comments in the React file explicitly show uncertainty about endpoint names.
Recommendation:
- Decide whether Excel import remains in the React admin.
- If yes, wire preview to `/import-export/preview` and confirm to `/import-export/questions` with required `tryout_id`.
- If snapshot promotion is required in React, add a JSON API for selected snapshot question IDs and update the UI accordingly.
- Hide `/admin/import` until the contract is implemented.
### P1-01: Student tryout portal is missing from React
Severity: P1
Category: Feature parity / core business flow
Evidence:
- `REACT_Migration_Plan.md:73-85` describes Phase 3 Student Portal construction: tryout listing, exam session, async answer submission, state recovery, server timer, and result page.
- Current `frontend/src/App.tsx:38-66` only defines `/login` and `/admin/*` routes.
- No student session routes or pages were found under `frontend/src/pages`.
Impact:
- The React migration does not yet cover the learner-facing tryout experience.
- If the goal is full Python frontend replacement, core user-facing functionality remains unmigrated.
Recommendation:
- Add student routes for tryout listing, active session, answer submission, completion, and result summary.
- Use existing backend session APIs under `/api/v1/session`.
- Add E2E coverage for refresh recovery and server-synced timer behavior.
### P1-02: AI generation workflow is incomplete and can save invalid variants
Severity: P1
Category: AI generation / operator workflow
Evidence:
- Global AI page uses manual basis item id and comments that a real template selector is missing in `frontend/src/pages/admin/ai/Workspace.tsx:26-27`.
- Global AI page has "Discard" and "Save & Queue Review" buttons with no handlers in `frontend/src/pages/admin/ai/Workspace.tsx:138-140`.
- Tryout AI workspace saves generated questions with `slot: basisItem ? 1 : 1` in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:64-76`.
- Tryout AI workspace "Review Variants" and "Batch Generation" tabs are placeholder text in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:233-253`.
- Legacy Python admin supported batch count, operator notes, note inclusion, run history, filters, review-bulk, and variant detail pages.
Impact:
- Operators cannot reliably save variants with correct slot linkage.
- Batch generation and review parity are missing.
- Duplicate slot conflicts are likely because saved AI variants always use slot `1`.
Recommendation:
- Use the selected basis item's real `slot`, `tryout_id`, `website_id`, and source snapshot metadata.
- Add JSON APIs if needed for batch generation, run history, review filtering, and bulk review.
- Disable save buttons until all required fields are present.
- Remove or implement the global AI workspace to avoid two partial AI workflows.
### P1-03: Imported and generated HTML is rendered without sanitization
Severity: P1
Category: Security / XSS
Evidence:
- React renders question HTML with `dangerouslySetInnerHTML` in `frontend/src/pages/admin/tryouts/QuestionManagement.tsx`.
- React renders AI preview stem/options/explanation with `dangerouslySetInnerHTML` in `frontend/src/pages/admin/tryouts/AIWorkspace.tsx:180-200`.
- The migration plan explicitly calls out HTML sanitization as a security checklist item in `REACT_Migration_Plan.md:204-208`.
Impact:
- Imported Sejoli payloads or AI-generated content could inject scripts or unsafe markup into admin pages.
- Admin XSS is high impact because admins hold cross-website operational access.
Recommendation:
- Add a sanitizer such as DOMPurify.
- Create a single `SafeHtml` component and forbid direct `dangerouslySetInnerHTML` in pages.
- Sanitize on render and consider backend-side validation for stored HTML.
### P1-04: CORS config does not include the default Vite dev origin
Severity: P1
Category: Local development / environment configuration
Evidence:
- Backend `.env` allows `http://localhost:3000` and `http://localhost:8000` in `backend/.env:15`.
- Vite dev normally serves at `http://localhost:5173`.
- `REACT_Migration_Plan.md:53` explicitly calls out adding frontend dev origins.
Impact:
- Local `npm run dev` can fail with CORS errors even after the API base URL is corrected.
- Developers may mistakenly debug auth/API code when the root cause is CORS.
Recommendation:
- Add `http://localhost:5173` to `ALLOWED_ORIGINS`.
- Keep Docker/static origin and Vite dev origin both represented in `.env.example`.
### P2-01: Question quality page is a static placeholder
Severity: P2
Category: Missing feature / reporting parity
Evidence:
- `frontend/src/pages/admin/questions/QuestionQuality.tsx:14-47` displays `...` for all metrics.
- `frontend/src/pages/admin/questions/QuestionQuality.tsx:61-65` says diagnostic charts are coming soon.
- Legacy Python admin had a real `/admin/question-quality` view that computed calibrated totals and per-tryout readiness.
Impact:
- Operators lose the prior calibration diagnostics workflow.
- The page appears present but does not provide operational data.
Recommendation:
- Either wire this page to `/reports/calibration/status` per selected tryout or add a dashboard-level quality summary API.
- Replace placeholder cards with real metrics and loading/error states.
### P2-02: Tryout settings and general/security settings are placeholders
Severity: P2
Category: Missing feature / admin configuration
Evidence:
- Tryout settings page contains only placeholder text in `frontend/src/pages/admin/tryouts/TryoutSettings.tsx:14-16`.
- Security settings form has inputs and button but no mutation in `frontend/src/pages/admin/settings/index.tsx:151-176`.
- General settings tab is placeholder text in `frontend/src/pages/admin/settings/index.tsx:203-211`.
- Legacy Python admin had `/admin/password` and website management.
Impact:
- Operators cannot update tryout scoring/selection/AI settings from React.
- Password update looks available but does nothing.
Recommendation:
- Implement tryout settings using `/tryout/{id}/config` plus update endpoints for scoring mode, selection mode, AI generation, and calibration thresholds.
- Add or expose a JSON password-change endpoint, or hide Security until implemented.
- Replace "General" with concrete settings or remove the tab.
### P2-03: Hierarchy/data overview was not migrated
Severity: P2
Category: Missing feature / operator orientation
Evidence:
- Legacy Python admin exposed `/admin/hierarchy`.
- `UX_AUDIT_ADMIN_FLOW.md` and `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md` identified hierarchy visibility as important.
- Current React sidebar has Dashboard, Questions, Tryouts, Reports, Settings only in `frontend/src/layouts/AdminLayout.tsx:10-16`.
- No React hierarchy page exists.
Impact:
- Operators lose the data relationship map for Website -> Tryout -> Snapshot -> Basis Item -> AI Run -> Variant.
- This was specifically identified as important for reducing confusion after import and AI generation.
Recommendation:
- Add a React Data Overview/Hierarchy page.
- Expose a JSON hierarchy API instead of relying on legacy HTML.
- Link it from Dashboard or Tryouts, not only Settings.
### P2-04: Route structure deviates from the planned tryout-centric URL model
Severity: P2
Category: Navigation / route consistency
Evidence:
- `ADMIN_TRYOUT_RESTRUCTURE_PLAN.md:19-28` planned singular `/admin/tryout/{tryout_id}/...` route depth.
- Current React uses plural `/admin/tryouts/:id/...` in `frontend/src/App.tsx:55-60`.
- Planned question workspace route includes question id, but current route is `/admin/tryouts/:id/questions/ai-workspace` without question id.
- TryoutLayout tabs omit Normalization from the visible tab list even though the route exists.
Impact:
- URL semantics differ from the planned hierarchy.
- AI workspace lacks clear parent question context.
- Users navigating to Normalization see a page that is not represented in the tab state.
Recommendation:
- Decide on singular or plural route convention and align docs, React routes, and links.
- Include `questionId` in AI workspace routes.
- Add a visible Normalization tab or move normalization under Settings consistently.
### P2-05: Legacy Python admin remains mounted, creating deployment ambiguity
Severity: P2
Category: Deployment / migration completeness
Evidence:
- `backend/app/main.py` still includes `admin_web_router` when admin is enabled.
- Docker serves React at port `3000` and backend at port `8000`.
- Backend still owns `/admin/*` on port `8000`; React owns `/admin/*` on port `3000`.
Impact:
- If production routing later places frontend and backend behind one host, `/admin` routing can easily point to the wrong application.
- Operators may accidentally use two different admin UIs with different feature coverage.
Recommendation:
- Define production routing explicitly:
- Frontend owns `/admin/*`.
- Backend owns `/api/v1/*`, `/docs`, `/health`, and possibly legacy admin only behind a temporary fallback path.
- Add a migration flag to disable legacy admin once React parity is reached.
### P2-06: Query invalidation and cache keys are not website-aware
Severity: P2
Category: State management / data freshness
Evidence:
- Examples: `['dashboard-stats']`, `['tryouts']`, `['admin-questions']`, `['ai-pending-reviews']`.
- The API interceptor changes `X-Website-ID` based on Zustand state, but React Query cache keys do not reflect that state.
Impact:
- After switching websites, React Query can return prior website data without refetching.
- The visible WebsiteSelector can imply a different scope than the data currently shown.
Recommendation:
- Include `websiteId` in all website-scoped query keys.
- Add a small helper for scoped keys to avoid drift.
- Consider clearing scoped query cache on logout and website switch.
### P3-01: Several buttons look actionable but do nothing
Severity: P3
Category: UX polish / trust
Evidence:
- Report export buttons have no click handlers.
- Excel "Download Template" button has no click handler.
- Global AI "Discard" and "Save & Queue Review" buttons have no click handlers.
- Settings "Update Password" button has no click handler.
Impact:
- The UI feels more complete than it is, which can mislead testers and operators.
Recommendation:
- Remove disabled/nonfunctional controls or wire them to real mutations/downloads.
- Prefer disabled buttons with explanatory tooltip only when the missing backend is intentional.
## 8. Feature Parity Checklist
| Area | Legacy Python admin | React status | Notes |
|---|---|---|---|
| Login/logout | Present | Partial | JWT login works by path only if base URL includes `/api/v1`; no remember-me equivalent. |
| Dashboard | Present | Partial | React has KPI cards, but first-load website scoping and query key issues affect data. |
| Website management | Present | Partial | React CRUD exists; confirm delete cascade semantics and query invalidation. |
| Tryout import JSON | Present | Mostly present | Modal maps to real JSON endpoints when base URL includes `/api/v1`. |
| Excel import | Present via API | Broken | React page calls nonexistent endpoints. |
| Snapshot question promotion | Present as legacy HTML | Missing JSON/React | React calls nonexistent API. |
| Global question list | Present with filters/detail | Partial | React list exists, but filters and detail page are missing. |
| Question detail | Present | Missing | No React route/page. |
| Question quality | Present | Placeholder | Static cards only. |
| Tryout list/tree | Present/planned | Partial | Accordion exists; average NM/NN and some plan details missing. |
| Tryout attempts | Present | Present basic | Filtered table exists. |
| Normalization | Present | Broken | Wrong API contract. |
| Tryout settings | Present via backend fields | Placeholder | No real form. |
| AI basis workspace | Present | Partial | Preview and single save partially exist; batch/review/run history missing. |
| AI pending review | Present | Partial | List and approve/reject exist; preview/detail missing. |
| Variant detail | Present | Missing | No React page. |
| Bulk variant review | Present | Missing | No React workflow. |
| Hierarchy/data overview | Present | Missing | Important operator context lost. |
| Reports dashboard | Present | Broken/placeholder | Wrong endpoints and no tryout filters. |
| Report exports | Present in backend | Missing in React | Buttons not wired. |
| Password update | Present in legacy HTML | Placeholder | No API/mutation. |
| Student tryout portal | Planned | Missing | No React student/session routes. |
## 9. Recommended Remediation Plan
### Phase 0: Stop the bleeding
1. Fix `VITE_API_URL` to include `/api/v1` in `frontend/.env`, Docker build args, and `.env.example`.
2. Add `http://localhost:5173` to backend CORS for Vite dev.
3. Fix system-admin website scoping so no-header system admin is global or explicitly blocked until a website is selected.
4. Gate website-scoped React queries until `websiteId` is available.
5. Add `websiteId` to all scoped query keys.
### Phase 1: Repair broken API contracts
1. Fix Reports paths and require a selected tryout.
2. Fix Normalization GET/PUT paths and payload schema.
3. Remove or fix the broken Excel import page.
4. Decide whether snapshot promotion needs a JSON API and add it if React owns the workflow.
5. Add API contract tests that compare frontend endpoint constants against OpenAPI paths.
### Phase 2: Recover feature parity
1. Implement real Tryout Settings.
2. Implement Question Detail and Variant Detail pages.
3. Implement AI run history, review filters, batch generation, and bulk review.
4. Implement Question Quality with real metrics.
5. Implement Data Overview/Hierarchy in React.
6. Wire report export buttons.
### Phase 3: Student portal
1. Add learner tryout listing.
2. Add active session page using `/session/{id}/next_item` and `/submit_answer`.
3. Add server-synced timer from `expires_at`.
4. Persist session recovery state.
5. Add completion and result pages.
### Phase 4: Migration hardening
1. Decide the final production routing split between frontend `/admin/*` and backend `/api/v1/*`.
2. Disable or move legacy Python admin once React parity is complete.
3. Add Playwright smoke tests for login, website switch, import preview, tryout drilldown, AI preview, normalization save, and reports.
4. Add a route/API smoke test that verifies every visible navigation target and button either works or is intentionally disabled.
## 10. Suggested Test Plan
Minimum tests before considering the React migration complete:
| Test | Expected result |
|---|---|
| `npm run build` | Passes with no TypeScript errors. |
| Login with local env | Hits `/api/v1/auth/admin-login`, stores token, lands on dashboard. |
| First dashboard load | Waits for or uses a real selected website, never website `0`. |
| Website switch | Dashboard, tryouts, questions, reports, and AI pending reviews refetch for the selected website. |
| Tryout JSON preview/import | Calls `/api/v1/import-export/tryout-json/preview` and `/tryout-json`; new tryouts appear. |
| Excel import | Calls real preview and import endpoints or the page is hidden. |
| Normalization save | GET `/api/v1/tryout/{id}/config`, PUT `/api/v1/tryout/{id}/normalization`, visible success/error state. |
| Reports | Requires tryout context and loads real calibration/item/student data. |
| AI preview/save | Saves with correct basis slot and displays generated variant in review queue. |
| AI review/bulk | Approve/reject/archive works and status updates are visible. |
| XSS smoke | Imported HTML and AI HTML are sanitized before rendering. |
| Student session | Start/resume/answer/complete/result works with server timer. |
## 11. Final Assessment
The migration has a good foundation: React, routing, TanStack Query, Zustand, shadcn-style components, Docker/Nginx serving, and a number of admin pages are already present. The main risk is that the UI currently looks further along than its backend integration really is.
The highest leverage next move is to stabilize the API boundary: fix the `/api/v1` base URL, align endpoint paths and methods, make website scoping deterministic, and add website-aware query keys. Once that is done, the team can safely fill the larger parity gaps without chasing confusing 404s, empty dashboards, or stale tenant data.

31
FRONTEND_MIGRATION_CUTOVER.md Normal file
View File

@@ -0,0 +1,31 @@
# Frontend Migration Cutover Notes
## Route Ownership
- React owns browser-facing admin routes under `/admin/*` and student routes under `/student/*`.
- FastAPI owns JSON APIs under `/api/v1/*`.
- The legacy Python admin remains available as fallback until React parity smoke tests are accepted.
## Local Development
- React Vite dev server: `http://127.0.0.1:5173`
- Backend API root: `http://localhost:8000/api/v1`
- Frontend API config should keep `VITE_API_URL` pointed at the FastAPI v1 root.
- System-admin tokens may be global with `website_id: null`; React sends `X-Website-ID` only when the website selector has an explicit website.
## Cutover Guardrails
- Do not disable the legacy admin until React covers import, snapshot promotion, question detail, AI review, reports, normalization, settings, and student session smoke tests.
- Avoid adding new frontend calls to legacy or nonexistent API paths. New React API calls should map to OpenAPI paths.
- Website-scoped React Query keys must include the selected website ID and should be gated until a website is selected.
- Any page rendering question HTML must use the shared `SafeHtml` component.
## Smoke Coverage Used During Migration Fix
- Admin dashboard
- Global questions list and question detail
- Data overview hierarchy
- AI review, variants, and run history
- Excel import
- Tryout questions, snapshot promotion, settings, normalization, and AI workspace
- Student tryout list, session start, next item, answer submission, completion, and result summary

439
UX_AUDIT_ADMIN_FLOW.md Normal file
View File

@@ -0,0 +1,439 @@
# UX Audit: Admin Flow - IRT Bank Soal
> **Audit Date:** 2026-06-17
> **Auditor:** Dev Agent
> **Focus:** Login → First-time experience → Navigation discoverability → Hierarchy visibility
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Login Flow Analysis](#login-flow-analysis)
3. [Post-Login Experience](#post-login-experience)
4. [Navigation & Discoverability](#navigation--discoverability)
5. [Hierarchy Visibility](#hierarchy-visibility)
6. [Issue Summary & Priority Matrix](#issue-summary--priority-matrix)
7. [Recommended Improvements](#recommended-improvements)
8. [Appendix: Current vs Proposed Flow](#appendix-current-vs-proposed-flow)
---
## Executive Summary
The current admin flow has significant UX gaps that make it difficult for new administrators to orient themselves and complete tasks efficiently. The main issues are:
| Category | Severity | Count |
|----------|----------|-------|
| Critical (blocks usage) | 🔴 High | 4 |
| Medium (confuses users) | 🟡 Medium | 6 |
| Low (minor friction) | 🟢 Low | 5 |
### Key Findings
1. **No onboarding guidance** after login - users land on Dashboard with no context
2. **Hierarchy is hidden** in Settings submenu - should be prominently visible
3. **Navigation labels are inconsistent** - mixed technical and human terms
4. **Login page lacks branding** - no visual connection to the product
5. **No breadcrumb navigation** - users get lost in deep pages
---
## Login Flow Analysis
### Current State
The login page (`/admin/login`) presents:
- Simple username/password form
- "Remember me" checkbox
- Minimal error messaging
- Help button (bottom-right corner)
```python
# Current login form elements
- Username field
- Password field
- Remember me checkbox
- Sign in button
```
### Issues Found
| # | Issue | Impact | Severity |
|---|-------|--------|----------|
| 1.1 | **No product branding/logo** | Users don't know what system they're logging into | 🟡 Medium |
| 1.2 | **No error state distinction** | Failed login looks same as rate limiting | 🟡 Medium |
| 1.3 | **"Remember me" is unclear** | Doesn't explain session duration or implications | 🟢 Low |
| 1.4 | **No "forgot password" path** | No recovery mechanism exists | 🟡 Medium |
| 1.5 | **Help button is discoverable** | Good: floating help exists but underutilized | 🟢 Positive |
### Login → Dashboard Redirect
**Current behavior:** After successful login → `/admin/dashboard`
**What users see:**
```
┌─────────────────────────────────────────┐
│ Good Morning, admin! 👋 │
│ Here's what's happening today. │
│ │
│ ⚠️ 25 questions need calibration │
│ 📝 3 AI-generated questions pending │
│ 💡 Tip: Start by importing questions... │
│ │
│ 📊 System Overview │
│ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │
│ │ 5 │ │ 150 │ │ 890 │ │ 2 │ │
│ │Exams │ │Quest │ │Tests │ │Sites │ │
│ └──────┘ └──────┘ └──────┘ └──────┘ │
└─────────────────────────────────────────┘
```
### Problems After Login
| # | Issue | Why It's a Problem |
|---|-------|-------------------|
| 2.1 | **No welcome message explaining the system** | First-time users don't know what IRT Bank Soal does |
| 2.2 | **"5 Exams" is meaningless without context** | Users don't know what an Exam/Tryout means |
| 2.3 | **Alerts are action-oriented but not instructive** | "Import questions" - but where? How? |
| 2.4 | **Quick Actions use technical language** | "Generate AI Questions" doesn't explain what happens |
| 2.5 | **No first-time setup wizard** | Empty state users have no guidance |
---
## Navigation & Discoverability
### Current Navigation Structure
```
Sidebar Navigation (collapsed view):
┌─────────────────────────┐
│ IRT Bank Soal Admin │
├─────────────────────────┤
│ 📊 Dashboard │ ← Always first
│ 📝 Questions │ ← What is this?
│ 📥 Import Questions │ ← Separate from Questions?
│ 🤖 AI Generator │ ← Is this part of Questions?
│ 📋 Exams │ ← Tryout = Exam?
│ 📈 Reports │
│ ⚙️ Settings │ ← Hierarchy buried here
│ ─────────────────────── │
│ 🚪 Logout │
└─────────────────────────┘
```
### Label Analysis
| Current Label | User Interpretation | Issue |
|---------------|---------------------|-------|
| Questions | "Where I view questions?" | ✅ Clear |
| Import Questions | "Is this separate from Questions?" | ⚠️ Unclear relationship |
| AI Generator | "What does AI Generate?" | ⚠️ Vague |
| Exams | "Same as Tryout?" | ⚠️ Mismatch with backend term |
| Reports | "Student scores?" | ✅ Clear |
| Settings → Hierarchy | "What is hierarchy?" | 🔴 Wrong place + wrong term |
### Missing Navigation Features
| # | Missing Feature | Impact |
|---|-----------------|--------|
| 3.1 | **No breadcrumbs** | Users can't trace their path back |
| 3.2 | **No "back to parent" links** | Deep pages have no escape route |
| 3.3 | **No search/global nav** | Can't jump to specific pages |
| 3.4 | **No recent pages** | Can't quickly return to work in progress |
| 3.5 | **Settings is a catch-all** | Mixes Website management, Hierarchy, Password |
---
## Hierarchy Visibility
### Current Hierarchy Location
Hierarchy is located at: **Settings → Data Structure** (`/admin/hierarchy`)
### Problems with Current Hierarchy Placement
| # | Issue | Why It Matters |
|---|-------|----------------|
| 4.1 | **Buried 2 levels deep** | First-time users never find it |
| 4.2 | **Label is technical** | "Data Structure" vs "How data connects" |
| 4.3 | **No explanation of the hierarchy concept** | Users don't know Website → Tryout → Questions → Variants |
| 4.4 | **No visual flowchart on Dashboard** | Users should see the big picture immediately |
### Expected Mental Model
```
┌─────────────────────────────────────────────────────────────┐
│ USER'S EXPECTED FLOW │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Website (where exams are hosted) │
│ │ │
│ ▼ │
│ 2. Tryout/Exam (the test itself) │
│ │ │
│ ▼ │
│ 3. Questions (individual items in the test) │
│ │ │
│ ├── Original/Basis Question ──────────────────────┐ │
│ │ │ │ │
│ │ ▼ │ │
│ │ AI Variant (different version) │ │
│ │ │ │
│ └── (repeated for each question slot) │ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Where Users Expect Hierarchy Info
| Location | User Expectation |
|----------|------------------|
| **Dashboard** | "Show me the big picture" - visual overview |
| **First-time tooltip** | "Here's how things connect" |
| **Help/Docs** | "Explain the data model" |
| **Settings sidebar** | ❌ Too late - user already lost |
---
## Issue Summary & Priority Matrix
### Priority Matrix
```
│ High Value │ Low Value │
────────────────────┼──────────────┼──────────────┤
High Effort │ [A] Refactor │ [B] Nice to │
│ Navigation │ have │
────────────────────┼──────────────┼──────────────┤
Low Effort │ [C] Quick │ [D] Ignore │
│ Wins │ │
────────────────────┼──────────────┼──────────────┤
```
### Cell [A] - High Value, High Effort (Do First)
| Issue ID | Description | Notes |
|----------|-------------|-------|
| P1 | **Add Dashboard onboarding section** | Explain the system + show hierarchy flow |
| P2 | **Move Hierarchy to prominent location** | Dashboard or separate nav item |
| P3 | **Redesign navigation labels** | Human-friendly, consistent terminology |
| P4 | **Add breadcrumbs** | Across all pages |
### Cell [C] - High Value, Low Effort (Quick Wins)
| Issue ID | Description | Effort |
|----------|-------------|--------|
| Q1 | Add product logo to login page | 15 min |
| Q2 | Improve dashboard welcome message | 10 min |
| Q3 | Add "How it works" section to Dashboard | 30 min |
| Q4 | Rename "Data Structure" → "Data Overview" in Settings | 5 min |
| Q5 | Add contextual tooltips to Quick Actions | 20 min |
### Cell [B] - Low Value, High Effort (Consider Later)
| Issue ID | Description |
|----------|-------------|
| L1 | Global search across all pages |
| L2 | Recent pages sidebar widget |
| L3 | Full first-time setup wizard |
### Cell [D] - Low Value, Low Effort (Ignore)
| Issue ID | Description |
|----------|-------------|
| N1 | Custom "Remember me" tooltip |
| N2 | Login page background gradient (cosmetic only) |
---
## Recommended Improvements
### Phase 1: Critical Fixes (Same Session)
#### 1. Login Page Enhancement
```html
<!-- Add to login page -->
<div class="login-header">
<img src="/static/logo.png" alt="IRT Bank Soal" class="login-logo">
<h1>IRT Bank Soal</h1>
<p>Adaptive Question Bank System</p>
</div>
```
#### 2. Dashboard - Add "How It Works" Section
Add this block to dashboard after greeting:
```html
<div class="onboarding-flow">
<h3>How Your Exam System Works</h3>
<div class="flow-steps">
<div class="step">
<span class="step-num">1</span>
<span class="step-title">Add Website</span>
<span class="step-desc">Connect your WordPress site</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">2</span>
<span class="step-title">Import Questions</span>
<span class="step-desc">Upload your exam questions</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">3</span>
<span class="step-title">Generate Variants</span>
<span class="step-desc">AI creates different versions</span>
</div>
<div class="step-arrow">→</div>
<div class="step">
<span class="step-num">4</span>
<span class="step-title">Students Take Tests</span>
<span class="step-desc">Adaptive difficulty adjusts</span>
</div>
</div>
<a href="/admin/hierarchy" class="flow-link">View full data structure →</a>
</div>
```
#### 3. Dashboard - Add "Get Started" for Empty State
When `tryouts_count == 0`:
```html
<div class="getting-started">
<h2>🚀 Welcome to IRT Bank Soal!</h2>
<p>Get started in 3 simple steps:</p>
<div class="steps">
<div class="step-card">
<span class="num">1</span>
<h3>Connect a Website</h3>
<p>Add your WordPress site to the system</p>
<a href="/admin/websites" class="btn">Add Website →</a>
</div>
<div class="step-card">
<span class="num">2</span>
<h3>Import Questions</h3>
<p>Upload questions from Excel or JSON</p>
<a href="/admin/tryout-import" class="btn">Import Questions →</a>
</div>
<div class="step-card">
<span class="num">3</span>
<h3>Generate Variants</h3>
<p>Use AI to create question variations</p>
<a href="/admin/basis-items" class="btn">Generate Variants →</a>
</div>
</div>
</div>
```
### Phase 2: Navigation Improvement (Next Sprint)
#### 4. Rename Navigation Items
| Current | Proposed | Reason |
|---------|----------|--------|
| Import Questions | Import from Excel | More specific |
| AI Generator | Generate AI Questions | Action-oriented |
| Settings → Hierarchy | (move to Dashboard) | Too hidden |
| Questions | Question Bank | Clarify scope |
#### 5. Add Breadcrumbs Component
```html
<nav class="breadcrumbs">
<a href="/admin/dashboard">Dashboard</a>
<span class="sep">›</span>
<a href="/admin/questions">Questions</a>
<span class="sep">›</span>
<span class="current">Question #123</span>
</nav>
```
### Phase 3: Advanced Features (Future)
#### 6. First-Time Setup Wizard
Modal that walks new admins through:
1. Website configuration
2. First import
3. Basic settings review
#### 7. Interactive Hierarchy Diagram
Replace static hierarchy view with interactive visualization:
```mermaid
graph LR
A[Website] --> B[Tryout]
B --> C[Questions]
C --> D[Variants]
C --> E[Student Answers]
D --> F[AI Generation]
```
---
## Appendix: Current vs Proposed Flow
### Current Flow (Confusing)
```
Login
↓
Dashboard (counts, no context)
↓ (guess where to go)
Settings? Questions? Import? (trial & error)
↓
Get lost → Leave → Ask for help
```
### Proposed Flow (Guided)
```
Login
↓
Dashboard
├─ "Here's how it works" (visual flow)
├─ Quick Stats (with explanations)
├─ Alerts (with direct action buttons)
└─ Recent Activity
↓
Follow guided steps OR jump to specific task
↓
Complete task → Return to Dashboard
↓
See updated progress
```
---
## Files to Modify
| File | Changes Needed |
|------|---------------|
| `app/admin_web.py` | Dashboard content, navigation labels, breadcrumbs |
| `app/admin_web_icons.py` | (No changes needed) |
| `app/templates/` | (Add if using templates) |
---
## Test Checklist
After implementing changes, verify:
- [ ] Login page shows product branding
- [ ] Dashboard explains the system for first-time users
- [ ] Empty state shows guided setup
- [ ] Navigation labels are consistent and clear
- [ ] Hierarchy is accessible from Dashboard
- [ ] Breadcrumbs appear on all sub-pages
- [ ] Quick Actions have explanatory tooltips
- [ ] User can complete first import without help
---
*End of Audit Report*

227
app/routers/admin.py
View File

@@ -1,227 +0,0 @@
"""
Admin API router for custom admin actions.
Provides admin-specific endpoints for triggering calibration,
toggling AI generation, and resetting normalization.
"""
from typing import Any, Dict
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.auth import AuthContext, get_auth_context, require_website_auth
from app.core.config import get_settings
from app.database import get_db
from app.models import Tryout, TryoutStats
from app.services.irt_calibration import (
calibrate_all,
CALIBRATION_SAMPLE_THRESHOLD,
)
router = APIRouter(prefix="/admin", tags=["admin"])
settings = get_settings()
@router.post(
"/{tryout_id}/calibrate",
summary="Trigger IRT calibration",
description="Trigger IRT calibration for all items in this tryout with sufficient response data.",
)
async def admin_trigger_calibration(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Trigger IRT calibration for all items in a tryout.
Runs calibration for items with >= min_calibration_sample responses.
Updates item.irt_b, item.irt_se, and item.calibrated status.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Calibration results summary
Raises:
HTTPException: If tryout not found or calibration fails
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Run calibration
result = await calibrate_all(
tryout_id=tryout_id,
website_id=website_id,
db=db,
min_sample_size=tryout.min_calibration_sample or CALIBRATION_SAMPLE_THRESHOLD,
)
return {
"tryout_id": tryout_id,
"total_items": result.total_items,
"calibrated_items": result.calibrated_items,
"failed_items": result.failed_items,
"calibration_percentage": round(result.calibration_percentage * 100, 2),
"ready_for_irt": result.ready_for_irt,
"message": f"Calibration complete: {result.calibrated_items}/{result.total_items} items calibrated",
}
@router.post(
"/{tryout_id}/toggle-ai-generation",
summary="Toggle AI generation",
description="Toggle AI question generation for a tryout.",
)
async def admin_toggle_ai_generation(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Toggle AI generation for a tryout.
Updates Tryout.AI_generation_enabled field.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Updated AI generation status
Raises:
HTTPException: If tryout not found
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Toggle AI generation
tryout.ai_generation_enabled = not tryout.ai_generation_enabled
await db.commit()
await db.refresh(tryout)
status = "enabled" if tryout.ai_generation_enabled else "disabled"
return {
"tryout_id": tryout_id,
"ai_generation_enabled": tryout.ai_generation_enabled,
"message": f"AI generation {status} for tryout {tryout_id}",
}
@router.post(
"/{tryout_id}/reset-normalization",
summary="Reset normalization",
description="Reset normalization to static values and clear incremental stats.",
)
async def admin_reset_normalization(
tryout_id: str,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> Dict[str, Any]:
"""
Reset normalization for a tryout.
Resets rataan, sb to static values and clears incremental stats.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Reset statistics
Raises:
HTTPException: If tryout or stats not found
"""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"TryoutStats for {tryout_id} not found for website {website_id}",
)
# Get tryout for static values
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout:
# Reset to static values
stats.rataan = tryout.static_rataan
stats.sb = tryout.static_sb
else:
# Reset to default values
stats.rataan = 500.0
stats.sb = 100.0
# Clear incremental stats
old_participant_count = stats.participant_count
stats.participant_count = 0
stats.total_nm_sum = 0.0
stats.total_nm_sq_sum = 0.0
stats.min_nm = None
stats.max_nm = None
stats.last_calculated = None
await db.commit()
await db.refresh(stats)
return {
"tryout_id": tryout_id,
"rataan": stats.rataan,
"sb": stats.sb,
"cleared_stats": {
"previous_participant_count": old_participant_count,
},
"message": f"Normalization reset to static values (rataan={stats.rataan}, sb={stats.sb}). Incremental stats cleared.",
}

102
app/schemas/ai.py
View File

@@ -1,102 +0,0 @@
"""
Pydantic schemas for AI generation endpoints.
Request/response models for admin AI generation playground.
"""
from typing import Dict, Literal, Optional
from pydantic import BaseModel, Field, field_validator
class AIGeneratePreviewRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated question"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
class AIGeneratePreviewResponse(BaseModel):
success: bool = Field(..., description="Whether generation was successful")
stem: Optional[str] = None
options: Optional[Dict[str, str]] = None
correct: Optional[str] = None
explanation: Optional[str] = None
ai_model: Optional[str] = None
basis_item_id: Optional[int] = None
target_level: Optional[str] = None
error: Optional[str] = None
cached: bool = False
class AISaveRequest(BaseModel):
stem: str = Field(..., description="Question stem")
options: Dict[str, str] = Field(
..., description="Answer options (A, B, C, D)"
)
correct: str = Field(..., description="Correct answer (A/B/C/D)")
explanation: Optional[str] = None
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
basis_item_id: int = Field(..., description="Basis item ID")
slot: int = Field(..., description="Question slot position")
level: Literal["mudah", "sedang", "sulit"] = Field(
..., description="Difficulty level"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model used for generation",
)
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()
@field_validator("options")
@classmethod
def validate_options(cls, v: Dict[str, str]) -> Dict[str, str]:
required_keys = {"A", "B", "C", "D"}
if not required_keys.issubset(set(v.keys())):
raise ValueError("Options must contain keys A, B, C, D")
return v
class AISaveResponse(BaseModel):
success: bool = Field(..., description="Whether save was successful")
item_id: Optional[int] = None
error: Optional[str] = None
class AIStatsResponse(BaseModel):
total_ai_items: int = Field(..., description="Total AI-generated items")
items_by_model: Dict[str, int] = Field(
default_factory=dict, description="Items count by AI model"
)
cache_hit_rate: float = Field(
default=0.0, description="Cache hit rate (0.0 to 1.0)"
)
total_cache_hits: int = Field(default=0, description="Total cache hits")
total_requests: int = Field(default=0, description="Total generation requests")
class GeneratedQuestion(BaseModel):
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()

0
.env.example → backend/.env.example
View File

2
Dockerfile → backend/Dockerfile
View File

@@ -15,4 +15,4 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Run migrations and start the app
CMD ["sh", "-c", "alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"]
CMD ["sh", "-c", "alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000"]

0
alembic.ini → backend/alembic.ini
View File

0
alembic/README → backend/alembic/README
View File

0
alembic/env.py → backend/alembic/env.py
View File

0
alembic/script.py.mako → backend/alembic/script.py.mako
View File

0
alembic/versions/20260331_000001_initial_schema.py → backend/alembic/versions/20260331_000001_initial_schema.py
View File

0
alembic/versions/20260402_000002_tryout_json_snapshots.py → backend/alembic/versions/20260402_000002_tryout_json_snapshots.py
View File

0
alembic/versions/20260404_000003_ai_runs_and_variant_status.py → backend/alembic/versions/20260404_000003_ai_runs_and_variant_status.py
View File

0
alembic/versions/20260405_000004_report_schedules.py → backend/alembic/versions/20260405_000004_report_schedules.py
View File

26
backend/alembic/versions/20260617_000005_session_expires_at.py Normal file
View File

@@ -0,0 +1,26 @@
"""add session expires at
Revision ID: 20260617_000005
Revises: 20260405_000004
Create Date: 2026-06-17 15:00:00
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260617_000005"
down_revision: Union[str, None] = "20260405_000004"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("sessions", sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True))
def downgrade() -> None:
op.drop_column("sessions", "expires_at")

0
app/__init__.py → backend/app/__init__.py
View File

0
app/admin.py → backend/app/admin.py
View File

1849
app/admin_web.py → backend/app/admin_web.py
View File

File diff suppressed because it is too large Load Diff

4
app/admin_web_icons.py → backend/app/admin_web_icons.py
View File

@@ -102,9 +102,7 @@ EMOJI_TO_ICON = {
# Navigation icon mapping
NAV_ICONS_SVG = {
"Dashboard": ICON_DASHBOARD,
"Questions": ICON_QUESTIONS,
"Import Questions": ICON_IMPORT,
"AI Generator": ICON_AI,
"Import": ICON_IMPORT,
"Exams": ICON_EXAMS,
"Reports": ICON_REPORTS,
"Settings": ICON_SETTINGS,

0
app/api/__init__.py → backend/app/api/__init__.py
View File

0
app/api/v1/__init__.py → backend/app/api/v1/__init__.py
View File

8
app/api/v1/session.py → backend/app/api/v1/session.py
View File

@@ -50,6 +50,9 @@ class NextItemResponse(BaseModel):
options: Optional[dict] = None
slot: Optional[int] = None
level: Optional[str] = None
display_level: Optional[str] = None
generated_by: Optional[str] = None
source_snapshot_question_id: Optional[int] = None
selection_method: Optional[str] = None
reason: Optional[str] = None
current_theta: Optional[float] = None
@@ -212,6 +215,11 @@ async def get_next_item_endpoint(
options=item.options,
slot=item.slot,
level=item.level,
display_level="Original"
if item.generated_by != "ai" and item.source_snapshot_question_id is not None
else item.level,
generated_by=item.generated_by,
source_snapshot_question_id=item.source_snapshot_question_id,
selection_method=result.selection_method,
reason=result.reason,
current_theta=session.theta,

0
app/core/__init__.py → backend/app/core/__init__.py
View File

42
app/core/auth.py → backend/app/core/auth.py
View File

@@ -21,7 +21,7 @@ settings = get_settings()
@dataclass
class AuthContext:
website_id: int
website_id: Optional[int]
role: str
wp_user_id: Optional[str] = None
@@ -36,13 +36,13 @@ def _b64url_decode(raw: str) -> bytes:
def issue_access_token(
website_id: int,
website_id: int | None,
role: str = "student",
wp_user_id: str | None = None,
expires_in_seconds: int = 3600,
) -> str:
payload = {
"website_id": int(website_id),
"website_id": int(website_id) if website_id is not None else None,
"role": role,
"wp_user_id": wp_user_id,
"exp": int(time.time()) + int(expires_in_seconds),
@@ -91,14 +91,19 @@ def decode_access_token(token: str) -> AuthContext:
website_id = payload.get("website_id")
role = payload.get("role")
if website_id is None or not role:
if not role:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token missing required claims",
)
if website_id is None and role != "system_admin":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token missing website scope",
)
return AuthContext(
website_id=int(website_id),
website_id=int(website_id) if website_id is not None else None,
role=str(role),
wp_user_id=payload.get("wp_user_id"),
)
@@ -106,6 +111,7 @@ def decode_access_token(token: str) -> AuthContext:
def get_auth_context(
authorization: str | None = Header(None, alias="Authorization"),
x_website_id: str | None = Header(None, alias="X-Website-ID"),
) -> AuthContext:
if authorization is None:
raise HTTPException(
@@ -118,25 +124,45 @@ def get_auth_context(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid Authorization header format. Use: Bearer {token}",
)
return decode_access_token(parts[1])
context = decode_access_token(parts[1])
# If system_admin explicitly sets a website context via header, use it
if context.role == "system_admin" and x_website_id and x_website_id.isdigit():
context.website_id = int(x_website_id)
return context
def require_website_auth(
auth: AuthContext,
allowed_roles: set[str] | None = None,
) -> int:
) -> Optional[int]:
"""
Check if the authenticated user has required roles.
Returns the website_id if scoped to a specific website.
Returns None if the user is a system_admin with global access and no specific website context.
"""
if allowed_roles is not None and auth.role not in allowed_roles:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Insufficient permissions for this endpoint",
)
if auth.role == "system_admin":
if auth.website_id is not None:
return auth.website_id
return None
return auth.website_id
def ensure_website_scope_matches(
auth_website_id: int,
auth_website_id: int | None,
payload_website_id: int,
) -> None:
if auth_website_id is None:
return
if int(auth_website_id) != int(payload_website_id):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,

8
app/core/config.py → backend/app/core/config.py
View File

@@ -4,10 +4,10 @@ Application configuration using Pydantic Settings.
Loads configuration from environment variables with validation.
"""
from typing import Literal, List, Union
from typing import Annotated, Literal, List, Union
from pydantic import Field, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
class Settings(BaseSettings):
@@ -98,8 +98,8 @@ class Settings(BaseSettings):
)
# CORS - stored as list, accepts comma-separated string from env
ALLOWED_ORIGINS: List[str] = Field(
default=["http://localhost:3000"],
ALLOWED_ORIGINS: Annotated[List[str], NoDecode] = Field(
default=["http://localhost:3000", "http://localhost:5173", "http://127.0.0.1:5173"],
description="List of allowed CORS origins",
)

0
app/core/rate_limit.py → backend/app/core/rate_limit.py
View File

0
app/database.py → backend/app/database.py
View File

10
app/main.py → backend/app/main.py
View File

@@ -31,11 +31,13 @@ from app.database import close_db, init_db
from app.routers import (
admin_router,
ai_router,
auth_router,
import_export_router,
reports_router,
sessions_router,
tryouts_router,
wordpress_router,
websites_router,
)
settings = get_settings()
@@ -190,6 +192,10 @@ async def health_check():
# Include API routers with version prefix
app.include_router(
auth_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
import_export_router,
)
@@ -213,6 +219,10 @@ app.include_router(
reports_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
websites_router,
prefix=f"{settings.API_V1_STR}",
)
if settings.ENABLE_ADMIN:
app.include_router(

0
app/models/__init__.py → backend/app/models/__init__.py
View File

0
app/models/ai_generation_run.py → backend/app/models/ai_generation_run.py
View File

0
app/models/item.py → backend/app/models/item.py
View File

0
app/models/report_schedule.py → backend/app/models/report_schedule.py
View File

3
app/models/session.py → backend/app/models/session.py
View File

@@ -89,6 +89,9 @@ class Session(Base):
end_time: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True), nullable=True, comment="Session end timestamp"
)
expires_at: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True), nullable=True, comment="Session expiration timestamp"
)
is_completed: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, comment="Completion status"
)

0
app/models/tryout.py → backend/app/models/tryout.py
View File

0
app/models/tryout_import_snapshot.py → backend/app/models/tryout_import_snapshot.py
View File

0
app/models/tryout_snapshot_question.py → backend/app/models/tryout_snapshot_question.py
View File

0
app/models/tryout_stats.py → backend/app/models/tryout_stats.py
View File

0
app/models/user.py → backend/app/models/user.py
View File

0
app/models/user_answer.py → backend/app/models/user_answer.py
View File

0
app/models/website.py → backend/app/models/website.py
View File

4
app/routers/__init__.py → backend/app/routers/__init__.py
View File

@@ -4,18 +4,22 @@ API routers package.
from app.routers.admin import router as admin_router
from app.routers.ai import router as ai_router
from app.routers.auth import router as auth_router
from app.routers.import_export import router as import_export_router
from app.routers.reports import router as reports_router
from app.routers.sessions import router as sessions_router
from app.routers.tryouts import router as tryouts_router
from app.routers.wordpress import router as wordpress_router
from app.routers.websites import router as websites_router
__all__ = [
"admin_router",
"ai_router",
"auth_router",
"import_export_router",
"reports_router",
"sessions_router",
"tryouts_router",
"wordpress_router",
"websites_router",
]

1077
backend/app/routers/admin.py Normal file
View File

File diff suppressed because it is too large Load Diff

260
app/routers/ai.py → backend/app/routers/ai.py
View File

@@ -8,7 +8,7 @@ import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request, status
from sqlalchemy import and_, select
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
@@ -22,6 +22,9 @@ from app.core.rate_limit import enforce_rate_limit
from app.database import get_db
from app.models.item import Item
from app.schemas.ai import (
AIBatchGeneratedItem,
AIGenerateBatchRequest,
AIGenerateBatchResponse,
AIGeneratePreviewRequest,
AIGeneratePreviewResponse,
AISaveRequest,
@@ -30,8 +33,13 @@ from app.schemas.ai import (
)
from app.services.ai_generation import (
SUPPORTED_MODELS,
combine_usage,
create_generation_run,
generate_question,
generate_questions_batch,
generated_matches_basis_options,
get_ai_stats,
get_model_pricing,
save_ai_question,
validate_ai_model,
)
@@ -42,6 +50,19 @@ settings = get_settings()
router = APIRouter(prefix="/admin/ai", tags=["admin", "ai-generation"])
def _validate_original_basis_item(basis_item: Item) -> None:
if basis_item.level != "sedang":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Basis item must be 'sedang' level, got: {basis_item.level}",
)
if basis_item.generated_by == "ai":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Basis item must be an original question, not an AI-generated variant.",
)
@router.post(
"/generate-preview",
response_model=AIGeneratePreviewResponse,
@@ -107,12 +128,7 @@ async def generate_preview(
)
ensure_website_scope_matches(website_id, basis_item.website_id)
# Validate basis item is sedang level
if basis_item.level != "sedang":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Basis item must be 'sedang' level, got: {basis_item.level}",
)
_validate_original_basis_item(basis_item)
# Generate question
try:
@@ -137,6 +153,7 @@ async def generate_preview(
options=generated.options,
correct=generated.correct,
explanation=generated.explanation,
usage=generated.usage,
ai_model=request.ai_model,
basis_item_id=request.basis_item_id,
target_level=request.target_level,
@@ -171,7 +188,6 @@ async def generate_preview(
200: {"description": "Question saved successfully"},
400: {"description": "Invalid request data"},
404: {"description": "Basis item or tryout not found"},
409: {"description": "Item already exists at this slot/level"},
500: {"description": "Database save failed"},
},
)
@@ -185,8 +201,8 @@ async def generate_save(
Save AI-generated question to database.
- **stem**: Question text
- **options**: Dict with A, B, C, D options
- **correct**: Correct answer (A/B/C/D)
- **options**: Dict with the same option labels as the basis item
- **correct**: Correct answer label from the generated options
- **explanation**: Answer explanation (optional)
- **tryout_id**: Tryout identifier
- **website_id**: Website identifier
@@ -216,26 +232,7 @@ async def generate_save(
detail=f"Basis item not found: {request.basis_item_id}",
)
ensure_website_scope_matches(website_id, basis_item.website_id)
# Check for duplicate (same tryout, website, slot, level)
existing_result = await db.execute(
select(Item).where(
and_(
Item.tryout_id == request.tryout_id,
Item.website_id == request.website_id,
Item.slot == request.slot,
Item.level == request.level,
)
)
)
existing = existing_result.scalar_one_or_none()
if existing:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"Item already exists at slot={request.slot}, level={request.level} "
f"for tryout={request.tryout_id}",
)
_validate_original_basis_item(basis_item)
# Create GeneratedQuestion from request
from app.schemas.ai import GeneratedQuestion
@@ -246,6 +243,21 @@ async def generate_save(
correct=request.correct,
explanation=request.explanation,
)
if not generated_matches_basis_options(generated_data, basis_item):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Generated options must match the basis question option labels exactly.",
)
run_id = await create_generation_run(
basis_item_id=basis_item.id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
target_level=request.level,
requested_count=1,
model=request.ai_model,
created_by=auth.wp_user_id or auth.role,
db=db,
)
# Save to database
item_id = await save_ai_question(
@@ -256,6 +268,9 @@ async def generate_save(
slot=request.slot,
level=request.level,
ai_model=request.ai_model,
generation_run_id=run_id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
variant_status=request.variant_status,
db=db,
)
@@ -268,6 +283,111 @@ async def generate_save(
return AISaveResponse(
success=True,
item_id=item_id,
run_id=run_id,
)
@router.post(
"/generate-batch",
response_model=AIGenerateBatchResponse,
summary="Generate and save AI question batch",
description="Generate multiple trusted active variants from one medium-level basis question and track the run.",
)
async def generate_batch(
request_http: Request,
request: AIGenerateBatchRequest,
db: Annotated[AsyncSession, Depends(get_db)],
auth: AuthContext = Depends(get_auth_context),
) -> AIGenerateBatchResponse:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
await enforce_rate_limit(
request_http,
scope="ai.generate_batch",
max_requests=10,
window_seconds=300,
)
if not validate_ai_model(request.ai_model):
supported = ", ".join(SUPPORTED_MODELS.keys())
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Unsupported AI model: {request.ai_model}. Supported models: {supported}",
)
result = await db.execute(select(Item).where(Item.id == request.basis_item_id))
basis_item = result.scalar_one_or_none()
if not basis_item:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Basis item not found: {request.basis_item_id}",
)
ensure_website_scope_matches(website_id, basis_item.website_id)
_validate_original_basis_item(basis_item)
run_id = await create_generation_run(
basis_item_id=basis_item.id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
target_level=request.target_level,
requested_count=request.count,
model=request.ai_model,
created_by=auth.wp_user_id or auth.role,
operator_notes=request.operator_notes,
db=db,
)
generated_questions = await generate_questions_batch(
basis_item=basis_item,
target_level=request.target_level,
ai_model=request.ai_model,
count=request.count,
operator_notes=request.operator_notes,
)
item_ids: list[int] = []
response_items: list[AIBatchGeneratedItem] = []
for generated in generated_questions:
item_id = await save_ai_question(
generated_data=generated,
tryout_id=basis_item.tryout_id,
website_id=basis_item.website_id,
basis_item_id=basis_item.id,
slot=basis_item.slot,
level=request.target_level,
ai_model=request.ai_model,
db=db,
generation_run_id=run_id,
source_snapshot_question_id=basis_item.source_snapshot_question_id,
variant_status="active",
)
if item_id is not None:
item_ids.append(item_id)
response_items.append(
AIBatchGeneratedItem(
item_id=item_id,
stem=generated.stem,
options=generated.options,
correct=generated.correct,
explanation=generated.explanation,
level=request.target_level,
variant_status="active",
usage=generated.usage,
)
)
if not item_ids:
return AIGenerateBatchResponse(
success=False,
run_id=run_id,
generated_count=0,
error="AI generation failed. No variants were saved.",
)
return AIGenerateBatchResponse(
success=True,
run_id=run_id,
item_ids=item_ids,
items=response_items,
generated_count=len(item_ids),
usage=combine_usage([item.usage for item in response_items]),
)
@@ -313,8 +433,7 @@ async def list_models(auth: AuthContext = Depends(get_auth_context)) -> dict:
List supported AI models.
"""
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
return {
"models": [
configured_models = [
{
"id": settings.OPENROUTER_MODEL_CHEAP,
"name": "Mistral Small 4",
@@ -331,4 +450,81 @@ async def list_models(auth: AuthContext = Depends(get_auth_context)) -> dict:
"description": "Premium fallback when you want better quality over cost",
},
]
models = []
for model in configured_models:
pricing = await get_model_pricing(model["id"])
models.append({**model, "pricing": pricing})
return {"models": models}
@router.get(
"/pending-reviews",
summary="Get pending AI generated questions",
description="Retrieve all AI generated questions that are pending review (variant_status='draft').",
)
async def admin_get_pending_reviews(
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> dict:
"""Retrieve pending reviews."""
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
query = (
select(Item)
.where(Item.generated_by == "ai", Item.variant_status == "draft")
.order_by(Item.created_at.desc())
.limit(200)
)
if website_id is not None:
query = query.where(Item.website_id == website_id)
result = await db.execute(query)
items = result.scalars().all()
return {
"items": [
{
"id": i.id,
"tryout_id": i.tryout_id,
"level": i.level,
"stem_text": i.stem_text if hasattr(i, 'stem_text') else i.stem[:100],
"ai_model": i.ai_model,
"basis_item_id": i.basis_item_id,
"created_at": i.created_at,
"status": i.variant_status,
}
for i in items
]
}
@router.post(
"/review/{item_id}",
summary="Approve or reject AI generated question",
description="Update the variant_status of an AI generated question.",
)
async def admin_review_ai_question(
item_id: int,
status: str, # "active", "rejected"
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> dict:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
result = await db.execute(select(Item).where(Item.id == item_id))
item = result.scalar_one_or_none()
if not item:
raise HTTPException(status_code=404, detail="Item not found")
if website_id is not None and item.website_id != website_id:
raise HTTPException(status_code=403, detail="Not authorized for this website")
if status not in ["active", "rejected"]:
raise HTTPException(status_code=400, detail="Status must be active or rejected")
item.variant_status = status
await db.commit()
return {"success": True, "item_id": item_id, "status": status}

60
backend/app/routers/auth.py Normal file
View File

@@ -0,0 +1,60 @@
"""
Authentication endpoints.
"""
from typing import Any, Dict
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel
from app.core.auth import issue_access_token
from app.core.config import get_settings
router = APIRouter(prefix="/auth", tags=["auth"])
settings = get_settings()
class LoginRequest(BaseModel):
username: str
password: str
@router.post(
"/admin-login",
summary="Admin Login",
description="Login for standalone app administration.",
)
async def admin_login(request: LoginRequest) -> Dict[str, Any]:
"""Authenticate an app admin and issue a JWT token."""
if not settings.ENABLE_ADMIN:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin functionality is disabled.",
)
if not settings.ADMIN_USERNAME or not settings.ADMIN_PASSWORD:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Admin credentials not configured.",
)
if (
request.username != settings.ADMIN_USERNAME
or request.password != settings.ADMIN_PASSWORD
):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid credentials",
)
token = issue_access_token(
website_id=None,
role="system_admin",
expires_in_seconds=86400 * 7, # 7 days
)
return {
"access_token": token,
"token_type": "bearer",
"role": "system_admin",
}

11
app/routers/import_export.py → backend/app/routers/import_export.py
View File

@@ -292,12 +292,6 @@ async def export_questions(
"""
Export questions to Excel file.
Creates Excel file with standardized format:
- Row 2: KUNCI (answer key)
- Row 4: TK (p-values)
- Row 5: BOBOT (weights)
- Rows 6+: Question data
Args:
tryout_id: Tryout identifier
website_id: Website ID from header
@@ -394,6 +388,11 @@ async def import_tryout_json(
db: AsyncSession = Depends(get_db),
) -> dict:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
if website_id is None:
x_website_id = request.headers.get("x-website-id")
if not x_website_id or not x_website_id.isdigit():
raise HTTPException(status_code=400, detail="X-Website-ID header is required for system_admin")
website_id = int(x_website_id)
await enforce_rate_limit(
request,
scope="import.tryout_json",

0
app/routers/normalization.py → backend/app/routers/normalization.py
View File

0
app/routers/reports.py → backend/app/routers/reports.py
View File

81
app/routers/sessions.py → backend/app/routers/sessions.py
View File

@@ -7,7 +7,7 @@ Endpoints:
- POST /session: Create new session
"""
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
@@ -25,6 +25,7 @@ from app.models.item import Item
from app.models.session import Session
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.user import User
from app.models.user_answer import UserAnswer
from app.schemas.session import (
SessionCompleteRequest,
@@ -83,14 +84,15 @@ async def complete_session(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get session with tryout relationship
result = await db.execute(
session_query = (
select(Session)
.options(selectinload(Session.tryout))
.where(
Session.session_id == session_id,
Session.website_id == website_id,
)
.where(Session.session_id == session_id)
)
if website_id is not None:
session_query = session_query.where(Session.website_id == website_id)
result = await db.execute(session_query)
session = result.scalar_one_or_none()
if session is None:
@@ -110,18 +112,25 @@ async def complete_session(
detail="Session does not belong to this authenticated user",
)
effective_website_id = session.website_id
# Get tryout configuration
tryout = session.tryout
# Get all items for this tryout to calculate bobot
items_result = await db.execute(
select(Item).where(
Item.website_id == website_id,
Item.website_id == effective_website_id,
Item.tryout_id == session.tryout_id,
)
)
items = {item.id: item for item in items_result.scalars().all()}
existing_answers_full_result = await db.execute(
select(UserAnswer).where(UserAnswer.session_id == session.session_id)
)
existing_answer_records = list(existing_answers_full_result.scalars().all())
# Process each answer
submitted_item_ids = [answer.item_id for answer in request.user_answers]
if len(submitted_item_ids) != len(set(submitted_item_ids)):
@@ -130,10 +139,7 @@ async def complete_session(
detail="Duplicate item answers are not allowed in a session completion",
)
existing_answers_result = await db.execute(
select(UserAnswer.item_id).where(UserAnswer.session_id == session.session_id)
)
existing_answered_item_ids = {row[0] for row in existing_answers_result.all()}
existing_answered_item_ids = {answer.item_id for answer in existing_answer_records}
duplicate_existing_ids = sorted(set(submitted_item_ids) & existing_answered_item_ids)
if duplicate_existing_ids:
raise HTTPException(
@@ -148,7 +154,15 @@ async def complete_session(
total_bobot_earned = 0.0
user_answer_records = []
for answer_input in request.user_answers:
if request.user_answers:
answers_to_score = request.user_answers
else:
answers_to_score = []
user_answer_records = existing_answer_records
total_benar = sum(1 for answer in existing_answer_records if answer.is_correct)
total_bobot_earned = sum(answer.bobot_earned or 0.0 for answer in existing_answer_records)
for answer_input in answers_to_score:
item = items.get(answer_input.item_id)
if item is None:
@@ -172,7 +186,7 @@ async def complete_session(
user_answer = UserAnswer(
session_id=session.session_id,
wp_user_id=session.wp_user_id,
website_id=website_id,
website_id=effective_website_id,
tryout_id=session.tryout_id,
item_id=item.id,
response=answer_input.response.upper(),
@@ -187,7 +201,7 @@ async def complete_session(
# Calculate total_bobot_max for NM calculation
try:
total_bobot_max = await get_total_bobot_max(
db, website_id, session.tryout_id, level="sedang"
db, effective_website_id, session.tryout_id, level="sedang"
)
except ValueError:
# Fallback: calculate from items we have
@@ -209,7 +223,7 @@ async def complete_session(
# Get current stats for dynamic normalization
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.website_id == effective_website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
@@ -226,7 +240,7 @@ async def complete_session(
# Hybrid: use dynamic if enough data, otherwise static
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.website_id == effective_website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
@@ -253,7 +267,7 @@ async def complete_session(
session.sb_used = sb
# Update tryout stats incrementally
await update_tryout_stats(db, website_id, session.tryout_id, nm)
await update_tryout_stats(db, effective_website_id, session.tryout_id, nm)
# Commit all changes
try:
@@ -276,6 +290,7 @@ async def complete_session(
tryout_id=session.tryout_id,
start_time=session.start_time,
end_time=session.end_time,
expires_at=session.expires_at,
is_completed=session.is_completed,
scoring_mode_used=session.scoring_mode_used,
total_benar=session.total_benar,
@@ -325,12 +340,11 @@ async def get_session(
"""
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
result = await db.execute(
select(Session).where(
Session.session_id == session_id,
Session.website_id == website_id,
)
)
session_query = select(Session).where(Session.session_id == session_id)
if website_id is not None:
session_query = session_query.where(Session.website_id == website_id)
result = await db.execute(session_query)
session = result.scalar_one_or_none()
if session is None:
@@ -375,6 +389,7 @@ async def create_session(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
ensure_website_scope_matches(website_id, request.website_id)
effective_website_id = website_id if website_id is not None else request.website_id
if auth.role == "student" and request.wp_user_id != auth.wp_user_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
@@ -384,7 +399,7 @@ async def create_session(
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.website_id == effective_website_id,
Tryout.tryout_id == request.tryout_id,
)
)
@@ -393,7 +408,7 @@ async def create_session(
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {request.tryout_id} not found for website {website_id}",
detail=f"Tryout {request.tryout_id} not found for website {effective_website_id}",
)
# Check if session already exists
@@ -408,14 +423,26 @@ async def create_session(
detail=f"Session {request.session_id} already exists",
)
user_result = await db.execute(
select(User).where(
User.wp_user_id == request.wp_user_id,
User.website_id == effective_website_id,
)
)
if user_result.scalar_one_or_none() is None:
db.add(User(wp_user_id=request.wp_user_id, website_id=effective_website_id))
started_at = datetime.now(timezone.utc)
# Create new session
session = Session(
session_id=request.session_id,
wp_user_id=request.wp_user_id,
website_id=website_id,
website_id=effective_website_id,
tryout_id=request.tryout_id,
scoring_mode_used=request.scoring_mode,
start_time=datetime.now(timezone.utc),
start_time=started_at,
expires_at=started_at + timedelta(hours=2),
is_completed=False,
total_benar=0,
total_bobot_earned=0.0,

190
app/routers/tryouts.py → backend/app/routers/tryouts.py
View File

@@ -19,11 +19,13 @@ from app.core.auth import AuthContext, get_auth_context, require_website_auth
from app.models.item import Item
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.tryout_snapshot_question import TryoutSnapshotQuestion
from app.schemas.tryout import (
NormalizationUpdateRequest,
NormalizationUpdateResponse,
TryoutConfigBrief,
TryoutConfigResponse,
TryoutConfigUpdateRequest,
TryoutStatsResponse,
)
@@ -53,14 +55,15 @@ async def get_tryout_config(
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get tryout with stats
result = await db.execute(
query = (
select(Tryout)
.options(selectinload(Tryout.stats))
.where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
.where(Tryout.tryout_id == tryout_id)
)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
@@ -104,6 +107,73 @@ async def get_tryout_config(
)
@router.put(
"/{tryout_id}/config",
response_model=TryoutConfigResponse,
summary="Update tryout configuration",
description="Update editable tryout configuration fields.",
)
async def update_tryout_config(
tryout_id: str,
request: TryoutConfigUpdateRequest,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
) -> TryoutConfigResponse:
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
query = select(Tryout).options(selectinload(Tryout.stats)).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
update_data = request.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(tryout, field, value)
await db.commit()
await db.refresh(tryout)
current_stats = None
if tryout.stats:
current_stats = TryoutStatsResponse(
participant_count=tryout.stats.participant_count,
rataan=tryout.stats.rataan,
sb=tryout.stats.sb,
min_nm=tryout.stats.min_nm,
max_nm=tryout.stats.max_nm,
last_calculated=tryout.stats.last_calculated,
)
return TryoutConfigResponse(
id=tryout.id,
website_id=tryout.website_id,
tryout_id=tryout.tryout_id,
name=tryout.name,
description=tryout.description,
scoring_mode=tryout.scoring_mode,
selection_mode=tryout.selection_mode,
normalization_mode=tryout.normalization_mode,
min_sample_for_dynamic=tryout.min_sample_for_dynamic,
static_rataan=tryout.static_rataan,
static_sb=tryout.static_sb,
ai_generation_enabled=tryout.ai_generation_enabled,
hybrid_transition_slot=tryout.hybrid_transition_slot,
min_calibration_sample=tryout.min_calibration_sample,
theta_estimation_method=tryout.theta_estimation_method,
fallback_to_ctt_on_error=tryout.fallback_to_ctt_on_error,
current_stats=current_stats,
created_at=tryout.created_at,
updated_at=tryout.updated_at,
)
@router.put(
"/{tryout_id}/normalization",
response_model=NormalizationUpdateResponse,
@@ -134,12 +204,11 @@ async def update_normalization(
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryout = result.scalar_one_or_none()
if tryout is None:
@@ -160,12 +229,11 @@ async def update_normalization(
tryout.static_sb = request.static_sb
# Get current stats for participant count
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats_query = select(TryoutStats).where(TryoutStats.tryout_id == tryout_id)
if website_id is not None:
stats_query = stats_query.where(TryoutStats.website_id == website_id)
stats_result = await db.execute(stats_query)
stats = stats_result.scalar_one_or_none()
current_participant_count = stats.participant_count if stats else 0
@@ -204,22 +272,42 @@ async def list_tryouts(
"""
website_id = require_website_auth(auth, allowed_roles={"student", "admin", "system_admin"})
# Get tryouts with stats
result = await db.execute(
select(Tryout)
.options(selectinload(Tryout.stats))
.where(Tryout.website_id == website_id)
)
# Get tryouts with stats and items
query = select(Tryout).options(selectinload(Tryout.stats), selectinload(Tryout.items))
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
result = await db.execute(query)
tryouts = result.scalars().all()
# Get snapshot counts for tryouts to show accurate item_count for JSON imports
snapshot_counts = {}
if tryouts:
tryout_ids = [t.tryout_id for t in tryouts]
count_query = (
select(TryoutSnapshotQuestion.source_tryout_id, func.count(TryoutSnapshotQuestion.id))
.where(TryoutSnapshotQuestion.source_tryout_id.in_(tryout_ids))
)
if website_id is not None:
count_query = count_query.where(TryoutSnapshotQuestion.website_id == website_id)
count_query = count_query.group_by(TryoutSnapshotQuestion.source_tryout_id)
count_result = await db.execute(count_query)
snapshot_counts = dict(count_result.all())
return [
TryoutConfigBrief(
website_id=t.website_id,
tryout_id=t.tryout_id,
name=t.name,
scoring_mode=t.scoring_mode,
selection_mode=t.selection_mode,
normalization_mode=t.normalization_mode,
participant_count=t.stats.participant_count if t.stats else 0,
rataan=t.stats.rataan if t.stats else None,
sb=t.stats.sb if t.stats else None,
item_count=len(t.items) or snapshot_counts.get(t.tryout_id, 0),
calibrated_item_count=sum(1 for i in t.items if i.calibrated),
)
for t in tryouts
]
@@ -254,12 +342,11 @@ async def get_calibration_status(
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -269,16 +356,16 @@ async def get_calibration_status(
)
# Get calibration statistics
stats_result = await db.execute(
select(
stats_query = select(
func.count().label("total_items"),
func.sum(cast(Item.calibrated, Integer)).label("calibrated_items"),
func.avg(Item.calibration_sample_size).label("avg_sample_size"),
).where(
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
).where(Item.tryout_id == tryout_id)
if website_id is not None:
stats_query = stats_query.where(Item.website_id == website_id)
stats_result = await db.execute(stats_query)
stats = stats_result.first()
total_items = stats.total_items or 0
@@ -331,12 +418,11 @@ async def trigger_calibration(
)
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -395,12 +481,11 @@ async def trigger_item_calibration(
from app.services.irt_calibration import calibrate_item, CALIBRATION_SAMPLE_THRESHOLD
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
query = select(Tryout).where(Tryout.tryout_id == tryout_id)
if website_id is not None:
query = query.where(Tryout.website_id == website_id)
tryout_result = await db.execute(query)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
@@ -410,13 +495,14 @@ async def trigger_item_calibration(
)
# Verify item belongs to this tryout
item_result = await db.execute(
select(Item).where(
item_query = select(Item).where(
Item.id == item_id,
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
if website_id is not None:
item_query = item_query.where(Item.website_id == website_id)
item_result = await db.execute(item_query)
item = item_result.scalar_one_or_none()
if item is None:

84
backend/app/routers/websites.py Normal file
View File

@@ -0,0 +1,84 @@
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.future import select
from pydantic import BaseModel
from app.database import get_db
from app.models import Website
from app.core.auth import AuthContext, get_auth_context, require_website_auth
router = APIRouter(tags=["websites"])
class WebsiteBase(BaseModel):
name: str
domain: str
class WebsiteResponse(WebsiteBase):
id: int
class Config:
from_attributes = True
@router.get("/websites", response_model=List[WebsiteResponse])
async def get_websites(
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
result = await db.execute(select(Website).order_by(Website.id.asc()))
websites = result.scalars().all()
# Map old columns (site_name, site_url) to new response format
return [
WebsiteResponse(
id=w.id,
name=w.site_name,
domain=w.site_url
) for w in websites
]
@router.post("/websites", response_model=WebsiteResponse)
async def create_website(
payload: WebsiteBase,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = Website(site_name=payload.name, site_url=payload.domain)
db.add(website)
await db.commit()
await db.refresh(website)
return WebsiteResponse(id=website.id, name=website.site_name, domain=website.site_url)
@router.put("/websites/{website_id}", response_model=WebsiteResponse)
async def update_website(
website_id: int,
payload: WebsiteBase,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = await db.get(Website, website_id)
if not website:
raise HTTPException(status_code=404, detail="Website not found")
website.site_name = payload.name
website.site_url = payload.domain
await db.commit()
await db.refresh(website)
return WebsiteResponse(id=website.id, name=website.site_name, domain=website.site_url)
@router.delete("/websites/{website_id}")
async def delete_website(
website_id: int,
db: AsyncSession = Depends(get_db),
auth: AuthContext = Depends(get_auth_context),
):
require_website_auth(auth, allowed_roles={"admin", "system_admin"})
website = await db.get(Website, website_id)
if not website:
raise HTTPException(status_code=404, detail="Website not found")
await db.delete(website)
await db.commit()
return {"status": "success", "message": "Website deleted"}

0
app/routers/wordpress.py → backend/app/routers/wordpress.py
View File

0
app/schemas/__init__.py → backend/app/schemas/__init__.py
View File

180
backend/app/schemas/ai.py Normal file
View File

@@ -0,0 +1,180 @@
"""
Pydantic schemas for AI generation endpoints.
Request/response models for admin AI generation playground.
"""
from typing import Dict, Literal, Optional
from pydantic import BaseModel, Field, field_validator
OPTION_LABELS = tuple("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
class AIGeneratePreviewRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated question"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
class AIModelPricing(BaseModel):
prompt: Optional[float] = Field(
default=None, description="Input token price in USD per token"
)
completion: Optional[float] = Field(
default=None, description="Output token price in USD per token"
)
prompt_per_million: Optional[float] = Field(
default=None, description="Input token price in USD per 1M tokens"
)
completion_per_million: Optional[float] = Field(
default=None, description="Output token price in USD per 1M tokens"
)
currency: str = "USD"
source: str = "openrouter"
class AIUsageInfo(BaseModel):
prompt_tokens: Optional[int] = None
completion_tokens: Optional[int] = None
total_tokens: Optional[int] = None
cost_usd: Optional[float] = None
class AIGeneratePreviewResponse(BaseModel):
success: bool = Field(..., description="Whether generation was successful")
stem: Optional[str] = None
options: Optional[Dict[str, str]] = None
correct: Optional[str] = None
explanation: Optional[str] = None
ai_model: Optional[str] = None
basis_item_id: Optional[int] = None
target_level: Optional[str] = None
usage: Optional[AIUsageInfo] = None
error: Optional[str] = None
cached: bool = False
class AISaveRequest(BaseModel):
stem: str = Field(..., description="Question stem")
options: Dict[str, str] = Field(
..., description="Answer options. Labels must match the basis item exactly."
)
correct: str = Field(..., description="Correct answer option label")
explanation: Optional[str] = None
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
basis_item_id: int = Field(..., description="Basis item ID")
slot: int = Field(..., description="Question slot position")
level: Literal["mudah", "sedang", "sulit"] = Field(
..., description="Difficulty level"
)
variant_status: Literal["active", "draft"] = Field(
default="active",
description="Lifecycle status for the saved variant. Workspace approvals save active variants.",
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model used for generation",
)
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
label = v.upper()
if label not in OPTION_LABELS:
raise ValueError("Correct answer must be an option label A-Z")
return label
@field_validator("options")
@classmethod
def validate_options(cls, v: Dict[str, str]) -> Dict[str, str]:
normalized = {
str(key).strip().upper(): str(value).strip()
for key, value in v.items()
if str(key).strip() and str(value).strip()
}
if len(normalized) < 2:
raise ValueError("Options must contain at least two non-empty choices")
invalid_keys = sorted(set(normalized) - set(OPTION_LABELS))
if invalid_keys:
raise ValueError(f"Options contain invalid labels: {', '.join(invalid_keys)}")
return normalized
class AISaveResponse(BaseModel):
success: bool = Field(..., description="Whether save was successful")
item_id: Optional[int] = None
run_id: Optional[int] = None
error: Optional[str] = None
class AIGenerateBatchRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated questions"
)
ai_model: str = Field(
default="qwen/qwen2.5-32b-instruct",
description="AI model to use for generation",
)
count: int = Field(default=3, ge=1, le=10, description="Number of variants to generate")
operator_notes: Optional[str] = None
class AIBatchGeneratedItem(BaseModel):
item_id: int
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
level: str
variant_status: str
usage: Optional[AIUsageInfo] = None
class AIGenerateBatchResponse(BaseModel):
success: bool
run_id: Optional[int] = None
item_ids: list[int] = Field(default_factory=list)
items: list[AIBatchGeneratedItem] = Field(default_factory=list)
generated_count: int = 0
usage: Optional[AIUsageInfo] = None
error: Optional[str] = None
class AIStatsResponse(BaseModel):
total_ai_items: int = Field(..., description="Total AI-generated items")
items_by_model: Dict[str, int] = Field(
default_factory=dict, description="Items count by AI model"
)
cache_hit_rate: float = Field(
default=0.0, description="Cache hit rate (0.0 to 1.0)"
)
total_cache_hits: int = Field(default=0, description="Total cache hits")
total_requests: int = Field(default=0, description="Total generation requests")
class GeneratedQuestion(BaseModel):
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
usage: Optional[AIUsageInfo] = None
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
label = v.upper()
if label not in OPTION_LABELS:
raise ValueError("Correct answer must be an option label A-Z")
return label

0
app/schemas/report.py → backend/app/schemas/report.py
View File

2
app/schemas/session.py → backend/app/schemas/session.py
View File

@@ -52,6 +52,7 @@ class SessionCompleteResponse(BaseModel):
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
expires_at: Optional[datetime] = None
is_completed: bool
scoring_mode_used: str
@@ -99,6 +100,7 @@ class SessionResponse(BaseModel):
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
expires_at: Optional[datetime] = None
is_completed: bool
scoring_mode_used: str

23
app/schemas/tryout.py → backend/app/schemas/tryout.py
View File

@@ -64,16 +64,39 @@ class TryoutStatsResponse(BaseModel):
class TryoutConfigBrief(BaseModel):
"""Brief tryout config for list responses."""
website_id: int
tryout_id: str
name: str
scoring_mode: str
selection_mode: str
normalization_mode: str
participant_count: Optional[int] = None
rataan: Optional[float] = None
sb: Optional[float] = None
item_count: int = 0
calibrated_item_count: int = 0
model_config = {"from_attributes": True}
class TryoutConfigUpdateRequest(BaseModel):
"""Request schema for updating editable tryout configuration."""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
scoring_mode: Optional[Literal["ctt", "irt", "hybrid"]] = None
selection_mode: Optional[Literal["fixed", "adaptive", "hybrid"]] = None
normalization_mode: Optional[Literal["static", "dynamic", "hybrid"]] = None
min_sample_for_dynamic: Optional[int] = Field(None, ge=1)
static_rataan: Optional[float] = Field(None, ge=0)
static_sb: Optional[float] = Field(None, gt=0)
ai_generation_enabled: Optional[bool] = None
hybrid_transition_slot: Optional[int] = Field(None, ge=1)
min_calibration_sample: Optional[int] = Field(None, ge=1)
theta_estimation_method: Optional[Literal["mle", "map", "eap"]] = None
fallback_to_ctt_on_error: Optional[bool] = None
class NormalizationUpdateRequest(BaseModel):
"""Request schema for updating normalization settings."""

0
app/schemas/wordpress.py → backend/app/schemas/wordpress.py
View File

0
app/services/__init__.py → backend/app/services/__init__.py
View File

251
app/services/ai_generation.py → backend/app/services/ai_generation.py
View File

@@ -9,6 +9,8 @@ import json
import logging
import re
import ast
import time
from dataclasses import dataclass
from typing import Any, Dict, Literal, Optional, Union
import httpx
@@ -20,13 +22,14 @@ from app.models.item import Item
from app.models.ai_generation_run import AIGenerationRun
from app.models.tryout import Tryout
from app.models.user_answer import UserAnswer
from app.schemas.ai import GeneratedQuestion
from app.schemas.ai import AIModelPricing, AIUsageInfo, GeneratedQuestion
logger = logging.getLogger(__name__)
settings = get_settings()
# OpenRouter API configuration
OPENROUTER_API_URL = "https://openrouter.ai/api/v1/chat/completions"
OPENROUTER_MODELS_URL = "https://openrouter.ai/api/v1/models"
# Supported AI models
SUPPORTED_MODELS = {
@@ -42,6 +45,159 @@ LEVEL_DESCRIPTIONS = {
"sulit": "harder (more complex concepts, multi-step reasoning)",
}
OPTION_LABELS = tuple("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
MODEL_PRICING_CACHE_TTL_SECONDS = 60 * 30
_model_pricing_cache: dict[str, tuple[float, AIModelPricing | None]] = {}
@dataclass
class OpenRouterCallResult:
content: str
usage: AIUsageInfo | None = None
def get_option_labels(options: Dict[str, str] | None) -> list[str]:
labels = {
str(key).strip().upper()
for key, value in (options or {}).items()
if str(key).strip() and str(value).strip()
}
return [label for label in OPTION_LABELS if label in labels]
def _parse_openrouter_price(value: Any) -> float | None:
if value is None:
return None
try:
price = float(value)
except (TypeError, ValueError):
return None
return price if price >= 0 else None
def _build_pricing(raw_pricing: dict[str, Any] | None) -> AIModelPricing | None:
if not raw_pricing:
return None
prompt = _parse_openrouter_price(raw_pricing.get("prompt"))
completion = _parse_openrouter_price(raw_pricing.get("completion"))
if prompt is None and completion is None:
return None
return AIModelPricing(
prompt=prompt,
completion=completion,
prompt_per_million=prompt * 1_000_000 if prompt is not None else None,
completion_per_million=completion * 1_000_000 if completion is not None else None,
)
async def get_model_pricing(model_id: str) -> AIModelPricing | None:
cached = _model_pricing_cache.get(model_id)
now = time.monotonic()
if cached and now - cached[0] < MODEL_PRICING_CACHE_TTL_SECONDS:
return cached[1]
headers = {"Content-Type": "application/json"}
if settings.OPENROUTER_API_KEY:
headers["Authorization"] = f"Bearer {settings.OPENROUTER_API_KEY}"
try:
timeout = httpx.Timeout(min(settings.OPENROUTER_TIMEOUT, 5))
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.get(OPENROUTER_MODELS_URL, headers=headers)
if response.status_code != 200:
logger.warning(
"OpenRouter models pricing request failed: %s - %s",
response.status_code,
response.text[:240],
)
_model_pricing_cache[model_id] = (now, None)
return None
for model in response.json().get("data", []):
if model.get("id") == model_id:
pricing = _build_pricing(model.get("pricing"))
_model_pricing_cache[model_id] = (now, pricing)
return pricing
except Exception as exc:
logger.warning("OpenRouter model pricing lookup failed for %s: %s", model_id, exc)
_model_pricing_cache[model_id] = (now, None)
return None
def _calculate_usage_cost(
prompt_tokens: int | None,
completion_tokens: int | None,
pricing: AIModelPricing | None,
provider_cost: Any = None,
) -> float | None:
provider_cost_value = _parse_openrouter_price(provider_cost)
if provider_cost_value is not None:
return provider_cost_value
if pricing is None:
return None
cost = 0.0
has_cost_component = False
if prompt_tokens is not None and pricing.prompt is not None:
cost += prompt_tokens * pricing.prompt
has_cost_component = True
if completion_tokens is not None and pricing.completion is not None:
cost += completion_tokens * pricing.completion
has_cost_component = True
return cost if has_cost_component else None
async def build_usage_info(raw_usage: dict[str, Any] | None, model_id: str) -> AIUsageInfo | None:
if not raw_usage:
return None
def token_count(key: str) -> int | None:
value = raw_usage.get(key)
if value is None:
return None
try:
return int(value)
except (TypeError, ValueError):
return None
prompt_tokens = token_count("prompt_tokens")
completion_tokens = token_count("completion_tokens")
total_tokens = token_count("total_tokens")
if total_tokens is None and (prompt_tokens is not None or completion_tokens is not None):
total_tokens = (prompt_tokens or 0) + (completion_tokens or 0)
pricing = await get_model_pricing(model_id)
cost_usd = _calculate_usage_cost(
prompt_tokens,
completion_tokens,
pricing,
provider_cost=raw_usage.get("cost"),
)
return AIUsageInfo(
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
total_tokens=total_tokens,
cost_usd=cost_usd,
)
def combine_usage(usages: list[AIUsageInfo | None]) -> AIUsageInfo | None:
filtered = [usage for usage in usages if usage is not None]
if not filtered:
return None
def summed(field: str) -> int | float | None:
values = [getattr(usage, field) for usage in filtered]
present = [value for value in values if value is not None]
return sum(present) if present else None
return AIUsageInfo(
prompt_tokens=summed("prompt_tokens"),
completion_tokens=summed("completion_tokens"),
total_tokens=summed("total_tokens"),
cost_usd=summed("cost_usd"),
)
def get_prompt_template(
basis_stem: str,
@@ -65,6 +221,10 @@ def get_prompt_template(
Formatted prompt string
"""
level_desc = LEVEL_DESCRIPTIONS.get(target_level, target_level)
option_labels = get_option_labels(basis_options) or ["A", "B", "C", "D"]
option_count = len(option_labels)
option_label_text = ", ".join(option_labels)
example_options = {label: f"Option {label} text" for label in option_labels}
options_text = "\n".join(
[f" {key}: {value}" for key, value in basis_options.items()]
@@ -103,17 +263,19 @@ Generate 1 new question that is {level_desc} than the basis question above.
REQUIREMENTS:
1. Keep the SAME topic/subject matter as the basis question
2. Use similar context and terminology
3. Create exactly 4 answer options (A, B, C, D)
4. Only ONE correct answer
5. Include a clear explanation of why the correct answer is correct
6. Make the question noticeably {level_desc} - not just a minor variation
7. Follow and preserve any HTML formatting (e.g., <p>, <br>, <b>) present in the basis question
3. Create exactly {option_count} answer options with labels exactly: {option_label_text}
4. Preserve the basis option count and option labels. Do not omit, add, rename, or merge answer options.
5. Only ONE correct answer, and it must be one of: {option_label_text}
6. Include a clear explanation of why the correct answer is correct
7. Make the question noticeably {level_desc} - not just a minor variation
8. Follow and preserve the basis question's inline HTML style. Keep structural and inline tags such as <p>, <br>, <strong>, <b>, <em>, <i>, <u>, <sub>, <sup>, and simple inline attributes such as text alignment when the basis uses them.
9. Do not escape HTML tags as text. Return HTML markup in the JSON string values exactly as markup.
OUTPUT FORMAT:
Return ONLY a valid JSON object with this exact structure (no markdown, no code blocks):
{{"stem": "Your question text here", "options": {{"A": "Option A text", "B": "Option B text", "C": "Option C text", "D": "Option D text"}}, "correct": "A", "explanation": "Explanation text here"}}
{{"stem": "Your question text here", "options": {json.dumps(example_options, ensure_ascii=False)}, "correct": "{option_labels[0]}", "explanation": "Explanation text here"}}
Remember: The correct field must be exactly "A", "B", "C", or "D"."""
Remember: The correct field must be exactly one of: {option_label_text}."""
return prompt
@@ -164,18 +326,13 @@ def validate_and_create_question(data: Dict[str, Any]) -> Optional[GeneratedQues
options = _normalize_options(data.get("options"))
if not options:
logger.warning("Options cannot be normalized to A/B/C/D map")
return None
required_options = {"A", "B", "C", "D"}
if not required_options.issubset(set(options.keys())):
logger.warning(f"Missing required options: {required_options - set(options.keys())}")
logger.warning("Options cannot be normalized to a labeled option map")
return None
correct = _normalize_correct_answer(
data.get("correct") or data.get("correct_answer") or data.get("answer")
)
if correct not in required_options:
if correct not in set(options.keys()):
logger.warning(f"Invalid correct answer: {correct}")
return None
@@ -258,7 +415,7 @@ def _try_parse_json_like(candidate: str) -> Any:
def _normalize_options(raw_options: Any) -> dict[str, str]:
if isinstance(raw_options, dict):
normalized = {str(k).strip().upper(): str(v).strip() for k, v in raw_options.items()}
return {k: normalized.get(k, "") for k in ["A", "B", "C", "D"] if normalized.get(k, "")}
return {k: normalized[k] for k in OPTION_LABELS if normalized.get(k, "")}
if isinstance(raw_options, list):
mapped: dict[str, str] = {}
@@ -269,9 +426,9 @@ def _normalize_options(raw_options: Any) -> dict[str, str]:
else:
key = ""
text = str(opt).strip()
if not key and idx < 4:
key = ["A", "B", "C", "D"][idx]
if key in {"A", "B", "C", "D"} and text:
if not key and idx < len(OPTION_LABELS):
key = OPTION_LABELS[idx]
if key in OPTION_LABELS and text:
mapped[key] = text
return mapped
@@ -282,24 +439,44 @@ def _normalize_correct_answer(raw_correct: Any) -> str:
if raw_correct is None:
return ""
raw_text = str(raw_correct).strip().upper()
if raw_text in {"A", "B", "C", "D"}:
if raw_text in OPTION_LABELS:
return raw_text
if raw_text.isdigit():
idx = int(raw_text)
if 1 <= idx <= 4:
return ["A", "B", "C", "D"][idx - 1]
if 0 <= idx <= 3:
return ["A", "B", "C", "D"][idx]
if raw_text in {"OPTION A", "OPTION B", "OPTION C", "OPTION D"}:
if 1 <= idx <= len(OPTION_LABELS):
return OPTION_LABELS[idx - 1]
if 0 <= idx < len(OPTION_LABELS):
return OPTION_LABELS[idx]
if raw_text.startswith("OPTION ") and raw_text[-1:] in OPTION_LABELS:
return raw_text[-1]
return raw_text[:1]
def generated_matches_basis_options(generated: GeneratedQuestion, basis_item: Item) -> bool:
basis_labels = get_option_labels(basis_item.options)
generated_labels = get_option_labels(generated.options)
if basis_labels != generated_labels:
logger.warning(
"Generated option labels do not match basis: basis=%s generated=%s",
basis_labels,
generated_labels,
)
return False
if generated.correct not in set(basis_labels):
logger.warning(
"Generated correct answer %s is outside basis labels %s",
generated.correct,
basis_labels,
)
return False
return True
async def call_openrouter_api(
prompt: str,
model: str,
max_retries: int = 3,
) -> Optional[str]:
) -> Optional[OpenRouterCallResult]:
"""
Call OpenRouter API to generate question.
@@ -309,7 +486,7 @@ async def call_openrouter_api(
max_retries: Maximum retry attempts
Returns:
API response text or None if failed
OpenRouterCallResult with response text and usage, or None if failed
"""
if not settings.OPENROUTER_API_KEY:
logger.error("OPENROUTER_API_KEY not configured")
@@ -362,7 +539,12 @@ async def call_openrouter_api(
choices = data.get("choices", [])
if choices:
message = choices[0].get("message", {})
return message.get("content")
content = message.get("content")
if not content:
logger.warning("OpenRouter response had no message content")
return None
usage = await build_usage_info(data.get("usage"), model)
return OpenRouterCallResult(content=content, usage=usage)
logger.warning("No choices in OpenRouter response")
return None
@@ -423,19 +605,20 @@ async def generate_question(
operator_notes=operator_notes,
)
max_generation_attempts = 2
max_generation_attempts = 3
for attempt in range(1, max_generation_attempts + 1):
response_text = await call_openrouter_api(prompt, ai_model)
if not response_text:
api_result = await call_openrouter_api(prompt, ai_model)
if not api_result:
logger.error("No response from OpenRouter API")
continue
generated = parse_ai_response(response_text)
if generated:
generated = parse_ai_response(api_result.content)
if generated and generated_matches_basis_options(generated, basis_item):
generated = generated.model_copy(update={"usage": api_result.usage})
return generated
logger.warning(
"Failed to parse AI response (attempt %s/%s), retrying",
"Failed to parse or validate AI response (attempt %s/%s), retrying",
attempt,
max_generation_attempts,
)

56
app/services/cat_selection.py → backend/app/services/cat_selection.py
View File

@@ -53,6 +53,30 @@ class TerminationCheck:
DEFAULT_SE_THRESHOLD = 0.5
# Default max items if not configured
DEFAULT_MAX_ITEMS = 50
SERVABLE_VARIANT_STATUSES = ("active", "approved")
def _servable_item_filter():
return Item.variant_status.in_(SERVABLE_VARIANT_STATUSES)
async def _get_user_answered_slot_levels(
db: AsyncSession,
wp_user_id: str,
website_id: int,
tryout_id: str,
) -> set[tuple[int, str]]:
"""Return slot/level pairs this user has already seen for this tryout."""
result = await db.execute(
select(Item.slot, Item.level)
.join(UserAnswer, UserAnswer.item_id == Item.id)
.where(
UserAnswer.wp_user_id == wp_user_id,
UserAnswer.website_id == website_id,
UserAnswer.tryout_id == tryout_id,
)
)
return {(int(slot), str(level)) for slot, level in result.all()}
async def get_next_item_fixed(
@@ -99,7 +123,8 @@ async def get_next_item_fixed(
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
Item.website_id == website_id,
_servable_item_filter(),
)
.order_by(Item.slot, Item.level)
)
@@ -113,7 +138,16 @@ async def get_next_item_fixed(
query = query.where(not_(Item.id.in_(answered_item_ids)))
result = await db.execute(query)
items = result.scalars().all()
items = list(result.scalars().all())
user_answered_slot_levels = await _get_user_answered_slot_levels(
db, session.wp_user_id, website_id, tryout_id
)
if user_answered_slot_levels:
items = [
item
for item in items
if (item.slot, item.level) not in user_answered_slot_levels
]
if not items:
return NextItemResult(
@@ -187,6 +221,7 @@ async def get_next_item_adaptive(
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
_servable_item_filter(),
Item.calibrated == True # Only calibrated items for IRT
)
)
@@ -204,7 +239,16 @@ async def get_next_item_adaptive(
query = query.where(Item.generated_by == 'manual')
result = await db.execute(query)
items = result.scalars().all()
items = list(result.scalars().all())
user_answered_slot_levels = await _get_user_answered_slot_levels(
db, session.wp_user_id, website_id, tryout_id
)
if user_answered_slot_levels:
items = [
item
for item in items
if (item.slot, item.level) not in user_answered_slot_levels
]
if not items:
return NextItemResult(
@@ -553,7 +597,8 @@ async def get_available_levels_for_slot(
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.slot == slot
Item.slot == slot,
_servable_item_filter(),
)
.distinct()
)
@@ -599,7 +644,8 @@ async def simulate_cat_selection(
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
Item.website_id == website_id,
_servable_item_filter(),
)
.order_by(Item.slot)
)

0
app/services/config_management.py → backend/app/services/config_management.py
View File

0
app/services/ctt_scoring.py → backend/app/services/ctt_scoring.py
View File

0
app/services/excel_import.py → backend/app/services/excel_import.py
View File

0
app/services/irt_calibration.py → backend/app/services/irt_calibration.py
View File

0
app/services/normalization.py → backend/app/services/normalization.py
View File

0
app/services/reporting.py → backend/app/services/reporting.py
View File

24
app/services/tryout_json_import.py → backend/app/services/tryout_json_import.py
View File

@@ -17,7 +17,7 @@ from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models import Item, TryoutImportSnapshot, TryoutSnapshotQuestion, Website
from app.models import Item, Tryout, TryoutImportSnapshot, TryoutSnapshotQuestion, Website
SOURCE_FORMAT = "sejoli_json"
DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
@@ -248,6 +248,28 @@ async def import_tryout_json_snapshot(payload: dict[str, Any], website_id: int,
db.add(snapshot)
await db.flush()
# Ensure operational tryout exists
result_tryout = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == source_tryout_id,
)
)
tryout = result_tryout.scalar_one_or_none()
if not tryout:
tryout = Tryout(
website_id=website_id,
tryout_id=source_tryout_id,
name=title,
description=f"Operational tryout basis created from imported snapshot #{snapshot.id}.",
scoring_mode="ctt",
selection_mode="fixed",
normalization_mode="static",
ai_generation_enabled=True,
)
db.add(tryout)
await db.flush()
existing_result = await db.execute(
select(TryoutSnapshotQuestion).where(
TryoutSnapshotQuestion.website_id == website_id,

0
app/services/wordpress_auth.py → backend/app/services/wordpress_auth.py
View File

0
docker-compose.dev.yml → backend/docker-compose.dev.yml
View File

0
docs/ALUR-APLIKASI-DAN-IRT.md → backend/docs/ALUR-APLIKASI-DAN-IRT.md
View File

0
error.html → backend/error.html
View File

0
irt_1pl_mle.py → backend/irt_1pl_mle.py
View File

0
patch_css.py → backend/patch_css.py
View File

0
patch_icons.py → backend/patch_icons.py
View File

0
requirements.txt → backend/requirements.txt
View File

0
run_local.sh → backend/run_local.sh
View File

275
backend/test_all_post_endpoints.py Normal file
View File

@@ -0,0 +1,275 @@
#!/usr/bin/env python3
"""
Comprehensive test of all form POST endpoints with proper authentication.
"""
import re
import sys
import httpx
BASE_URL = "http://localhost:8000"
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
response = client.get("/admin/login")
if response.status_code != 200:
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
return False
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
return response.status_code == 200 and "/admin/dashboard" in str(response.url)
def get_csrf_token(client: httpx.Client, page_url: str) -> str:
"""Extract CSRF token from a page."""
response = client.get(page_url)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1)
return ""
def test_endpoint(client: httpx.Client, name: str, url: str, data: dict) -> dict:
"""Test a single POST endpoint."""
csrf_token = get_csrf_token(client, url)
# Get the base URL (strip query params) for CSRF token extraction
base_url = url.split("?")[0] if "?" in url else url
# If we're on a different page, get CSRF token from there
if not csrf_token:
# Try to get CSRF from dashboard if it's a subpage
csrf_token = get_csrf_token(client, "/admin/dashboard")
if not csrf_token:
return {
"name": name,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Add CSRF token to data
test_data = data.copy()
test_data["csrf_token"] = csrf_token
response = client.post(
url,
data=test_data,
follow_redirects=True,
)
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
idx = response.text.find("Traceback")
traceback_text = response.text[idx : idx + 2000]
print(f"\n ⚠️ TRACEBACK on {name}:")
print(f" {traceback_text[:500]}...")
return {
"name": name,
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"error": None,
"response_preview": response.text[:500],
}
def main():
print("=" * 80)
print("Testing All Form POST Endpoints for Internal Server Errors")
print("=" * 80)
results = []
with httpx.Client(base_url=BASE_URL, timeout=60.0) as client:
print("\nStep 1: Logging in...")
if not login(client):
print("❌ Login failed")
return 1
print("✅ Login successful")
# Test 1: Variant approval (with item ID 4)
print("\nStep 2: Testing variant approval...")
result = test_endpoint(
client,
"Variant approval (/admin/questions/4/generate/review-bulk)",
"/admin/questions/4/generate?tab=review",
{"item_ids": "4", "action": "approved", "tab": "review"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 2: Basis item review
print("\nStep 3: Testing basis item review...")
result = test_endpoint(
client,
"Basis item review (/admin/basis-items/4/review-bulk)",
"/admin/basis-items/4",
{"item_ids": "4", "action": "approved"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 3: Generate variants for question
print("\nStep 4: Testing generate variants...")
result = test_endpoint(
client,
"Generate variants (/admin/questions/4/generate)",
"/admin/questions/4/generate?tab=generate",
{
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"include_note_for_admin": "on",
"include_note_in_prompt": "",
},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 5: Website creation
print("\nStep 5: Testing website creation...")
result = test_endpoint(
client,
"Website creation (/admin/websites)",
"/admin/websites",
{"site_name": "Test Site API", "site_url": "https://test-api.example.com"},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 6: Website deletion (with test ID)
print("\nStep 6: Testing website deletion...")
# First create a website
result_create = test_endpoint(
client,
"Create test website",
"/admin/websites",
{
"site_name": "Delete Test Site",
"site_url": "https://delete-test.example.com",
},
)
# Now delete it (using website ID 2 if exists)
result = test_endpoint(
client,
"Website deletion (/admin/websites/2/delete)",
"/admin/websites/2/delete",
{},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 7: Tryout import preview (without file - should get validation error not server error)
print("\nStep 7: Testing tryout import preview...")
result = test_endpoint(
client,
"Tryout import preview (/admin/tryout-import/preview)",
"/admin/tryout-import",
{"website_id": "1"},
)
results.append(result)
print(f" Status: {result['status_code']} (validation error expected: 422)")
# Test 8: Snapshot promote bulk
print("\nStep 8: Testing snapshot promote bulk...")
result = test_endpoint(
client,
"Snapshot promote (/admin/snapshot-questions/promote-bulk)",
"/admin/snapshot-questions",
{"snapshot_id": "1", "snapshot_question_ids": ""},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Test 9: AI generation basis item
print("\nStep 9: Testing AI generation for basis item...")
result = test_endpoint(
client,
"Basis item generate (/admin/basis-items/4/generate)",
"/admin/basis-items/4",
{
"target_level": "mudah",
"ai_model": "",
"generation_count": "1",
"operator_notes": "",
},
)
results.append(result)
print(
f" Status: {result['status_code']} {'✅' if result['status_code'] in [200, 303] else '❌'}"
)
# Summary
print("\n" + "=" * 80)
print("RESULTS SUMMARY")
print("=" * 80)
errors = []
for result in results:
if result.get("has_traceback"):
errors.append(f"❌ {result['name']}: TRACEBACK")
print(f"❌ {result['name']}: TRACEBACK")
elif result.get("has_ise"):
errors.append(f"❌ {result['name']}: INTERNAL SERVER ERROR")
print(f"❌ {result['name']}: INTERNAL SERVER ERROR")
elif result.get("error"):
print(f"⚠️ {result['name']}: {result['error']}")
elif result["status_code"] in [200, 303]:
print(f"✅ {result['name']}: OK ({result['status_code']})")
elif result["status_code"] == 422:
print(f"✅ {result['name']}: Validation Error (expected)")
else:
print(f"⚠️ {result['name']}: Status {result['status_code']}")
print()
if errors:
print("❌ Some endpoints have INTERNAL SERVER ERRORS:")
for error in errors:
print(f" {error}")
return 1
else:
print("✅ All form POST endpoints tested successfully!")
print(" No Internal Server Errors detected.")
return 0
if __name__ == "__main__":
sys.exit(main())

346
backend/test_all_routes.py Normal file
View File

@@ -0,0 +1,346 @@
#!/usr/bin/env python3
"""
Test all routes in the IRT Bank Soal application.
Tests each endpoint and checks for Internal Server Errors.
"""
import json
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from urllib.parse import urlparse
import httpx
BASE_URL = "http://localhost:8000"
# All routes from OpenAPI spec
API_ROUTES = [
# Root endpoints
("GET", "/"),
("GET", "/health"),
# Session endpoints
("POST", "/api/v1/session/"),
("GET", "/api/v1/session/{session_id}"),
("POST", "/api/v1/session/{session_id}/complete"),
("GET", "/api/v1/session/{session_id}/next_item"),
("POST", "/api/v1/session/{session_id}/submit_answer"),
# Tryout endpoints
("GET", "/api/v1/tryout/"),
("GET", "/api/v1/tryout/{tryout_id}/config"),
("PUT", "/api/v1/tryout/{tryout_id}/normalization"),
("GET", "/api/v1/tryout/{tryout_id}/calibration-status"),
("POST", "/api/v1/tryout/{tryout_id}/calibrate"),
("POST", "/api/v1/tryout/{tryout_id}/calibrate/{item_id}"),
# WordPress endpoints
("POST", "/api/v1/wordpress/sync_users"),
("POST", "/api/v1/wordpress/verify_session"),
("GET", "/api/v1/wordpress/website/{website_id}/users"),
("GET", "/api/v1/wordpress/website/{website_id}/user/{wp_user_id}"),
# Reports endpoints
("POST", "/api/v1/reports/schedule"),
("GET", "/api/v1/reports/schedule/{schedule_id}"),
("GET", "/api/v1/reports/schedules"),
("DELETE", "/api/v1/reports/schedule/{schedule_id}"),
("POST", "/api/v1/reports/schedule/{schedule_id}/export"),
("GET", "/api/v1/reports/student/performance"),
("GET", "/api/v1/reports/student/performance/export/{format}"),
("GET", "/api/v1/reports/items/analysis"),
("GET", "/api/v1/reports/items/analysis/export/{format}"),
("GET", "/api/v1/reports/calibration/status"),
("GET", "/api/v1/reports/calibration/status/export/{format}"),
("GET", "/api/v1/reports/tryout/comparison"),
("GET", "/api/v1/reports/tryout/comparison/export/{format}"),
("GET", "/api/v1/reports/export/{schedule_id}/{format}"),
# Import/Export endpoints
("POST", "/api/v1/import-export/preview"),
("POST", "/api/v1/import-export/questions"),
("GET", "/api/v1/import-export/export/questions"),
("POST", "/api/v1/import-export/tryout-json/preview"),
("POST", "/api/v1/import-export/tryout-json"),
# Admin AI endpoints
("POST", "/api/v1/admin/ai/generate-preview"),
("POST", "/api/v1/admin/ai/generate-save"),
("GET", "/api/v1/admin/ai/stats"),
("GET", "/api/v1/admin/ai/models"),
# Admin endpoints
("POST", "/api/v1/admin/{tryout_id}/calibrate"),
("POST", "/api/v1/admin/{tryout_id}/toggle-ai-generation"),
("POST", "/api/v1/admin/{tryout_id}/reset-normalization"),
# Admin CAT endpoints
("POST", "/api/v1/admin/cat/test"),
("GET", "/api/v1/admin/session/{session_id}/status"),
# Admin web routes (HTML pages)
("GET", "/admin"),
("GET", "/admin/login"),
("POST", "/admin/login"),
("POST", "/admin/logout"),
("GET", "/admin/password"),
("POST", "/admin/password"),
("GET", "/admin/dashboard"),
("GET", "/admin/questions"),
("GET", "/admin/questions/{item_id}"),
("GET", "/admin/questions/{item_id}/quality"),
("GET", "/admin/exams"),
("GET", "/admin/exams/{tryout_id}"),
("GET", "/admin/reports"),
("GET", "/admin/settings"),
("GET", "/admin/hierarchy"),
("GET", "/admin/websites"),
("POST", "/admin/websites"),
("GET", "/admin/websites/new"),
("GET", "/admin/websites/{website_id}"),
("POST", "/admin/websites/{website_id}"),
("POST", "/admin/websites/{website_id}/delete"),
("GET", "/admin/tryout-import"),
("GET", "/admin/tryout-import/preview"),
("POST", "/admin/tryout-import"),
("GET", "/admin/snapshot-questions"),
("POST", "/admin/snapshot-questions/promote-bulk"),
("GET", "/admin/calibration-status"),
("GET", "/admin/item-statistics"),
("GET", "/admin/sessions"),
("GET", "/admin/basis-items"),
("GET", "/admin/basis-items/{item_id}"),
("POST", "/admin/basis-items/{item_id}/generate"),
("POST", "/admin/basis-items/{item_id}/generate/review-bulk"),
("GET", "/admin/basis-items/{item_id}/generate/variants/{variant_id}"),
]
# Placeholder values for path parameters
PLACEHOLDERS = {
"{session_id}": "test-session-123",
"{tryout_id}": "test-tryout-123",
"{item_id}": "1",
"{website_id}": "1",
"{wp_user_id}": "123",
"{schedule_id}": "test-schedule-123",
"{format}": "xlsx",
"{variant_id}": "test-variant-123",
}
# Minimal request bodies for POST endpoints
REQUEST_BODIES = {
"/api/v1/session/": {
"session_id": "test",
"tryout_id": "test",
"wp_user_id": "123",
"website_id": 1,
"scoring_mode": "ctt",
},
"/api/v1/session/{session_id}/complete": {
"end_time": "2024-01-01T00:00:00Z",
"user_answers": [],
},
"/api/v1/session/{session_id}/submit_answer": {
"item_id": 1,
"response": "A",
"time_spent": 10,
},
"/api/v1/tryout/{tryout_id}/normalization": {
"normalization_mode": "static",
"static_rataan": 500,
"static_sb": 100,
},
"/api/v1/wordpress/sync_users": {}, # Requires proper auth header
"/api/v1/wordpress/verify_session": {
"website_id": 1,
"wp_user_id": "123",
"token": "test",
},
"/api/v1/reports/schedule": {
"tryout_id": "test",
"report_type": "student_performance",
},
"/api/v1/admin/ai/generate-preview": {
"basis_item_id": 1,
"target_level": "sulit",
"ai_model": "qwen/qwen2.5-32b-instruct",
},
"/api/v1/admin/ai/generate-save": {
"stem": "Test?",
"options": {"A": "a", "B": "b", "C": "c", "D": "d"},
"correct": "A",
"tryout_id": "test",
"website_id": 1,
"basis_item_id": 1,
"slot": 1,
"level": "sulit",
"ai_model": "qwen/qwen2.5-32b-instruct",
},
"/api/v1/admin/cat/test": {"tryout_id": "test", "website_id": 1},
"/api/v1/admin/{tryout_id}/calibrate": {},
"/api/v1/admin/{tryout_id}/toggle-ai-generation": {},
"/api/v1/admin/{tryout_id}/reset-normalization": {},
"/api/v1/import-export/preview": None, # Requires file upload
"/api/v1/import-export/questions": None, # Requires file upload
"/api/v1/import-export/tryout-json/preview": None, # Requires file upload
"/api/v1/import-export/tryout-json": None, # Requires file upload
}
def expand_route(method: str, route: str) -> list:
"""Expand route with placeholders."""
expanded = []
test_route = route
for placeholder, value in PLACEHOLDERS.items():
if placeholder in test_route:
test_route = test_route.replace(placeholder, value)
expanded.append((method, test_route))
return expanded
def test_route(client: httpx.Client, method: str, route: str) -> dict:
"""Test a single route."""
# Expand placeholders
expanded = expand_route(method, route)
if not expanded:
return {
"route": route,
"method": method,
"error": "Could not expand route",
"status_code": None,
}
method, test_route = expanded[0]
# Determine request body
body = None
request_body = REQUEST_BODIES.get(route, REQUEST_BODIES.get(test_route, {}))
if request_body is not None:
body = request_body
# Determine query params
params = {}
if "export/questions" in route:
params = {"tryout_id": "test"}
headers = {"X-Website-ID": "1"}
try:
response = client.request(
method=method,
url=BASE_URL + test_route,
json=body if body and method in ["POST", "PUT", "PATCH"] else None,
params=params,
headers=headers,
timeout=10.0,
follow_redirects=True,
)
is_500 = response.status_code == 500
is_ise = "Internal Server Error" in response.text
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": response.status_code,
"has_500": is_500,
"has_ise": is_ise,
"response_preview": response.text[:200] if response.text else "",
"error": None,
}
except httpx.TimeoutException:
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": None,
"has_500": False,
"has_ise": False,
"response_preview": "",
"error": "Timeout",
}
except Exception as e:
return {
"route": route,
"method": method,
"expanded_route": test_route,
"status_code": None,
"has_500": False,
"has_ise": False,
"response_preview": "",
"error": str(e),
}
def main():
print("=" * 80)
print("Testing all IRT Bank Soal routes for Internal Server Errors")
print("=" * 80)
print()
results = []
has_errors = False
with httpx.Client(timeout=30.0) as client:
for method, route in API_ROUTES:
result = test_route(client, method, route)
results.append(result)
status = result["status_code"]
error_marker = ""
if result["error"]:
error_marker = f" [ERROR: {result['error']}]"
has_errors = True
elif status and status >= 500:
error_marker = f" [INTERNAL SERVER ERROR!]"
has_errors = True
elif status and status == 500:
error_marker = f" [500 - INTERNAL SERVER ERROR!]"
has_errors = True
elif "Internal Server Error" in str(result.get("response_preview", "")):
error_marker = " [500 - INTERNAL SERVER ERROR!]"
has_errors = True
status_str = str(status) if status else "N/A"
print(f"{method:6} {route:<60} -> {status_str}{error_marker}")
print()
print("=" * 80)
print("SUMMARY")
print("=" * 80)
total = len(results)
successful = sum(1 for r in results if r["status_code"] and r["status_code"] < 500)
client_errors = sum(
1 for r in results if r["status_code"] and 400 <= r["status_code"] < 500
)
server_errors = sum(
1 for r in results if r["status_code"] and r["status_code"] >= 500
)
timeouts = sum(1 for r in results if r["error"] == "Timeout")
exceptions = sum(1 for r in results if r["error"] and r["error"] != "Timeout")
ise_errors = sum(1 for r in results if r.get("has_ise") or r.get("has_500"))
print(f"Total routes tested: {total}")
print(f"Successful (2xx): {successful}")
print(f"Client errors (4xx): {client_errors}")
print(f"Server errors (5xx): {server_errors}")
print(f"Timeouts: {timeouts}")
print(f"Exceptions: {exceptions}")
print(f"Internal Server Errors: {ise_errors}")
print()
if has_errors:
print("Routes with issues:")
for r in results:
if r["status_code"] and r["status_code"] >= 500:
print(f" - {r['method']} {r['route']} -> {r['status_code']}")
elif r["error"]:
print(f" - {r['method']} {r['route']} -> ERROR: {r['error']}")
elif r.get("has_ise"):
print(f" - {r['method']} {r['route']} -> Internal Server Error")
print()
if ise_errors == 0 and exceptions == 0:
print("✅ All routes passed! No Internal Server Errors detected.")
return 0
else:
print("❌ Some routes have issues. Please review the output above.")
return 1
if __name__ == "__main__":
sys.exit(main())

50
backend/test_debug_login.py Normal file
View File

@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""
Debug login issue.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging login issue...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Get login page
response = client.get("/admin/login")
print(f"Login page status: {response.status_code}")
# Extract CSRF token
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
print(f"CSRF token: {csrf_token[:30]}...")
# Look for any error messages in the page
if "error" in response.text.lower():
print("\n=== Error messages in login page ===")
# Extract error div content
error_match = re.search(
r'<div class="error">(.*?)</div>', response.text, re.DOTALL
)
if error_match:
print(error_match.group(1))
else:
# Print a portion of the page around "error"
idx = response.text.lower().find("error")
print(response.text[max(0, idx - 50) : idx + 200])
# Try to check if Redis is accessible via the health endpoint
health = client.get("/health")
print(f"\nHealth check: {health.text}")
# Print login page content for inspection
print("\n=== Login page content (first 2000 chars) ===")
print(response.text[:2000])
if __name__ == "__main__":
main()

73
backend/test_debug_login2.py Normal file
View File

@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""
Debug login issue - check Redis.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging login issue - detailed...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Get login page
response = client.get("/admin/login")
print(f"Login page status: {response.status_code}")
# Extract CSRF token
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
print(f"CSRF token: {csrf_token}")
# Print ALL cookies
print(f"\nCookies before login: {dict(client.cookies)}")
# Submit login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=False, # Don't follow redirect to see the response
)
print(f"\nLogin response status: {response.status_code}")
print(f"Login response headers: {dict(response.headers)}")
print(f"Cookies after login: {dict(client.cookies)}")
# Check if response has any content
print(f"\nLogin response content (first 1000 chars):")
print(response.text[:1000])
# Now try with a redirect follow
print("\n\n=== Trying with redirect follow ===")
client2 = httpx.Client(base_url=BASE_URL, timeout=30.0)
response = client2.get("/admin/login")
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
response = client2.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f"Final status after redirect: {response.status_code}")
print(f"Final URL: {response.url}")
print(f"Final cookies: {dict(client2.cookies)}")
print(f"Final content (first 500 chars): {response.text[:500]}")
if __name__ == "__main__":
main()

142
backend/test_debug_traceback.py Normal file
View File

@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""
Debug the 500 Internal Server Error on variant approval - fixed CSRF.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
response = client.get("/admin/login")
if response.status_code != 200:
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
return False
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
return response.status_code == 200 and "/admin/dashboard" in str(response.url)
def get_csrf_from_page(client: httpx.Client, page_url: str) -> tuple:
"""Get CSRF token from a specific page and return both token and response."""
response = client.get(page_url, follow_redirects=True)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1), response
return "", response
def main():
print("=" * 80)
print("Debugging 500 Internal Server Error on Variant Approval")
print("=" * 80)
with httpx.Client(base_url=BASE_URL, timeout=60.0) as client:
print("\n1. Logging in...")
if not login(client):
print(" ❌ Login failed")
return
print(" ✅ Login successful")
# Test 1: Variant approval - get CSRF from the actual review page
print("\n2. Testing variant approval...")
# First access the review page to get the CSRF token
csrf_token, page_response = get_csrf_from_page(
client, "/admin/questions/4/generate?tab=review"
)
print(f" Page URL: {page_response.url}")
print(f" Page status: {page_response.status_code}")
print(f" CSRF token: {csrf_token[:30] if csrf_token else 'None'}...")
# If we got redirected, we can't test this endpoint
if "/generate" not in str(page_response.url):
print(
" ⚠️ Redirected away from AI playground - item may not exist or not be AI-generated"
)
print(" Skipping this test...")
else:
# Submit the form
response = client.post(
"/admin/questions/4/generate/review-bulk",
data={
"item_ids": "4",
"action": "approved",
"tab": "review",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
# Extract and print the full traceback
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print("\n" + "=" * 80)
print("FULL TRACEBACK:")
print("=" * 80)
print(response.text[idx:])
print("=" * 80)
elif response.status_code == 500:
print("\n ⚠️ Got 500 error but no traceback in response")
print(f" Response preview: {response.text[:500]}")
else:
print(f" Response preview: {response.text[:500]}")
# Test 2: Generate variants
print("\n3. Testing generate variants...")
csrf_token, page_response = get_csrf_from_page(
client, "/admin/questions/4/generate?tab=generate"
)
print(f" Page URL: {page_response.url}")
print(f" Page status: {page_response.status_code}")
if "/generate" not in str(page_response.url):
print(" ⚠️ Redirected away from AI playground")
else:
response = client.post(
"/admin/questions/4/generate",
data={
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print("\n" + "=" * 80)
print("FULL TRACEBACK:")
print("=" * 80)
print(response.text[idx:])
print("=" * 80)
if __name__ == "__main__":
main()

0
test_error.py → backend/test_error.py
View File

0
test_fetch.py → backend/test_fetch.py
View File

404
backend/test_form_posts.py Normal file
View File

@@ -0,0 +1,404 @@
#!/usr/bin/env python3
"""
Test all form POST endpoints for Internal Server Errors.
"""
import json
import sys
import httpx
BASE_URL = "http://localhost:8000"
# All form POST endpoints from admin_web.py
FORM_POST_ENDPOINTS = [
# (endpoint, method, form_data, description)
(
"/admin/login",
"POST",
{"username": "admin", "password": "admin123"},
"Admin login",
),
(
"/admin/password",
"POST",
{
"old_password": "admin123",
"new_password": "admin123",
"re_new_password": "admin123",
},
"Change password",
),
(
"/admin/websites",
"POST",
{
"site_name": "Test Site",
"site_url": "https://test.example.com",
},
"Create website",
),
(
"/admin/websites/1/edit",
"POST",
{
"site_name": "Updated Test Site",
"site_url": "https://updated.example.com",
},
"Edit website",
),
("/admin/websites/1/delete", "POST", {}, "Delete website"),
(
"/admin/tryout-import/preview",
"POST",
{
"website_id": "1",
},
"Tryout import preview (no file)",
),
(
"/admin/tryout-import",
"POST",
{
"website_id": "1",
"preview_token": "invalid-token",
},
"Tryout import submit",
),
(
"/admin/snapshot-questions/promote-bulk",
"POST",
{
"snapshot_id": "1",
"snapshot_question_ids": [],
},
"Promote snapshot questions bulk",
),
(
"/admin/basis-items/1/generate",
"POST",
{
"target_level": "mudah",
"ai_model": "",
"generation_count": "1",
"operator_notes": "",
},
"Generate variants for basis item",
),
(
"/admin/basis-items/1/review-bulk",
"POST",
{
"item_ids": ["1"],
"action": "approved",
},
"Review bulk variants",
),
(
"/admin/questions/1/generate",
"POST",
{
"target_level": "mudah",
"ai_model": "meta-llama/llama-4-maverick:free",
"generation_count": "1",
"operator_notes": "",
"include_note_for_admin": True,
"include_note_in_prompt": False,
},
"Generate question variants",
),
(
"/admin/questions/1/generate/review-bulk",
"POST",
{
"item_ids": ["1"],
"action": "approved",
"tab": "review",
},
"Review question variants bulk",
),
]
# API POST endpoints
API_POST_ENDPOINTS = [
(
"/api/v1/session/",
{
"session_id": "test-session-123",
"tryout_id": "test",
"wp_user_id": "123",
"website_id": 1,
"scoring_mode": "ctt",
},
"Create session",
),
(
"/api/v1/session/test-session-123/complete",
{
"end_time": "2024-01-01T00:00:00Z",
"user_answers": [],
},
"Complete session",
),
(
"/api/v1/session/test-session-123/submit_answer",
{
"item_id": 1,
"response": "A",
"time_spent": 10,
},
"Submit answer",
),
(
"/api/v1/wordpress/verify_session",
{
"website_id": 1,
"wp_user_id": "123",
"token": "test",
},
"Verify WordPress session",
),
(
"/api/v1/reports/schedule",
{
"tryout_id": "test",
"report_type": "student_performance",
},
"Schedule report",
),
(
"/api/v1/admin/cat/test",
{
"tryout_id": "test",
"website_id": 1,
},
"Test CAT algorithm",
),
("/api/v1/admin/1/calibrate", {}, "Calibrate tryout"),
("/api/v1/admin/1/toggle-ai-generation", {}, "Toggle AI generation"),
("/api/v1/admin/1/reset-normalization", {}, "Reset normalization"),
]
def get_admin_session():
"""Login and get session cookies for admin access."""
with httpx.Client(base_url=BASE_URL, follow_redirects=True, timeout=30.0) as client:
# Try to login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
},
)
print(f"Login response: {response.status_code}")
# Check if we have admin access
response = client.get("/admin")
print(f"Admin page response: {response.status_code}")
# Return cookies
return client.cookies
def test_endpoint(
client: httpx.Client, endpoint: str, method: str, data: dict, cookies: dict = None
) -> dict:
"""Test a single endpoint."""
headers = {"X-Website-ID": "1"}
try:
if method == "POST":
# Check if this looks like form data or JSON
if isinstance(data, dict) and all(
isinstance(v, str) or v is None for v in data.values()
):
# Form data
response = client.post(
endpoint,
data=data,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
else:
# JSON data
response = client.post(
endpoint,
json=data,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
else:
response = client.request(
method,
endpoint,
headers=headers,
cookies=cookies,
timeout=30.0,
follow_redirects=True,
)
# Check for internal server error
has_ise = (
response.status_code == 500
or "Internal Server Error" in response.text
or "500 Internal Server Error" in response.text
)
# Check for traceback
has_traceback = "Traceback" in response.text
return {
"endpoint": endpoint,
"method": method,
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:500] if response.text else "",
"redirect_location": response.headers.get("location", ""),
}
except httpx.TimeoutException:
return {
"endpoint": endpoint,
"method": method,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"response_preview": "",
"error": "Timeout",
}
except Exception as e:
return {
"endpoint": endpoint,
"method": method,
"status_code": None,
"has_ise": False,
"has_traceback": False,
"response_preview": "",
"error": str(e),
}
def main():
print("=" * 80)
print("Testing all Form POST endpoints for Internal Server Errors")
print("=" * 80)
print()
# Get admin session
print("Getting admin session...")
cookies = get_admin_session()
print()
results = []
has_errors = False
with httpx.Client(base_url=BASE_URL, follow_redirects=True, timeout=30.0) as client:
# Test admin form POST endpoints
print("-" * 80)
print("ADMIN FORM POST ENDPOINTS")
print("-" * 80)
for endpoint, method, data, description in FORM_POST_ENDPOINTS:
print(f"\nTesting: {description}")
print(f" Endpoint: {endpoint}")
result = test_endpoint(client, endpoint, method, data, cookies)
results.append((description, result))
status = result["status_code"]
error_details = ""
if result.get("error"):
error_details = f" [ERROR: {result['error']}]"
has_errors = True
elif result.get("has_traceback"):
error_details = f" [TRACEBACK!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
elif result.get("has_ise"):
error_details = f" [INTERNAL SERVER ERROR!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
status_str = str(status) if status else "N/A"
print(f" Status: {status_str}{error_details}")
if result.get("redirect_location"):
print(f" Redirect: {result['redirect_location']}")
# Test API POST endpoints
print()
print("-" * 80)
print("API POST ENDPOINTS")
print("-" * 80)
for endpoint, data, description in API_POST_ENDPOINTS:
print(f"\nTesting: {description}")
print(f" Endpoint: {endpoint}")
result = test_endpoint(client, endpoint, "POST", data, cookies)
results.append((description, result))
status = result["status_code"]
error_details = ""
if result.get("error"):
error_details = f" [ERROR: {result['error']}]"
has_errors = True
elif result.get("has_traceback"):
error_details = f" [TRACEBACK!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
elif result.get("has_ise"):
error_details = f" [INTERNAL SERVER ERROR!]"
has_errors = True
print(f" Response: {result['response_preview'][:1000]}")
status_str = str(status) if status else "N/A"
print(f" Status: {status_str}{error_details}")
# Summary
print()
print("=" * 80)
print("SUMMARY")
print("=" * 80)
total = len(results)
ise_errors = sum(1 for _, r in results if r.get("has_ise"))
tracebacks = sum(1 for _, r in results if r.get("has_traceback"))
timeouts = sum(1 for _, r in results if r.get("error") == "Timeout")
exceptions = sum(
1 for _, r in results if r.get("error") and r.get("error") != "Timeout"
)
print(f"Total endpoints tested: {total}")
print(f"Internal Server Errors: {ise_errors}")
print(f"Tracebacks: {tracebacks}")
print(f"Timeouts: {timeouts}")
print(f"Exceptions: {exceptions}")
print()
if ise_errors > 0 or tracebacks > 0:
print("Endpoints with issues:")
for desc, r in results:
if r.get("has_ise") or r.get("has_traceback"):
print(f" - {desc}: {r['endpoint']} -> {r['status_code']}")
if r.get("has_traceback"):
print(f" Traceback detected in response")
print()
if has_errors:
print("❌ Some endpoints have issues. Please review the output above.")
return 1
else:
print("✅ All endpoints passed! No Internal Server Errors detected.")
return 0
if __name__ == "__main__":
sys.exit(main())

68
backend/test_session_debug.py Normal file
View File

@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""
Debug redirect on AI playground page.
"""
import re
import httpx
BASE_URL = "http://localhost:8000"
def main():
print("Debugging redirect on AI playground page...")
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Login first
response = client.get("/admin/login")
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f"Logged in, URL: {response.url}")
# Get AI playground page without following redirects
print("\nGetting AI playground page without following redirects...")
response = client.get(
"/admin/questions/1/generate?tab=review", follow_redirects=False
)
print(f"Status: {response.status_code}")
print(f"Location header: {response.headers.get('location', 'None')}")
# Follow the redirect
if response.headers.get("location"):
redirect_url = response.headers["location"]
print(f"\nFollowing redirect to: {redirect_url}")
response = client.get(redirect_url, follow_redirects=True)
print(f"Final status: {response.status_code}")
print(f"Final URL: {response.url}")
# Check for forms
post_forms = re.findall(
r'<form[^>]*method="post"[^>]*>', response.text, re.IGNORECASE
)
print(f"\nFound {len(post_forms)} POST forms")
# Look for CSRF token
csrf_inputs = re.findall(
r'<input[^>]*name="csrf_token"[^>]*>', response.text, re.IGNORECASE
)
if csrf_inputs:
print(f"Found {len(csrf_inputs)} CSRF token inputs:")
for inp in csrf_inputs[:3]:
print(f" {inp}")
else:
print("No CSRF token inputs found")
if __name__ == "__main__":
main()

374
backend/test_variant_approval.py Normal file
View File

@@ -0,0 +1,374 @@
#!/usr/bin/env python3
"""
Test variant approval endpoints with proper session handling.
"""
import re
import sys
import httpx
BASE_URL = "http://localhost:8000"
def get_csrf_token(client: httpx.Client, page_url: str) -> str:
"""Extract CSRF token from a page."""
try:
response = client.get(page_url)
if response.status_code == 200:
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
if match:
return match.group(1)
except Exception as e:
print(f" Error getting CSRF token from {page_url}: {e}")
return ""
def login(client: httpx.Client) -> bool:
"""Login and maintain session."""
# Get login page
response = client.get("/admin/login")
if response.status_code != 200:
print(f" Failed to get login page: {response.status_code}")
return False
match = re.search(r'name="csrf_token" value="([^"]+)"', response.text)
csrf_token = match.group(1) if match else ""
if not csrf_token:
print(" Failed to get CSRF token")
return False
# Submit login - follow redirects to complete login
response = client.post(
"/admin/login",
data={
"username": "admin",
"password": "admin123",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
if response.status_code == 200 and "/admin/dashboard" in str(response.url):
print(" ✅ Successfully logged in!")
return True
print(f" Login failed: {response.status_code}, URL: {response.url}")
return False
def test_variant_approval(client: httpx.Client) -> dict:
"""Test the variant approval endpoint."""
# Get CSRF token from the review page
csrf_token = get_csrf_token(client, "/admin/questions/1/generate?tab=review")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit variant approval
response = client.post(
"/admin/questions/1/generate/review-bulk",
data={
"item_ids": "1",
"action": "approved",
"tab": "review",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
print(f" Final URL: {response.url}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
# Extract just the traceback part
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_basis_item_review(client: httpx.Client) -> dict:
"""Test the basis item review bulk endpoint."""
# Get CSRF token from the basis item page
csrf_token = get_csrf_token(client, "/admin/basis-items/1")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit basis item review
response = client.post(
"/admin/basis-items/1/review-bulk",
data={
"item_ids": "1",
"action": "approved",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
print(f" Final URL: {response.url}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_snapshot_promote(client: httpx.Client) -> dict:
"""Test the snapshot questions promote bulk endpoint."""
# Get CSRF token from the hierarchy page
csrf_token = get_csrf_token(client, "/admin/hierarchy")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token - likely not authenticated",
"response_preview": "",
}
# Submit snapshot promote (with empty list)
response = client.post(
"/admin/snapshot-questions/promote-bulk",
data={
"snapshot_id": "1",
"snapshot_question_ids": "",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
# Check for errors
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_tryout_import_preview(client: httpx.Client) -> dict:
"""Test the tryout import preview endpoint."""
csrf_token = get_csrf_token(client, "/admin/tryout-import")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Submit tryout import preview (without file)
response = client.post(
"/admin/tryout-import/preview",
data={
"website_id": "1",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def test_website_crud(client: httpx.Client) -> dict:
"""Test website creation endpoint."""
csrf_token = get_csrf_token(client, "/admin/websites")
if not csrf_token:
return {
"status_code": None,
"has_ise": False,
"has_traceback": False,
"error": "Could not get CSRF token",
"response_preview": "",
}
# Submit website creation
response = client.post(
"/admin/websites",
data={
"site_name": "Test Site",
"site_url": "https://test.example.com",
"csrf_token": csrf_token,
},
follow_redirects=True,
)
print(f" Response status: {response.status_code}")
has_ise = response.status_code == 500 or "Internal Server Error" in response.text
has_traceback = "Traceback" in response.text
if has_traceback:
print("\n === TRACEBACK DETECTED ===")
if "Traceback" in response.text:
idx = response.text.find("Traceback")
print(response.text[idx : idx + 3000])
print(" ==========================\n")
return {
"status_code": response.status_code,
"has_ise": has_ise,
"has_traceback": has_traceback,
"response_preview": response.text[:1000],
}
def main():
print("=" * 80)
print("Testing Form POST Endpoints for Internal Server Errors")
print("=" * 80)
print()
results = []
with httpx.Client(base_url=BASE_URL, timeout=30.0) as client:
# Login
print("Step 1: Logging in...")
if not login(client):
print("❌ Login failed")
return 1
print()
# Test 1: Variant approval
print(
"Step 2: Testing variant approval (/admin/questions/1/generate/review-bulk)..."
)
result1 = test_variant_approval(client)
results.append(("Variant approval", result1))
print()
# Test 2: Basis item review
print("Step 3: Testing basis item review (/admin/basis-items/1/review-bulk)...")
result2 = test_basis_item_review(client)
results.append(("Basis item review", result2))
print()
# Test 3: Snapshot promote
print(
"Step 4: Testing snapshot promote (/admin/snapshot-questions/promote-bulk)..."
)
result3 = test_snapshot_promote(client)
results.append(("Snapshot promote", result3))
print()
# Test 4: Tryout import preview
print("Step 5: Testing tryout import preview (/admin/tryout-import/preview)...")
result4 = test_tryout_import_preview(client)
results.append(("Tryout import preview", result4))
print()
# Test 5: Website creation
print("Step 6: Testing website creation (/admin/websites)...")
result5 = test_website_crud(client)
results.append(("Website creation", result5))
print()
# Summary
print("=" * 80)
print("RESULTS SUMMARY")
print("=" * 80)
all_good = True
for name, result in results:
if result.get("has_ise") or result.get("has_traceback"):
print(f"❌ {name}: INTERNAL SERVER ERROR!")
print(f" Status: {result['status_code']}")
print(f" Preview: {result['response_preview'][:200]}...")
all_good = False
elif result.get("error"):
print(f"⚠️ {name}: {result['error']}")
elif result["status_code"] in [200, 303]:
print(f"✅ {name}: OK ({result['status_code']})")
else:
print(f"⚠️ {name}: Unexpected status {result['status_code']}")
print()
if all_good:
print("✅ All form POST endpoints passed! No Internal Server Errors detected.")
return 0
else:
print("❌ Some endpoints have issues. Please review the output above.")
return 1
if __name__ == "__main__":
sys.exit(main())

10
tests/test_auth_scope.py → backend/tests/test_auth_scope.py
View File

@@ -19,6 +19,12 @@ def test_require_website_auth_returns_scoped_website_for_allowed_role():
assert website_id == 5
def test_require_website_auth_allows_global_system_admin_scope():
auth = AuthContext(website_id=None, role="system_admin", wp_user_id=None)
website_id = require_website_auth(auth, allowed_roles={"admin", "system_admin"})
assert website_id is None
def test_require_website_auth_rejects_disallowed_role():
auth = AuthContext(website_id=5, role="student", wp_user_id="u1")
with pytest.raises(HTTPException) as exc_info:
@@ -30,3 +36,7 @@ def test_cross_website_payload_mismatch_is_blocked():
with pytest.raises(HTTPException) as exc_info:
ensure_website_scope_matches(auth_website_id=10, payload_website_id=11)
assert exc_info.value.status_code == 403
def test_global_system_admin_scope_can_write_any_payload_website():
ensure_website_scope_matches(auth_website_id=None, payload_website_id=11)

24
tests/test_auth_tokens.py → backend/tests/test_auth_tokens.py
View File

@@ -23,6 +23,30 @@ def test_issue_and_decode_access_token_round_trip():
assert auth.wp_user_id == "wp-1001"
def test_system_admin_token_can_be_global_without_website_scope():
token = issue_access_token(
website_id=None,
role="system_admin",
wp_user_id=None,
expires_in_seconds=3600,
)
auth = decode_access_token(token)
assert auth.website_id is None
assert auth.role == "system_admin"
def test_non_system_admin_token_requires_website_scope():
token = issue_access_token(
website_id=None,
role="admin",
wp_user_id=None,
expires_in_seconds=3600,
)
with pytest.raises(HTTPException) as exc_info:
decode_access_token(token)
assert exc_info.value.status_code == 401
def test_decode_access_token_rejects_tampered_signature():
token = issue_access_token(
website_id=7,

0
tests/test_model_mappings.py → backend/tests/test_model_mappings.py
View File

0
tests/test_normalization.py → backend/tests/test_normalization.py
View File

59
tests/test_operational_hardening.py → backend/tests/test_operational_hardening.py
View File

@@ -8,12 +8,14 @@ from app.core import rate_limit
from app.core.config import Settings
from app.models.report_schedule import ReportScheduleModel
from app.services import ai_generation
from app.services import cat_selection
from app.services.reporting import (
cancel_scheduled_report,
get_scheduled_report,
list_scheduled_reports,
schedule_report,
)
from app.schemas.ai import GeneratedQuestion
class DummyRequest:
@@ -101,6 +103,63 @@ def test_ai_stats_accepts_website_scope(monkeypatch):
assert all("items.website_id" in query for query in captured_queries)
def test_ai_prompt_preserves_basis_option_labels():
prompt = ai_generation.get_prompt_template(
basis_stem="<p>Basis question?</p>",
basis_options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
"E": "Option E",
},
basis_correct="A",
basis_explanation="<p>Because A.</p>",
target_level="mudah",
)
assert "Create exactly 5 answer options with labels exactly: A, B, C, D, E" in prompt
assert '"E": "Option E text"' in prompt
assert "The correct field must be exactly one of: A, B, C, D, E" in prompt
def test_generated_question_must_match_basis_option_labels():
basis_item = SimpleNamespace(
options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
"E": "Option E",
}
)
generated = GeneratedQuestion(
stem="Generated",
options={
"A": "Option A",
"B": "Option B",
"C": "Option C",
"D": "Option D",
},
correct="A",
)
assert not ai_generation.generated_matches_basis_options(generated, basis_item)
def test_cat_selection_only_serves_active_or_approved_variants():
compiled = str(
cat_selection._servable_item_filter().compile(
compile_kwargs={"literal_binds": True}
)
)
assert "active" in compiled
assert "approved" in compiled
assert "draft" not in compiled
assert "rejected" not in compiled
def test_production_init_db_skips_create_all(monkeypatch):
import app.database as database

2
tests/test_route_wiring.py → backend/tests/test_route_wiring.py
View File

@@ -7,5 +7,5 @@ from app.main import app
def test_next_item_route_is_registered():
paths = {route.path for route in app.routes}
paths = set(app.openapi()["paths"])
assert "/api/v1/session/{session_id}/next_item" in paths

0
tests/test_security_regressions.py → backend/tests/test_security_regressions.py
View File

0
tests/test_tryout_json_import.py → backend/tests/test_tryout_json_import.py
View File

71
docker-compose.yml Normal file
View File

@@ -0,0 +1,71 @@
version: '3.8'
services:
# 1. FastAPI Backend
backend:
build:
context: ./backend
ports:
- "8000:8000"
env_file:
- ./backend/.env
environment:
- DATABASE_URL=postgresql+asyncpg://irt_user:dev_password@postgres:5432/irt_bank_soal
- REDIS_URL=redis://redis:6379/0
depends_on:
- postgres
- redis
restart: unless-stopped
# 2. Redis Message Broker (Required by Celery)
redis:
image: redis:7-alpine
ports:
- "6380:6379"
restart: unless-stopped
volumes:
- redis_data:/data
# 2.5 PostgreSQL Database
postgres:
image: postgres:15-alpine
environment:
POSTGRES_USER: irt_user
POSTGRES_PASSWORD: dev_password
POSTGRES_DB: irt_bank_soal
ports:
- "5433:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
restart: unless-stopped
# 3. Celery Worker (For AI Generation Background Tasks)
celery_worker:
build:
context: ./backend
command: ["celery", "-A", "app.core.celery_app", "worker", "--loglevel=info"]
env_file:
- ./backend/.env
environment:
- DATABASE_URL=postgresql+asyncpg://irt_user:dev_password@postgres:5432/irt_bank_soal
- REDIS_URL=redis://redis:6379/0
depends_on:
- backend
- redis
restart: unless-stopped
# 4. React Frontend SPA
frontend:
build:
context: ./frontend
args:
VITE_API_URL: "http://localhost:8000/api/v1"
ports:
- "3000:80"
depends_on:
- backend
restart: unless-stopped
volumes:
postgres_data:
redis_data:

7198
form_posts_check_and_fixes.md Normal file
View File

File diff suppressed because it is too large Load Diff

1
frontend/.env.example Normal file
View File

@@ -0,0 +1 @@
VITE_API_URL=http://localhost:8000/api/v1

24
frontend/.gitignore vendored Normal file
View File

@@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

Some files were not shown because too many files have changed in this diff Show More