dwindown/yellow-bank-soal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Dwindi Ramadhana
2026-03-21 23:32:59 +07:00
commit cf193d7ea0
57 changed files with 17871 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.claude/settings.local.json Normal file
View File

@@ -0,0 +1,6 @@
{
"enabledMcpjsonServers": [
"brave-search"
],
"enableAllProjectMcpServers": true
}

31
.env.example Normal file
View File

@@ -0,0 +1,31 @@
# Database
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/irt_bank_soal
DB_HOST=localhost
DB_PORT=5432
DB_NAME=irt_bank_soal
DB_USER=postgres
DB_PASSWORD=your_password_here
# FastAPI
SECRET_KEY=your-secret-key-here-change-in-production
API_V1_STR=/api/v1
PROJECT_NAME=IRT Bank Soal
ENVIRONMENT=development
# OpenRouter (AI Generation)
OPENROUTER_API_KEY=your-openrouter-api-key-here
OPENROUTER_MODEL_QWEN=qwen/qwen-2.5-coder-32b-instruct
OPENROUTER_MODEL_LLAMA=meta-llama/llama-3.3-70b-instruct
OPENROUTER_TIMEOUT=30
# WordPress Integration
WORDPRESS_API_URL=https://your-wordpress-site.com/wp-json
WORDPRESS_AUTH_TOKEN=your-wordpress-jwt-token
# Redis (Celery)
REDIS_URL=redis://localhost:6379/0
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0
# CORS
ALLOWED_ORIGINS=https://site1.com,https://site2.com,https://site3.com

30
.gitignore vendored Normal file
View File

@@ -0,0 +1,30 @@
*.pyc
__pycache__/
*.py[cod]
*$py.class
.env
.venv/
venv/
ENV/
env/
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
.pytest_cache/
.coverage
htmlcov/
.DS_Store

952
AAPANEL_DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,952 @@
# IRT Bank Soal - AaPanel Deployment Guide
**Document Version:** 1.1
**Date:** March 21, 2026
**Project:** IRT-Powered Adaptive Question Bank System v1.2.0
**Updated:** Clarified PostgreSQL setup using Databases > PgSQL menu
---
## Table of Contents
1. [Prerequisites](#1-prerequisites)
2. [AaPanel Installation](#2-aapanel-installation)
3. [Install Required Software via AaPanel](#3-install-required-software-via-aapanel)
4. [PostgreSQL Setup](#4-postgresql-setup)
5. [Python Manager Setup](#5-python-manager-setup)
6. [Project Deployment](#6-project-deployment)
7. [Environment Configuration](#7-environment-configuration)
8. [Database Migration](#8-database-migration)
9. [Running the Application](#9-running-the-application)
10. [Nginx Reverse Proxy Configuration](#10-nginx-reverse-proxy-configuration)
11. [SSL Configuration](#11-ssl-configuration)
12. [Post-Deployment Verification](#12-post-deployment-verification)
13. [Troubleshooting](#13-troubleshooting)
---
## 1. Prerequisites
### Server Requirements
| Requirement | Minimum | Recommended |
|-------------|---------|-------------|
| OS | Ubuntu 20.04 / CentOS 7+ | Ubuntu 22.04 LTS |
| RAM | 2 GB | 4 GB+ |
| Storage | 20 GB | 50 GB+ |
| CPU | 1 vCPU | 2+ vCPU |
### Domain Requirements
- A domain name pointed to your server IP
- Subdomain recommended (e.g., `api.yourdomain.com`)
---
## 2. AaPanel Installation
### Step 2.1: Install AaPanel
**For Ubuntu/Debian:**
```bash
# Login to your server via SSH
ssh root@your-server-ip
# Install AaPanel
wget -O install.sh http://www.aapanel.com/script/install-ubuntu_6.0_en.sh && bash install.sh
```
**For CentOS:**
```bash
# Install AaPanel
yum install -y wget && wget -O install.sh http://www.aapanel.com/script/install_6.0_en.sh && sh install.sh
```
### Step 2.2: Access AaPanel
1. After installation completes, note the panel URL and credentials
2. Access AaPanel via browser: `http://your-server-ip:8888`
3. Login with provided credentials
4. **Important:** Change default port and password after first login
---
## 3. Install Required Software via AaPanel
### Step 3.1: Install Nginx
1. In AaPanel, go to **App Store**
2. Find **Nginx** and click **Install**
3. Select version (recommended: 1.24+)
4. Click **Submit** and wait for installation
### Step 3.2: Install Python Manager
1. Go to **App Store**
2. Search for **Python Manager** (or **PM2 Manager**)
3. Click **Install**
### Step 3.3: Install Redis (Optional, for Celery)
1. Go to **App Store**
2. Find **Redis** and click **Install**
3. Click **Submit**
---
## 4. PostgreSQL Setup
> **IMPORTANT:** Use **Databases > PgSQL** menu from AaPanel sidebar.
>
> This menu supports both:
> - **Local server** - PostgreSQL installed on your AaPanel server
> - **Remote server** - External PostgreSQL (Supabase, Neon, AWS RDS, etc.)
### Step 4.1: Choose Your Database Type
You have two options:
| Option | Description | Best For |
|--------|-------------|----------|
| **Remote Database** | External PostgreSQL service (Supabase, Neon, etc.) | Easy setup, managed, free tier available |
| **Local Database** | PostgreSQL on your AaPanel server | Full control, no external dependency |
---
### Option A: Remote PostgreSQL Database (RECOMMENDED)
Use an external PostgreSQL service:
- **Supabase** - https://supabase.com (free tier: 500MB)
- **Neon** - https://neon.tech (free tier: 3GB)
- **AWS RDS** - https://aws.amazon.com/rds/postgresql/
- **DigitalOcean** - https://www.digitalocean.com/products/managed-databases-postgresql
- **Railway** - https://railway.app
#### Step 4.A.1: Create Database on Provider
1. Sign up on your chosen provider
2. Create a new PostgreSQL project/database
3. Note down the connection details from dashboard:
- **Host** (e.g., `db.xxxxx.supabase.co` or `ep-xxx.us-east-2.aws.neon.tech`)
- **Port** (usually `5432`, Supabase uses `6543` for pooler)
- **Database name** (e.g., `postgres` or `neondb`)
- **Username** (e.g., `postgres.xxxxx`)
- **Password**
#### Step 4.A.2: Add Remote Server to AaPanel PgSQL
1. In AaPanel, go to **Databases** > **PgSQL**
2. Click **Remote DB** button
3. Fill in the form:
- **Server Name:** `my-remote-db` (any name you like)
- **Server Address:** `db.xxxxx.supabase.co` (your host)
- **Port:** `5432` or `6543` (check your provider)
- **Root User:** `postgres` or your username
- **Root Password:** your password
4. Click **Submit**
#### Step 4.A.3: Sync Databases from Remote Server
1. After adding remote server, click **Get DB from server**
2. Select your remote server from dropdown
3. Click **Submit**
4. Your remote databases will appear in the list
#### Step 4.A.4: Note Your Connection String
Your connection string format:
```
postgresql+asyncpg://username:password@host:port/database_name
```
**Example (Supabase):**
```
postgresql+asyncpg://postgres.xxxxx:YourPassword@aws-0-ap-southeast-1.pooler.supabase.com:6543/postgres
```
**Example (Neon):**
```
postgresql+asyncpg://neondb_owner:YourPassword@ep-xxxx.us-east-2.aws.neon.tech/neondb?sslmode=require
```
---
### Option B: Local PostgreSQL Database
Install PostgreSQL directly on your AaPanel server.
#### Step 4.B.1: Install PostgreSQL via Terminal
```bash
# SSH into your server
ssh root@your-server-ip
# Ubuntu/Debian
apt update
apt install -y postgresql postgresql-contrib
# Start and enable PostgreSQL
systemctl start postgresql
systemctl enable postgresql
# Check status
systemctl status postgresql
```
#### Step 4.B.2: Create Database and User via Terminal
```bash
# Switch to postgres user
su - postgres
# Enter PostgreSQL CLI
psql
# Run SQL commands:
CREATE DATABASE irt_bank_soal;
CREATE USER irt_user WITH ENCRYPTED PASSWORD 'your_secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE irt_bank_soal TO irt_user;
# Connect to database and grant schema
\c irt_bank_soal
GRANT ALL ON SCHEMA public TO irt_user;
# Exit
\q
exit
```
#### Step 4.B.3: Add Local Server to AaPanel PgSQL
1. In AaPanel, go to **Databases** > **PgSQL**
2. Click **Root Password** to view/change postgres password
3. If your local PostgreSQL is not showing, click **Get DB from server**
4. Select **Local server**
5. Click **Submit**
#### Step 4.B.4: Create Additional Database via AaPanel (Optional)
1. In **Databases** > **PgSQL**
2. Click **Add DB**
3. Fill in:
- **Database name:** `irt_bank_soal`
- **Username:** `irt_user` (or same as DB name)
- **Password:** (click generate or enter custom)
- **Add to:** `Local server`
4. Click **Submit**
#### Step 4.B.5: Note Your Connection String
```
postgresql+asyncpg://irt_user:your_password@127.0.0.1:5432/irt_bank_soal
```
---
## 4.1 Test Database Connection
Before proceeding, verify your database connection works.
### For Remote Database:
```bash
# Install psql client if needed
apt install -y postgresql-client
# Test connection (replace with your details)
psql "postgresql://username:password@host:port/database_name" -c "SELECT version();"
```
### For Local Database:
```bash
# Test connection
psql -U irt_user -d irt_bank_soal -h 127.0.0.1 -c "SELECT version();"
# If prompted for password, enter it
```
---
## 4.2 Connection String Quick Reference
| Database Type | Connection String Format |
|---------------|-------------------------|
| **Remote (Supabase)** | `postgresql+asyncpg://postgres.xxxx:password@aws-0-region.pooler.supabase.com:6543/postgres` |
| **Remote (Neon)** | `postgresql+asyncpg://user:password@ep-xxxx.region.aws.neon.tech/neondb?sslmode=require` |
| **Local** | `postgresql+asyncpg://irt_user:password@127.0.0.1:5432/irt_bank_soal` |
> **Note:** We use `postgresql+asyncpg://` because our app uses async SQLAlchemy with `asyncpg` driver.
---
## 5. Python Manager Setup
### Step 5.1: Open Python Manager
1. In AaPanel, go to **App Store**
2. Find **Python Manager** and click **Settings**
### Step 5.2: Install Python Version
1. Click **Version Management**
2. Select **Python 3.11** (or latest stable)
3. Click **Install**
4. Wait for installation to complete
---
## 6. Project Deployment
### Step 6.1: Create Project Directory
```bash
# Create project directory
mkdir -p /www/wwwroot/irt-bank-soal
# Navigate to directory
cd /www/wwwroot/irt-bank-soal
```
### Step 6.2: Upload Project Files
**Option A: Upload via File Manager**
1. In AaPanel, go to **Files**
2. Navigate to `/www/wwwroot/irt-bank-soal`
3. Upload your project ZIP file
4. Extract the archive
**Option B: Clone from Git (if applicable)**
```bash
cd /www/wwwroot/irt-bank-soal
# If using Git
git clone https://github.com/your-repo/irt-bank-soal.git .
# Or copy from local
# scp -r /Users/dwindown/Applications/tryout-system/* root@your-server-ip:/www/wwwroot/irt-bank-soal/
```
### Step 6.3: Verify Project Structure
```bash
# Expected structure:
ls -la /www/wwwroot/irt-bank-soal/
# app/
# app/models/
# app/routers/
# app/services/
# app/core/
# tests/
# requirements.txt
# .env.example
# alembic/
```
---
## 7. Environment Configuration
### Step 7.1: Create Virtual Environment via Python Manager
1. In AaPanel **Python Manager**, click **Add Project**
2. Configure:
- **Project Name:** `irt-bank-soal`
- **Project Path:** `/www/wwwroot/irt-bank-soal`
- **Python Version:** `Python 3.11`
- **Framework:** `FastAPI`
- **Startup Method:** `uvicorn`
3. Click **Submit**
### Step 7.2: Create Environment File
```bash
# Copy example file
cp /www/wwwroot/irt-bank-soal/.env.example /www/wwwroot/irt-bank-soal/.env
# Edit .env file
nano /www/wwwroot/irt-bank-soal/.env
```
### Step 7.3: Configure .env File
```env
# Database Configuration
# For Remote Database (Supabase example):
# DATABASE_URL=postgresql+asyncpg://postgres.xxxx:password@aws-0-ap-southeast-1.pooler.supabase.com:6543/postgres
# For Remote Database (Neon example):
# DATABASE_URL=postgresql+asyncpg://neondb_owner:password@ep-xxxx.us-east-2.aws.neon.tech/neondb?sslmode=require
# For Local Database:
DATABASE_URL=postgresql+asyncpg://irt_user:your_secure_password_here@127.0.0.1:5432/irt_bank_soal
# Security
SECRET_KEY=your-production-secret-key-min-32-characters-random-string
# Environment
ENVIRONMENT=production
DEBUG=false
# API Configuration
API_V1_STR=/api/v1
PROJECT_NAME=IRT Bank Soal
PROJECT_VERSION=1.2.0
# CORS - Add your WordPress domains
ALLOWED_ORIGINS=https://yourdomain.com,https://www.yourdomain.com
# OpenRouter API (for AI Generation)
OPENROUTER_API_KEY=your-openrouter-api-key-here
OPENROUTER_API_URL=https://openrouter.ai/api/v1
OPENROUTER_MODEL_QWEN=qwen/qwen-2.5-coder-32b-instruct
OPENROUTER_MODEL_LLAMA=meta-llama/llama-3.3-70b-instruct
OPENROUTER_TIMEOUT=60
# WordPress Integration
WORDPRESS_API_URL=https://yourdomain.com/wp-json
WORDPRESS_AUTH_TOKEN=your-wordpress-jwt-token
# Redis (for Celery task queue)
REDIS_URL=redis://127.0.0.1:6379/0
# Admin Panel
ADMIN_USER=admin
ADMIN_PASSWORD=your-secure-admin-password
# Normalization Defaults
DEFAULT_RATAAN=500
DEFAULT_SB=100
MIN_SAMPLE_FOR_DYNAMIC=100
```
### Step 7.4: Generate Secret Key
```bash
# Generate a secure secret key
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# Copy the output and paste into SECRET_KEY in .env
```
---
## 8. Database Migration
### Step 8.1: Activate Virtual Environment
```bash
# Via Python Manager, the venv is usually at:
source /www/wwwroot/irt-bank-soal/venv/bin/activate
# Or check Python Manager for exact venv path
```
### Step 8.2: Install Dependencies
```bash
# Ensure you're in project directory
cd /www/wwwroot/irt-bank-soal
# Install dependencies
pip install -r requirements.txt
# Verify installation
pip list | grep -E "fastapi|sqlalchemy|numpy|scipy|httpx|openpyxl"
```
### Step 8.3: Initialize Alembic (First Time Setup)
```bash
# Initialize Alembic if not already done
alembic init alembic
# Generate initial migration
alembic revision --autogenerate -m "Initial migration"
# Apply migration
alembic upgrade head
```
### Step 8.4: Verify Database Tables
```bash
# Check tables were created
psql -U irt_user -d irt_bank_soal -h 127.0.0.1 -c "\dt"
# Expected output: websites, users, tryouts, items, sessions, user_answers, tryout_stats
```
---
## 9. Running the Application
### Step 9.1: Configure Python Project in AaPanel
1. In **Python Manager**, find your project `irt-bank-soal`
2. Click **Settings**
3. Configure startup:
- **Startup File:** `app/main.py`
- **Startup Method:** `uvicorn`
- **Port:** `8000`
- **Modules:** `uvicorn[standard]`
### Step 9.2: Set Startup Command
In Python Manager settings, set the startup command:
```bash
# Startup command
uvicorn app.main:app --host 127.0.0.1 --port 8000 --workers 4
# Or for development:
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
```
### Step 9.3: Start the Application
1. In Python Manager, click **Start** on your project
2. Check logs for any errors
3. Verify the application is running:
```bash
# Test health endpoint
curl http://127.0.0.1:8000/
# Expected response:
# {"status": "healthy", "project_name": "IRT Bank Soal", "version": "1.2.0"}
```
### Step 9.4: Configure Auto-Start on Boot
1. In Python Manager, enable **Auto-start on boot**
2. Or manually via terminal:
```bash
# Using systemd (create service file)
nano /etc/systemd/system/irt-bank-soal.service
```
```ini
[Unit]
Description=IRT Bank Soal FastAPI Application
After=network.target
# Uncomment below if using LOCAL PostgreSQL:
# After=network.target postgresql.service
[Service]
Type=simple
User=www
Group=www
WorkingDirectory=/www/wwwroot/irt-bank-soal
Environment="PATH=/www/wwwroot/irt-bank-soal/venv/bin"
ExecStart=/www/wwwroot/irt-bank-soal/venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000 --workers 4
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash
# Enable and start service
systemctl daemon-reload
systemctl enable irt-bank-soal
systemctl start irt-bank-soal
systemctl status irt-bank-soal
```
---
## 10. Nginx Reverse Proxy Configuration
### Step 10.1: Create Website in AaPanel
1. In AaPanel, go to **Website**
2. Click **Add Site**
3. Configure:
- **Domain:** `api.yourdomain.com` (or your subdomain)
- **PHP Version:** Pure Static (not needed)
- **Database:** None (already created)
4. Click **Submit**
### Step 10.2: Configure Reverse Proxy
1. Click **Settings** on the newly created website
2. Go to **Reverse Proxy**
3. Click **Add Reverse Proxy**
4. Configure:
- **Proxy Name:** `irt-api`
- **Target URL:** `http://127.0.0.1:8000`
5. Click **Submit**
### Step 10.3: Manual Nginx Configuration (Alternative)
```bash
# Edit Nginx config
nano /www/server/panel/vhost/nginx/api.yourdomain.com.conf
```
```nginx
server {
listen 80;
server_name api.yourdomain.com;
# Access and error logs
access_log /www/wwwlogs/api.yourdomain.com.log;
error_log /www/wwwlogs/api.yourdomain.com.error.log;
# Client body size (for Excel uploads)
client_max_body_size 50M;
# Proxy to FastAPI
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Static files (if any)
location /static/ {
alias /www/wwwroot/irt-bank-soal/static/;
expires 30d;
}
}
```
### Step 10.4: Test and Reload Nginx
```bash
# Test Nginx configuration
nginx -t
# Reload Nginx
nginx -s reload
# Or via AaPanel: Website > Settings > Config > Save
```
---
## 11. SSL Configuration
### Step 11.1: Install SSL Certificate
1. In AaPanel, go to **Website**
2. Click **Settings** on your site
3. Go to **SSL**
4. Choose method:
- **Let's Encrypt:** Free, auto-renewal
- **Own Certificate:** Upload your own
- **Buy:** Purchase through AaPanel
### Step 11.2: Configure Let's Encrypt
1. Click **Let's Encrypt**
2. Enter your email
3. Select domain `api.yourdomain.com`
4. Click **Apply**
5. Enable **Force HTTPS**
### Step 11.3: Update .env for HTTPS
```bash
# Edit .env
nano /www/wwwroot/irt-bank-soal/.env
# Update CORS to use HTTPS
ALLOWED_ORIGINS=https://yourdomain.com,https://www.yourdomain.com
```
---
## 12. Post-Deployment Verification
### Step 12.1: Test API Endpoints
```bash
# Test health endpoint
curl https://api.yourdomain.com/
# Test detailed health
curl https://api.yourdomain.com/health
# Test API documentation
# Open in browser: https://api.yourdomain.com/docs
```
### Step 12.2: Test Database Connection
```bash
# Via API
curl https://api.yourdomain.com/health
# Expected response includes database status:
# {"status": "healthy", "database": "connected", "api_version": "v1"}
```
### Step 12.3: Test Admin Panel
```bash
# Access admin panel
# Open in browser: https://api.yourdomain.com/admin
# Login with credentials from .env
```
### Step 12.4: Load Test Data (Optional)
```bash
# SSH into server
ssh root@your-server-ip
# Navigate to project
cd /www/wwwroot/irt-bank-soal
# Activate venv
source venv/bin/activate
# Run test data script
python3 -c "
import asyncio
from app.database import init_db
asyncio.run(init_db())
print('Database initialized successfully')
"
```
---
## 13. Troubleshooting
### Issue: Python Manager Not Starting Application
**Solution:**
```bash
# Check logs
tail -f /www/wwwroot/irt-bank-soal/logs/error.log
# Check if port is in use
lsof -i :8000
# Manually test startup
cd /www/wwwroot/irt-bank-soal
source venv/bin/activate
uvicorn app.main:app --host 127.0.0.1 --port 8000
```
### Issue: Database Connection Failed
**For Remote Database:**
```bash
# Test connection from server
apt install -y postgresql-client
psql "postgresql://username:password@remote-host:port/database" -c "SELECT 1;"
# Check if firewall allows outbound connection
# Most remote DBs use port 5432 or 6543
# Verify DATABASE_URL in .env
cat /www/wwwroot/irt-bank-soal/.env | grep DATABASE_URL
# Common issues:
# - Wrong port (Supabase pooler uses 6543, direct uses 5432)
# - Missing sslmode=require (Neon requires this)
# - IP not whitelisted (check provider dashboard)
```
**For Local Database:**
```bash
# Check PostgreSQL status
systemctl status postgresql
# Test connection manually
psql -U irt_user -d irt_bank_soal -h 127.0.0.1 -W
# Check pg_hba.conf allows connections
cat /etc/postgresql/*/main/pg_hba.conf | grep -v "^#" | grep -v "^$"
# Verify DATABASE_URL in .env
cat /www/wwwroot/irt-bank-soal/.env | grep DATABASE_URL
```
### Issue: 502 Bad Gateway
**Solution:**
```bash
# Check if FastAPI is running
ps aux | grep uvicorn
# Check Nginx error logs
tail -f /www/wwwlogs/api.yourdomain.com.error.log
# Verify proxy configuration
cat /www/server/panel/vhost/nginx/api.yourdomain.com.conf | grep proxy_pass
```
### Issue: CORS Errors
**Solution:**
```bash
# Check ALLOWED_ORIGINS in .env
cat /www/wwwroot/irt-bank-soal/.env | grep ALLOWED_ORIGINS
# Ensure WordPress domain is included
# Example: ALLOWED_ORIGINS=https://site1.com,https://site2.com
# Restart application after changes
# Via Python Manager: Stop > Start
```
### Issue: SSL Certificate Not Working
**Solution:**
```bash
# Check certificate
openssl s_client -connect api.yourdomain.com:443
# Force HTTPS in Nginx config
# Add to server block:
# return 301 https://$host$request_uri;
# Reload Nginx
nginx -s reload
```
### Issue: Large File Upload Failed
**Solution:**
```bash
# Increase Nginx client body size
nano /www/server/panel/vhost/nginx/api.yourdomain.com.conf
# Add/modify:
# client_max_body_size 100M;
# Also check PHP settings if using PHP
# In AaPanel: PHP > Settings > Upload Max Filesize
```
---
## Quick Reference Commands
```bash
# Application Management
systemctl start irt-bank-soal
systemctl stop irt-bank-soal
systemctl restart irt-bank-soal
systemctl status irt-bank-soal
# Local Database Management (if using local PostgreSQL)
systemctl start postgresql
systemctl stop postgresql
systemctl restart postgresql
systemctl status postgresql
# Nginx Management
nginx -t # Test config
nginx -s reload # Reload config
systemctl restart nginx # Restart Nginx
# View Logs
tail -f /www/wwwlogs/api.yourdomain.com.log
tail -f /www/wwwlogs/api.yourdomain.com.error.log
# Application Logs (if configured)
tail -f /www/wwwroot/irt-bank-soal/logs/app.log
# Test Database Connection
# Local:
psql -U irt_user -d irt_bank_soal -h 127.0.0.1 -c "SELECT version();"
# Remote:
psql "postgresql://user:pass@host:port/db" -c "SELECT version();"
```
---
## Security Checklist
- [ ] Changed AaPanel default port and password
- [ ] Database user has strong password
- [ ] SECRET_KEY is unique and 32+ characters
- [ ] SSL certificate installed and forced HTTPS
- [ ] CORS restricted to production domains only
- [ ] Firewall configured (only 80, 443, 22, 8888 open)
- [ ] Admin password is strong
- [ ] For local DB: PostgreSQL not exposed to internet
- [ ] For remote DB: IP whitelist configured (if supported)
- [ ] Regular backups configured
---
## Backup Configuration
### Database Backup
**For Local Database:**
```bash
# Create backup directory
mkdir -p /www/backup
# Manual backup
pg_dump -U irt_user -h 127.0.0.1 irt_bank_soal > /www/backup/irt_bank_soal_$(date +%Y%m%d).sql
# Automated backup (cron)
crontab -e
# Add: 0 2 * * * pg_dump -U irt_user -h 127.0.0.1 irt_bank_soal > /www/backup/irt_bank_soal_$(date +\%Y\%m\%d).sql
```
**For Remote Database:**
Most managed PostgreSQL providers have built-in backup features:
- **Supabase:** Dashboard > Database > Backups (daily automatic)
- **Neon:** Automatic point-in-time recovery
- **AWS RDS:** Automated backups with retention period
You can also backup manually:
```bash
# Manual backup from remote (requires postgresql-client)
pg_dump "postgresql://username:password@host:port/database" > /www/backup/irt_bank_soal_$(date +%Y%m%d).sql
# Or with SSL for providers like Neon
pg_dump "postgresql://username:password@host:port/database?sslmode=require" > /www/backup/irt_bank_soal_$(date +%Y%m%d).sql
```
### Project Backup
```bash
# Backup project files
tar -czvf /www/backup/irt_project_$(date +%Y%m%d).tar.gz /www/wwwroot/irt-bank-soal
# Exclude venv to save space
tar -czvf /www/backup/irt_project_$(date +%Y%m%d).tar.gz --exclude='venv' /www/wwwroot/irt-bank-soal
```
---
**Document End**
**Status:** Ready for Deployment
**Support:** Refer to TEST.md for testing procedures and PRD.md for requirements.

746
PRD.md Normal file
View File

@@ -0,0 +1,746 @@
# Product Requirements Document (PRD)
## IRT-Powered Adaptive Question Bank System
**Document Version:** 1.1
**Date:** March 21, 2026 (Updated)
**Product Name:** IRT Bank Soal (Adaptive Question Bank with AI Generation)
**Client:** Sejoli Tryout Multi-Website Platform
**Status:** Draft - Clarifications Incorporated
---
## Changelog
### v1.1 (March 21, 2026)
- Added **AI Generation**: 1 request = 1 question, no approval workflow
- Added **Admin Playground**: Admin can test AI generation without saving to DB
- Updated **Normalization Control**: Optional manual/automatic mode, system handles auto when sufficient data
- Updated **IRT → CTT Rollback**: Historical IRT scores preserved, CTT applied to new sessions only
- Removed **Admin Permissions/Role-based Access**: Not needed (each admin per site via WordPress)
- Updated **Custom Dashboards**: Use FastAPI Admin only (no custom dashboards)
- Added **AI Generation Toggle**: Global on/off switch for cost control
- Added **User-level Question Reuse**: Check if student already answered at difficulty level
- Updated **Student UX**: Admin sees internal metrics, students see only primary score
- Added **Data Retention**: Keep all data (no policy yet)
- Added **Reporting Section**: Student performance, Item analysis, Calibration status, Tryout comparison
- Updated **Admin Persona Note**: This project is backend tool for IRT/CTT calculation; WordPress handles static questions
---
## 1. Product Vision
### 1.1 Vision Statement
To provide an adaptive, intelligent question bank system that seamlessly integrates with Sejoli's existing Excel-based workflow while introducing modern Item Response Theory (IRT) capabilities and AI-powered question generation, enabling more accurate and efficient student assessment.
### 1.1.1 Primary Goals
- **100% Excel Compatibility**: Maintain exact formula compatibility with client's existing Excel workflow (CTT scoring with p, bobot, NM, NN)
- **Gradual Modernization**: Enable smooth transition from Classical Test Theory (CTT) to Item Response Theory (IRT)
- **Adaptive Assessment**: Provide Computerized Adaptive Testing (CAT) capabilities for more efficient and accurate measurement
- **AI-Enhanced Content**: Automatically generate question variants (Mudah/Sulit) from base Sedang questions
- **Multi-Site Support**: Single backend serving multiple WordPress-powered educational sites
- **Non-Destructive**: Zero disruption to existing operations - all enhancements are additive
### 1.1.2 Success Metrics
- **Technical**: CTT scores match client Excel 100%, IRT calibration >80% coverage
- **Educational**: 30% reduction in test length with IRT vs CTT, measurement precision (SE < 0.5 after 15 items)
- **Adoption**: >70% tryouts use hybrid mode within 3 months, >80% student satisfaction with adaptive mode
- **Efficiency**: 99.9% question reuse rate via AI-generated variants
---
## 2. User Personas
### 2.1 Administrators (School/Guru)
**Profile:** Non-technical education professionals managing tryouts
**Pain Points:**
- Excel-based scoring is manual and time-consuming
- Static questions require constant new content creation
- Difficulty normalization requires manual calculation
- Limited ability to compare student performance across groups
**Needs:**
- Simple, transparent scoring formulas (CTT mode)
- Easy Excel import/export workflow
- Clear visualizations of student performance
- Configurable normalization (static vs dynamic)
- Optional advanced features (IRT) without complexity
### 2.2 Students
**Profile:** Students taking tryouts for assessment
**Pain Points:**
- Fixed-length tests regardless of ability level
- Question difficulty may not match their skill
- Long testing sessions with low-value questions
**Needs:**
- Adaptive tests that match their ability level
- Shorter, more efficient assessment
- Clear feedback on strengths/weaknesses
- Consistent scoring across attempts
### 2.3 Content Creators
**Profile:** Staff creating and managing question banks
**Pain Points:**
- Creating 3 difficulty variants per question is time-consuming
- Limited question pool for repeated assessments
- Manual categorization of difficulty levels
**Needs:**
- AI-assisted question generation
- Easy difficulty level adjustment
- Reuse of base questions with variant generation
- Bulk question management tools
### 2.4 Technical Administrators
**Profile:** IT staff managing the platform
**Pain Points:**
- Multiple WordPress sites with separate databases
- Difficulty scaling question pools
- Maintenance of complex scoring systems
**Needs:**
- Centralized backend for multiple sites
- Scalable architecture (AA-panel VPS)
- REST API for WordPress integration
- Automated calibration and normalization
- **Note**: Each admin manages static questions within WordPress; this project provides the backend tool for IRT/CTT calculation and dynamic question selection
---
## 3. Functional Requirements
### 3.1 CTT Scoring (Classical Test Theory)
**FR-1.1** System must calculate tingkat kesukaran (p) per question using exact client Excel formula:
```
p = Σ Benar / Total Peserta
```
**Acceptance Criteria:**
- p-value calculated per question for each tryout
- Values stored in database (items.ctt_p)
- Results match client Excel to 4 decimal places
**FR-1.2** System must calculate bobot (weight) per question:
```
Bobot = 1 - p
```
**Acceptance Criteria:**
- Bobot calculated and stored (items.ctt_bobot)
- Easy questions (p > 0.70) have low bobot (< 0.30)
- Difficult questions (p < 0.30) have high bobot (> 0.70)
**FR-1.3** System must calculate Nilai Mentah (NM) per student:
```
NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000
```
**Acceptance Criteria:**
- NM ranges 0-1000
- SUMPRODUCT equivalent implemented correctly
- Results stored per response (user_answers.ctt_nm)
**FR-1.4** System must calculate Nilai Nasional (NN) with normalization:
```
NN = 500 + 100 × ((NM - Rataan) / SB)
```
**Acceptance Criteria:**
- NN normalized to mean=500, SD=100
- Support static (hardcoded rataan/SB) and dynamic (real-time) modes
- NN clipped to 0-1000 range
**FR-1.5** System must categorize question difficulty per CTT standards:
- p < 0.30 → Sukar (Sulit)
- 0.30 ≤ p ≤ 0.70 → Sedang
- p > 0.70 → Mudah
**Acceptance Criteria:**
- Category assigned (items.ctt_category)
- Used for level field (items.level)
### 3.2 IRT Scoring (Item Response Theory)
**FR-2.1** System must implement 1PL Rasch model:
```
P(θ) = 1 / (1 + e^-(θ - b))
```
**Acceptance Criteria:**
- θ (ability) estimated per student
- b (difficulty) calibrated per question
- Ranges: θ, b ∈ [-3, +3]
**FR-2.2** System must estimate θ using Maximum Likelihood Estimation (MLE)
**Acceptance Criteria:**
- Initial guess θ = 0
- Optimization bounds [-3, +3]
- Standard error (SE) calculated using Fisher information
**FR-2.3** System must calibrate b parameters from response data
**Acceptance Criteria:**
- Minimum 100-500 responses per item for calibration
- Calibration status tracked (items.calibrated)
- Auto-convert CTT p to initial b: `b ≈ -ln((1-p)/p)`
**FR-2.4** System must map θ to NN for CTT comparison
**Acceptance Criteria:**
- θ ∈ [-3, +3] mapped to NN ∈ [0, 1000]
- Formula: `NN = 500 + (θ / 3) × 500`
- Secondary score returned in API responses
### 3.3 Hybrid Mode
**FR-3.1** System must support dual scoring (CTT + IRT parallel)
**Acceptance Criteria:**
- Both scores calculated per response
- Primary/secondary score returned
- Admin can choose which to display
**FR-3.2** System must support hybrid item selection
**Acceptance Criteria:**
- First N items: fixed order (CTT mode)
- Remaining items: adaptive (IRT mode)
- Configurable transition point (tryout_config.hybrid_transition_slot)
**FR-3.3** System must support hybrid normalization
**Acceptance Criteria:**
- Static mode for small samples (< threshold)
- Dynamic mode for large samples (≥ threshold)
- Configurable threshold (tryout_config.min_sample_for_dynamic)
### 3.4 Dynamic Normalization
**FR-4.1** System must maintain running statistics per tryout
**Acceptance Criteria:**
- Track: participant_count, total_nm_sum, total_nm_sq_sum
- Update on each completed session
- Stored in tryout_stats table
**FR-4.2** System must calculate real-time rataan and SB
**Acceptance Criteria:**
- Rataan = mean(all NM)
- SB = sqrt(variance(all NM))
- Updated incrementally (no full recalc)
**FR-4.3** System must support optional normalization control (manual vs automatic)
**Acceptance Criteria:**
- Admin can choose manual mode (static normalization with hardcoded values)
- Admin can choose automatic mode (dynamic normalization when sufficient data)
- When automatic selected and sufficient data reached: system handles normalization automatically
- Configurable threshold: min_sample_for_dynamic (default: 100)
- Admin can switch between manual/automatic at any time
- System displays current data readiness (participant count vs threshold)
### 3.5 AI Question Generation
**FR-5.1** System must generate question variants via OpenRouter API
**Acceptance Criteria:**
- Generate Mudah variant from Sedang base
- Generate Sulit variant from Sedang base
- Generate same-level variant from Sedang base
- Use Qwen3 Coder 480B or Llama 3.3 70B
- **1 request = 1 question** (not batch generation)
**FR-5.2** System must use standardized prompt template
**Acceptance Criteria:**
- Include context (tryout_id, slot, level)
- Include basis soal for reference (provides topic/context)
- Request 1 question with 4 options
- Include explanation
- Maintain same context, vary only difficulty level
**FR-5.3** System must implement question reuse/caching with user-level tracking
**Acceptance Criteria:**
- Check DB for existing variant before generating
- Check if student user_id already answered question at specific difficulty level
- Reuse if found (same tryout_id, slot, level)
- Generate only if cache miss OR user hasn't answered at this difficulty
**FR-5.4** System must provide admin playground for AI testing
**Acceptance Criteria:**
- Admin can request AI generation without saving to database
- Admin can re-request unlimited times until satisfied (no approval workflow)
- Preview mode shows generated question before saving
- Admin can edit content before saving
- Purpose: Build admin trust in AI quality before enabling for students
**FR-5.5** System must parse and store AI-generated questions
**Acceptance Criteria:**
- Parse stem, options, correct answer, explanation
- Store in items table with generated_by='ai'
- Link to basis_item_id
- No approval workflow required for student tests
**FR-5.6** System must support AI generation toggle
**Acceptance Criteria:**
- Global toggle to enable/disable AI generation (config.AI_generation_enabled)
- When disabled: reuse DB questions regardless of repetition
- When enabled: generate new variants if cache miss
- Admin can toggle on/off based on cost/budget
### 3.6 Item Selection
**FR-6.1** System must support fixed order selection (CTT mode)
**Acceptance Criteria:**
- Items delivered in slot order (1, 2, 3, ...)
- No adaptive logic
- Used when selection_mode='fixed'
**FR-6.2** System must support adaptive selection (IRT mode)
**Acceptance Criteria:**
- Select item where b ≈ current θ
- Prioritize calibrated items
- Use item information to maximize precision
**FR-6.3** System must support level-based selection (hybrid mode)
**Acceptance Criteria:**
- Select from specified level (Mudah/Sedang/Sulit)
- Check if level variant exists in DB
- Generate via AI if not exists
### 3.7 Excel Import
**FR-7.1** System must import from client Excel format
**Acceptance Criteria:**
- Parse answer key (Row 2, KUNCI)
- Extract calculated p-values (Row 4, data_only=True)
- Extract bobot values (Row 5)
- Import student responses (Row 6+)
**FR-7.2** System must create items from Excel import
**Acceptance Criteria:**
- Create item per question slot
- Set ctt_p, ctt_bobot, ctt_category
- Auto-calculate irt_b from ctt_p
- Set calibrated=False
**FR-7.3** System must configure tryout from Excel import
**Acceptance Criteria:**
- Create tryout_config with CTT settings
- Set normalization_mode='static' (default)
- Set static_rataan=500, static_sb=100
### 3.8 API Endpoints
**FR-8.1** System must provide Next Item endpoint
**Acceptance Criteria:**
- POST /api/v1/session/{session_id}/next_item
- Accept mode (ctt/irt/hybrid)
- Accept current_responses array
- Return item with selection_method metadata
**FR-8.2** System must provide Complete Session endpoint
**Acceptance Criteria:**
- POST /api/v1/session/{session_id}/complete
- Return primary_score (CTT or IRT)
- Return secondary_score (parallel calculation)
- Return comparison (NN difference, agreement)
**FR-8.3** System must provide Get Tryout Config endpoint
**Acceptance Criteria:**
- GET /api/v1/tryout/{tryout_id}/config
- Return scoring_mode, normalization_mode
- Return current_stats (participant_count, rataan, SB)
- Return calibration_status
**FR-8.4** System must provide Update Normalization endpoint
**Acceptance Criteria:**
- PUT /api/v1/tryout/{tryout_id}/normalization
- Accept normalization_mode update
- Accept static_rataan, static_sb overrides
- Return will_switch_to_dynamic_at threshold
### 3.9 Multi-Site Support
**FR-9.1** System must support multiple WordPress sites
**Acceptance Criteria:**
- Each site has unique website_id
- Shared backend, isolated data per site
- API responses scoped to website_id
**FR-9.2** System must support per-site configuration
**Acceptance Criteria:**
- Each (website_id, tryout_id) pair unique
- Independent tryout_config per tryout
- Independent tryout_stats per tryout
---
## 4. Non-Functional Requirements
### 4.1 Performance
**NFR-4.1.1** Next Item API response time < 500ms
**NFR-4.1.2** Complete Session API response time < 2s
**NFR-4.1.3** AI question generation < 10s (OpenRouter timeout)
**NFR-4.1.4** Support 1000 concurrent students
### 4.2 Scalability
**NFR-4.2.1** Support 10,000+ items in database
**NFR-4.2.2** Support 100,000+ student responses
**NFR-4.2.3** Question reuse: 99.9% cache hit rate after initial generation
**NFR-4.2.4** Horizontal scaling via PostgreSQL read replicas
### 4.3 Reliability
**NFR-4.3.1** 99.9% uptime for tryout periods
**NFR-4.3.2** Automatic fallback to CTT if IRT fails
**NFR-4.3.3** Database transaction consistency
**NFR-4.3.4** Graceful degradation if AI API unavailable
### 4.4 Security
**NFR-4.4.1** API authentication via WordPress tokens
**NFR-4.4.2** Website_id isolation (no cross-site data access)
**NFR-4.4.3** Rate limiting per API key
**NFR-4.4.4** Audit trail for all scoring changes
### 4.5 Compatibility
**NFR-4.5.1** 100% formula match with client Excel
**NFR-4.5.2** Non-destructive: zero data loss during transitions
**NFR-4.5.3** Reversible: can disable IRT features anytime
**NFR-4.5.4** WordPress REST API integration
### 4.6 Maintainability
**NFR-4.6.1** FastAPI Admin auto-generated UI for CRUD
**NFR-4.6.2** Alembic migrations for schema changes
**NFR-4.6.3** Comprehensive API documentation (OpenAPI)
**NFR-4.6.4** Logging for debugging scoring calculations
---
## 5. Data Requirements
### 5.1 Core Entities
#### Items
- **id**: Primary key
- **website_id, tryout_id**: Composite key for multi-site
- **slot, level**: Position and difficulty
- **stem, options, correct, explanation**: Question content
- **ctt_p, ctt_bobot, ctt_category**: CTT parameters
- **irt_b, irt_a, irt_c**: IRT parameters
- **calibrated, calibration_sample_size**: Calibration status
- **generated_by, ai_model, basis_item_id**: AI generation metadata
#### User Answers
- **id**: Primary key
- **wp_user_id, website_id, tryout_id, slot, level**: Composite key
- **item_id, response**: Question and answer
- **ctt_bobot_earned, ctt_total_bobot_cumulative, ctt_nm, ctt_nn**: CTT scores
- **rataan_used, sb_used, normalization_mode_used**: Normalization metadata
- **irt_theta, irt_theta_se, irt_information**: IRT scores
- **scoring_mode_used**: Which mode was used
#### Tryout Config
- **id**: Primary key
- **website_id, tryout_id**: Composite key
- **scoring_mode**: 'ctt', 'irt', 'hybrid'
- **selection_mode**: 'fixed', 'adaptive', 'hybrid'
- **normalization_mode**: 'static', 'dynamic', 'hybrid'
- **static_rataan, static_sb, min_sample_for_dynamic**: Normalization settings
- **min_calibration_sample, theta_estimation_method**: IRT settings
- **hybrid_transition_slot, fallback_to_ctt_on_error**: Transition settings
#### Tryout Stats
- **id**: Primary key
- **website_id, tryout_id**: Composite key
- **participant_count**: Number of completed sessions
- **total_nm_sum, total_nm_sq_sum**: Running sums for mean/SD calc
- **current_rataan, current_sb**: Calculated values
- **min_nm, max_nm**: Score range
- **last_calculated_at, last_participant_id**: Metadata
### 5.2 Data Relationships
- Items → User Answers (1:N, CASCADE delete)
- Items → Items (self-reference via basis_item_id for AI generation)
- Tryout Config → User Answers (1:N via website_id, tryout_id)
- Tryout Stats → User Answers (1:N via website_id, tryout_id)
---
## 6. Technical Constraints
### 6.1 Tech Stack (Fixed)
- **Backend**: FastAPI (Python)
- **Database**: PostgreSQL (via aaPanel PgSQL Manager)
- **ORM**: SQLAlchemy
- **Admin**: FastAPI Admin
- **AI**: OpenRouter API (Qwen3 Coder 480B, Llama 3.3 70B)
- **Deployment**: aaPanel VPS (Python Manager)
### 6.2 External Dependencies
- **OpenRouter API**: Must handle rate limits, timeouts, errors
- **WordPress**: REST API integration, authentication
- **Excel**: openpyxl for import, pandas for data processing
### 6.3 Mathematical Constraints
- **CTT**: Must use EXACT client formulas (p, bobot, NM, NN)
- **IRT**: 1PL Rasch model only (no a, c parameters initially)
- **Normalization**: Mean=500, SD=100 target
- **Ranges**: θ, b ∈ [-3, +3], NM, NN ∈ [0, 1000]
---
## 7. User Stories
### 7.1 Administrator Stories
**US-7.1.1** As an administrator, I want to import questions from Excel so that I can migrate existing content without manual entry.
- Priority: High
- Acceptance: FR-7.1, FR-7.2, FR-7.3
**US-7.1.2** As an administrator, I want to configure normalization mode (static/dynamic/hybrid) so that I can control how scores are normalized.
- Priority: High
- Acceptance: FR-4.3, FR-8.4
**US-7.1.3** As an administrator, I want to view calibration status so that I can know when IRT is ready for production.
- Priority: Medium
- Acceptance: FR-8.3
**US-7.1.4** As an administrator, I want to choose scoring mode (CTT/IRT/hybrid) so that I can gradually adopt advanced features.
- Priority: High
- Acceptance: FR-3.1, FR-3.2, FR-3.3
### 7.2 Student Stories
**US-7.2.1** As a student, I want to take adaptive tests so that I get questions matching my ability level.
- Priority: High
- Acceptance: FR-6.2, FR-2.1, FR-2.2
**US-7.2.2** As a student, I want to see my normalized score (NN) so that I can compare my performance with others.
- Priority: High
- Acceptance: FR-1.4, FR-4.2
**US-7.2.3** As a student, I want a seamless experience where any technical issues (IRT fallback, AI generation failures) are handled without interrupting my test.
- Priority: High
- Acceptance: Seamless fallback (student unaware of internal mode switching), no error messages visible to students
### 7.3 Content Creator Stories
**US-7.3.1** As a content creator, I want to generate question variants via AI so that I don't have to manually create 3 difficulty levels.
- Priority: High
- Acceptance: FR-5.1, FR-5.2, FR-5.3, FR-5.4
**US-7.3.2** As a content creator, I want to reuse existing questions with different difficulty levels so that I can maximize question pool efficiency.
- Priority: Medium
- Acceptance: FR-5.3, FR-6.3
### 7.4 Technical Administrator Stories
**US-7.4.1** As a technical administrator, I want to manage multiple WordPress sites from one backend so that I don't have to duplicate infrastructure.
- Priority: High
- Acceptance: FR-9.1, FR-9.2
**US-7.4.2** As a technical administrator, I want to monitor calibration progress so that I can plan IRT rollout.
- Priority: Medium
- Acceptance: FR-2.3, FR-8.3
**US-7.4.3** As a technical administrator, I want access to internal scoring details (CTT vs IRT comparison, normalization metrics) for debugging and monitoring, while students only see primary scores.
- Priority: Medium
- Acceptance: Admin visibility of all internal metrics, student visibility limited to final NN score only
---
## 8. Success Criteria
### 8.1 Technical Validation
- ✅ CTT scores match client Excel to 4 decimal places (100% formula accuracy)
- ✅ Dynamic normalization produces mean=500±5, SD=100±5 after 100 users
- ✅ IRT calibration covers >80% items with 500+ responses per item
- ✅ CTT vs IRT NN difference <20 points (moderate agreement)
- ✅ Fallback rate <5% (IRT → CTT on error)
### 8.2 Educational Validation
- ✅ IRT measurement precision: SE <0.5 after 15 items
- ✅ Normalization quality: Distribution skewness <0.5
- ✅ Adaptive efficiency: 30% reduction in test length (15 IRT = 30 CTT items for same precision)
- ✅ Student satisfaction: >80% prefer adaptive mode in surveys
- ✅ Admin adoption: >70% tryouts use hybrid mode within 3 months
### 8.3 Business Validation
- ✅ Zero data loss during CTT→IRT transition
- ✅ Reversible: Can disable IRT and revert to CTT anytime
- ✅ Non-destructive: Existing Excel workflow remains functional
- ✅ Cost efficiency: 99.9% question reuse vs 90,000 unique questions for 1000 users
- ✅ Multi-site scalability: One backend supports unlimited WordPress sites
---
## 9. Risk Mitigation
### 9.1 Technical Risks
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| IRT calibration fails (insufficient data) | High | Medium | Fallback to CTT mode, enable hybrid transition |
| OpenRouter API down/unavailable | Medium | Low | Cache questions, serve static variants |
| Excel formula mismatch | High | Low | Unit tests with client Excel data |
| Database performance degradation | Medium | Low | Indexing, read replicas, query optimization |
### 9.2 Business Risks
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| Administrators refuse to use IRT (too complex) | High | Medium | Hybrid mode with CTT-first UI |
| Students dislike adaptive tests | Medium | Low | A/B testing, optional mode |
| Excel workflow changes (client updates) | High | Low | Version control, flexible import parser |
| Multi-site data isolation failure | Critical | Low | Website_id validation, RBAC |
---
## 10. Migration Strategy
### 10.1 Phase 1: Import Existing Data (Week 1)
- Export current Sejoli Tryout data to Excel
- Run import script to load items and configurations
- Configure CTT mode with static normalization
- Validate: CTT scores match Excel 100%
### 10.2 Phase 2: Collect Calibration Data (Week 2-4)
- Students use tryout normally (CTT mode)
- Backend logs all responses
- Monitor calibration progress (items.calibrated status)
- Collect running statistics (tryout_stats)
### 10.3 Phase 3: Enable Dynamic Normalization (Week 5)
- Check participant count ≥ 100
- Update normalization_mode='hybrid'
- Test with 10-20 new students
- Verify: Normalized distribution has mean≈500, SD≈100
### 10.4 Phase 4: Enable IRT Adaptive (Week 6+)
- After 90% items calibrated + 1000+ responses
- Update scoring_mode='irt', selection_mode='adaptive'
- Enable AI generation for Mudah/Sulit variants
- Monitor fallback rate, measurement precision
### 10.5 Rollback Plan
- Any phase is reversible
- Revert to CTT mode if IRT issues occur
- **Score preservation**: Historical IRT scores kept as-is; CTT applied only to new sessions after rollback
- Disable AI generation if costs too high
- Revert to static normalization if dynamic unstable
---
## 11. Future Enhancements
### 11.1 Short-term (3-6 months)
- **2PL/3PL IRT**: Add discrimination (a) and guessing (c) parameters
- **Item Response Categorization**: Bloom's Taxonomy, cognitive domains
- **Advanced AI Models**: Fine-tune models for specific subjects
- **Data Retention Policy**: Define archival and anonymization strategy (currently: keep all data)
### 11.2 Long-term (6-12 months)
- **Multi-dimensional IRT**: Measure multiple skills per question
- **Automatic Item Difficulty Adjustment**: AI calibrates b parameters
- **Predictive Analytics**: Student performance forecasting
- **Integration with LMS**: Moodle, Canvas API support
---
## 12. Glossary
| Term | Definition |
|------|------------|
| **p (TK)** | Proportion correct / Tingkat Kesukaran (CTT difficulty) |
| **Bobot** | 1-p weight (CTT scoring weight) |
| **NM** | Nilai Mentah (raw score 0-1000) |
| **NN** | Nilai Nasional (normalized 500±100) |
| **Rataan** | Mean of NM scores |
| **SB** | Simpangan Baku (standard deviation of NM) |
| **θ (theta)** | IRT ability (-3 to +3) |
| **b** | IRT difficulty (-3 to +3) |
| **SE** | Standard error (precision) |
| **CAT** | Computerized Adaptive Testing |
| **MLE** | Maximum Likelihood Estimation |
| **CTT** | Classical Test Theory |
| **IRT** | Item Response Theory |
---
## 13. Appendices
### 13.1 Formula Reference
- **CTT p**: `p = Σ Benar / Total Peserta`
- **CTT Bobot**: `Bobot = 1 - p`
- **CTT NM**: `NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000`
- **CTT NN**: `NN = 500 + 100 × ((NM - Rataan) / SB)`
- **IRT 1PL**: `P(θ) = 1 / (1 + e^-(θ - b))`
- **CTT→IRT conversion**: `b ≈ -ln((1-p)/p)`
- **θ→NN mapping**: `NN = 500 + (θ / 3) × 500`
### 13.2 Difficulty Categories
| CTT p | CTT Category | Level | IRT b Range |
|-------|--------------|-------|-------------|
| p < 0.30 | Sukar | Sulit | b > 0.85 |
| 0.30 ≤ p ≤ 0.70 | Sedang | Sedang | -0.85 ≤ b ≤ 0.85 |
| p > 0.70 | Mudah | Mudah | b < -0.85 |
### 13.3 API Quick Reference
- `POST /api/v1/session/{session_id}/next_item` - Get next question
- `POST /api/v1/session/{session_id}/complete` - Submit and score
- `GET /api/v1/tryout/{tryout_id}/config` - Get configuration
- `PUT /api/v1/tryout/{tryout_id}/normalization` - Update normalization
---
## 14. Reporting Requirements
### 14.1 Student Performance Reports
**FR-14.1.1** System must provide individual student performance reports
**Acceptance Criteria:**
- Report all student sessions (CTT, IRT, hybrid)
- Include NM, NN scores per session
- Include time spent per question
- Include total_benar, total_bobot_earned
- Export to CSV/Excel
**FR-14.1.2** System must provide aggregate student performance reports
**Acceptance Criteria:**
- Group by tryout, website_id, date range
- Show average NM, NN, theta per group
- Show distribution (min, max, median, std dev)
- Show pass/fail rates
- Export to CSV/Excel
### 14.2 Item Analysis Reports
**FR-14.2.1** System must provide item difficulty reports
**Acceptance Criteria:**
- Show CTT p-value per item
- Show IRT b-parameter per item
- Show calibration status
- Show discrimination index (if available)
- Filter by difficulty category (Mudah/Sedang/Sulit)
**FR-14.2.2** System must provide item information function reports
**Acceptance Criteria:**
- Show item information value at different theta levels
- Visualize item characteristic curves (optional)
- Show optimal theta range for each item
### 14.3 Calibration Status Reports
**FR-14.3.1** System must provide calibration progress reports
**Acceptance Criteria:**
- Show total items per tryout
- Show calibrated items count and percentage
- Show items awaiting calibration
- Show average calibration sample size
- Show estimated time to reach calibration threshold
- Highlight ready-for-IRT rollout status (≥90% calibrated)
### 14.4 Tryout Comparison Reports
**FR-14.4.1** System must provide tryout comparison across dates
**Acceptance Criteria:**
- Compare NM/NN distributions across different tryout dates
- Show trends over time (e.g., monthly averages)
- Show normalization changes impact (static → dynamic)
**FR-14.4.2** System must provide tryout comparison across subjects
**Acceptance Criteria:**
- Compare performance across different subjects (Mat SD vs Bahasa SMA)
- Show subject-specific calibration status
- Show IRT accuracy differences per subject
### 14.5 Reporting Infrastructure
**FR-14.5.1** System must provide report scheduling
**Acceptance Criteria:**
- Admin can schedule daily/weekly/monthly reports
- Reports emailed to admin on schedule
- Report templates configurable (e.g., calibration status every Monday)
**FR-14.5.2** System must provide report export formats
**Acceptance Criteria:**
- Export to CSV
- Export to Excel (.xlsx)
- Export to PDF (with charts if available)
---
**Document End**
**Document Version:** 1.1
**Created:** March 21, 2026
**Updated:** March 21, 2026 (Clarifications Incorporated)
**Author:** Product Team (based on Technical Specification v1.2.0)
**Status:** Draft - Ready for Implementation
**Status:** Draft for Review

1395
TEST.md Normal file
View File

File diff suppressed because it is too large Load Diff

147
alembic.ini Normal file
View File

@@ -0,0 +1,147 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts.
# this is typically a path given in POSIX (e.g. forward slashes)
# format, relative to the token %(here)s which refers to the location of this
# ini file
script_location = %(here)s/alembic
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory. for multiple paths, the path separator
# is defined by "path_separator" below.
prepend_sys_path = .
# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =
# max length of characters to apply to the "slug" field
# truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; This defaults
# to <script_location>/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "path_separator"
# below.
# version_locations = %(here)s/bar:%(here)s/bat:%(here)s/alembic/versions
# path_separator; This indicates what character is used to split lists of file
# paths, including version_locations and prepend_sys_path within configparser
# files such as alembic.ini.
# The default rendered in new alembic.ini files is "os", which uses os.pathsep
# to provide os-dependent path splitting.
#
# Note that in order to support legacy alembic.ini files, this default does NOT
# take place if path_separator is not present in alembic.ini. If this
# option is omitted entirely, fallback logic is as follows:
#
# 1. Parsing of the version_locations option falls back to using the legacy
# "version_path_separator" key, which if absent then falls back to the legacy
# behavior of splitting on spaces and/or commas.
# 2. Parsing of the prepend_sys_path option falls back to the legacy
# behavior of splitting on spaces, commas, or colons.
#
# Valid values for path_separator are:
#
# path_separator = :
# path_separator = ;
# path_separator = space
# path_separator = newline
#
# Use os.pathsep. Default configuration used for new projects.
path_separator = os
# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
# database URL. This is consumed by the user-maintained env.py script only.
# other means of configuring database URLs may be customized within the env.py
# file.
sqlalchemy.url = postgresql+asyncpg://postgres:postgres@localhost:5432/irt_bank_soal
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples
# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME
# lint with attempts to fix using "ruff" - use the module runner, against the "ruff" module
# hooks = ruff
# ruff.type = module
# ruff.module = ruff
# ruff.options = check --fix REVISION_SCRIPT_FILENAME
# Alternatively, use the exec runner to execute a binary found on your PATH
# hooks = ruff
# ruff.type = exec
# ruff.executable = ruff
# ruff.options = check --fix REVISION_SCRIPT_FILENAME
# Logging configuration. This is also consumed by the user-maintained
# env.py script only.
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

1
alembic/README Normal file
View File

@@ -0,0 +1 @@
Generic single-database configuration.

99
alembic/env.py Normal file
View File

@@ -0,0 +1,99 @@
"""
Alembic environment configuration for async PostgreSQL migrations.
Configures Alembic to work with SQLAlchemy async engine and models.
"""
import asyncio
import sys
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
# Import models and Base
sys.path.insert(0, ".")
from app.database import Base
from app.models import * # noqa: F401, F403
# Import settings for database URL
from app.core.config import get_settings
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# Get settings and set database URL
settings = get_settings()
config.set_main_option("sqlalchemy.url", settings.DATABASE_URL)
# add your model's MetaData object here
# for 'autogenerate' support
target_metadata = Base.metadata
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = async_engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode."""
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

28
alembic/script.py.mako Normal file
View File

@@ -0,0 +1,28 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
"""Upgrade schema."""
${upgrades if upgrades else "pass"}
def downgrade() -> None:
"""Downgrade schema."""
${downgrades if downgrades else "pass"}

7
app/__init__.py Normal file
View File

@@ -0,0 +1,7 @@
"""
IRT Bank Soal - Adaptive Question Bank System
Main application package.
"""
__version__ = "1.0.0"

625
app/admin.py Normal file
View File

@@ -0,0 +1,625 @@
"""
FastAPI Admin configuration for IRT Bank Soal system.
Provides admin panel for managing tryouts, items, sessions, users, and tryout stats.
Includes custom actions for calibration, AI generation toggle, and normalization reset.
"""
from typing import Any, Dict, Optional
from fastapi import Request
from fastapi_admin.app import app as admin_app
from fastapi_admin.resources import (
Field,
Link,
Model,
)
from fastapi_admin.widgets import displays, inputs
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.database import get_db
from app.models import Item, Session, Tryout, TryoutStats, User
settings = get_settings()
# =============================================================================
# Authentication Provider
# =============================================================================
class AdminAuthProvider:
"""
Authentication provider for FastAPI Admin.
Supports two modes:
1. WordPress JWT token integration (production)
2. Basic auth for testing (development)
"""
async def login(
self,
username: str,
password: str,
) -> Optional[str]:
"""
Authenticate user and return token.
Args:
username: Username
password: Password
Returns:
Access token if authentication successful, None otherwise
"""
# Development mode: basic auth
if settings.ENVIRONMENT == "development":
# Allow admin/admin or admin/password for testing
if (username == "admin" and password in ["admin", "password"]):
return f"dev_token_{username}"
# Production mode: WordPress JWT token validation
# For now, return None - implement WordPress integration when needed
return None
async def logout(self, request: Request) -> bool:
"""
Logout user.
Args:
request: FastAPI request
Returns:
True if logout successful
"""
return True
async def get_current_user(self, request: Request) -> Optional[dict]:
"""
Get current authenticated user.
Args:
request: FastAPI request
Returns:
User data if authenticated, None otherwise
"""
token = request.cookies.get("admin_token") or request.headers.get("Authorization")
if not token:
return None
# Development mode: validate dev token
if settings.ENVIRONMENT == "development" and token.startswith("dev_token_"):
username = token.replace("dev_token_", "")
return {
"id": 1,
"username": username,
"is_superuser": True,
}
return None
# =============================================================================
# Admin Model Resources
# =============================================================================
class TryoutResource(Model):
"""
Admin resource for Tryout model.
Displays tryout configuration and provides calibration and AI generation actions.
"""
label = "Tryouts"
model = Tryout
page_size = 20
# Fields to display
fields = [
Field(name="id", label="ID", input_=inputs.Input(), display=displays.Display()),
Field(name="website_id", label="Website ID", input_=inputs.Input(), display=displays.Display()),
Field(name="tryout_id", label="Tryout ID", input_=inputs.Input(), display=displays.Display()),
Field(name="name", label="Name", input_=inputs.Input(), display=displays.Display()),
Field(
name="description",
label="Description",
input_=inputs.TextArea(),
display=displays.Display(),
),
Field(
name="scoring_mode",
label="Scoring Mode",
input_=inputs.Select(options=["ctt", "irt", "hybrid"], default="ctt"),
display=displays.Select(choices=["ctt", "irt", "hybrid"]),
),
Field(
name="selection_mode",
label="Selection Mode",
input_=inputs.Select(options=["fixed", "adaptive", "hybrid"], default="fixed"),
display=displays.Select(choices=["fixed", "adaptive", "hybrid"]),
),
Field(
name="normalization_mode",
label="Normalization Mode",
input_=inputs.Select(options=["static", "dynamic", "hybrid"], default="static"),
display=displays.Select(choices=["static", "dynamic", "hybrid"]),
),
Field(
name="min_sample_for_dynamic",
label="Min Sample for Dynamic",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="static_rataan",
label="Static Mean (Rataan)",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="static_sb",
label="Static Std Dev (SB)",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="ai_generation_enabled",
label="Enable AI Generation",
input_=inputs.Switch(),
display=displays.Boolean(true_text="Enabled", false_text="Disabled"),
),
Field(
name="hybrid_transition_slot",
label="Hybrid Transition Slot",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="min_calibration_sample",
label="Min Calibration Sample",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="theta_estimation_method",
label="Theta Estimation Method",
input_=inputs.Select(options=["mle", "map", "eap"], default="mle"),
display=displays.Select(choices=["mle", "map", "eap"]),
),
Field(
name="fallback_to_ctt_on_error",
label="Fallback to CTT on Error",
input_=inputs.Switch(),
display=displays.Boolean(true_text="Yes", false_text="No"),
),
Field(name="created_at", label="Created At", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="updated_at", label="Updated At", input_=inputs.DateTime(), display=displays.DateTime()),
]
class ItemResource(Model):
"""
Admin resource for Item model.
Displays items with CTT and IRT parameters, and calibration status.
"""
label = "Items"
model = Item
page_size = 50
# Fields to display
fields = [
Field(name="id", label="ID", input_=inputs.Input(), display=displays.Display()),
Field(name="tryout_id", label="Tryout ID", input_=inputs.Input(), display=displays.Display()),
Field(name="website_id", label="Website ID", input_=inputs.Input(), display=displays.Display()),
Field(name="slot", label="Slot", input_=inputs.Input(type="number"), display=displays.Display()),
Field(
name="level",
label="Difficulty Level",
input_=inputs.Select(options=["mudah", "sedang", "sulit"], default="sedang"),
display=displays.Display(),
),
Field(
name="stem",
label="Question Stem",
input_=inputs.TextArea(),
display=displays.Text(maxlen=100),
),
Field(name="options", label="Options", input_=inputs.Json(), display=displays.Json()),
Field(name="correct_answer", label="Correct Answer", input_=inputs.Input(), display=displays.Display()),
Field(
name="explanation",
label="Explanation",
input_=inputs.TextArea(),
display=displays.Text(maxlen=100),
),
Field(
name="ctt_p",
label="CTT p-value",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="ctt_bobot",
label="CTT Bobot",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="ctt_category",
label="CTT Category",
input_=inputs.Select(options=["mudah", "sedang", "sulit"]),
display=displays.Display(),
),
Field(
name="irt_b",
label="IRT b-parameter",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="irt_se",
label="IRT SE",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="calibrated",
label="Calibrated",
input_=inputs.Switch(),
display=displays.Boolean(true_text="Yes", false_text="No"),
),
Field(
name="calibration_sample_size",
label="Calibration Sample Size",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="generated_by",
label="Generated By",
input_=inputs.Select(options=["manual", "ai"], default="manual"),
display=displays.Display(),
),
Field(name="ai_model", label="AI Model", input_=inputs.Input(), display=displays.Display()),
Field(
name="basis_item_id",
label="Basis Item ID",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(name="created_at", label="Created At", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="updated_at", label="Updated At", input_=inputs.DateTime(), display=displays.DateTime()),
]
class UserResource(Model):
"""
Admin resource for User model.
Displays WordPress users and their tryout sessions.
"""
label = "Users"
model = User
page_size = 50
# Fields
fields = [
Field(name="id", label="ID", input_=inputs.Input(), display=displays.Display()),
Field(name="wp_user_id", label="WordPress User ID", input_=inputs.Input(), display=displays.Display()),
Field(name="website_id", label="Website ID", input_=inputs.Input(), display=displays.Display()),
Field(name="created_at", label="Created At", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="updated_at", label="Updated At", input_=inputs.DateTime(), display=displays.DateTime()),
]
class SessionResource(Model):
"""
Admin resource for Session model.
Displays tryout sessions with scoring results (NM, NN, theta).
"""
label = "Sessions"
model = Session
page_size = 50
# Fields
fields = [
Field(name="id", label="ID", input_=inputs.Input(), display=displays.Display()),
Field(name="session_id", label="Session ID", input_=inputs.Input(), display=displays.Display()),
Field(name="wp_user_id", label="WordPress User ID", input_=inputs.Input(), display=displays.Display()),
Field(name="website_id", label="Website ID", input_=inputs.Input(), display=displays.Display()),
Field(name="tryout_id", label="Tryout ID", input_=inputs.Input(), display=displays.Display()),
Field(name="start_time", label="Start Time", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="end_time", label="End Time", input_=inputs.DateTime(), display=displays.DateTime()),
Field(
name="is_completed",
label="Completed",
input_=inputs.Switch(),
display=displays.Boolean(true_text="Yes", false_text="No"),
),
Field(
name="scoring_mode_used",
label="Scoring Mode Used",
input_=inputs.Select(options=["ctt", "irt", "hybrid"]),
display=displays.Display(),
),
Field(name="total_benar", label="Total Benar", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="total_bobot_earned", label="Total Bobot Earned", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="NM", label="NM Score", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="NN", label="NN Score", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="theta", label="Theta", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="theta_se", label="Theta SE", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="rataan_used", label="Rataan Used", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="sb_used", label="SB Used", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="created_at", label="Created At", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="updated_at", label="Updated At", input_=inputs.DateTime(), display=displays.DateTime()),
]
class TryoutStatsResource(Model):
"""
Admin resource for TryoutStats model.
Displays tryout-level statistics and provides normalization reset action.
"""
label = "Tryout Stats"
model = TryoutStats
page_size = 20
# Fields
fields = [
Field(name="id", label="ID", input_=inputs.Input(), display=displays.Display()),
Field(name="website_id", label="Website ID", input_=inputs.Input(), display=displays.Display()),
Field(name="tryout_id", label="Tryout ID", input_=inputs.Input(), display=displays.Display()),
Field(
name="participant_count",
label="Participant Count",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="total_nm_sum",
label="Total NM Sum",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(
name="total_nm_sq_sum",
label="Total NM Squared Sum",
input_=inputs.Input(type="number"),
display=displays.Display(),
),
Field(name="rataan", label="Rataan", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="sb", label="SB", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="min_nm", label="Min NM", input_=inputs.Input(type="number"), display=displays.Display()),
Field(name="max_nm", label="Max NM", input_=inputs.Input(type="number"), display=displays.Display()),
Field(
name="last_calculated",
label="Last Calculated",
input_=inputs.DateTime(),
display=displays.DateTime(),
),
Field(name="created_at", label="Created At", input_=inputs.DateTime(), display=displays.DateTime()),
Field(name="updated_at", label="Updated At", input_=inputs.DateTime(), display=displays.DateTime()),
]
# =============================================================================
# Custom Dashboard Views
# =============================================================================
class CalibrationDashboardLink(Link):
"""
Link to calibration status dashboard.
Displays calibration percentage and items awaiting calibration.
"""
label = "Calibration Status"
icon = "fas fa-chart-line"
url = "/admin/calibration_status"
async def get(self, request: Request) -> Dict[str, Any]:
"""Get calibration status for all tryouts."""
# Get all tryouts
db_gen = get_db()
db = await db_gen.__anext__()
try:
result = await db.execute(
select(
Tryout.id,
Tryout.tryout_id,
Tryout.name,
)
)
tryouts = result.all()
calibration_data = []
for tryout_id, tryout_str, name in tryouts:
# Get calibration status
from app.services.irt_calibration import get_calibration_status
status = await get_calibration_status(tryout_str, 1, db)
calibration_data.append({
"tryout_id": tryout_str,
"name": name,
"total_items": status["total_items"],
"calibrated_items": status["calibrated_items"],
"calibration_percentage": status["calibration_percentage"],
"ready_for_irt": status["ready_for_irt"],
})
return {
"status": "success",
"data": calibration_data,
}
finally:
await db_gen.aclose()
class ItemStatisticsLink(Link):
"""
Link to item statistics view.
Displays items grouped by difficulty level with calibration status.
"""
label = "Item Statistics"
icon = "fas fa-chart-bar"
url = "/admin/item_statistics"
async def get(self, request: Request) -> Dict[str, Any]:
"""Get item statistics grouped by difficulty level."""
db_gen = get_db()
db = await db_gen.__anext__()
try:
# Get items grouped by level
result = await db.execute(
select(
Item.level,
)
.distinct()
)
levels = result.scalars().all()
stats = []
for level in levels:
# Get items for this level
item_result = await db.execute(
select(Item)
.where(Item.level == level)
.order_by(Item.slot)
.limit(10)
)
items = item_result.scalars().all()
# Calculate average correctness rate
total_responses = sum(item.calibration_sample_size for item in items)
calibrated_count = sum(1 for item in items if item.calibrated)
level_stats = {
"level": level,
"total_items": len(items),
"calibrated_items": calibrated_count,
"calibration_percentage": (calibrated_count / len(items) * 100) if len(items) > 0 else 0,
"total_responses": total_responses,
"avg_correctness": sum(item.ctt_p or 0 for item in items) / len(items) if len(items) > 0 else 0,
"items": [
{
"id": item.id,
"slot": item.slot,
"calibrated": item.calibrated,
"ctt_p": item.ctt_p,
"irt_b": item.irt_b,
"calibration_sample_size": item.calibration_sample_size,
}
for item in items
],
}
stats.append(level_stats)
return {
"status": "success",
"data": stats,
}
finally:
await db_gen.aclose()
class SessionOverviewLink(Link):
"""
Link to session overview view.
Displays sessions with scores (NM, NN, theta) and completion status.
"""
label = "Session Overview"
icon = "fas fa-users"
url = "/admin/session_overview"
async def get(self, request: Request) -> Dict[str, Any]:
"""Get session overview with filters."""
db_gen = get_db()
db = await db_gen.__anext__()
try:
# Get recent sessions
result = await db.execute(
select(Session)
.order_by(Session.created_at.desc())
.limit(50)
)
sessions = result.scalars().all()
session_data = [
{
"session_id": session.session_id,
"wp_user_id": session.wp_user_id,
"tryout_id": session.tryout_id,
"is_completed": session.is_completed,
"scoring_mode_used": session.scoring_mode_used,
"total_benar": session.total_benar,
"NM": session.NM,
"NN": session.NN,
"theta": session.theta,
"theta_se": session.theta_se,
"start_time": session.start_time.isoformat() if session.start_time else None,
"end_time": session.end_time.isoformat() if session.end_time else None,
}
for session in sessions
]
return {
"status": "success",
"data": session_data,
}
finally:
await db_gen.aclose()
# =============================================================================
# Initialize FastAPI Admin
# =============================================================================
def create_admin_app() -> Any:
"""
Create and configure FastAPI Admin application.
Returns:
FastAPI app with admin panel
"""
# Configure admin app
admin_app.settings.logo_url = "/static/logo.png"
admin_app.settings.site_title = "IRT Bank Soal Admin"
admin_app.settings.site_description = "Admin Panel for Adaptive Question Bank System"
# Register authentication provider
admin_app.settings.auth_provider = AdminAuthProvider()
# Register model resources
admin_app.register(TryoutResource)
admin_app.register(ItemResource)
admin_app.register(UserResource)
admin_app.register(SessionResource)
admin_app.register(TryoutStatsResource)
# Register dashboard links
admin_app.register(CalibrationDashboardLink)
admin_app.register(ItemStatisticsLink)
admin_app.register(SessionOverviewLink)
return admin_app
# Export admin app for mounting in main.py
admin = create_admin_app()

5
app/api/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""
API module for IRT Bank Soal.
Contains FastAPI routers and endpoint definitions.
"""

25
app/api/v1/__init__.py Normal file
View File

@@ -0,0 +1,25 @@
"""
API v1 Router configuration.
Defines all API v1 endpoints and their prefixes.
"""
from fastapi import APIRouter
from app.api.v1 import session
api_router = APIRouter()
# Include session endpoints
api_router.include_router(
session.router,
prefix="/session",
tags=["session"]
)
# Include admin endpoints
api_router.include_router(
session.admin_router,
prefix="/admin",
tags=["admin"]
)

388
app/api/v1/session.py Normal file
View File

@@ -0,0 +1,388 @@
"""
Session API endpoints for CAT item selection.
Provides endpoints for:
- GET /api/v1/session/{session_id}/next_item - Get next question
- POST /api/v1/admin/cat/test - Admin playground for testing CAT
"""
from typing import Literal, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Item, Session, Tryout
from app.services.cat_selection import (
CATSelectionError,
get_next_item,
should_terminate,
simulate_cat_selection,
update_theta,
)
# Default SE threshold for termination
DEFAULT_SE_THRESHOLD = 0.5
# Session router for student-facing endpoints
router = APIRouter()
# Admin router for admin-only endpoints
admin_router = APIRouter()
# ============== Request/Response Models ==============
class NextItemResponse(BaseModel):
"""Response for next item endpoint."""
status: Literal["item", "completed"] = "item"
item_id: Optional[int] = None
stem: Optional[str] = None
options: Optional[dict] = None
slot: Optional[int] = None
level: Optional[str] = None
selection_method: Optional[str] = None
reason: Optional[str] = None
current_theta: Optional[float] = None
current_se: Optional[float] = None
items_answered: Optional[int] = None
class SubmitAnswerRequest(BaseModel):
"""Request for submitting an answer."""
item_id: int = Field(..., description="Item ID being answered")
response: str = Field(..., description="User's answer (A, B, C, D)")
time_spent: int = Field(default=0, ge=0, description="Time spent on question (seconds)")
class SubmitAnswerResponse(BaseModel):
"""Response for submitting an answer."""
is_correct: bool
correct_answer: str
explanation: Optional[str] = None
theta: Optional[float] = None
theta_se: Optional[float] = None
class CATTestRequest(BaseModel):
"""Request for admin CAT test endpoint."""
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
initial_theta: float = Field(default=0.0, ge=-3.0, le=3.0, description="Initial theta value")
selection_mode: Literal["fixed", "adaptive", "hybrid"] = Field(
default="adaptive", description="Selection mode"
)
max_items: int = Field(default=15, ge=1, le=100, description="Maximum items to simulate")
se_threshold: float = Field(
default=0.5, ge=0.1, le=3.0, description="SE threshold for termination"
)
hybrid_transition_slot: int = Field(
default=10, ge=1, description="Slot to transition in hybrid mode"
)
class CATTestResponse(BaseModel):
"""Response for admin CAT test endpoint."""
tryout_id: str
website_id: int
initial_theta: float
selection_mode: str
total_items: int
final_theta: float
final_se: float
se_threshold_met: bool
items: list
# ============== Session Endpoints ==============
@router.get(
"/{session_id}/next_item",
response_model=NextItemResponse,
summary="Get next item for session",
description="Returns the next question for a session based on the tryout's selection mode."
)
async def get_next_item_endpoint(
session_id: str,
db: AsyncSession = Depends(get_db)
) -> NextItemResponse:
"""
Get the next item for a session.
Validates session exists and is not completed.
Gets Tryout config (scoring_mode, selection_mode, max_items).
Calls appropriate selection function based on selection_mode.
Returns item or completion status.
"""
# Get session
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Session {session_id} not found"
)
if session.is_completed:
return NextItemResponse(
status="completed",
reason="Session already completed"
)
# Get tryout config
tryout_query = select(Tryout).where(
Tryout.tryout_id == session.tryout_id,
Tryout.website_id == session.website_id
)
tryout_result = await db.execute(tryout_query)
tryout = tryout_result.scalar_one_or_none()
if not tryout:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {session.tryout_id} not found"
)
# Check termination conditions
termination = await should_terminate(
db,
session_id,
max_items=None, # Will be set from tryout config if needed
se_threshold=DEFAULT_SE_THRESHOLD
)
if termination.should_terminate:
return NextItemResponse(
status="completed",
reason=termination.reason,
current_theta=session.theta,
current_se=session.theta_se,
items_answered=termination.items_answered
)
# Get next item based on selection mode
try:
result = await get_next_item(
db,
session_id,
selection_mode=tryout.selection_mode,
hybrid_transition_slot=tryout.hybrid_transition_slot or 10,
ai_generation_enabled=tryout.ai_generation_enabled
)
except CATSelectionError as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=str(e)
)
if result.item is None:
return NextItemResponse(
status="completed",
reason=result.reason,
current_theta=session.theta,
current_se=session.theta_se,
items_answered=termination.items_answered
)
item = result.item
return NextItemResponse(
status="item",
item_id=item.id,
stem=item.stem,
options=item.options,
slot=item.slot,
level=item.level,
selection_method=result.selection_method,
reason=result.reason,
current_theta=session.theta,
current_se=session.theta_se,
items_answered=termination.items_answered
)
@router.post(
"/{session_id}/submit_answer",
response_model=SubmitAnswerResponse,
summary="Submit answer for item",
description="Submit an answer for an item and update theta estimate."
)
async def submit_answer_endpoint(
session_id: str,
request: SubmitAnswerRequest,
db: AsyncSession = Depends(get_db)
) -> SubmitAnswerResponse:
"""
Submit an answer for an item.
Validates session and item.
Checks correctness.
Updates theta estimate.
Records response time.
"""
# Get session
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Session {session_id} not found"
)
if session.is_completed:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Session already completed"
)
# Get item
item_query = select(Item).where(Item.id == request.item_id)
item_result = await db.execute(item_query)
item = item_result.scalar_one_or_none()
if not item:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Item {request.item_id} not found"
)
# Check correctness
is_correct = request.response.upper() == item.correct_answer.upper()
# Update theta
theta, theta_se = await update_theta(db, session_id, request.item_id, is_correct)
# Create user answer record
from app.models import UserAnswer
user_answer = UserAnswer(
session_id=session_id,
wp_user_id=session.wp_user_id,
website_id=session.website_id,
tryout_id=session.tryout_id,
item_id=request.item_id,
response=request.response.upper(),
is_correct=is_correct,
time_spent=request.time_spent,
scoring_mode_used=session.scoring_mode_used,
bobot_earned=item.ctt_bobot if is_correct and item.ctt_bobot else 0.0
)
db.add(user_answer)
await db.commit()
return SubmitAnswerResponse(
is_correct=is_correct,
correct_answer=item.correct_answer,
explanation=item.explanation,
theta=theta,
theta_se=theta_se
)
# ============== Admin Endpoints ==============
@admin_router.post(
"/cat/test",
response_model=CATTestResponse,
summary="Test CAT selection algorithm",
description="Admin playground for testing adaptive selection behavior."
)
async def test_cat_endpoint(
request: CATTestRequest,
db: AsyncSession = Depends(get_db)
) -> CATTestResponse:
"""
Test CAT selection algorithm.
Simulates CAT selection for a tryout and returns
the sequence of selected items with theta progression.
"""
# Verify tryout exists
tryout_query = select(Tryout).where(
Tryout.tryout_id == request.tryout_id,
Tryout.website_id == request.website_id
)
tryout_result = await db.execute(tryout_query)
tryout = tryout_result.scalar_one_or_none()
if not tryout:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {request.tryout_id} not found for website {request.website_id}"
)
# Run simulation
result = await simulate_cat_selection(
db,
tryout_id=request.tryout_id,
website_id=request.website_id,
initial_theta=request.initial_theta,
selection_mode=request.selection_mode,
max_items=request.max_items,
se_threshold=request.se_threshold,
hybrid_transition_slot=request.hybrid_transition_slot
)
if "error" in result:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=result["error"]
)
return CATTestResponse(**result)
@admin_router.get(
"/session/{session_id}/status",
summary="Get session status",
description="Get detailed session status including theta and SE."
)
async def get_session_status_endpoint(
session_id: str,
db: AsyncSession = Depends(get_db)
) -> dict:
"""
Get session status for admin monitoring.
"""
# Get session
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Session {session_id} not found"
)
# Count answers
from sqlalchemy import func
from app.models import UserAnswer
count_query = select(func.count(UserAnswer.id)).where(
UserAnswer.session_id == session_id
)
count_result = await db.execute(count_query)
items_answered = count_result.scalar() or 0
return {
"session_id": session.session_id,
"wp_user_id": session.wp_user_id,
"tryout_id": session.tryout_id,
"is_completed": session.is_completed,
"theta": session.theta,
"theta_se": session.theta_se,
"items_answered": items_answered,
"scoring_mode_used": session.scoring_mode_used,
"NM": session.NM,
"NN": session.NN,
"start_time": session.start_time.isoformat() if session.start_time else None,
"end_time": session.end_time.isoformat() if session.end_time else None
}

3
app/core/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Core configuration and database utilities.
"""

115
app/core/config.py Normal file
View File

@@ -0,0 +1,115 @@
"""
Application configuration using Pydantic Settings.
Loads configuration from environment variables with validation.
"""
from typing import Literal, List, Union
from pydantic import Field, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
"""Application settings loaded from environment variables."""
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
case_sensitive=False,
)
# Database
DATABASE_URL: str = Field(
default="postgresql+asyncpg://postgres:postgres@localhost:5432/irt_bank_soal",
description="PostgreSQL database URL with asyncpg driver",
)
# FastAPI
SECRET_KEY: str = Field(
default="dev-secret-key-change-in-production",
description="Secret key for JWT token signing",
)
API_V1_STR: str = Field(default="/api/v1", description="API v1 prefix")
PROJECT_NAME: str = Field(default="IRT Bank Soal", description="Project name")
ENVIRONMENT: Literal["development", "staging", "production"] = Field(
default="development", description="Environment name"
)
# OpenRouter (AI Generation)
OPENROUTER_API_KEY: str = Field(
default="", description="OpenRouter API key for AI generation"
)
OPENROUTER_MODEL_QWEN: str = Field(
default="qwen/qwen-2.5-coder-32b-instruct",
description="Qwen model identifier",
)
OPENROUTER_MODEL_LLAMA: str = Field(
default="meta-llama/llama-3.3-70b-instruct",
description="Llama model identifier",
)
OPENROUTER_TIMEOUT: int = Field(default=30, description="OpenRouter API timeout in seconds")
# WordPress Integration
WORDPRESS_API_URL: str = Field(
default="", description="WordPress REST API base URL"
)
WORDPRESS_AUTH_TOKEN: str = Field(
default="", description="WordPress JWT authentication token"
)
# Redis (Celery)
REDIS_URL: str = Field(
default="redis://localhost:6379/0", description="Redis connection URL"
)
CELERY_BROKER_URL: str = Field(
default="redis://localhost:6379/0", description="Celery broker URL"
)
CELERY_RESULT_BACKEND: str = Field(
default="redis://localhost:6379/0", description="Celery result backend URL"
)
# CORS - stored as list, accepts comma-separated string from env
ALLOWED_ORIGINS: List[str] = Field(
default=["http://localhost:3000"],
description="List of allowed CORS origins",
)
@field_validator("ALLOWED_ORIGINS", mode="before")
@classmethod
def parse_allowed_origins(cls, v: Union[str, List[str]]) -> List[str]:
"""Parse comma-separated origins into list."""
if isinstance(v, str):
return [origin.strip() for origin in v.split(",") if origin.strip()]
return v
# Global settings instance
_settings: Union[Settings, None] = None
def get_settings() -> Settings:
"""
Get application settings instance.
Returns:
Settings: Application settings
Raises:
ValueError: If settings not initialized
"""
global _settings
if _settings is None:
_settings = Settings()
return _settings
def init_settings(settings: Settings) -> None:
"""
Initialize settings with custom instance (useful for testing).
Args:
settings: Settings instance to use
"""
global _settings
_settings = settings

85
app/database.py Normal file
View File

@@ -0,0 +1,85 @@
"""
Database configuration and session management for async PostgreSQL.
Uses SQLAlchemy 2.0 async ORM with asyncpg driver.
"""
from typing import AsyncGenerator
from sqlalchemy.ext.asyncio import (
AsyncSession,
async_sessionmaker,
create_async_engine,
)
from sqlalchemy.orm import DeclarativeBase
from app.core.config import get_settings
settings = get_settings()
# Create async engine with connection pooling
engine = create_async_engine(
settings.DATABASE_URL,
echo=settings.ENVIRONMENT == "development", # Log SQL in development
pool_pre_ping=True, # Verify connections before using
pool_size=10, # Number of connections to maintain
max_overflow=20, # Max additional connections beyond pool_size
pool_recycle=3600, # Recycle connections after 1 hour
)
# Create async session factory
AsyncSessionLocal = async_sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False, # Prevent attributes from being expired after commit
autocommit=False,
autoflush=False,
)
class Base(DeclarativeBase):
"""Base class for all database models."""
pass
async def get_db() -> AsyncGenerator[AsyncSession, None]:
"""
Dependency for getting async database session.
Yields:
AsyncSession: Database session
Example:
```python
@app.get("/items/")
async def get_items(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(Item))
return result.scalars().all()
```
"""
async with AsyncSessionLocal() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
finally:
await session.close()
async def init_db() -> None:
"""
Initialize database - create all tables.
Note: In production, use Alembic migrations instead.
This is useful for development and testing.
"""
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
async def close_db() -> None:
"""Close database connections."""
await engine.dispose()

204
app/main.py Normal file
View File

@@ -0,0 +1,204 @@
"""
IRT Bank Soal - Adaptive Question Bank System
Main FastAPI application entry point.
Features:
- CTT (Classical Test Theory) scoring with exact Excel formulas
- IRT (Item Response Theory) support for adaptive testing
- Multi-website support for WordPress integration
- AI-powered question generation
"""
from contextlib import asynccontextmanager
from typing import AsyncGenerator
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.admin import admin as admin_app
from app.core.config import get_settings
from app.database import close_db, init_db
from app.routers import (
admin_router,
ai_router,
import_export_router,
reports_router,
sessions_router,
tryouts_router,
wordpress_router,
)
settings = get_settings()
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
"""
Application lifespan manager.
Handles startup and shutdown events.
"""
# Startup: Initialize database
await init_db()
yield
# Shutdown: Close database connections
await close_db()
# Initialize FastAPI application
app = FastAPI(
title="IRT Bank Soal",
description="""
## Adaptive Question Bank System with IRT/CTT Scoring
This API provides a comprehensive backend for adaptive assessment systems.
### Features
- **CTT Scoring**: Classical Test Theory with exact Excel formula compatibility
- **IRT Support**: Item Response Theory for adaptive testing (1PL Rasch model)
- **Multi-Site**: Single backend serving multiple WordPress sites
- **AI Generation**: Automatic question variant generation
### Scoring Formulas (PRD Section 13.1)
- **CTT p-value**: `p = Σ Benar / Total Peserta`
- **CTT Bobot**: `Bobot = 1 - p`
- **CTT NM**: `NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000`
- **CTT NN**: `NN = 500 + 100 × ((NM - Rataan) / SB)`
### Authentication
Most endpoints require `X-Website-ID` header for multi-site isolation.
""",
version="1.0.0",
docs_url="/docs",
redoc_url="/redoc",
openapi_url="/openapi.json",
lifespan=lifespan,
)
# Configure CORS middleware
# Parse ALLOWED_ORIGINS from settings (comma-separated string)
allowed_origins = settings.ALLOWED_ORIGINS
if isinstance(allowed_origins, str):
allowed_origins = [origin.strip() for origin in allowed_origins.split(",") if origin.strip()]
app.add_middleware(
CORSMiddleware,
allow_origins=allowed_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Health check endpoint
@app.get(
"/",
summary="Health check",
description="Returns API status and version information.",
tags=["health"],
)
async def root():
"""
Health check endpoint.
Returns basic API information for monitoring and load balancer checks.
"""
return {
"status": "healthy",
"service": "IRT Bank Soal",
"version": "1.0.0",
"docs": "/docs",
}
@app.get(
"/health",
summary="Detailed health check",
description="Returns detailed health status including database connectivity.",
tags=["health"],
)
async def health_check():
"""
Detailed health check endpoint.
Includes database connectivity verification.
"""
from app.database import engine
from sqlalchemy import text
db_status = "unknown"
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
db_status = "connected"
except Exception as e:
db_status = f"error: {str(e)}"
return {
"status": "healthy" if db_status == "connected" else "degraded",
"service": "IRT Bank Soal",
"version": "1.0.0",
"database": db_status,
"environment": settings.ENVIRONMENT,
}
# Include API routers with version prefix
app.include_router(
import_export_router,
)
app.include_router(
sessions_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
tryouts_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
wordpress_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
ai_router,
prefix=f"{settings.API_V1_STR}",
)
app.include_router(
reports_router,
prefix=f"{settings.API_V1_STR}",
)
# Mount FastAPI Admin panel
app.mount("/admin", admin_app)
# Include admin API router for custom actions
app.include_router(
admin_router,
prefix=f"{settings.API_V1_STR}",
)
# Placeholder routers for future implementation
# These will be implemented in subsequent phases
# app.include_router(
# items_router,
# prefix=f"{settings.API_V1_STR}",
# tags=["items"],
# )
if __name__ == "__main__":
import uvicorn
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=8000,
reload=settings.ENVIRONMENT == "development",
)

25
app/models/__init__.py Normal file
View File

@@ -0,0 +1,25 @@
"""
Database models for IRT Bank Soal system.
Exports all SQLAlchemy ORM models for use in the application.
"""
from app.database import Base
from app.models.item import Item
from app.models.session import Session
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.user import User
from app.models.user_answer import UserAnswer
from app.models.website import Website
__all__ = [
"Base",
"User",
"Website",
"Tryout",
"Item",
"Session",
"UserAnswer",
"TryoutStats",
]

222
app/models/item.py Normal file
View File

@@ -0,0 +1,222 @@
"""
Item model for questions with CTT and IRT parameters.
Represents individual questions with both classical test theory (CTT)
and item response theory (IRT) parameters.
"""
from datetime import datetime
from typing import Literal, Union
from sqlalchemy import (
Boolean,
CheckConstraint,
DateTime,
Float,
ForeignKey,
Index,
Integer,
JSON,
String,
Text,
)
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Item(Base):
"""
Item model representing individual questions.
Supports both CTT (p, bobot, category) and IRT (b, se) parameters.
Tracks AI generation metadata and calibration status.
Attributes:
id: Primary key
tryout_id: Tryout identifier
website_id: Website identifier
slot: Question position in tryout
level: Difficulty level (mudah, sedang, sulit)
stem: Question text
options: JSON array of answer options
correct_answer: Correct option (A, B, C, D)
explanation: Answer explanation
ctt_p: CTT difficulty (proportion correct)
ctt_bobot: CTT weight (1 - p)
ctt_category: CTT difficulty category
irt_b: IRT difficulty parameter [-3, +3]
irt_se: IRT standard error
calibrated: Calibration status
calibration_sample_size: Sample size for calibration
generated_by: Generation source (manual, ai)
ai_model: AI model used (if generated by AI)
basis_item_id: Original item ID (for AI variants)
created_at: Record creation timestamp
updated_at: Record update timestamp
tryout: Tryout relationship
user_answers: User responses to this item
"""
__tablename__ = "items"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Foreign keys
tryout_id: Mapped[str] = mapped_column(
String(255), nullable=False, index=True, comment="Tryout identifier"
)
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
# Position and difficulty
slot: Mapped[int] = mapped_column(
Integer, nullable=False, comment="Question position in tryout"
)
level: Mapped[Literal["mudah", "sedang", "sulit"]] = mapped_column(
String(50), nullable=False, comment="Difficulty level"
)
# Question content
stem: Mapped[str] = mapped_column(Text, nullable=False, comment="Question text")
options: Mapped[dict] = mapped_column(
JSON,
nullable=False,
comment="JSON object with options (e.g., {\"A\": \"option1\", \"B\": \"option2\"})",
)
correct_answer: Mapped[str] = mapped_column(
String(10), nullable=False, comment="Correct option (A, B, C, D)"
)
explanation: Mapped[Union[str, None]] = mapped_column(
Text, nullable=True, comment="Answer explanation"
)
# CTT parameters
ctt_p: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="CTT difficulty (proportion correct)",
)
ctt_bobot: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="CTT weight (1 - p)",
)
ctt_category: Mapped[Union[Literal["mudah", "sedang", "sulit"], None]] = mapped_column(
String(50),
nullable=True,
comment="CTT difficulty category",
)
# IRT parameters (1PL Rasch model)
irt_b: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="IRT difficulty parameter [-3, +3]",
)
irt_se: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="IRT standard error",
)
# Calibration status
calibrated: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, comment="Calibration status"
)
calibration_sample_size: Mapped[int] = mapped_column(
Integer,
nullable=False,
default=0,
comment="Sample size for calibration",
)
# AI generation metadata
generated_by: Mapped[Literal["manual", "ai"]] = mapped_column(
String(50),
nullable=False,
default="manual",
comment="Generation source",
)
ai_model: Mapped[Union[str, None]] = mapped_column(
String(255),
nullable=True,
comment="AI model used (if generated by AI)",
)
basis_item_id: Mapped[Union[int, None]] = mapped_column(
ForeignKey("items.id", ondelete="SET NULL", onupdate="CASCADE"),
nullable=True,
index=True,
comment="Original item ID (for AI variants)",
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
tryout: Mapped["Tryout"] = relationship(
"Tryout", back_populates="items", lazy="selectin"
)
user_answers: Mapped[list["UserAnswer"]] = relationship(
"UserAnswer", back_populates="item", lazy="selectin", cascade="all, delete-orphan"
)
basis_item: Mapped[Union["Item", None]] = relationship(
"Item",
remote_side=[id],
back_populates="variants",
lazy="selectin",
single_parent=True,
)
variants: Mapped[list["Item"]] = relationship(
"Item",
back_populates="basis_item",
lazy="selectin",
cascade="all, delete-orphan",
)
# Constraints and indexes
__table_args__ = (
Index(
"ix_items_tryout_id_website_id_slot",
"tryout_id",
"website_id",
"slot",
"level",
unique=True,
),
Index("ix_items_calibrated", "calibrated"),
Index("ix_items_basis_item_id", "basis_item_id"),
# IRT b parameter constraint [-3, +3]
CheckConstraint(
"irt_b IS NULL OR (irt_b >= -3 AND irt_b <= 3)",
"ck_irt_b_range",
),
# CTT p constraint [0, 1]
CheckConstraint(
"ctt_p IS NULL OR (ctt_p >= 0 AND ctt_p <= 1)",
"ck_ctt_p_range",
),
# CTT bobot constraint [0, 1]
CheckConstraint(
"ctt_bobot IS NULL OR (ctt_bobot >= 0 AND ctt_bobot <= 1)",
"ck_ctt_bobot_range",
),
# Slot must be positive
CheckConstraint("slot > 0", "ck_slot_positive"),
)
def __repr__(self) -> str:
return f"<Item(id={self.id}, slot={self.slot}, level={self.level})>"

193
app/models/session.py Normal file
View File

@@ -0,0 +1,193 @@
"""
Session model for tryout attempt tracking.
Represents a student's attempt at a tryout with scoring information.
"""
from datetime import datetime
from typing import Literal, Union
from sqlalchemy import (
Boolean,
CheckConstraint,
DateTime,
Float,
ForeignKey,
Index,
Integer,
String,
)
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Session(Base):
"""
Session model representing a student's tryout attempt.
Tracks session metadata, scoring results, and IRT estimates.
Attributes:
id: Primary key
session_id: Unique session identifier
wp_user_id: WordPress user ID
website_id: Website identifier
tryout_id: Tryout identifier
start_time: Session start timestamp
end_time: Session end timestamp
is_completed: Completion status
scoring_mode_used: Scoring mode used for this session
total_benar: Total correct answers
total_bobot_earned: Total weight earned
NM: Nilai Mentah (raw score) [0, 1000]
NN: Nilai Nasional (normalized score) [0, 1000]
theta: IRT ability estimate [-3, +3]
theta_se: IRT standard error
rataan_used: Mean value used for normalization
sb_used: Standard deviation used for normalization
created_at: Record creation timestamp
updated_at: Record update timestamp
user: User relationship
tryout: Tryout relationship
user_answers: User's responses in this session
"""
__tablename__ = "sessions"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Session identifier (globally unique)
session_id: Mapped[str] = mapped_column(
String(255),
nullable=False,
unique=True,
index=True,
comment="Unique session identifier",
)
# Foreign keys
wp_user_id: Mapped[str] = mapped_column(
String(255), nullable=False, index=True, comment="WordPress user ID"
)
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
tryout_id: Mapped[str] = mapped_column(
String(255), nullable=False, index=True, comment="Tryout identifier"
)
# Timestamps
start_time: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
end_time: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True), nullable=True, comment="Session end timestamp"
)
is_completed: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, comment="Completion status"
)
# Scoring metadata
scoring_mode_used: Mapped[Literal["ctt", "irt", "hybrid"]] = mapped_column(
String(50),
nullable=False,
comment="Scoring mode used for this session",
)
# CTT scoring results
total_benar: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, comment="Total correct answers"
)
total_bobot_earned: Mapped[float] = mapped_column(
Float, nullable=False, default=0.0, comment="Total weight earned"
)
NM: Mapped[Union[int, None]] = mapped_column(
Integer,
nullable=True,
comment="Nilai Mentah (raw score) [0, 1000]",
)
NN: Mapped[Union[int, None]] = mapped_column(
Integer,
nullable=True,
comment="Nilai Nasional (normalized score) [0, 1000]",
)
# IRT scoring results
theta: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="IRT ability estimate [-3, +3]",
)
theta_se: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="IRT standard error",
)
# Normalization metadata
rataan_used: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="Mean value used for normalization",
)
sb_used: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="Standard deviation used for normalization",
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
user: Mapped["User"] = relationship(
"User", back_populates="sessions", lazy="selectin"
)
tryout: Mapped["Tryout"] = relationship(
"Tryout", back_populates="sessions", lazy="selectin"
)
user_answers: Mapped[list["UserAnswer"]] = relationship(
"UserAnswer", back_populates="session", lazy="selectin", cascade="all, delete-orphan"
)
# Constraints and indexes
__table_args__ = (
Index("ix_sessions_wp_user_id", "wp_user_id"),
Index("ix_sessions_website_id", "website_id"),
Index("ix_sessions_tryout_id", "tryout_id"),
Index("ix_sessions_is_completed", "is_completed"),
# Score constraints [0, 1000]
CheckConstraint(
"NM IS NULL OR (NM >= 0 AND NM <= 1000)",
"ck_nm_range",
),
CheckConstraint(
"NN IS NULL OR (NN >= 0 AND NN <= 1000)",
"ck_nn_range",
),
# IRT theta constraint [-3, +3]
CheckConstraint(
"theta IS NULL OR (theta >= -3 AND theta <= 3)",
"ck_theta_range",
),
# Total correct must be non-negative
CheckConstraint("total_benar >= 0", "ck_total_benar_non_negative"),
# Total bobot must be non-negative
CheckConstraint("total_bobot_earned >= 0", "ck_total_bobot_non_negative"),
)
def __repr__(self) -> str:
return f"<Session(session_id={self.session_id}, tryout_id={self.tryout_id})>"

184
app/models/tryout.py Normal file
View File

@@ -0,0 +1,184 @@
"""
Tryout model with configuration for assessment sessions.
Represents tryout exams with configurable scoring, selection, and normalization modes.
"""
from datetime import datetime
from typing import Literal, Union
from sqlalchemy import Boolean, CheckConstraint, DateTime, Float, ForeignKey, Index, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Tryout(Base):
"""
Tryout model with configuration for assessment sessions.
Supports multiple scoring modes (CTT, IRT, hybrid), selection strategies
(fixed, adaptive, hybrid), and normalization modes (static, dynamic, hybrid).
Attributes:
id: Primary key
website_id: Website identifier
tryout_id: Tryout identifier (unique per website)
name: Tryout name
description: Tryout description
scoring_mode: Scoring algorithm (ctt, irt, hybrid)
selection_mode: Item selection strategy (fixed, adaptive, hybrid)
normalization_mode: Normalization method (static, dynamic, hybrid)
min_sample_for_dynamic: Minimum sample size for dynamic normalization
static_rataan: Static mean value for manual normalization
static_sb: Static standard deviation for manual normalization
AI_generation_enabled: Enable/disable AI question generation
hybrid_transition_slot: Slot number to transition from fixed to adaptive
min_calibration_sample: Minimum responses needed for IRT calibration
theta_estimation_method: Method for estimating theta (mle, map, eap)
fallback_to_ctt_on_error: Fallback to CTT if IRT fails
created_at: Record creation timestamp
updated_at: Record update timestamp
website: Website relationship
items: Items in this tryout
sessions: Sessions for this tryout
stats: Tryout statistics
"""
__tablename__ = "tryouts"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Foreign keys
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
# Tryout identifier (unique per website)
tryout_id: Mapped[str] = mapped_column(
String(255),
nullable=False,
index=True,
comment="Tryout identifier (unique per website)",
)
# Basic information
name: Mapped[str] = mapped_column(
String(255), nullable=False, comment="Tryout name"
)
description: Mapped[Union[str, None]] = mapped_column(
String(1000), nullable=True, comment="Tryout description"
)
# Scoring mode: ctt (Classical Test Theory), irt (Item Response Theory), hybrid
scoring_mode: Mapped[Literal["ctt", "irt", "hybrid"]] = mapped_column(
String(50), nullable=False, default="ctt", comment="Scoring mode"
)
# Selection mode: fixed (slot order), adaptive (CAT), hybrid (mixed)
selection_mode: Mapped[Literal["fixed", "adaptive", "hybrid"]] = mapped_column(
String(50), nullable=False, default="fixed", comment="Item selection mode"
)
# Normalization mode: static (hardcoded), dynamic (real-time), hybrid
normalization_mode: Mapped[Literal["static", "dynamic", "hybrid"]] = mapped_column(
String(50), nullable=False, default="static", comment="Normalization mode"
)
# Normalization settings
min_sample_for_dynamic: Mapped[int] = mapped_column(
Integer,
nullable=False,
default=100,
comment="Minimum sample size for dynamic normalization",
)
static_rataan: Mapped[float] = mapped_column(
Float,
nullable=False,
default=500.0,
comment="Static mean value for manual normalization",
)
static_sb: Mapped[float] = mapped_column(
Float,
nullable=False,
default=100.0,
comment="Static standard deviation for manual normalization",
)
# AI generation settings
ai_generation_enabled: Mapped[bool] = mapped_column(
Boolean,
nullable=False,
default=False,
comment="Enable/disable AI question generation",
)
# Hybrid mode settings
hybrid_transition_slot: Mapped[Union[int, None]] = mapped_column(
Integer,
nullable=True,
comment="Slot number to transition from fixed to adaptive (hybrid mode)",
)
# IRT settings
min_calibration_sample: Mapped[int] = mapped_column(
Integer,
nullable=False,
default=100,
comment="Minimum responses needed for IRT calibration",
)
theta_estimation_method: Mapped[Literal["mle", "map", "eap"]] = mapped_column(
String(50),
nullable=False,
default="mle",
comment="Method for estimating theta",
)
fallback_to_ctt_on_error: Mapped[bool] = mapped_column(
Boolean,
nullable=False,
default=True,
comment="Fallback to CTT if IRT fails",
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
website: Mapped["Website"] = relationship(
"Website", back_populates="tryouts", lazy="selectin"
)
items: Mapped[list["Item"]] = relationship(
"Item", back_populates="tryout", lazy="selectin", cascade="all, delete-orphan"
)
sessions: Mapped[list["Session"]] = relationship(
"Session", back_populates="tryout", lazy="selectin", cascade="all, delete-orphan"
)
stats: Mapped["TryoutStats"] = relationship(
"TryoutStats", back_populates="tryout", lazy="selectin", uselist=False
)
# Constraints and indexes
__table_args__ = (
Index(
"ix_tryouts_website_id_tryout_id", "website_id", "tryout_id", unique=True
),
CheckConstraint("min_sample_for_dynamic > 0", "ck_min_sample_positive"),
CheckConstraint("static_rataan > 0", "ck_static_rataan_positive"),
CheckConstraint("static_sb > 0", "ck_static_sb_positive"),
CheckConstraint("min_calibration_sample > 0", "ck_min_calibration_positive"),
)
def __repr__(self) -> str:
return f"<Tryout(id={self.id}, tryout_id={self.tryout_id}, website_id={self.website_id})>"

151
app/models/tryout_stats.py Normal file
View File

@@ -0,0 +1,151 @@
"""
TryoutStats model for tracking tryout-level statistics.
Maintains running statistics for dynamic normalization and reporting.
"""
from datetime import datetime
from typing import Union
from sqlalchemy import CheckConstraint, DateTime, Float, ForeignKey, Index, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class TryoutStats(Base):
"""
TryoutStats model for maintaining tryout-level statistics.
Tracks participant counts, score distributions, and calculated
normalization parameters (rataan, sb) for dynamic normalization.
Attributes:
id: Primary key
website_id: Website identifier
tryout_id: Tryout identifier
participant_count: Number of completed sessions
total_nm_sum: Running sum of NM scores
total_nm_sq_sum: Running sum of squared NM scores (for variance calc)
rataan: Calculated mean of NM scores
sb: Calculated standard deviation of NM scores
min_nm: Minimum NM score observed
max_nm: Maximum NM score observed
last_calculated: Timestamp of last statistics update
created_at: Record creation timestamp
updated_at: Record update timestamp
tryout: Tryout relationship
"""
__tablename__ = "tryout_stats"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Foreign keys
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
tryout_id: Mapped[str] = mapped_column(
String(255),
nullable=False,
index=True,
comment="Tryout identifier",
)
# Running statistics
participant_count: Mapped[int] = mapped_column(
Integer,
nullable=False,
default=0,
comment="Number of completed sessions",
)
total_nm_sum: Mapped[float] = mapped_column(
Float,
nullable=False,
default=0.0,
comment="Running sum of NM scores",
)
total_nm_sq_sum: Mapped[float] = mapped_column(
Float,
nullable=False,
default=0.0,
comment="Running sum of squared NM scores",
)
# Calculated statistics
rataan: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="Calculated mean of NM scores",
)
sb: Mapped[Union[float, None]] = mapped_column(
Float,
nullable=True,
comment="Calculated standard deviation of NM scores",
)
# Score range
min_nm: Mapped[Union[int, None]] = mapped_column(
Integer,
nullable=True,
comment="Minimum NM score observed",
)
max_nm: Mapped[Union[int, None]] = mapped_column(
Integer,
nullable=True,
comment="Maximum NM score observed",
)
# Timestamps
last_calculated: Mapped[Union[datetime, None]] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="Timestamp of last statistics update",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
tryout: Mapped["Tryout"] = relationship(
"Tryout", back_populates="stats", lazy="selectin"
)
# Constraints and indexes
__table_args__ = (
Index(
"ix_tryout_stats_website_id_tryout_id",
"website_id",
"tryout_id",
unique=True,
),
# Participant count must be non-negative
CheckConstraint("participant_count >= 0", "ck_participant_count_non_negative"),
# Min and max NM must be within valid range [0, 1000]
CheckConstraint(
"min_nm IS NULL OR (min_nm >= 0 AND min_nm <= 1000)",
"ck_min_nm_range",
),
CheckConstraint(
"max_nm IS NULL OR (max_nm >= 0 AND max_nm <= 1000)",
"ck_max_nm_range",
),
# Min must be less than or equal to max
CheckConstraint(
"min_nm IS NULL OR max_nm IS NULL OR min_nm <= max_nm",
"ck_min_max_nm_order",
),
)
def __repr__(self) -> str:
return f"<TryoutStats(tryout_id={self.tryout_id}, participant_count={self.participant_count})>"

72
app/models/user.py Normal file
View File

@@ -0,0 +1,72 @@
"""
User model for WordPress user integration.
Represents users from WordPress that can take tryouts.
"""
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Index, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class User(Base):
"""
User model representing WordPress users.
Attributes:
id: Primary key
wp_user_id: WordPress user ID (unique per site)
website_id: Website identifier (for multi-site support)
created_at: Record creation timestamp
updated_at: Record update timestamp
sessions: User's tryout sessions
"""
__tablename__ = "users"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# WordPress user ID (unique within website context)
wp_user_id: Mapped[int] = mapped_column(
String(255), nullable=False, index=True, comment="WordPress user ID"
)
# Website identifier (for multi-site support)
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
website: Mapped["Website"] = relationship(
"Website", back_populates="users", lazy="selectin"
)
sessions: Mapped[list["Session"]] = relationship(
"Session", back_populates="user", lazy="selectin", cascade="all, delete-orphan"
)
# Indexes
__table_args__ = (
Index("ix_users_wp_user_id_website_id", "wp_user_id", "website_id", unique=True),
Index("ix_users_website_id", "website_id"),
)
def __repr__(self) -> str:
return f"<User(wp_user_id={self.wp_user_id}, website_id={self.website_id})>"

137
app/models/user_answer.py Normal file
View File

@@ -0,0 +1,137 @@
"""
UserAnswer model for tracking individual question responses.
Represents a student's response to a single question with scoring metadata.
"""
from datetime import datetime
from typing import Literal, Union
from sqlalchemy import Boolean, CheckConstraint, DateTime, Float, ForeignKey, Index, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class UserAnswer(Base):
"""
UserAnswer model representing a student's response to a question.
Tracks response, correctness, scoring, and timing information.
Attributes:
id: Primary key
session_id: Session identifier
wp_user_id: WordPress user ID
website_id: Website identifier
tryout_id: Tryout identifier
item_id: Item identifier
response: User's answer (A, B, C, D)
is_correct: Whether answer is correct
time_spent: Time spent on this question (seconds)
scoring_mode_used: Scoring mode used
bobot_earned: Weight earned for this answer
created_at: Record creation timestamp
updated_at: Record update timestamp
session: Session relationship
item: Item relationship
"""
__tablename__ = "user_answers"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Foreign keys
session_id: Mapped[str] = mapped_column(
ForeignKey("sessions.session_id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Session identifier",
)
wp_user_id: Mapped[str] = mapped_column(
String(255), nullable=False, index=True, comment="WordPress user ID"
)
website_id: Mapped[int] = mapped_column(
ForeignKey("websites.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Website identifier",
)
tryout_id: Mapped[str] = mapped_column(
String(255), nullable=False, index=True, comment="Tryout identifier"
)
item_id: Mapped[int] = mapped_column(
ForeignKey("items.id", ondelete="CASCADE", onupdate="CASCADE"),
nullable=False,
index=True,
comment="Item identifier",
)
# Response information
response: Mapped[str] = mapped_column(
String(10), nullable=False, comment="User's answer (A, B, C, D)"
)
is_correct: Mapped[bool] = mapped_column(
Boolean, nullable=False, comment="Whether answer is correct"
)
time_spent: Mapped[int] = mapped_column(
Integer,
nullable=False,
default=0,
comment="Time spent on this question (seconds)",
)
# Scoring metadata
scoring_mode_used: Mapped[Literal["ctt", "irt", "hybrid"]] = mapped_column(
String(50),
nullable=False,
comment="Scoring mode used",
)
bobot_earned: Mapped[float] = mapped_column(
Float,
nullable=False,
default=0.0,
comment="Weight earned for this answer",
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
session: Mapped["Session"] = relationship(
"Session", back_populates="user_answers", lazy="selectin"
)
item: Mapped["Item"] = relationship(
"Item", back_populates="user_answers", lazy="selectin"
)
# Constraints and indexes
__table_args__ = (
Index("ix_user_answers_session_id", "session_id"),
Index("ix_user_answers_wp_user_id", "wp_user_id"),
Index("ix_user_answers_website_id", "website_id"),
Index("ix_user_answers_tryout_id", "tryout_id"),
Index("ix_user_answers_item_id", "item_id"),
Index(
"ix_user_answers_session_id_item_id",
"session_id",
"item_id",
unique=True,
),
# Time spent must be non-negative
CheckConstraint("time_spent >= 0", "ck_time_spent_non_negative"),
# Bobot earned must be non-negative
CheckConstraint("bobot_earned >= 0", "ck_bobot_earned_non_negative"),
)
def __repr__(self) -> str:
return f"<UserAnswer(id={self.id}, session_id={self.session_id}, item_id={self.item_id})>"

69
app/models/website.py Normal file
View File

@@ -0,0 +1,69 @@
"""
Website model for multi-site support.
Represents WordPress websites that use the IRT Bank Soal system.
"""
from datetime import datetime
from sqlalchemy import DateTime, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Website(Base):
"""
Website model representing WordPress sites.
Enables multi-site support where a single backend serves multiple
WordPress-powered educational sites.
Attributes:
id: Primary key
site_url: WordPress site URL
site_name: Human-readable site name
created_at: Record creation timestamp
updated_at: Record update timestamp
users: Users belonging to this website
tryouts: Tryouts available on this website
"""
__tablename__ = "websites"
# Primary key
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Site information
site_url: Mapped[str] = mapped_column(
String(512),
nullable=False,
unique=True,
index=True,
comment="WordPress site URL",
)
site_name: Mapped[str] = mapped_column(
String(255), nullable=False, comment="Human-readable site name"
)
# Timestamps
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default="NOW()"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
server_default="NOW()",
onupdate="NOW()",
)
# Relationships
users: Mapped[list["User"]] = relationship(
"User", back_populates="website", lazy="selectin", cascade="all, delete-orphan"
)
tryouts: Mapped[list["Tryout"]] = relationship(
"Tryout", back_populates="website", lazy="selectin", cascade="all, delete-orphan"
)
def __repr__(self) -> str:
return f"<Website(id={self.id}, site_url={self.site_url})>"

13
app/routers/__init__.py Normal file
View File

@@ -0,0 +1,13 @@
"""
API routers package.
"""
from app.routers.sessions import router as sessions_router
from app.routers.tryouts import router as tryouts_router
from app.routers.reports import router as reports_router
__all__ = [
"sessions_router",
"tryouts_router",
"reports_router",
]

249
app/routers/admin.py Normal file
View File

@@ -0,0 +1,249 @@
"""
Admin API router for custom admin actions.
Provides admin-specific endpoints for triggering calibration,
toggling AI generation, and resetting normalization.
"""
from typing import Dict, Optional
from fastapi import APIRouter, Depends, Header, HTTPException, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.database import get_db
from app.models import Tryout, TryoutStats
from app.services.irt_calibration import (
calibrate_all,
CALIBRATION_SAMPLE_THRESHOLD,
)
router = APIRouter(prefix="/admin", tags=["admin"])
settings = get_settings()
def get_admin_website_id(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header for admin operations.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
@router.post(
"/{tryout_id}/calibrate",
summary="Trigger IRT calibration",
description="Trigger IRT calibration for all items in this tryout with sufficient response data.",
)
async def admin_trigger_calibration(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_admin_website_id),
) -> Dict[str, any]:
"""
Trigger IRT calibration for all items in a tryout.
Runs calibration for items with >= min_calibration_sample responses.
Updates item.irt_b, item.irt_se, and item.calibrated status.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Calibration results summary
Raises:
HTTPException: If tryout not found or calibration fails
"""
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Run calibration
result = await calibrate_all(
tryout_id=tryout_id,
website_id=website_id,
db=db,
min_sample_size=tryout.min_calibration_sample or CALIBRATION_SAMPLE_THRESHOLD,
)
return {
"tryout_id": tryout_id,
"total_items": result.total_items,
"calibrated_items": result.calibrated_items,
"failed_items": result.failed_items,
"calibration_percentage": round(result.calibration_percentage * 100, 2),
"ready_for_irt": result.ready_for_irt,
"message": f"Calibration complete: {result.calibrated_items}/{result.total_items} items calibrated",
}
@router.post(
"/{tryout_id}/toggle-ai-generation",
summary="Toggle AI generation",
description="Toggle AI question generation for a tryout.",
)
async def admin_toggle_ai_generation(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_admin_website_id),
) -> Dict[str, any]:
"""
Toggle AI generation for a tryout.
Updates Tryout.AI_generation_enabled field.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Updated AI generation status
Raises:
HTTPException: If tryout not found
"""
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Toggle AI generation
tryout.ai_generation_enabled = not tryout.ai_generation_enabled
await db.commit()
await db.refresh(tryout)
status = "enabled" if tryout.ai_generation_enabled else "disabled"
return {
"tryout_id": tryout_id,
"ai_generation_enabled": tryout.ai_generation_enabled,
"message": f"AI generation {status} for tryout {tryout_id}",
}
@router.post(
"/{tryout_id}/reset-normalization",
summary="Reset normalization",
description="Reset normalization to static values and clear incremental stats.",
)
async def admin_reset_normalization(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_admin_website_id),
) -> Dict[str, any]:
"""
Reset normalization for a tryout.
Resets rataan, sb to static values and clears incremental stats.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Reset statistics
Raises:
HTTPException: If tryout or stats not found
"""
# Get tryout stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"TryoutStats for {tryout_id} not found for website {website_id}",
)
# Get tryout for static values
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout:
# Reset to static values
stats.rataan = tryout.static_rataan
stats.sb = tryout.static_sb
else:
# Reset to default values
stats.rataan = 500.0
stats.sb = 100.0
# Clear incremental stats
old_participant_count = stats.participant_count
stats.participant_count = 0
stats.total_nm_sum = 0.0
stats.total_nm_sq_sum = 0.0
stats.min_nm = None
stats.max_nm = None
stats.last_calculated = None
await db.commit()
await db.refresh(stats)
return {
"tryout_id": tryout_id,
"rataan": stats.rataan,
"sb": stats.sb,
"cleared_stats": {
"previous_participant_count": old_participant_count,
},
"message": f"Normalization reset to static values (rataan={stats.rataan}, sb={stats.sb}). Incremental stats cleared.",
}

292
app/routers/ai.py Normal file
View File

@@ -0,0 +1,292 @@
"""
AI Generation Router.
Admin endpoints for AI question generation playground.
"""
import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import and_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.item import Item
from app.schemas.ai import (
AIGeneratePreviewRequest,
AIGeneratePreviewResponse,
AISaveRequest,
AISaveResponse,
AIStatsResponse,
)
from app.services.ai_generation import (
generate_question,
get_ai_stats,
save_ai_question,
validate_ai_model,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/admin/ai", tags=["admin", "ai-generation"])
@router.post(
"/generate-preview",
response_model=AIGeneratePreviewResponse,
summary="Preview AI-generated question",
description="""
Generate a question preview using AI without saving to database.
This is an admin playground endpoint for testing AI generation quality.
Admins can retry unlimited times until satisfied with the result.
Requirements:
- basis_item_id must reference an existing item at 'sedang' level
- target_level must be 'mudah' or 'sulit'
- ai_model must be a supported OpenRouter model
""",
responses={
200: {"description": "Question generated successfully (preview mode)"},
400: {"description": "Invalid request (wrong level, unsupported model)"},
404: {"description": "Basis item not found"},
500: {"description": "AI generation failed"},
},
)
async def generate_preview(
request: AIGeneratePreviewRequest,
db: Annotated[AsyncSession, Depends(get_db)],
) -> AIGeneratePreviewResponse:
"""
Generate AI question preview (no database save).
- **basis_item_id**: ID of the sedang-level question to base generation on
- **target_level**: Target difficulty (mudah/sulit)
- **ai_model**: OpenRouter model to use (default: qwen/qwen-2.5-coder-32b-instruct)
"""
# Validate AI model
if not validate_ai_model(request.ai_model):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Unsupported AI model: {request.ai_model}. "
f"Supported models: qwen/qwen-2.5-coder-32b-instruct, meta-llama/llama-3.3-70b-instruct",
)
# Fetch basis item
result = await db.execute(
select(Item).where(Item.id == request.basis_item_id)
)
basis_item = result.scalar_one_or_none()
if not basis_item:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Basis item not found: {request.basis_item_id}",
)
# Validate basis item is sedang level
if basis_item.level != "sedang":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Basis item must be 'sedang' level, got: {basis_item.level}",
)
# Generate question
try:
generated = await generate_question(
basis_item=basis_item,
target_level=request.target_level,
ai_model=request.ai_model,
)
if not generated:
return AIGeneratePreviewResponse(
success=False,
error="AI generation failed. Please check logs or try again.",
ai_model=request.ai_model,
basis_item_id=request.basis_item_id,
target_level=request.target_level,
)
return AIGeneratePreviewResponse(
success=True,
stem=generated.stem,
options=generated.options,
correct=generated.correct,
explanation=generated.explanation,
ai_model=request.ai_model,
basis_item_id=request.basis_item_id,
target_level=request.target_level,
cached=False,
)
except Exception as e:
logger.error(f"AI preview generation failed: {e}")
return AIGeneratePreviewResponse(
success=False,
error=f"AI generation error: {str(e)}",
ai_model=request.ai_model,
basis_item_id=request.basis_item_id,
target_level=request.target_level,
)
@router.post(
"/generate-save",
response_model=AISaveResponse,
summary="Save AI-generated question",
description="""
Save an AI-generated question to the database.
This endpoint creates a new Item record with:
- generated_by='ai'
- ai_model from request
- basis_item_id linking to original question
- calibrated=False (will be calculated later)
""",
responses={
200: {"description": "Question saved successfully"},
400: {"description": "Invalid request data"},
404: {"description": "Basis item or tryout not found"},
409: {"description": "Item already exists at this slot/level"},
500: {"description": "Database save failed"},
},
)
async def generate_save(
request: AISaveRequest,
db: Annotated[AsyncSession, Depends(get_db)],
) -> AISaveResponse:
"""
Save AI-generated question to database.
- **stem**: Question text
- **options**: Dict with A, B, C, D options
- **correct**: Correct answer (A/B/C/D)
- **explanation**: Answer explanation (optional)
- **tryout_id**: Tryout identifier
- **website_id**: Website identifier
- **basis_item_id**: Original item ID this was generated from
- **slot**: Question slot position
- **level**: Difficulty level
- **ai_model**: AI model used for generation
"""
# Verify basis item exists
basis_result = await db.execute(
select(Item).where(Item.id == request.basis_item_id)
)
basis_item = basis_result.scalar_one_or_none()
if not basis_item:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Basis item not found: {request.basis_item_id}",
)
# Check for duplicate (same tryout, website, slot, level)
existing_result = await db.execute(
select(Item).where(
and_(
Item.tryout_id == request.tryout_id,
Item.website_id == request.website_id,
Item.slot == request.slot,
Item.level == request.level,
)
)
)
existing = existing_result.scalar_one_or_none()
if existing:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"Item already exists at slot={request.slot}, level={request.level} "
f"for tryout={request.tryout_id}",
)
# Create GeneratedQuestion from request
from app.schemas.ai import GeneratedQuestion
generated_data = GeneratedQuestion(
stem=request.stem,
options=request.options,
correct=request.correct,
explanation=request.explanation,
)
# Save to database
item_id = await save_ai_question(
generated_data=generated_data,
tryout_id=request.tryout_id,
website_id=request.website_id,
basis_item_id=request.basis_item_id,
slot=request.slot,
level=request.level,
ai_model=request.ai_model,
db=db,
)
if not item_id:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to save AI-generated question",
)
return AISaveResponse(
success=True,
item_id=item_id,
)
@router.get(
"/stats",
response_model=AIStatsResponse,
summary="Get AI generation statistics",
description="""
Get statistics about AI-generated questions.
Returns:
- Total AI-generated items count
- Items count by model
- Cache hit rate (placeholder)
""",
)
async def get_stats(
db: Annotated[AsyncSession, Depends(get_db)],
) -> AIStatsResponse:
"""
Get AI generation statistics.
"""
stats = await get_ai_stats(db)
return AIStatsResponse(
total_ai_items=stats["total_ai_items"],
items_by_model=stats["items_by_model"],
cache_hit_rate=stats["cache_hit_rate"],
total_cache_hits=stats["total_cache_hits"],
total_requests=stats["total_requests"],
)
@router.get(
"/models",
summary="List supported AI models",
description="Returns list of supported AI models for question generation.",
)
async def list_models() -> dict:
"""
List supported AI models.
"""
return {
"models": [
{
"id": "qwen/qwen-2.5-coder-32b-instruct",
"name": "Qwen 2.5 Coder 32B",
"description": "Fast and efficient model for question generation",
},
{
"id": "meta-llama/llama-3.3-70b-instruct",
"name": "Llama 3.3 70B",
"description": "High-quality model with better reasoning",
},
]
}

324
app/routers/import_export.py Normal file
View File

@@ -0,0 +1,324 @@
"""
Import/Export API router for Excel question migration.
Endpoints:
- POST /api/v1/import/preview: Preview Excel import without saving
- POST /api/v1/import/questions: Import questions from Excel to database
- GET /api/v1/export/questions: Export questions to Excel file
"""
import os
import tempfile
from typing import Optional
from fastapi import APIRouter, Depends, File, Form, Header, HTTPException, UploadFile, status
from fastapi.responses import FileResponse
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.services.excel_import import (
bulk_insert_items,
export_questions_to_excel,
parse_excel_import,
validate_excel_structure,
)
router = APIRouter(prefix="/api/v1/import-export", tags=["import-export"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
@router.post(
"/preview",
summary="Preview Excel import",
description="Parse Excel file and return preview without saving to database.",
)
async def preview_import(
file: UploadFile = File(..., description="Excel file (.xlsx)"),
website_id: int = Depends(get_website_id_from_header),
) -> dict:
"""
Preview Excel import without saving to database.
Args:
file: Excel file upload (.xlsx format)
website_id: Website ID from header
Returns:
Dict with:
- items_count: Number of items parsed
- preview: List of item previews
- validation_errors: List of validation errors if any
Raises:
HTTPException: If file format is invalid or parsing fails
"""
# Validate file format
if not file.filename or not file.filename.lower().endswith('.xlsx'):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="File must be .xlsx format",
)
# Save uploaded file to temporary location
try:
with tempfile.NamedTemporaryFile(delete=False, suffix=".xlsx") as temp_file:
content = await file.read()
temp_file.write(content)
temp_file_path = temp_file.name
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to save uploaded file: {str(e)}",
)
try:
# Validate Excel structure
validation = validate_excel_structure(temp_file_path)
if not validation["valid"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"error": "Invalid Excel structure",
"validation_errors": validation["errors"],
},
)
# Parse Excel (tryout_id is optional for preview)
tryout_id = "preview" # Use dummy tryout_id for preview
result = parse_excel_import(
temp_file_path,
website_id=website_id,
tryout_id=tryout_id
)
if result["validation_errors"]:
return {
"items_count": result["items_count"],
"preview": result["items"],
"validation_errors": result["validation_errors"],
"has_errors": True,
}
# Return limited preview (first 5 items)
preview_items = result["items"][:5]
return {
"items_count": result["items_count"],
"preview": preview_items,
"validation_errors": [],
"has_errors": False,
}
finally:
# Clean up temporary file
if os.path.exists(temp_file_path):
os.unlink(temp_file_path)
@router.post(
"/questions",
summary="Import questions from Excel",
description="Parse Excel file and import questions to database with 100% data integrity.",
)
async def import_questions(
file: UploadFile = File(..., description="Excel file (.xlsx)"),
website_id: int = Depends(get_website_id_from_header),
tryout_id: str = Form(..., description="Tryout identifier"),
db: AsyncSession = Depends(get_db),
) -> dict:
"""
Import questions from Excel to database.
Validates file format, parses Excel content, checks for duplicates,
and performs bulk insert with rollback on error.
Args:
file: Excel file upload (.xlsx format)
website_id: Website ID from header
tryout_id: Tryout identifier
db: Async database session
Returns:
Dict with:
- imported: Number of items successfully imported
- duplicates: Number of duplicate items skipped
- errors: List of errors if any
Raises:
HTTPException: If file format is invalid, validation fails, or import fails
"""
# Validate file format
if not file.filename or not file.filename.lower().endswith('.xlsx'):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="File must be .xlsx format",
)
# Save uploaded file to temporary location
try:
with tempfile.NamedTemporaryFile(delete=False, suffix=".xlsx") as temp_file:
content = await file.read()
temp_file.write(content)
temp_file_path = temp_file.name
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to save uploaded file: {str(e)}",
)
try:
# Validate Excel structure
validation = validate_excel_structure(temp_file_path)
if not validation["valid"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"error": "Invalid Excel structure",
"validation_errors": validation["errors"],
},
)
# Parse Excel
result = parse_excel_import(
temp_file_path,
website_id=website_id,
tryout_id=tryout_id
)
# Check for validation errors
if result["validation_errors"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"error": "Validation failed",
"validation_errors": result["validation_errors"],
},
)
# Check if items were parsed
if result["items_count"] == 0:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No items found in Excel file",
)
# Bulk insert items
insert_result = await bulk_insert_items(result["items"], db)
# Check for insertion errors
if insert_result["errors"]:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"error": "Import failed",
"errors": insert_result["errors"],
},
)
# Check for conflicts (duplicates)
if insert_result["duplicate_count"] > 0:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail={
"message": f"Import completed with {insert_result['duplicate_count']} duplicate(s) skipped",
"imported": insert_result["inserted_count"],
"duplicates": insert_result["duplicate_count"],
},
)
return {
"message": "Import successful",
"imported": insert_result["inserted_count"],
"duplicates": insert_result["duplicate_count"],
}
finally:
# Clean up temporary file
if os.path.exists(temp_file_path):
os.unlink(temp_file_path)
@router.get(
"/export/questions",
summary="Export questions to Excel",
description="Export questions for a tryout to Excel file in standardized format.",
)
async def export_questions(
tryout_id: str,
website_id: int = Depends(get_website_id_from_header),
db: AsyncSession = Depends(get_db),
) -> FileResponse:
"""
Export questions to Excel file.
Creates Excel file with standardized format:
- Row 2: KUNCI (answer key)
- Row 4: TK (p-values)
- Row 5: BOBOT (weights)
- Rows 6+: Question data
Args:
tryout_id: Tryout identifier
website_id: Website ID from header
db: Async database session
Returns:
FileResponse with Excel file
Raises:
HTTPException: If tryout has no questions or export fails
"""
try:
# Export questions to Excel
output_path = await export_questions_to_excel(
tryout_id=tryout_id,
website_id=website_id,
db=db
)
# Return file for download
filename = f"tryout_{tryout_id}_questions.xlsx"
return FileResponse(
path=output_path,
media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
filename=filename,
)
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Export failed: {str(e)}",
)

279
app/routers/normalization.py Normal file
View File

@@ -0,0 +1,279 @@
"""
Normalization API router for dynamic normalization management.
Endpoints:
- GET /tryout/{tryout_id}/normalization: Get normalization configuration
- PUT /tryout/{tryout_id}/normalization: Update normalization settings
- POST /tryout/{tryout_id}/normalization/reset: Reset normalization stats
- GET /tryout/{tryout_id}/normalization/validate: Validate dynamic normalization
"""
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Header, status
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.services.config_management import (
get_normalization_config,
reset_normalization_stats,
toggle_normalization_mode,
update_config,
)
from app.services.normalization import (
validate_dynamic_normalization,
)
router = APIRouter(prefix="/tryout", tags=["normalization"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
@router.get(
"/{tryout_id}/normalization",
summary="Get normalization configuration",
description="Retrieve current normalization configuration including mode, static values, dynamic values, and threshold status.",
)
async def get_normalization_endpoint(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Get normalization configuration for a tryout.
Returns:
Normalization configuration with:
- mode (static/dynamic/hybrid)
- current rataan, sb (from TryoutStats)
- static_rataan, static_sb (from Tryout config)
- participant_count
- threshold_status (ready for dynamic or not)
Raises:
HTTPException: If tryout not found
"""
try:
config = await get_normalization_config(db, website_id, tryout_id)
return config
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
@router.put(
"/{tryout_id}/normalization",
summary="Update normalization settings",
description="Update normalization mode and static values for a tryout.",
)
async def update_normalization_endpoint(
tryout_id: str,
normalization_mode: Optional[str] = None,
static_rataan: Optional[float] = None,
static_sb: Optional[float] = None,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Update normalization settings for a tryout.
Args:
tryout_id: Tryout identifier
normalization_mode: New normalization mode (static/dynamic/hybrid)
static_rataan: New static mean value
static_sb: New static standard deviation
db: Database session
website_id: Website ID from header
Returns:
Updated normalization configuration
Raises:
HTTPException: If tryout not found or validation fails
"""
# Build updates dictionary
updates = {}
if normalization_mode is not None:
if normalization_mode not in ["static", "dynamic", "hybrid"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid normalization_mode: {normalization_mode}. Must be 'static', 'dynamic', or 'hybrid'",
)
updates["normalization_mode"] = normalization_mode
if static_rataan is not None:
if static_rataan <= 0:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="static_rataan must be greater than 0",
)
updates["static_rataan"] = static_rataan
if static_sb is not None:
if static_sb <= 0:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="static_sb must be greater than 0",
)
updates["static_sb"] = static_sb
if not updates:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No updates provided",
)
try:
# Update configuration
await update_config(db, website_id, tryout_id, updates)
# Get updated configuration
config = await get_normalization_config(db, website_id, tryout_id)
return config
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
@router.post(
"/{tryout_id}/normalization/reset",
summary="Reset normalization stats",
description="Reset TryoutStats to initial values and switch to static normalization mode.",
)
async def reset_normalization_endpoint(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Reset normalization stats for a tryout.
Resets TryoutStats to initial values (participant_count=0, sums cleared)
and temporarily switches normalization_mode to "static".
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Success message with updated configuration
Raises:
HTTPException: If tryout not found
"""
try:
stats = await reset_normalization_stats(db, website_id, tryout_id)
config = await get_normalization_config(db, website_id, tryout_id)
return {
"message": "Normalization stats reset successfully",
"tryout_id": tryout_id,
"participant_count": stats.participant_count,
"normalization_mode": config["normalization_mode"],
}
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
@router.get(
"/{tryout_id}/normalization/validate",
summary="Validate dynamic normalization",
description="Validate that dynamic normalization produces expected distribution (mean≈500±5, SD≈100±5).",
)
async def validate_normalization_endpoint(
tryout_id: str,
target_mean: float = 500.0,
target_sd: float = 100.0,
mean_tolerance: float = 5.0,
sd_tolerance: float = 5.0,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Validate dynamic normalization for a tryout.
Checks if calculated rataan and sb are close to target values.
Returns validation status, deviations, warnings, and suggestions.
Args:
tryout_id: Tryout identifier
target_mean: Target mean (default: 500)
target_sd: Target standard deviation (default: 100)
mean_tolerance: Allowed deviation from target mean (default: 5)
sd_tolerance: Allowed deviation from target SD (default: 5)
db: Database session
website_id: Website ID from header
Returns:
Validation result with:
- is_valid: True if within tolerance
- details: Full validation details
Raises:
HTTPException: If tryout not found
"""
try:
is_valid, details = await validate_dynamic_normalization(
db=db,
website_id=website_id,
tryout_id=tryout_id,
target_mean=target_mean,
target_sd=target_sd,
mean_tolerance=mean_tolerance,
sd_tolerance=sd_tolerance,
)
return {
"tryout_id": tryout_id,
"is_valid": is_valid,
"target_mean": target_mean,
"target_sd": target_sd,
"mean_tolerance": mean_tolerance,
"sd_tolerance": sd_tolerance,
"details": details,
}
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)

792
app/routers/reports.py Normal file
View File

@@ -0,0 +1,792 @@
"""
Reports API router for comprehensive reporting.
Endpoints:
- GET /reports/student/performance: Get student performance report
- GET /reports/items/analysis: Get item analysis report
- GET /reports/calibration/status: Get calibration status report
- GET /reports/tryout/comparison: Get tryout comparison report
- POST /reports/schedule: Schedule a report
- GET /reports/export/{schedule_id}/{format}: Export scheduled report
"""
import os
from datetime import datetime
from typing import List, Literal, Optional
from fastapi import APIRouter, Depends, HTTPException, Header, status
from fastapi.responses import FileResponse
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.schemas.report import (
StudentPerformanceReportOutput,
AggregatePerformanceStatsOutput,
StudentPerformanceRecordOutput,
ItemAnalysisReportOutput,
ItemAnalysisRecordOutput,
CalibrationStatusReportOutput,
CalibrationItemStatusOutput,
TryoutComparisonReportOutput,
TryoutComparisonRecordOutput,
ReportScheduleRequest,
ReportScheduleOutput,
ReportScheduleResponse,
ExportResponse,
)
from app.services.reporting import (
generate_student_performance_report,
generate_item_analysis_report,
generate_calibration_status_report,
generate_tryout_comparison_report,
export_report_to_csv,
export_report_to_excel,
export_report_to_pdf,
schedule_report,
get_scheduled_report,
list_scheduled_reports,
cancel_scheduled_report,
StudentPerformanceReport,
ItemAnalysisReport,
CalibrationStatusReport,
TryoutComparisonReport,
)
router = APIRouter(prefix="/reports", tags=["reports"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
# =============================================================================
# Student Performance Report Endpoints
# =============================================================================
@router.get(
"/student/performance",
response_model=StudentPerformanceReportOutput,
summary="Get student performance report",
description="Generate student performance report with individual and aggregate statistics.",
)
async def get_student_performance_report(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
date_start: Optional[datetime] = None,
date_end: Optional[datetime] = None,
format_type: Literal["individual", "aggregate", "both"] = "both",
) -> StudentPerformanceReportOutput:
"""
Get student performance report.
Returns individual student records and/or aggregate statistics.
"""
date_range = None
if date_start or date_end:
date_range = {}
if date_start:
date_range["start"] = date_start
if date_end:
date_range["end"] = date_end
report = await generate_student_performance_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
date_range=date_range,
format_type=format_type,
)
return _convert_student_performance_report(report)
def _convert_student_performance_report(report: StudentPerformanceReport) -> StudentPerformanceReportOutput:
"""Convert dataclass report to Pydantic output."""
date_range_str = None
if report.date_range:
date_range_str = {}
if report.date_range.get("start"):
date_range_str["start"] = report.date_range["start"].isoformat()
if report.date_range.get("end"):
date_range_str["end"] = report.date_range["end"].isoformat()
return StudentPerformanceReportOutput(
generated_at=report.generated_at,
tryout_id=report.tryout_id,
website_id=report.website_id,
date_range=date_range_str,
aggregate=AggregatePerformanceStatsOutput(
tryout_id=report.aggregate.tryout_id,
participant_count=report.aggregate.participant_count,
avg_nm=report.aggregate.avg_nm,
std_nm=report.aggregate.std_nm,
min_nm=report.aggregate.min_nm,
max_nm=report.aggregate.max_nm,
median_nm=report.aggregate.median_nm,
avg_nn=report.aggregate.avg_nn,
std_nn=report.aggregate.std_nn,
avg_theta=report.aggregate.avg_theta,
pass_rate=report.aggregate.pass_rate,
avg_time_spent=report.aggregate.avg_time_spent,
),
individual_records=[
StudentPerformanceRecordOutput(
session_id=r.session_id,
wp_user_id=r.wp_user_id,
tryout_id=r.tryout_id,
NM=r.NM,
NN=r.NN,
theta=r.theta,
theta_se=r.theta_se,
total_benar=r.total_benar,
time_spent=r.time_spent,
start_time=r.start_time,
end_time=r.end_time,
scoring_mode_used=r.scoring_mode_used,
rataan_used=r.rataan_used,
sb_used=r.sb_used,
)
for r in report.individual_records
],
)
# =============================================================================
# Item Analysis Report Endpoints
# =============================================================================
@router.get(
"/items/analysis",
response_model=ItemAnalysisReportOutput,
summary="Get item analysis report",
description="Generate item analysis report with difficulty, discrimination, and information functions.",
)
async def get_item_analysis_report(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
filter_by: Optional[Literal["difficulty", "calibrated", "discrimination"]] = None,
difficulty_level: Optional[Literal["mudah", "sedang", "sulit"]] = None,
) -> ItemAnalysisReportOutput:
"""
Get item analysis report.
Returns item difficulty, discrimination, and information function data.
"""
report = await generate_item_analysis_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
filter_by=filter_by,
difficulty_level=difficulty_level,
)
return ItemAnalysisReportOutput(
generated_at=report.generated_at,
tryout_id=report.tryout_id,
website_id=report.website_id,
total_items=report.total_items,
items=[
ItemAnalysisRecordOutput(
item_id=r.item_id,
slot=r.slot,
level=r.level,
ctt_p=r.ctt_p,
ctt_bobot=r.ctt_bobot,
ctt_category=r.ctt_category,
irt_b=r.irt_b,
irt_se=r.irt_se,
calibrated=r.calibrated,
calibration_sample_size=r.calibration_sample_size,
correctness_rate=r.correctness_rate,
item_total_correlation=r.item_total_correlation,
information_values=r.information_values,
optimal_theta_range=r.optimal_theta_range,
)
for r in report.items
],
summary=report.summary,
)
# =============================================================================
# Calibration Status Report Endpoints
# =============================================================================
@router.get(
"/calibration/status",
response_model=CalibrationStatusReportOutput,
summary="Get calibration status report",
description="Generate calibration status report with progress tracking and readiness metrics.",
)
async def get_calibration_status_report(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> CalibrationStatusReportOutput:
"""
Get calibration status report.
Returns calibration progress, items awaiting calibration, and IRT readiness status.
"""
report = await generate_calibration_status_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
)
return CalibrationStatusReportOutput(
generated_at=report.generated_at,
tryout_id=report.tryout_id,
website_id=report.website_id,
total_items=report.total_items,
calibrated_items=report.calibrated_items,
calibration_percentage=report.calibration_percentage,
items_awaiting_calibration=[
CalibrationItemStatusOutput(
item_id=r.item_id,
slot=r.slot,
level=r.level,
sample_size=r.sample_size,
calibrated=r.calibrated,
irt_b=r.irt_b,
irt_se=r.irt_se,
ctt_p=r.ctt_p,
)
for r in report.items_awaiting_calibration
],
avg_calibration_sample_size=report.avg_calibration_sample_size,
estimated_time_to_90_percent=report.estimated_time_to_90_percent,
ready_for_irt_rollout=report.ready_for_irt_rollout,
items=[
CalibrationItemStatusOutput(
item_id=r.item_id,
slot=r.slot,
level=r.level,
sample_size=r.sample_size,
calibrated=r.calibrated,
irt_b=r.irt_b,
irt_se=r.irt_se,
ctt_p=r.ctt_p,
)
for r in report.items
],
)
# =============================================================================
# Tryout Comparison Report Endpoints
# =============================================================================
@router.get(
"/tryout/comparison",
response_model=TryoutComparisonReportOutput,
summary="Get tryout comparison report",
description="Generate tryout comparison report across dates or subjects.",
)
async def get_tryout_comparison_report(
tryout_ids: str, # Comma-separated list
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
group_by: Literal["date", "subject"] = "date",
) -> TryoutComparisonReportOutput:
"""
Get tryout comparison report.
Compares tryouts across dates or subjects.
"""
tryout_id_list = [tid.strip() for tid in tryout_ids.split(",")]
if len(tryout_id_list) < 2:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="At least 2 tryout IDs are required for comparison",
)
report = await generate_tryout_comparison_report(
tryout_ids=tryout_id_list,
website_id=website_id,
db=db,
group_by=group_by,
)
return TryoutComparisonReportOutput(
generated_at=report.generated_at,
comparison_type=report.comparison_type,
tryouts=[
TryoutComparisonRecordOutput(
tryout_id=r.tryout_id,
date=r.date,
subject=r.subject,
participant_count=r.participant_count,
avg_nm=r.avg_nm,
avg_nn=r.avg_nn,
avg_theta=r.avg_theta,
std_nm=r.std_nm,
calibration_percentage=r.calibration_percentage,
)
for r in report.tryouts
],
trends=report.trends,
normalization_impact=report.normalization_impact,
)
# =============================================================================
# Report Scheduling Endpoints
# =============================================================================
@router.post(
"/schedule",
response_model=ReportScheduleResponse,
summary="Schedule a report",
description="Schedule a report for automatic generation on a daily, weekly, or monthly basis.",
)
async def create_report_schedule(
request: ReportScheduleRequest,
db: AsyncSession = Depends(get_db),
) -> ReportScheduleResponse:
"""
Schedule a report.
Creates a scheduled report that will be generated automatically.
"""
schedule_id = schedule_report(
report_type=request.report_type,
schedule=request.schedule,
tryout_ids=request.tryout_ids,
website_id=request.website_id,
recipients=request.recipients,
export_format=request.export_format,
)
scheduled = get_scheduled_report(schedule_id)
return ReportScheduleResponse(
schedule_id=schedule_id,
message=f"Report scheduled successfully for {request.schedule} generation",
next_run=scheduled.next_run if scheduled else None,
)
@router.get(
"/schedule/{schedule_id}",
response_model=ReportScheduleOutput,
summary="Get scheduled report details",
description="Get details of a scheduled report.",
)
async def get_scheduled_report_details(
schedule_id: str,
website_id: int = Depends(get_website_id_from_header),
) -> ReportScheduleOutput:
"""
Get scheduled report details.
Returns the configuration and status of a scheduled report.
"""
scheduled = get_scheduled_report(schedule_id)
if not scheduled:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Scheduled report {schedule_id} not found",
)
if scheduled.website_id != website_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this scheduled report",
)
return ReportScheduleOutput(
schedule_id=scheduled.schedule_id,
report_type=scheduled.report_type,
schedule=scheduled.schedule,
tryout_ids=scheduled.tryout_ids,
website_id=scheduled.website_id,
recipients=scheduled.recipients,
format=scheduled.format,
created_at=scheduled.created_at,
last_run=scheduled.last_run,
next_run=scheduled.next_run,
is_active=scheduled.is_active,
)
@router.get(
"/schedule",
response_model=List[ReportScheduleOutput],
summary="List scheduled reports",
description="List all scheduled reports for a website.",
)
async def list_scheduled_reports_endpoint(
website_id: int = Depends(get_website_id_from_header),
) -> List[ReportScheduleOutput]:
"""
List all scheduled reports.
Returns all scheduled reports for the current website.
"""
reports = list_scheduled_reports(website_id=website_id)
return [
ReportScheduleOutput(
schedule_id=r.schedule_id,
report_type=r.report_type,
schedule=r.schedule,
tryout_ids=r.tryout_ids,
website_id=r.website_id,
recipients=r.recipients,
format=r.format,
created_at=r.created_at,
last_run=r.last_run,
next_run=r.next_run,
is_active=r.is_active,
)
for r in reports
]
@router.delete(
"/schedule/{schedule_id}",
summary="Cancel scheduled report",
description="Cancel a scheduled report.",
)
async def cancel_scheduled_report_endpoint(
schedule_id: str,
website_id: int = Depends(get_website_id_from_header),
) -> dict:
"""
Cancel a scheduled report.
Removes the scheduled report from the system.
"""
scheduled = get_scheduled_report(schedule_id)
if not scheduled:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Scheduled report {schedule_id} not found",
)
if scheduled.website_id != website_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this scheduled report",
)
success = cancel_scheduled_report(schedule_id)
if not success:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to cancel scheduled report",
)
return {
"message": f"Scheduled report {schedule_id} cancelled successfully",
"schedule_id": schedule_id,
}
# =============================================================================
# Report Export Endpoints
# =============================================================================
@router.get(
"/export/{schedule_id}/{format}",
summary="Export scheduled report",
description="Generate and export a scheduled report in the specified format.",
)
async def export_scheduled_report(
schedule_id: str,
format: Literal["csv", "xlsx", "pdf"],
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Export a scheduled report.
Generates the report and returns it as a file download.
"""
scheduled = get_scheduled_report(schedule_id)
if not scheduled:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Scheduled report {schedule_id} not found",
)
if scheduled.website_id != website_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this scheduled report",
)
# Generate report based on type
report = None
base_filename = f"report_{scheduled.report_type}_{schedule_id}"
try:
if scheduled.report_type == "student_performance":
if len(scheduled.tryout_ids) > 0:
report = await generate_student_performance_report(
tryout_id=scheduled.tryout_ids[0],
website_id=website_id,
db=db,
)
elif scheduled.report_type == "item_analysis":
if len(scheduled.tryout_ids) > 0:
report = await generate_item_analysis_report(
tryout_id=scheduled.tryout_ids[0],
website_id=website_id,
db=db,
)
elif scheduled.report_type == "calibration_status":
if len(scheduled.tryout_ids) > 0:
report = await generate_calibration_status_report(
tryout_id=scheduled.tryout_ids[0],
website_id=website_id,
db=db,
)
elif scheduled.report_type == "tryout_comparison":
report = await generate_tryout_comparison_report(
tryout_ids=scheduled.tryout_ids,
website_id=website_id,
db=db,
)
if not report:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to generate report",
)
# Export to requested format
if format == "csv":
file_path = export_report_to_csv(report, base_filename)
media_type = "text/csv"
elif format == "xlsx":
file_path = export_report_to_excel(report, base_filename)
media_type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
else: # pdf
file_path = export_report_to_pdf(report, base_filename)
media_type = "application/pdf"
# Return file
return FileResponse(
path=file_path,
media_type=media_type,
filename=os.path.basename(file_path),
)
except HTTPException:
raise
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to export report: {str(e)}",
)
# =============================================================================
# Direct Export Endpoints (without scheduling)
# =============================================================================
@router.get(
"/student/performance/export/{format}",
summary="Export student performance report directly",
description="Generate and export student performance report directly without scheduling.",
)
async def export_student_performance_direct(
format: Literal["csv", "xlsx", "pdf"],
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
date_start: Optional[datetime] = None,
date_end: Optional[datetime] = None,
):
"""Export student performance report directly."""
date_range = None
if date_start or date_end:
date_range = {}
if date_start:
date_range["start"] = date_start
if date_end:
date_range["end"] = date_end
report = await generate_student_performance_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
date_range=date_range,
)
base_filename = f"student_performance_{tryout_id}"
if format == "csv":
file_path = export_report_to_csv(report, base_filename)
media_type = "text/csv"
elif format == "xlsx":
file_path = export_report_to_excel(report, base_filename)
media_type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
else:
file_path = export_report_to_pdf(report, base_filename)
media_type = "application/pdf"
return FileResponse(
path=file_path,
media_type=media_type,
filename=os.path.basename(file_path),
)
@router.get(
"/items/analysis/export/{format}",
summary="Export item analysis report directly",
description="Generate and export item analysis report directly without scheduling.",
)
async def export_item_analysis_direct(
format: Literal["csv", "xlsx", "pdf"],
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
filter_by: Optional[Literal["difficulty", "calibrated", "discrimination"]] = None,
difficulty_level: Optional[Literal["mudah", "sedang", "sulit"]] = None,
):
"""Export item analysis report directly."""
report = await generate_item_analysis_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
filter_by=filter_by,
difficulty_level=difficulty_level,
)
base_filename = f"item_analysis_{tryout_id}"
if format == "csv":
file_path = export_report_to_csv(report, base_filename)
media_type = "text/csv"
elif format == "xlsx":
file_path = export_report_to_excel(report, base_filename)
media_type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
else:
file_path = export_report_to_pdf(report, base_filename)
media_type = "application/pdf"
return FileResponse(
path=file_path,
media_type=media_type,
filename=os.path.basename(file_path),
)
@router.get(
"/calibration/status/export/{format}",
summary="Export calibration status report directly",
description="Generate and export calibration status report directly without scheduling.",
)
async def export_calibration_status_direct(
format: Literal["csv", "xlsx", "pdf"],
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""Export calibration status report directly."""
report = await generate_calibration_status_report(
tryout_id=tryout_id,
website_id=website_id,
db=db,
)
base_filename = f"calibration_status_{tryout_id}"
if format == "csv":
file_path = export_report_to_csv(report, base_filename)
media_type = "text/csv"
elif format == "xlsx":
file_path = export_report_to_excel(report, base_filename)
media_type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
else:
file_path = export_report_to_pdf(report, base_filename)
media_type = "application/pdf"
return FileResponse(
path=file_path,
media_type=media_type,
filename=os.path.basename(file_path),
)
@router.get(
"/tryout/comparison/export/{format}",
summary="Export tryout comparison report directly",
description="Generate and export tryout comparison report directly without scheduling.",
)
async def export_tryout_comparison_direct(
format: Literal["csv", "xlsx", "pdf"],
tryout_ids: str, # Comma-separated
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
group_by: Literal["date", "subject"] = "date",
):
"""Export tryout comparison report directly."""
tryout_id_list = [tid.strip() for tid in tryout_ids.split(",")]
if len(tryout_id_list) < 2:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="At least 2 tryout IDs are required for comparison",
)
report = await generate_tryout_comparison_report(
tryout_ids=tryout_id_list,
website_id=website_id,
db=db,
group_by=group_by,
)
base_filename = "tryout_comparison"
if format == "csv":
file_path = export_report_to_csv(report, base_filename)
media_type = "text/csv"
elif format == "xlsx":
file_path = export_report_to_excel(report, base_filename)
media_type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
else:
file_path = export_report_to_pdf(report, base_filename)
media_type = "application/pdf"
return FileResponse(
path=file_path,
media_type=media_type,
filename=os.path.basename(file_path),
)

402
app/routers/sessions.py Normal file
View File

@@ -0,0 +1,402 @@
"""
Session API router for tryout session management.
Endpoints:
- POST /session/{session_id}/complete: Submit answers and complete session
- GET /session/{session_id}: Get session details
- POST /session: Create new session
"""
from datetime import datetime, timezone
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Header, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.models.item import Item
from app.models.session import Session
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.models.user_answer import UserAnswer
from app.schemas.session import (
SessionCompleteRequest,
SessionCompleteResponse,
SessionCreateRequest,
SessionResponse,
UserAnswerOutput,
)
from app.services.ctt_scoring import (
calculate_ctt_bobot,
calculate_ctt_nm,
calculate_ctt_nn,
get_total_bobot_max,
update_tryout_stats,
)
router = APIRouter(prefix="/session", tags=["sessions"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
@router.post(
"/{session_id}/complete",
response_model=SessionCompleteResponse,
summary="Complete session with answers",
description="Submit user answers, calculate CTT scores, and complete the session.",
)
async def complete_session(
session_id: str,
request: SessionCompleteRequest,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> SessionCompleteResponse:
"""
Complete a session by submitting answers and calculating CTT scores.
Process:
1. Validate session exists and is not completed
2. For each answer: check is_correct, calculate bobot_earned
3. Save UserAnswer records
4. Calculate CTT scores (total_benar, total_bobot_earned, NM)
5. Update Session with CTT results
6. Update TryoutStats incrementally
7. Return session with scores
Args:
session_id: Unique session identifier
request: Session completion request with end_time and user_answers
db: Database session
website_id: Website ID from header
Returns:
SessionCompleteResponse with CTT scores
Raises:
HTTPException: If session not found, already completed, or validation fails
"""
# Get session with tryout relationship
result = await db.execute(
select(Session)
.options(selectinload(Session.tryout))
.where(
Session.session_id == session_id,
Session.website_id == website_id,
)
)
session = result.scalar_one_or_none()
if session is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Session {session_id} not found",
)
if session.is_completed:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Session is already completed",
)
# Get tryout configuration
tryout = session.tryout
# Get all items for this tryout to calculate bobot
items_result = await db.execute(
select(Item).where(
Item.website_id == website_id,
Item.tryout_id == session.tryout_id,
)
)
items = {item.id: item for item in items_result.scalars().all()}
# Process each answer
total_benar = 0
total_bobot_earned = 0.0
user_answer_records = []
for answer_input in request.user_answers:
item = items.get(answer_input.item_id)
if item is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Item {answer_input.item_id} not found in tryout {session.tryout_id}",
)
# Check if answer is correct
is_correct = answer_input.response.upper() == item.correct_answer.upper()
# Calculate bobot_earned (only if correct)
bobot_earned = 0.0
if is_correct:
total_benar += 1
if item.ctt_bobot is not None:
bobot_earned = item.ctt_bobot
total_bobot_earned += bobot_earned
# Create UserAnswer record
user_answer = UserAnswer(
session_id=session.session_id,
wp_user_id=session.wp_user_id,
website_id=website_id,
tryout_id=session.tryout_id,
item_id=item.id,
response=answer_input.response.upper(),
is_correct=is_correct,
time_spent=answer_input.time_spent,
scoring_mode_used=session.scoring_mode_used,
bobot_earned=bobot_earned,
)
user_answer_records.append(user_answer)
db.add(user_answer)
# Calculate total_bobot_max for NM calculation
try:
total_bobot_max = await get_total_bobot_max(
db, website_id, session.tryout_id, level="sedang"
)
except ValueError:
# Fallback: calculate from items we have
total_bobot_max = sum(
item.ctt_bobot or 0 for item in items.values() if item.level == "sedang"
)
if total_bobot_max == 0:
# If no bobot values, use count of questions
total_bobot_max = len(items)
# Calculate CTT NM (Nilai Mentah)
nm = calculate_ctt_nm(total_bobot_earned, total_bobot_max)
# Get normalization parameters based on tryout configuration
if tryout.normalization_mode == "static":
rataan = tryout.static_rataan
sb = tryout.static_sb
elif tryout.normalization_mode == "dynamic":
# Get current stats for dynamic normalization
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats and stats.participant_count >= tryout.min_sample_for_dynamic:
rataan = stats.rataan or tryout.static_rataan
sb = stats.sb or tryout.static_sb
else:
# Not enough data, use static values
rataan = tryout.static_rataan
sb = tryout.static_sb
else: # hybrid
# Hybrid: use dynamic if enough data, otherwise static
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == session.tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats and stats.participant_count >= tryout.min_sample_for_dynamic:
rataan = stats.rataan or tryout.static_rataan
sb = stats.sb or tryout.static_sb
else:
rataan = tryout.static_rataan
sb = tryout.static_sb
# Calculate CTT NN (Nilai Nasional)
nn = calculate_ctt_nn(nm, rataan, sb)
# Update session with results
session.end_time = request.end_time
session.is_completed = True
session.total_benar = total_benar
session.total_bobot_earned = total_bobot_earned
session.NM = nm
session.NN = nn
session.rataan_used = rataan
session.sb_used = sb
# Update tryout stats incrementally
await update_tryout_stats(db, website_id, session.tryout_id, nm)
# Commit all changes
await db.commit()
# Refresh to get updated relationships
await db.refresh(session)
# Build response
return SessionCompleteResponse(
id=session.id,
session_id=session.session_id,
wp_user_id=session.wp_user_id,
website_id=session.website_id,
tryout_id=session.tryout_id,
start_time=session.start_time,
end_time=session.end_time,
is_completed=session.is_completed,
scoring_mode_used=session.scoring_mode_used,
total_benar=session.total_benar,
total_bobot_earned=session.total_bobot_earned,
NM=session.NM,
NN=session.NN,
rataan_used=session.rataan_used,
sb_used=session.sb_used,
user_answers=[
UserAnswerOutput(
id=ua.id,
item_id=ua.item_id,
response=ua.response,
is_correct=ua.is_correct,
time_spent=ua.time_spent,
bobot_earned=ua.bobot_earned,
scoring_mode_used=ua.scoring_mode_used,
)
for ua in user_answer_records
],
)
@router.get(
"/{session_id}",
response_model=SessionResponse,
summary="Get session details",
description="Retrieve session details including scores if completed.",
)
async def get_session(
session_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> SessionResponse:
"""
Get session details.
Args:
session_id: Unique session identifier
db: Database session
website_id: Website ID from header
Returns:
SessionResponse with session details
Raises:
HTTPException: If session not found
"""
result = await db.execute(
select(Session).where(
Session.session_id == session_id,
Session.website_id == website_id,
)
)
session = result.scalar_one_or_none()
if session is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Session {session_id} not found",
)
return SessionResponse.model_validate(session)
@router.post(
"/",
response_model=SessionResponse,
status_code=status.HTTP_201_CREATED,
summary="Create new session",
description="Create a new tryout session for a student.",
)
async def create_session(
request: SessionCreateRequest,
db: AsyncSession = Depends(get_db),
) -> SessionResponse:
"""
Create a new session.
Args:
request: Session creation request
db: Database session
Returns:
SessionResponse with created session
Raises:
HTTPException: If tryout not found or session already exists
"""
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == request.website_id,
Tryout.tryout_id == request.tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {request.tryout_id} not found for website {request.website_id}",
)
# Check if session already exists
existing_result = await db.execute(
select(Session).where(Session.session_id == request.session_id)
)
existing_session = existing_result.scalar_one_or_none()
if existing_session:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"Session {request.session_id} already exists",
)
# Create new session
session = Session(
session_id=request.session_id,
wp_user_id=request.wp_user_id,
website_id=request.website_id,
tryout_id=request.tryout_id,
scoring_mode_used=request.scoring_mode,
start_time=datetime.now(timezone.utc),
is_completed=False,
total_benar=0,
total_bobot_earned=0.0,
)
db.add(session)
await db.commit()
await db.refresh(session)
return SessionResponse.model_validate(session)

458
app/routers/tryouts.py Normal file
View File

@@ -0,0 +1,458 @@
"""
Tryout API router for tryout configuration and management.
Endpoints:
- GET /tryout/{tryout_id}/config: Get tryout configuration
- PUT /tryout/{tryout_id}/normalization: Update normalization settings
- GET /tryout: List tryouts for a website
"""
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException, Header, status
from sqlalchemy import select, func
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.models.item import Item
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
from app.schemas.tryout import (
NormalizationUpdateRequest,
NormalizationUpdateResponse,
TryoutConfigBrief,
TryoutConfigResponse,
TryoutStatsResponse,
)
router = APIRouter(prefix="/tryout", tags=["tryouts"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
@router.get(
"/{tryout_id}/config",
response_model=TryoutConfigResponse,
summary="Get tryout configuration",
description="Retrieve tryout configuration including scoring mode, normalization settings, and current stats.",
)
async def get_tryout_config(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> TryoutConfigResponse:
"""
Get tryout configuration.
Returns:
TryoutConfigResponse with scoring_mode, normalization_mode, and current_stats
Raises:
HTTPException: If tryout not found
"""
# Get tryout with stats
result = await db.execute(
select(Tryout)
.options(selectinload(Tryout.stats))
.where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Build stats response
current_stats = None
if tryout.stats:
current_stats = TryoutStatsResponse(
participant_count=tryout.stats.participant_count,
rataan=tryout.stats.rataan,
sb=tryout.stats.sb,
min_nm=tryout.stats.min_nm,
max_nm=tryout.stats.max_nm,
last_calculated=tryout.stats.last_calculated,
)
return TryoutConfigResponse(
id=tryout.id,
website_id=tryout.website_id,
tryout_id=tryout.tryout_id,
name=tryout.name,
description=tryout.description,
scoring_mode=tryout.scoring_mode,
selection_mode=tryout.selection_mode,
normalization_mode=tryout.normalization_mode,
min_sample_for_dynamic=tryout.min_sample_for_dynamic,
static_rataan=tryout.static_rataan,
static_sb=tryout.static_sb,
ai_generation_enabled=tryout.ai_generation_enabled,
hybrid_transition_slot=tryout.hybrid_transition_slot,
min_calibration_sample=tryout.min_calibration_sample,
theta_estimation_method=tryout.theta_estimation_method,
fallback_to_ctt_on_error=tryout.fallback_to_ctt_on_error,
current_stats=current_stats,
created_at=tryout.created_at,
updated_at=tryout.updated_at,
)
@router.put(
"/{tryout_id}/normalization",
response_model=NormalizationUpdateResponse,
summary="Update normalization settings",
description="Update normalization mode and static values for a tryout.",
)
async def update_normalization(
tryout_id: str,
request: NormalizationUpdateRequest,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> NormalizationUpdateResponse:
"""
Update normalization settings for a tryout.
Args:
tryout_id: Tryout identifier
request: Normalization update request
db: Database session
website_id: Website ID from header
Returns:
NormalizationUpdateResponse with updated settings
Raises:
HTTPException: If tryout not found or validation fails
"""
# Get tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Update normalization mode if provided
if request.normalization_mode is not None:
tryout.normalization_mode = request.normalization_mode
# Update static values if provided
if request.static_rataan is not None:
tryout.static_rataan = request.static_rataan
if request.static_sb is not None:
tryout.static_sb = request.static_sb
# Get current stats for participant count
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
current_participant_count = stats.participant_count if stats else 0
await db.commit()
await db.refresh(tryout)
return NormalizationUpdateResponse(
tryout_id=tryout.tryout_id,
normalization_mode=tryout.normalization_mode,
static_rataan=tryout.static_rataan,
static_sb=tryout.static_sb,
will_switch_to_dynamic_at=tryout.min_sample_for_dynamic,
current_participant_count=current_participant_count,
)
@router.get(
"/",
response_model=List[TryoutConfigBrief],
summary="List tryouts",
description="List all tryouts for a website.",
)
async def list_tryouts(
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
) -> List[TryoutConfigBrief]:
"""
List all tryouts for a website.
Args:
db: Database session
website_id: Website ID from header
Returns:
List of TryoutConfigBrief
"""
# Get tryouts with stats
result = await db.execute(
select(Tryout)
.options(selectinload(Tryout.stats))
.where(Tryout.website_id == website_id)
)
tryouts = result.scalars().all()
return [
TryoutConfigBrief(
tryout_id=t.tryout_id,
name=t.name,
scoring_mode=t.scoring_mode,
selection_mode=t.selection_mode,
normalization_mode=t.normalization_mode,
participant_count=t.stats.participant_count if t.stats else 0,
)
for t in tryouts
]
@router.get(
"/{tryout_id}/calibration-status",
summary="Get calibration status",
description="Get IRT calibration status for items in this tryout.",
)
async def get_calibration_status(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Get calibration status for items in a tryout.
Returns statistics on how many items are calibrated and ready for IRT.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Calibration status summary
Raises:
HTTPException: If tryout not found
"""
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Get calibration statistics
stats_result = await db.execute(
select(
func.count().label("total_items"),
func.sum(func.cast(Item.calibrated, type_=func.INTEGER)).label("calibrated_items"),
func.avg(Item.calibration_sample_size).label("avg_sample_size"),
).where(
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
stats = stats_result.first()
total_items = stats.total_items or 0
calibrated_items = stats.calibrated_items or 0
calibration_percentage = (calibrated_items / total_items * 100) if total_items > 0 else 0
return {
"tryout_id": tryout_id,
"total_items": total_items,
"calibrated_items": calibrated_items,
"calibration_percentage": round(calibration_percentage, 2),
"avg_sample_size": round(stats.avg_sample_size, 2) if stats.avg_sample_size else 0,
"min_calibration_sample": tryout.min_calibration_sample,
"ready_for_irt": calibration_percentage >= 90,
}
@router.post(
"/{tryout_id}/calibrate",
summary="Trigger IRT calibration",
description="Trigger IRT calibration for all items in this tryout with sufficient response data.",
)
async def trigger_calibration(
tryout_id: str,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Trigger IRT calibration for all items in a tryout.
Runs calibration for items with >= min_calibration_sample responses.
Updates item.irt_b, item.irt_se, and item.calibrated status.
Args:
tryout_id: Tryout identifier
db: Database session
website_id: Website ID from header
Returns:
Calibration results summary
Raises:
HTTPException: If tryout not found or calibration fails
"""
from app.services.irt_calibration import (
calibrate_all,
CALIBRATION_SAMPLE_THRESHOLD,
)
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Run calibration
result = await calibrate_all(
tryout_id=tryout_id,
website_id=website_id,
db=db,
min_sample_size=tryout.min_calibration_sample or CALIBRATION_SAMPLE_THRESHOLD,
)
return {
"tryout_id": tryout_id,
"total_items": result.total_items,
"calibrated_items": result.calibrated_items,
"failed_items": result.failed_items,
"calibration_percentage": round(result.calibration_percentage * 100, 2),
"ready_for_irt": result.ready_for_irt,
"message": f"Calibration complete: {result.calibrated_items}/{result.total_items} items calibrated",
}
@router.post(
"/{tryout_id}/calibrate/{item_id}",
summary="Trigger IRT calibration for single item",
description="Trigger IRT calibration for a specific item.",
)
async def trigger_item_calibration(
tryout_id: str,
item_id: int,
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
):
"""
Trigger IRT calibration for a single item.
Args:
tryout_id: Tryout identifier
item_id: Item ID to calibrate
db: Database session
website_id: Website ID from header
Returns:
Calibration result for the item
Raises:
HTTPException: If tryout or item not found
"""
from app.services.irt_calibration import calibrate_item, CALIBRATION_SAMPLE_THRESHOLD
# Verify tryout exists
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Tryout {tryout_id} not found for website {website_id}",
)
# Verify item belongs to this tryout
item_result = await db.execute(
select(Item).where(
Item.id == item_id,
Item.website_id == website_id,
Item.tryout_id == tryout_id,
)
)
item = item_result.scalar_one_or_none()
if item is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Item {item_id} not found in tryout {tryout_id}",
)
# Run calibration
result = await calibrate_item(
item_id=item_id,
db=db,
min_sample_size=tryout.min_calibration_sample or CALIBRATION_SAMPLE_THRESHOLD,
)
return {
"item_id": result.item_id,
"status": result.status.value,
"irt_b": result.irt_b,
"irt_se": result.irt_se,
"sample_size": result.sample_size,
"message": result.message,
}

384
app/routers/wordpress.py Normal file
View File

@@ -0,0 +1,384 @@
"""
WordPress Integration API Router.
Endpoints:
- POST /wordpress/sync_users: Synchronize users from WordPress
- POST /wordpress/verify_session: Verify WordPress session/token
- GET /wordpress/website/{website_id}/users: Get all users for a website
"""
import logging
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Header, status
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.user import User
from app.models.website import Website
from app.schemas.wordpress import (
SyncUsersResponse,
SyncStatsResponse,
UserListResponse,
VerifySessionRequest,
VerifySessionResponse,
WordPressUserResponse,
)
from app.services.wordpress_auth import (
get_wordpress_user,
sync_wordpress_users,
verify_website_exists,
verify_wordpress_token,
get_or_create_user,
WordPressAPIError,
WordPressRateLimitError,
WordPressTokenInvalidError,
WebsiteNotFoundError,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/wordpress", tags=["wordpress"])
def get_website_id_from_header(
x_website_id: Optional[str] = Header(None, alias="X-Website-ID"),
) -> int:
"""
Extract and validate website_id from request header.
Args:
x_website_id: Website ID from header
Returns:
Validated website ID as integer
Raises:
HTTPException: If header is missing or invalid
"""
if x_website_id is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID header is required",
)
try:
return int(x_website_id)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="X-Website-ID must be a valid integer",
)
async def get_valid_website(
website_id: int,
db: AsyncSession,
) -> Website:
"""
Validate website_id exists and return Website model.
Args:
website_id: Website identifier
db: Database session
Returns:
Website model instance
Raises:
HTTPException: If website not found
"""
try:
return await verify_website_exists(website_id, db)
except WebsiteNotFoundError:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Website {website_id} not found",
)
@router.post(
"/sync_users",
response_model=SyncUsersResponse,
summary="Synchronize users from WordPress",
description="Fetch all users from WordPress API and sync to local database. Requires admin WordPress token.",
)
async def sync_users_endpoint(
db: AsyncSession = Depends(get_db),
website_id: int = Depends(get_website_id_from_header),
authorization: Optional[str] = Header(None, alias="Authorization"),
) -> SyncUsersResponse:
"""
Synchronize users from WordPress to local database.
Process:
1. Validate website_id exists
2. Extract admin token from Authorization header
3. Fetch all users from WordPress API
4. Upsert: Update existing users, insert new users
5. Return sync statistics
Args:
db: Database session
website_id: Website ID from header
authorization: Authorization header with Bearer token
Returns:
SyncUsersResponse with sync statistics
Raises:
HTTPException: If website not found, token invalid, or API error
"""
# Validate website exists
await get_valid_website(website_id, db)
# Extract token from Authorization header
if authorization is None:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Authorization header is required",
)
# Parse Bearer token
parts = authorization.split()
if len(parts) != 2 or parts[0].lower() != "bearer":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid Authorization header format. Use: Bearer {token}",
)
admin_token = parts[1]
try:
sync_stats = await sync_wordpress_users(
website_id=website_id,
admin_token=admin_token,
db=db,
)
return SyncUsersResponse(
synced=SyncStatsResponse(
inserted=sync_stats.inserted,
updated=sync_stats.updated,
total=sync_stats.total,
errors=sync_stats.errors,
),
website_id=website_id,
message=f"Sync completed: {sync_stats.inserted} inserted, {sync_stats.updated} updated",
)
except WordPressTokenInvalidError as e:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail=str(e),
)
except WordPressRateLimitError as e:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=str(e),
)
except WordPressAPIError as e:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=str(e),
)
except WebsiteNotFoundError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
@router.post(
"/verify_session",
response_model=VerifySessionResponse,
summary="Verify WordPress session",
description="Verify WordPress JWT token and user identity.",
)
async def verify_session_endpoint(
request: VerifySessionRequest,
db: AsyncSession = Depends(get_db),
) -> VerifySessionResponse:
"""
Verify WordPress session/token.
Process:
1. Validate website_id exists
2. Call WordPress API to verify token
3. Verify wp_user_id matches token owner
4. Get or create local user
5. Return validation result
Args:
request: VerifySessionRequest with wp_user_id, token, website_id
db: Database session
Returns:
VerifySessionResponse with validation result
Raises:
HTTPException: If website not found or API error
"""
# Validate website exists
await get_valid_website(request.website_id, db)
try:
# Verify token with WordPress
wp_user_info = await verify_wordpress_token(
token=request.token,
website_id=request.website_id,
wp_user_id=request.wp_user_id,
db=db,
)
if wp_user_info is None:
return VerifySessionResponse(
valid=False,
error="User ID mismatch or invalid credentials",
)
# Get or create local user
user = await get_or_create_user(
wp_user_id=request.wp_user_id,
website_id=request.website_id,
db=db,
)
return VerifySessionResponse(
valid=True,
user=WordPressUserResponse.model_validate(user),
wp_user_info={
"username": wp_user_info.username,
"email": wp_user_info.email,
"display_name": wp_user_info.display_name,
"roles": wp_user_info.roles,
},
)
except WordPressTokenInvalidError as e:
return VerifySessionResponse(
valid=False,
error=f"Invalid credentials: {str(e)}",
)
except WordPressRateLimitError as e:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=str(e),
)
except WordPressAPIError as e:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=str(e),
)
except WebsiteNotFoundError as e:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(e),
)
@router.get(
"/website/{website_id}/users",
response_model=UserListResponse,
summary="Get users for website",
description="Retrieve all users for a specific website from local database with pagination.",
)
async def get_website_users(
website_id: int,
db: AsyncSession = Depends(get_db),
page: int = 1,
page_size: int = 50,
) -> UserListResponse:
"""
Get all users for a website.
Args:
website_id: Website identifier
db: Database session
page: Page number (default: 1)
page_size: Number of users per page (default: 50, max: 100)
Returns:
UserListResponse with paginated user list
Raises:
HTTPException: If website not found
"""
# Validate website exists
await get_valid_website(website_id, db)
# Clamp page_size
page_size = min(max(1, page_size), 100)
page = max(1, page)
# Get total count
count_result = await db.execute(
select(func.count()).select_from(User).where(User.website_id == website_id)
)
total = count_result.scalar() or 0
# Calculate pagination
offset = (page - 1) * page_size
total_pages = (total + page_size - 1) // page_size if total > 0 else 1
# Get users
result = await db.execute(
select(User)
.where(User.website_id == website_id)
.order_by(User.id)
.offset(offset)
.limit(page_size)
)
users = result.scalars().all()
return UserListResponse(
users=[WordPressUserResponse.model_validate(user) for user in users],
total=total,
page=page,
page_size=page_size,
total_pages=total_pages,
)
@router.get(
"/website/{website_id}/user/{wp_user_id}",
response_model=WordPressUserResponse,
summary="Get specific user",
description="Retrieve a specific user by WordPress user ID.",
)
async def get_user_endpoint(
website_id: int,
wp_user_id: str,
db: AsyncSession = Depends(get_db),
) -> WordPressUserResponse:
"""
Get a specific user by WordPress user ID.
Args:
website_id: Website identifier
wp_user_id: WordPress user ID
db: Database session
Returns:
WordPressUserResponse with user data
Raises:
HTTPException: If website or user not found
"""
# Validate website exists
await get_valid_website(website_id, db)
# Get user
user = await get_wordpress_user(
wp_user_id=wp_user_id,
website_id=website_id,
db=db,
)
if user is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"User {wp_user_id} not found for website {website_id}",
)
return WordPressUserResponse.model_validate(user)

65
app/schemas/__init__.py Normal file
View File

@@ -0,0 +1,65 @@
"""
Pydantic schemas package.
"""
from app.schemas.ai import (
AIGeneratePreviewRequest,
AIGeneratePreviewResponse,
AISaveRequest,
AISaveResponse,
AIStatsResponse,
GeneratedQuestion,
)
from app.schemas.session import (
SessionCompleteRequest,
SessionCompleteResponse,
SessionCreateRequest,
SessionResponse,
UserAnswerInput,
UserAnswerOutput,
)
from app.schemas.tryout import (
NormalizationUpdateRequest,
NormalizationUpdateResponse,
TryoutConfigBrief,
TryoutConfigResponse,
TryoutStatsResponse,
)
from app.schemas.wordpress import (
SyncStatsResponse,
SyncUsersResponse,
UserListResponse,
VerifySessionRequest,
VerifySessionResponse,
WordPressUserResponse,
)
__all__ = [
# AI schemas
"AIGeneratePreviewRequest",
"AIGeneratePreviewResponse",
"AISaveRequest",
"AISaveResponse",
"AIStatsResponse",
"GeneratedQuestion",
# Session schemas
"UserAnswerInput",
"UserAnswerOutput",
"SessionCompleteRequest",
"SessionCompleteResponse",
"SessionCreateRequest",
"SessionResponse",
# Tryout schemas
"TryoutConfigResponse",
"TryoutStatsResponse",
"TryoutConfigBrief",
"NormalizationUpdateRequest",
"NormalizationUpdateResponse",
# WordPress schemas
"SyncStatsResponse",
"SyncUsersResponse",
"UserListResponse",
"VerifySessionRequest",
"VerifySessionResponse",
"WordPressUserResponse",
]

102
app/schemas/ai.py Normal file
View File

@@ -0,0 +1,102 @@
"""
Pydantic schemas for AI generation endpoints.
Request/response models for admin AI generation playground.
"""
from typing import Dict, Literal, Optional
from pydantic import BaseModel, Field, field_validator
class AIGeneratePreviewRequest(BaseModel):
basis_item_id: int = Field(
..., description="ID of the basis item (must be sedang level)"
)
target_level: Literal["mudah", "sulit"] = Field(
..., description="Target difficulty level for generated question"
)
ai_model: str = Field(
default="qwen/qwen-2.5-coder-32b-instruct",
description="AI model to use for generation",
)
class AIGeneratePreviewResponse(BaseModel):
success: bool = Field(..., description="Whether generation was successful")
stem: Optional[str] = None
options: Optional[Dict[str, str]] = None
correct: Optional[str] = None
explanation: Optional[str] = None
ai_model: Optional[str] = None
basis_item_id: Optional[int] = None
target_level: Optional[str] = None
error: Optional[str] = None
cached: bool = False
class AISaveRequest(BaseModel):
stem: str = Field(..., description="Question stem")
options: Dict[str, str] = Field(
..., description="Answer options (A, B, C, D)"
)
correct: str = Field(..., description="Correct answer (A/B/C/D)")
explanation: Optional[str] = None
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
basis_item_id: int = Field(..., description="Basis item ID")
slot: int = Field(..., description="Question slot position")
level: Literal["mudah", "sedang", "sulit"] = Field(
..., description="Difficulty level"
)
ai_model: str = Field(
default="qwen/qwen-2.5-coder-32b-instruct",
description="AI model used for generation",
)
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()
@field_validator("options")
@classmethod
def validate_options(cls, v: Dict[str, str]) -> Dict[str, str]:
required_keys = {"A", "B", "C", "D"}
if not required_keys.issubset(set(v.keys())):
raise ValueError("Options must contain keys A, B, C, D")
return v
class AISaveResponse(BaseModel):
success: bool = Field(..., description="Whether save was successful")
item_id: Optional[int] = None
error: Optional[str] = None
class AIStatsResponse(BaseModel):
total_ai_items: int = Field(..., description="Total AI-generated items")
items_by_model: Dict[str, int] = Field(
default_factory=dict, description="Items count by AI model"
)
cache_hit_rate: float = Field(
default=0.0, description="Cache hit rate (0.0 to 1.0)"
)
total_cache_hits: int = Field(default=0, description="Total cache hits")
total_requests: int = Field(default=0, description="Total generation requests")
class GeneratedQuestion(BaseModel):
stem: str
options: Dict[str, str]
correct: str
explanation: Optional[str] = None
@field_validator("correct")
@classmethod
def validate_correct(cls, v: str) -> str:
if v.upper() not in ["A", "B", "C", "D"]:
raise ValueError("Correct answer must be A, B, C, or D")
return v.upper()

264
app/schemas/report.py Normal file
View File

@@ -0,0 +1,264 @@
"""
Pydantic schemas for Report API endpoints.
"""
from datetime import datetime
from typing import Any, Dict, List, Literal, Optional
from pydantic import BaseModel, Field
# =============================================================================
# Student Performance Report Schemas
# =============================================================================
class StudentPerformanceRecordOutput(BaseModel):
"""Individual student performance record output."""
session_id: str
wp_user_id: str
tryout_id: str
NM: Optional[int] = None
NN: Optional[int] = None
theta: Optional[float] = None
theta_se: Optional[float] = None
total_benar: int
time_spent: int # Total time in seconds
start_time: Optional[datetime] = None
end_time: Optional[datetime] = None
scoring_mode_used: str
rataan_used: Optional[float] = None
sb_used: Optional[float] = None
class AggregatePerformanceStatsOutput(BaseModel):
"""Aggregate statistics for student performance output."""
tryout_id: str
participant_count: int
avg_nm: Optional[float] = None
std_nm: Optional[float] = None
min_nm: Optional[int] = None
max_nm: Optional[int] = None
median_nm: Optional[float] = None
avg_nn: Optional[float] = None
std_nn: Optional[float] = None
avg_theta: Optional[float] = None
pass_rate: float # Percentage with NN >= 500
avg_time_spent: float # Average time in seconds
class StudentPerformanceReportOutput(BaseModel):
"""Complete student performance report output."""
generated_at: datetime
tryout_id: str
website_id: int
date_range: Optional[Dict[str, str]] = None
aggregate: AggregatePerformanceStatsOutput
individual_records: List[StudentPerformanceRecordOutput] = []
class StudentPerformanceReportRequest(BaseModel):
"""Request schema for student performance report."""
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
date_start: Optional[datetime] = Field(None, description="Filter by start date")
date_end: Optional[datetime] = Field(None, description="Filter by end date")
format_type: Literal["individual", "aggregate", "both"] = Field(
default="both", description="Report format"
)
# =============================================================================
# Item Analysis Report Schemas
# =============================================================================
class ItemAnalysisRecordOutput(BaseModel):
"""Item analysis record output for a single item."""
item_id: int
slot: int
level: str
ctt_p: Optional[float] = None
ctt_bobot: Optional[float] = None
ctt_category: Optional[str] = None
irt_b: Optional[float] = None
irt_se: Optional[float] = None
calibrated: bool
calibration_sample_size: int
correctness_rate: float
item_total_correlation: Optional[float] = None
information_values: Dict[float, float] = Field(default_factory=dict)
optimal_theta_range: str = "N/A"
class ItemAnalysisReportOutput(BaseModel):
"""Complete item analysis report output."""
generated_at: datetime
tryout_id: str
website_id: int
total_items: int
items: List[ItemAnalysisRecordOutput]
summary: Dict[str, Any]
class ItemAnalysisReportRequest(BaseModel):
"""Request schema for item analysis report."""
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
filter_by: Optional[Literal["difficulty", "calibrated", "discrimination"]] = Field(
None, description="Filter items by category"
)
difficulty_level: Optional[Literal["mudah", "sedang", "sulit"]] = Field(
None, description="Filter by difficulty level (only when filter_by='difficulty')"
)
# =============================================================================
# Calibration Status Report Schemas
# =============================================================================
class CalibrationItemStatusOutput(BaseModel):
"""Calibration status for a single item output."""
item_id: int
slot: int
level: str
sample_size: int
calibrated: bool
irt_b: Optional[float] = None
irt_se: Optional[float] = None
ctt_p: Optional[float] = None
class CalibrationStatusReportOutput(BaseModel):
"""Complete calibration status report output."""
generated_at: datetime
tryout_id: str
website_id: int
total_items: int
calibrated_items: int
calibration_percentage: float
items_awaiting_calibration: List[CalibrationItemStatusOutput]
avg_calibration_sample_size: float
estimated_time_to_90_percent: Optional[str] = None
ready_for_irt_rollout: bool
items: List[CalibrationItemStatusOutput]
class CalibrationStatusReportRequest(BaseModel):
"""Request schema for calibration status report."""
tryout_id: str = Field(..., description="Tryout identifier")
website_id: int = Field(..., description="Website identifier")
# =============================================================================
# Tryout Comparison Report Schemas
# =============================================================================
class TryoutComparisonRecordOutput(BaseModel):
"""Tryout comparison data point output."""
tryout_id: str
date: Optional[str] = None
subject: Optional[str] = None
participant_count: int
avg_nm: Optional[float] = None
avg_nn: Optional[float] = None
avg_theta: Optional[float] = None
std_nm: Optional[float] = None
calibration_percentage: float
class TryoutComparisonReportOutput(BaseModel):
"""Complete tryout comparison report output."""
generated_at: datetime
comparison_type: Literal["date", "subject"]
tryouts: List[TryoutComparisonRecordOutput]
trends: Optional[Dict[str, Any]] = None
normalization_impact: Optional[Dict[str, Any]] = None
class TryoutComparisonReportRequest(BaseModel):
"""Request schema for tryout comparison report."""
tryout_ids: List[str] = Field(..., min_length=2, description="List of tryout IDs to compare")
website_id: int = Field(..., description="Website identifier")
group_by: Literal["date", "subject"] = Field(
default="date", description="Group comparison by date or subject"
)
# =============================================================================
# Report Scheduling Schemas
# =============================================================================
class ReportScheduleRequest(BaseModel):
"""Request schema for scheduling a report."""
report_type: Literal["student_performance", "item_analysis", "calibration_status", "tryout_comparison"] = Field(
..., description="Type of report to generate"
)
schedule: Literal["daily", "weekly", "monthly"] = Field(
..., description="Schedule frequency"
)
tryout_ids: List[str] = Field(..., description="List of tryout IDs for the report")
website_id: int = Field(..., description="Website identifier")
recipients: List[str] = Field(..., description="List of email addresses to send report to")
export_format: Literal["csv", "xlsx", "pdf"] = Field(
default="xlsx", description="Export format for the report"
)
class ReportScheduleOutput(BaseModel):
"""Output schema for scheduled report."""
schedule_id: str
report_type: str
schedule: str
tryout_ids: List[str]
website_id: int
recipients: List[str]
format: str
created_at: datetime
last_run: Optional[datetime] = None
next_run: Optional[datetime] = None
is_active: bool
class ReportScheduleResponse(BaseModel):
"""Response schema for schedule creation."""
schedule_id: str
message: str
next_run: Optional[datetime] = None
# =============================================================================
# Export Schemas
# =============================================================================
class ExportRequest(BaseModel):
"""Request schema for exporting a report."""
schedule_id: str = Field(..., description="Schedule ID to generate report for")
export_format: Literal["csv", "xlsx", "pdf"] = Field(
default="xlsx", description="Export format"
)
class ExportResponse(BaseModel):
"""Response schema for export request."""
file_path: str
file_name: str
format: str
generated_at: datetime
download_url: Optional[str] = None

108
app/schemas/session.py Normal file
View File

@@ -0,0 +1,108 @@
"""
Pydantic schemas for Session API endpoints.
"""
from datetime import datetime
from typing import List, Literal, Optional
from pydantic import BaseModel, Field
class UserAnswerInput(BaseModel):
"""Input schema for a single user answer."""
item_id: int = Field(..., description="Item/question ID")
response: str = Field(..., min_length=1, max_length=10, description="User's answer (A, B, C, D)")
time_spent: int = Field(default=0, ge=0, description="Time spent on this question (seconds)")
class SessionCompleteRequest(BaseModel):
"""Request schema for completing a session."""
end_time: datetime = Field(..., description="Session end timestamp")
user_answers: List[UserAnswerInput] = Field(..., description="List of user answers")
class UserAnswerOutput(BaseModel):
"""Output schema for a single user answer."""
id: int
item_id: int
response: str
is_correct: bool
time_spent: int
bobot_earned: float
scoring_mode_used: str
model_config = {"from_attributes": True}
class SessionCompleteResponse(BaseModel):
"""Response schema for completed session with CTT scores."""
id: int
session_id: str
wp_user_id: str
website_id: int
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
is_completed: bool
scoring_mode_used: str
# CTT scores
total_benar: int = Field(description="Total correct answers")
total_bobot_earned: float = Field(description="Total weight earned")
NM: Optional[int] = Field(description="Nilai Mentah (raw score) [0, 1000]")
NN: Optional[int] = Field(description="Nilai Nasional (normalized score) [0, 1000]")
# Normalization metadata
rataan_used: Optional[float] = Field(description="Mean value used for normalization")
sb_used: Optional[float] = Field(description="Standard deviation used for normalization")
# User answers
user_answers: List[UserAnswerOutput]
model_config = {"from_attributes": True}
class SessionCreateRequest(BaseModel):
"""Request schema for creating a new session."""
session_id: str = Field(..., description="Unique session identifier")
wp_user_id: str = Field(..., description="WordPress user ID")
website_id: int = Field(..., description="Website identifier")
tryout_id: str = Field(..., description="Tryout identifier")
scoring_mode: Literal["ctt", "irt", "hybrid"] = Field(
default="ctt", description="Scoring mode for this session"
)
class SessionResponse(BaseModel):
"""Response schema for session data."""
id: int
session_id: str
wp_user_id: str
website_id: int
tryout_id: str
start_time: datetime
end_time: Optional[datetime]
is_completed: bool
scoring_mode_used: str
# CTT scores (populated after completion)
total_benar: int
total_bobot_earned: float
NM: Optional[int]
NN: Optional[int]
# IRT scores (populated after completion)
theta: Optional[float]
theta_se: Optional[float]
# Normalization metadata
rataan_used: Optional[float]
sb_used: Optional[float]
model_config = {"from_attributes": True}

97
app/schemas/tryout.py Normal file
View File

@@ -0,0 +1,97 @@
"""
Pydantic schemas for Tryout API endpoints.
"""
from datetime import datetime
from typing import List, Literal, Optional
from pydantic import BaseModel, Field
class TryoutConfigResponse(BaseModel):
"""Response schema for tryout configuration."""
id: int
website_id: int
tryout_id: str
name: str
description: Optional[str]
# Scoring configuration
scoring_mode: Literal["ctt", "irt", "hybrid"]
selection_mode: Literal["fixed", "adaptive", "hybrid"]
normalization_mode: Literal["static", "dynamic", "hybrid"]
# Normalization settings
min_sample_for_dynamic: int
static_rataan: float
static_sb: float
# AI generation
ai_generation_enabled: bool
# Hybrid mode settings
hybrid_transition_slot: Optional[int]
# IRT settings
min_calibration_sample: int
theta_estimation_method: Literal["mle", "map", "eap"]
fallback_to_ctt_on_error: bool
# Current stats
current_stats: Optional["TryoutStatsResponse"]
# Timestamps
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class TryoutStatsResponse(BaseModel):
"""Response schema for tryout statistics."""
participant_count: int
rataan: Optional[float]
sb: Optional[float]
min_nm: Optional[int]
max_nm: Optional[int]
last_calculated: Optional[datetime]
model_config = {"from_attributes": True}
class TryoutConfigBrief(BaseModel):
"""Brief tryout config for list responses."""
tryout_id: str
name: str
scoring_mode: str
selection_mode: str
normalization_mode: str
participant_count: Optional[int] = None
model_config = {"from_attributes": True}
class NormalizationUpdateRequest(BaseModel):
"""Request schema for updating normalization settings."""
normalization_mode: Optional[Literal["static", "dynamic", "hybrid"]] = None
static_rataan: Optional[float] = Field(None, ge=0)
static_sb: Optional[float] = Field(None, gt=0)
class NormalizationUpdateResponse(BaseModel):
"""Response schema for normalization update."""
tryout_id: str
normalization_mode: str
static_rataan: float
static_sb: float
will_switch_to_dynamic_at: int
current_participant_count: int
# Update forward reference
TryoutConfigResponse.model_rebuild()

86
app/schemas/wordpress.py Normal file
View File

@@ -0,0 +1,86 @@
"""
Pydantic schemas for WordPress Integration API endpoints.
"""
from datetime import datetime
from typing import Any, List, Optional
from pydantic import BaseModel, Field
class VerifySessionRequest(BaseModel):
"""Request schema for verifying WordPress session."""
wp_user_id: str = Field(..., description="WordPress user ID")
token: str = Field(..., description="WordPress JWT authentication token")
website_id: int = Field(..., description="Website identifier")
class WordPressUserResponse(BaseModel):
"""Response schema for WordPress user data."""
id: int = Field(..., description="Local database user ID")
wp_user_id: str = Field(..., description="WordPress user ID")
website_id: int = Field(..., description="Website identifier")
created_at: datetime = Field(..., description="User creation timestamp")
updated_at: datetime = Field(..., description="User last update timestamp")
model_config = {"from_attributes": True}
class VerifySessionResponse(BaseModel):
"""Response schema for session verification."""
valid: bool = Field(..., description="Whether the session is valid")
user: Optional[WordPressUserResponse] = Field(
default=None, description="User data if session is valid"
)
error: Optional[str] = Field(
default=None, description="Error message if session is invalid"
)
wp_user_info: Optional[dict[str, Any]] = Field(
default=None, description="WordPress user info from API"
)
class SyncUsersRequest(BaseModel):
"""Request schema for user synchronization (optional body)."""
pass
class SyncStatsResponse(BaseModel):
"""Response schema for user synchronization statistics."""
inserted: int = Field(..., description="Number of users inserted")
updated: int = Field(..., description="Number of users updated")
total: int = Field(..., description="Total users processed")
errors: int = Field(default=0, description="Number of errors during sync")
class SyncUsersResponse(BaseModel):
"""Response schema for user synchronization."""
synced: SyncStatsResponse = Field(..., description="Synchronization statistics")
website_id: int = Field(..., description="Website identifier")
message: str = Field(default="Sync completed", description="Status message")
class UserListResponse(BaseModel):
"""Response schema for paginated user list."""
users: List[WordPressUserResponse] = Field(..., description="List of users")
total: int = Field(..., description="Total number of users")
page: int = Field(default=1, description="Current page number")
page_size: int = Field(default=50, description="Number of users per page")
total_pages: int = Field(default=1, description="Total number of pages")
class WordPressErrorDetail(BaseModel):
"""Detail schema for WordPress errors."""
code: str = Field(..., description="Error code")
message: str = Field(..., description="Error message")
details: Optional[dict[str, Any]] = Field(
default=None, description="Additional error details"
)

155
app/services/__init__.py Normal file
View File

@@ -0,0 +1,155 @@
"""
Services module for IRT Bank Soal.
Contains business logic services for:
- IRT calibration
- CAT selection
- WordPress authentication
- AI question generation
- Reporting
"""
from app.services.irt_calibration import (
IRTCalibrationError,
calculate_fisher_information,
calculate_item_information,
calculate_probability,
calculate_theta_se,
estimate_b_from_ctt_p,
estimate_theta_mle,
get_session_responses,
nn_to_theta,
theta_to_nn,
update_session_theta,
update_theta_after_response,
)
from app.services.cat_selection import (
CATSelectionError,
NextItemResult,
TerminationCheck,
check_user_level_reuse,
get_available_levels_for_slot,
get_next_item,
get_next_item_adaptive,
get_next_item_fixed,
get_next_item_hybrid,
should_terminate,
simulate_cat_selection,
update_theta,
)
from app.services.wordpress_auth import (
WordPressAPIError,
WordPressAuthError,
WordPressRateLimitError,
WordPressTokenInvalidError,
WordPressUserInfo,
WebsiteNotFoundError,
SyncStats,
fetch_wordpress_users,
get_or_create_user,
get_wordpress_user,
sync_wordpress_users,
verify_website_exists,
verify_wordpress_token,
)
from app.services.ai_generation import (
call_openrouter_api,
check_cache_reuse,
generate_question,
generate_with_cache_check,
get_ai_stats,
get_prompt_template,
parse_ai_response,
save_ai_question,
validate_ai_model,
SUPPORTED_MODELS,
)
from app.services.reporting import (
generate_student_performance_report,
generate_item_analysis_report,
generate_calibration_status_report,
generate_tryout_comparison_report,
export_report_to_csv,
export_report_to_excel,
export_report_to_pdf,
schedule_report,
get_scheduled_report,
list_scheduled_reports,
cancel_scheduled_report,
StudentPerformanceReport,
ItemAnalysisReport,
CalibrationStatusReport,
TryoutComparisonReport,
ReportSchedule,
)
__all__ = [
# IRT Calibration
"IRTCalibrationError",
"calculate_fisher_information",
"calculate_item_information",
"calculate_probability",
"calculate_theta_se",
"estimate_b_from_ctt_p",
"estimate_theta_mle",
"get_session_responses",
"nn_to_theta",
"theta_to_nn",
"update_session_theta",
"update_theta_after_response",
# CAT Selection
"CATSelectionError",
"NextItemResult",
"TerminationCheck",
"check_user_level_reuse",
"get_available_levels_for_slot",
"get_next_item",
"get_next_item_adaptive",
"get_next_item_fixed",
"get_next_item_hybrid",
"should_terminate",
"simulate_cat_selection",
"update_theta",
# WordPress Auth
"WordPressAPIError",
"WordPressAuthError",
"WordPressRateLimitError",
"WordPressTokenInvalidError",
"WordPressUserInfo",
"WebsiteNotFoundError",
"SyncStats",
"fetch_wordpress_users",
"get_or_create_user",
"get_wordpress_user",
"sync_wordpress_users",
"verify_website_exists",
"verify_wordpress_token",
# AI Generation
"call_openrouter_api",
"check_cache_reuse",
"generate_question",
"generate_with_cache_check",
"get_ai_stats",
"get_prompt_template",
"parse_ai_response",
"save_ai_question",
"validate_ai_model",
"SUPPORTED_MODELS",
# Reporting
"generate_student_performance_report",
"generate_item_analysis_report",
"generate_calibration_status_report",
"generate_tryout_comparison_report",
"export_report_to_csv",
"export_report_to_excel",
"export_report_to_pdf",
"schedule_report",
"get_scheduled_report",
"list_scheduled_reports",
"cancel_scheduled_report",
"StudentPerformanceReport",
"ItemAnalysisReport",
"CalibrationStatusReport",
"TryoutComparisonReport",
"ReportSchedule",
]

595
app/services/ai_generation.py Normal file
View File

@@ -0,0 +1,595 @@
"""
AI Question Generation Service.
Handles OpenRouter API integration for generating question variants.
Implements caching, user-level reuse checking, and prompt engineering.
"""
import json
import logging
import re
from typing import Any, Dict, Literal, Optional, Union
import httpx
from sqlalchemy import and_, func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.models.item import Item
from app.models.tryout import Tryout
from app.models.user_answer import UserAnswer
from app.schemas.ai import GeneratedQuestion
logger = logging.getLogger(__name__)
settings = get_settings()
# OpenRouter API configuration
OPENROUTER_API_URL = "https://openrouter.ai/api/v1/chat/completions"
# Supported AI models
SUPPORTED_MODELS = {
"qwen/qwen-2.5-coder-32b-instruct": "Qwen 2.5 Coder 32B",
"meta-llama/llama-3.3-70b-instruct": "Llama 3.3 70B",
}
# Level mapping for prompts
LEVEL_DESCRIPTIONS = {
"mudah": "easier (simpler concepts, more straightforward calculations)",
"sedang": "medium difficulty",
"sulit": "harder (more complex concepts, multi-step reasoning)",
}
def get_prompt_template(
basis_stem: str,
basis_options: Dict[str, str],
basis_correct: str,
basis_explanation: Optional[str],
target_level: Literal["mudah", "sulit"],
) -> str:
"""
Generate standardized prompt for AI question generation.
Args:
basis_stem: The basis question stem
basis_options: The basis question options
basis_correct: The basis correct answer
basis_explanation: The basis explanation
target_level: Target difficulty level
Returns:
Formatted prompt string
"""
level_desc = LEVEL_DESCRIPTIONS.get(target_level, target_level)
options_text = "\n".join(
[f" {key}: {value}" for key, value in basis_options.items()]
)
explanation_text = (
f"Explanation: {basis_explanation}"
if basis_explanation
else "Explanation: (not provided)"
)
prompt = f"""You are an educational content creator specializing in creating assessment questions.
Given a "Sedang" (medium difficulty) question, generate a new question at a different difficulty level.
BASIS QUESTION (Sedang level):
Question: {basis_stem}
Options:
{options_text}
Correct Answer: {basis_correct}
{explanation_text}
TASK:
Generate 1 new question that is {level_desc} than the basis question above.
REQUIREMENTS:
1. Keep the SAME topic/subject matter as the basis question
2. Use similar context and terminology
3. Create exactly 4 answer options (A, B, C, D)
4. Only ONE correct answer
5. Include a clear explanation of why the correct answer is correct
6. Make the question noticeably {level_desc} - not just a minor variation
OUTPUT FORMAT:
Return ONLY a valid JSON object with this exact structure (no markdown, no code blocks):
{{"stem": "Your question text here", "options": {{"A": "Option A text", "B": "Option B text", "C": "Option C text", "D": "Option D text"}}, "correct": "A", "explanation": "Explanation text here"}}
Remember: The correct field must be exactly "A", "B", "C", or "D"."""
return prompt
def parse_ai_response(response_text: str) -> Optional[GeneratedQuestion]:
"""
Parse AI response to extract question data.
Handles various response formats including JSON code blocks.
Args:
response_text: Raw AI response text
Returns:
GeneratedQuestion if parsing successful, None otherwise
"""
if not response_text:
return None
# Clean the response text
cleaned = response_text.strip()
# Try to extract JSON from code blocks if present
json_patterns = [
r"```json\s*([\s\S]*?)\s*```", # ```json ... ```
r"```\s*([\s\S]*?)\s*```", # ``` ... ```
r"(\{[\s\S]*\})", # Raw JSON object
]
for pattern in json_patterns:
match = re.search(pattern, cleaned)
if match:
json_str = match.group(1).strip()
try:
data = json.loads(json_str)
return validate_and_create_question(data)
except json.JSONDecodeError:
continue
# Try parsing the entire response as JSON
try:
data = json.loads(cleaned)
return validate_and_create_question(data)
except json.JSONDecodeError:
pass
logger.warning(f"Failed to parse AI response: {cleaned[:200]}...")
return None
def validate_and_create_question(data: Dict[str, Any]) -> Optional[GeneratedQuestion]:
"""
Validate parsed data and create GeneratedQuestion.
Args:
data: Parsed JSON data
Returns:
GeneratedQuestion if valid, None otherwise
"""
required_fields = ["stem", "options", "correct"]
if not all(field in data for field in required_fields):
logger.warning(f"Missing required fields in AI response: {data.keys()}")
return None
# Validate options
options = data.get("options", {})
if not isinstance(options, dict):
logger.warning("Options is not a dictionary")
return None
required_options = {"A", "B", "C", "D"}
if not required_options.issubset(set(options.keys())):
logger.warning(f"Missing required options: {required_options - set(options.keys())}")
return None
# Validate correct answer
correct = str(data.get("correct", "")).upper()
if correct not in required_options:
logger.warning(f"Invalid correct answer: {correct}")
return None
return GeneratedQuestion(
stem=str(data["stem"]).strip(),
options={k: str(v).strip() for k, v in options.items()},
correct=correct,
explanation=str(data.get("explanation", "")).strip() or None,
)
async def call_openrouter_api(
prompt: str,
model: str,
max_retries: int = 3,
) -> Optional[str]:
"""
Call OpenRouter API to generate question.
Args:
prompt: The prompt to send
model: AI model to use
max_retries: Maximum retry attempts
Returns:
API response text or None if failed
"""
if not settings.OPENROUTER_API_KEY:
logger.error("OPENROUTER_API_KEY not configured")
return None
if model not in SUPPORTED_MODELS:
logger.error(f"Unsupported AI model: {model}")
return None
headers = {
"Authorization": f"Bearer {settings.OPENROUTER_API_KEY}",
"Content-Type": "application/json",
"HTTP-Referer": "https://github.com/irt-bank-soal",
"X-Title": "IRT Bank Soal",
}
payload = {
"model": model,
"messages": [
{
"role": "user",
"content": prompt,
}
],
"max_tokens": 2000,
"temperature": 0.7,
}
timeout = httpx.Timeout(settings.OPENROUTER_TIMEOUT)
for attempt in range(max_retries):
try:
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.post(
OPENROUTER_API_URL,
headers=headers,
json=payload,
)
if response.status_code == 200:
data = response.json()
choices = data.get("choices", [])
if choices:
message = choices[0].get("message", {})
return message.get("content")
logger.warning("No choices in OpenRouter response")
return None
elif response.status_code == 429:
# Rate limited - wait and retry
logger.warning(f"Rate limited, attempt {attempt + 1}/{max_retries}")
if attempt < max_retries - 1:
import asyncio
await asyncio.sleep(2 ** attempt)
continue
return None
else:
logger.error(
f"OpenRouter API error: {response.status_code} - {response.text}"
)
return None
except httpx.TimeoutException:
logger.warning(f"OpenRouter timeout, attempt {attempt + 1}/{max_retries}")
if attempt < max_retries - 1:
continue
return None
except Exception as e:
logger.error(f"OpenRouter API call failed: {e}")
if attempt < max_retries - 1:
continue
return None
return None
async def generate_question(
basis_item: Item,
target_level: Literal["mudah", "sulit"],
ai_model: str = "qwen/qwen-2.5-coder-32b-instruct",
) -> Optional[GeneratedQuestion]:
"""
Generate a new question based on a basis item.
Args:
basis_item: The basis item (must be sedang level)
target_level: Target difficulty level
ai_model: AI model to use
Returns:
GeneratedQuestion if successful, None otherwise
"""
# Build prompt
prompt = get_prompt_template(
basis_stem=basis_item.stem,
basis_options=basis_item.options,
basis_correct=basis_item.correct_answer,
basis_explanation=basis_item.explanation,
target_level=target_level,
)
# Call OpenRouter API
response_text = await call_openrouter_api(prompt, ai_model)
if not response_text:
logger.error("No response from OpenRouter API")
return None
# Parse response
generated = parse_ai_response(response_text)
if not generated:
logger.error("Failed to parse AI response")
return None
return generated
async def check_cache_reuse(
tryout_id: str,
slot: int,
level: str,
wp_user_id: str,
website_id: int,
db: AsyncSession,
) -> Optional[Item]:
"""
Check if there's a cached item that the user hasn't answered yet.
Query DB for existing item matching (tryout_id, slot, level).
Check if user already answered this item at this difficulty level.
Args:
tryout_id: Tryout identifier
slot: Question slot
level: Difficulty level
wp_user_id: WordPress user ID
website_id: Website identifier
db: Database session
Returns:
Cached item if found and user hasn't answered, None otherwise
"""
# Find existing items at this slot/level
result = await db.execute(
select(Item).where(
and_(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.slot == slot,
Item.level == level,
)
)
)
existing_items = result.scalars().all()
if not existing_items:
return None
# Check each item to find one the user hasn't answered
for item in existing_items:
# Check if user has answered this item
answer_result = await db.execute(
select(UserAnswer).where(
and_(
UserAnswer.item_id == item.id,
UserAnswer.wp_user_id == wp_user_id,
)
)
)
user_answer = answer_result.scalar_one_or_none()
if user_answer is None:
# User hasn't answered this item - can reuse
logger.info(
f"Cache hit for tryout={tryout_id}, slot={slot}, level={level}, "
f"item_id={item.id}, user={wp_user_id}"
)
return item
# All items have been answered by this user
logger.info(
f"Cache miss (user answered all) for tryout={tryout_id}, slot={slot}, "
f"level={level}, user={wp_user_id}"
)
return None
async def generate_with_cache_check(
tryout_id: str,
slot: int,
level: Literal["mudah", "sulit"],
wp_user_id: str,
website_id: int,
db: AsyncSession,
ai_model: str = "qwen/qwen-2.5-coder-32b-instruct",
) -> tuple[Optional[Union[Item, GeneratedQuestion]], bool]:
"""
Generate question with cache checking.
First checks if AI generation is enabled for the tryout.
Then checks for cached items the user hasn't answered.
If cache miss, generates new question via AI.
Args:
tryout_id: Tryout identifier
slot: Question slot
level: Target difficulty level
wp_user_id: WordPress user ID
website_id: Website identifier
db: Database session
ai_model: AI model to use
Returns:
Tuple of (item/question or None, is_cached)
"""
# Check if AI generation is enabled for this tryout
tryout_result = await db.execute(
select(Tryout).where(
and_(
Tryout.tryout_id == tryout_id,
Tryout.website_id == website_id,
)
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout and not tryout.ai_generation_enabled:
logger.info(f"AI generation disabled for tryout={tryout_id}")
# Still check cache even if AI disabled
cached_item = await check_cache_reuse(
tryout_id, slot, level, wp_user_id, website_id, db
)
if cached_item:
return cached_item, True
return None, False
# Check cache for reusable item
cached_item = await check_cache_reuse(
tryout_id, slot, level, wp_user_id, website_id, db
)
if cached_item:
return cached_item, True
# Cache miss - need to generate
# Get basis item (sedang level at same slot)
basis_result = await db.execute(
select(Item).where(
and_(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.slot == slot,
Item.level == "sedang",
)
).limit(1)
)
basis_item = basis_result.scalar_one_or_none()
if not basis_item:
logger.error(
f"No basis item found for tryout={tryout_id}, slot={slot}"
)
return None, False
# Generate new question
generated = await generate_question(basis_item, level, ai_model)
if not generated:
logger.error(
f"Failed to generate question for tryout={tryout_id}, slot={slot}, level={level}"
)
return None, False
return generated, False
async def save_ai_question(
generated_data: GeneratedQuestion,
tryout_id: str,
website_id: int,
basis_item_id: int,
slot: int,
level: Literal["mudah", "sedang", "sulit"],
ai_model: str,
db: AsyncSession,
) -> Optional[int]:
"""
Save AI-generated question to database.
Args:
generated_data: Generated question data
tryout_id: Tryout identifier
website_id: Website identifier
basis_item_id: Basis item ID
slot: Question slot
level: Difficulty level
ai_model: AI model used
db: Database session
Returns:
Created item ID or None if failed
"""
try:
new_item = Item(
tryout_id=tryout_id,
website_id=website_id,
slot=slot,
level=level,
stem=generated_data.stem,
options=generated_data.options,
correct_answer=generated_data.correct,
explanation=generated_data.explanation,
generated_by="ai",
ai_model=ai_model,
basis_item_id=basis_item_id,
calibrated=False,
ctt_p=None,
ctt_bobot=None,
ctt_category=None,
irt_b=None,
irt_se=None,
calibration_sample_size=0,
)
db.add(new_item)
await db.flush() # Get the ID without committing
logger.info(
f"Saved AI-generated item: id={new_item.id}, tryout={tryout_id}, "
f"slot={slot}, level={level}, model={ai_model}"
)
return new_item.id
except Exception as e:
logger.error(f"Failed to save AI-generated question: {e}")
return None
async def get_ai_stats(db: AsyncSession) -> Dict[str, Any]:
"""
Get AI generation statistics.
Args:
db: Database session
Returns:
Statistics dictionary
"""
# Total AI-generated items
total_result = await db.execute(
select(func.count(Item.id)).where(Item.generated_by == "ai")
)
total_ai_items = total_result.scalar() or 0
# Items by model
model_result = await db.execute(
select(Item.ai_model, func.count(Item.id))
.where(Item.generated_by == "ai")
.where(Item.ai_model.isnot(None))
.group_by(Item.ai_model)
)
items_by_model = {row[0]: row[1] for row in model_result.all()}
# Note: Cache hit rate would need to be tracked separately
# This is a placeholder for now
return {
"total_ai_items": total_ai_items,
"items_by_model": items_by_model,
"cache_hit_rate": 0.0,
"total_cache_hits": 0,
"total_requests": 0,
}
def validate_ai_model(model: str) -> bool:
"""
Validate that the AI model is supported.
Args:
model: AI model identifier
Returns:
True if model is supported
"""
return model in SUPPORTED_MODELS

702
app/services/cat_selection.py Normal file
View File

@@ -0,0 +1,702 @@
"""
CAT (Computerized Adaptive Testing) Selection Service.
Implements adaptive item selection algorithms for IRT-based testing.
Supports three modes: CTT (fixed), IRT (adaptive), and hybrid.
"""
import math
from dataclasses import dataclass
from datetime import datetime
from typing import Literal, Optional
from sqlalchemy import and_, not_, or_, select, func
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.models import Item, Session, Tryout, UserAnswer
from app.services.irt_calibration import (
calculate_item_information,
estimate_b_from_ctt_p,
estimate_theta_mle,
update_theta_after_response,
)
class CATSelectionError(Exception):
"""Exception raised for CAT selection errors."""
pass
@dataclass
class NextItemResult:
"""Result of next item selection."""
item: Optional[Item]
selection_method: str # 'fixed', 'adaptive', 'hybrid'
slot: Optional[int]
level: Optional[str]
reason: str # Why this item was selected
@dataclass
class TerminationCheck:
"""Result of termination condition check."""
should_terminate: bool
reason: str
items_answered: int
current_se: Optional[float]
max_items: Optional[int]
se_threshold_met: bool
# Default SE threshold for termination
DEFAULT_SE_THRESHOLD = 0.5
# Default max items if not configured
DEFAULT_MAX_ITEMS = 50
async def get_next_item_fixed(
db: AsyncSession,
session_id: str,
tryout_id: str,
website_id: int,
level_filter: Optional[str] = None
) -> NextItemResult:
"""
Get next item in fixed order (CTT mode).
Returns items in slot order (1, 2, 3, ...).
Filters by level if specified.
Checks if student already answered this item.
Args:
db: Database session
session_id: Session identifier
tryout_id: Tryout identifier
website_id: Website identifier
level_filter: Optional difficulty level filter ('mudah', 'sedang', 'sulit')
Returns:
NextItemResult with selected item or None if no more items
"""
# Get session to find current position and answered items
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise CATSelectionError(f"Session {session_id} not found")
# Get all item IDs already answered by this user in this session
answered_query = select(UserAnswer.item_id).where(
UserAnswer.session_id == session_id
)
answered_result = await db.execute(answered_query)
answered_item_ids = [row[0] for row in answered_result.all()]
# Build query for available items
query = (
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
)
.order_by(Item.slot, Item.level)
)
# Apply level filter if specified
if level_filter:
query = query.where(Item.level == level_filter)
# Exclude already answered items
if answered_item_ids:
query = query.where(not_(Item.id.in_(answered_item_ids)))
result = await db.execute(query)
items = result.scalars().all()
if not items:
return NextItemResult(
item=None,
selection_method="fixed",
slot=None,
level=None,
reason="No more items available"
)
# Return first available item (lowest slot)
next_item = items[0]
return NextItemResult(
item=next_item,
selection_method="fixed",
slot=next_item.slot,
level=next_item.level,
reason=f"Fixed order selection - slot {next_item.slot}"
)
async def get_next_item_adaptive(
db: AsyncSession,
session_id: str,
tryout_id: str,
website_id: int,
ai_generation_enabled: bool = False,
level_filter: Optional[str] = None
) -> NextItemResult:
"""
Get next item using adaptive selection (IRT mode).
Finds item where b ≈ current theta.
Only uses calibrated items (calibrated=True).
Filters: student hasn't answered this item.
Filters: AI-generated items only if AI generation is enabled.
Args:
db: Database session
session_id: Session identifier
tryout_id: Tryout identifier
website_id: Website identifier
ai_generation_enabled: Whether to include AI-generated items
level_filter: Optional difficulty level filter
Returns:
NextItemResult with selected item or None if no suitable items
"""
# Get session for current theta
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise CATSelectionError(f"Session {session_id} not found")
# Get current theta (default to 0.0 for first item)
current_theta = session.theta if session.theta is not None else 0.0
# Get all item IDs already answered by this user in this session
answered_query = select(UserAnswer.item_id).where(
UserAnswer.session_id == session_id
)
answered_result = await db.execute(answered_query)
answered_item_ids = [row[0] for row in answered_result.all()]
# Build query for available calibrated items
query = (
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.calibrated == True # Only calibrated items for IRT
)
)
# Apply level filter if specified
if level_filter:
query = query.where(Item.level == level_filter)
# Exclude already answered items
if answered_item_ids:
query = query.where(not_(Item.id.in_(answered_item_ids)))
# Filter AI-generated items if AI generation is disabled
if not ai_generation_enabled:
query = query.where(Item.generated_by == 'manual')
result = await db.execute(query)
items = result.scalars().all()
if not items:
return NextItemResult(
item=None,
selection_method="adaptive",
slot=None,
level=None,
reason="No calibrated items available"
)
# Find item with b closest to current theta
# Also consider item information (prefer items with higher information at current theta)
best_item = None
best_score = float('inf')
for item in items:
if item.irt_b is None:
# Skip items without b parameter (shouldn't happen with calibrated=True)
continue
# Calculate distance from theta
b_distance = abs(item.irt_b - current_theta)
# Calculate item information at current theta
information = calculate_item_information(current_theta, item.irt_b)
# Score: minimize distance, maximize information
# Use weighted combination: lower score is better
# Add small penalty for lower information
score = b_distance - (0.1 * information)
if score < best_score:
best_score = score
best_item = item
if not best_item:
return NextItemResult(
item=None,
selection_method="adaptive",
slot=None,
level=None,
reason="No items with valid IRT parameters available"
)
return NextItemResult(
item=best_item,
selection_method="adaptive",
slot=best_item.slot,
level=best_item.level,
reason=f"Adaptive selection - b={best_item.irt_b:.3f} ≈ θ={current_theta:.3f}"
)
async def get_next_item_hybrid(
db: AsyncSession,
session_id: str,
tryout_id: str,
website_id: int,
hybrid_transition_slot: int = 10,
ai_generation_enabled: bool = False,
level_filter: Optional[str] = None
) -> NextItemResult:
"""
Get next item using hybrid selection.
Uses fixed order for first N items, then switches to adaptive.
Falls back to CTT if no calibrated items available.
Args:
db: Database session
session_id: Session identifier
tryout_id: Tryout identifier
website_id: Website identifier
hybrid_transition_slot: Slot number to transition from fixed to adaptive
ai_generation_enabled: Whether to include AI-generated items
level_filter: Optional difficulty level filter
Returns:
NextItemResult with selected item or None if no items available
"""
# Get session to check current position
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise CATSelectionError(f"Session {session_id} not found")
# Count answered items to determine current position
count_query = select(func.count(UserAnswer.id)).where(
UserAnswer.session_id == session_id
)
count_result = await db.execute(count_query)
items_answered = count_result.scalar() or 0
# Determine current slot (next slot to fill)
current_slot = items_answered + 1
# Check if we're still in fixed phase
if current_slot <= hybrid_transition_slot:
# Use fixed selection for initial items
result = await get_next_item_fixed(
db, session_id, tryout_id, website_id, level_filter
)
result.selection_method = "hybrid_fixed"
result.reason = f"Hybrid mode (fixed phase) - slot {current_slot}"
return result
# Try adaptive selection
adaptive_result = await get_next_item_adaptive(
db, session_id, tryout_id, website_id, ai_generation_enabled, level_filter
)
if adaptive_result.item is not None:
adaptive_result.selection_method = "hybrid_adaptive"
adaptive_result.reason = f"Hybrid mode (adaptive phase) - {adaptive_result.reason}"
return adaptive_result
# Fallback to fixed selection if no calibrated items available
fixed_result = await get_next_item_fixed(
db, session_id, tryout_id, website_id, level_filter
)
fixed_result.selection_method = "hybrid_fallback"
fixed_result.reason = f"Hybrid mode (CTT fallback) - {fixed_result.reason}"
return fixed_result
async def update_theta(
db: AsyncSession,
session_id: str,
item_id: int,
is_correct: bool
) -> tuple[float, float]:
"""
Update session theta estimate based on response.
Calls estimate_theta from irt_calibration.py.
Updates session.theta and session.theta_se.
Handles initial theta (uses 0.0 for first item).
Clamps theta to [-3, +3].
Args:
db: Database session
session_id: Session identifier
item_id: Item that was answered
is_correct: Whether the answer was correct
Returns:
Tuple of (theta, theta_se)
"""
return await update_theta_after_response(db, session_id, item_id, is_correct)
async def should_terminate(
db: AsyncSession,
session_id: str,
max_items: Optional[int] = None,
se_threshold: float = DEFAULT_SE_THRESHOLD
) -> TerminationCheck:
"""
Check if session should terminate.
Termination conditions:
- Reached max_items
- Reached SE threshold (theta_se < se_threshold)
- No more items available
Args:
db: Database session
session_id: Session identifier
max_items: Maximum items allowed (None = no limit)
se_threshold: SE threshold for termination
Returns:
TerminationCheck with termination status and reason
"""
# Get session
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise CATSelectionError(f"Session {session_id} not found")
# Count answered items
count_query = select(func.count(UserAnswer.id)).where(
UserAnswer.session_id == session_id
)
count_result = await db.execute(count_query)
items_answered = count_result.scalar() or 0
# Check max items
max_items_reached = False
if max_items is not None and items_answered >= max_items:
max_items_reached = True
# Check SE threshold
current_se = session.theta_se
se_threshold_met = False
if current_se is not None and current_se < se_threshold:
se_threshold_met = True
# Check if we have enough items for SE threshold (at least 15 items per PRD)
min_items_for_se = 15
se_threshold_met = se_threshold_met and items_answered >= min_items_for_se
# Determine termination
should_term = max_items_reached or se_threshold_met
# Build reason
reasons = []
if max_items_reached:
reasons.append(f"max items reached ({items_answered}/{max_items})")
if se_threshold_met:
reasons.append(f"SE threshold met ({current_se:.3f} < {se_threshold})")
if not reasons:
reasons.append("continuing")
return TerminationCheck(
should_terminate=should_term,
reason="; ".join(reasons),
items_answered=items_answered,
current_se=current_se,
max_items=max_items,
se_threshold_met=se_threshold_met
)
async def get_next_item(
db: AsyncSession,
session_id: str,
selection_mode: Literal["fixed", "adaptive", "hybrid"] = "fixed",
hybrid_transition_slot: int = 10,
ai_generation_enabled: bool = False,
level_filter: Optional[str] = None
) -> NextItemResult:
"""
Get next item based on selection mode.
Main entry point for item selection.
Args:
db: Database session
session_id: Session identifier
selection_mode: Selection mode ('fixed', 'adaptive', 'hybrid')
hybrid_transition_slot: Slot to transition in hybrid mode
ai_generation_enabled: Whether AI generation is enabled
level_filter: Optional difficulty level filter
Returns:
NextItemResult with selected item
"""
# Get session for tryout info
session_query = select(Session).where(Session.session_id == session_id)
session_result = await db.execute(session_query)
session = session_result.scalar_one_or_none()
if not session:
raise CATSelectionError(f"Session {session_id} not found")
tryout_id = session.tryout_id
website_id = session.website_id
if selection_mode == "fixed":
return await get_next_item_fixed(
db, session_id, tryout_id, website_id, level_filter
)
elif selection_mode == "adaptive":
return await get_next_item_adaptive(
db, session_id, tryout_id, website_id, ai_generation_enabled, level_filter
)
elif selection_mode == "hybrid":
return await get_next_item_hybrid(
db, session_id, tryout_id, website_id,
hybrid_transition_slot, ai_generation_enabled, level_filter
)
else:
raise CATSelectionError(f"Unknown selection mode: {selection_mode}")
async def check_user_level_reuse(
db: AsyncSession,
wp_user_id: str,
website_id: int,
tryout_id: str,
slot: int,
level: str
) -> bool:
"""
Check if user has already answered a question at this difficulty level.
Per PRD FR-5.3: Check if student user_id already answered question
at specific difficulty level.
Args:
db: Database session
wp_user_id: WordPress user ID
website_id: Website identifier
tryout_id: Tryout identifier
slot: Question slot
level: Difficulty level
Returns:
True if user has answered at this level, False otherwise
"""
# Check if user has answered any item at this slot/level combination
query = (
select(func.count(UserAnswer.id))
.join(Item, UserAnswer.item_id == Item.id)
.where(
UserAnswer.wp_user_id == wp_user_id,
UserAnswer.website_id == website_id,
UserAnswer.tryout_id == tryout_id,
Item.slot == slot,
Item.level == level
)
)
result = await db.execute(query)
count = result.scalar() or 0
return count > 0
async def get_available_levels_for_slot(
db: AsyncSession,
tryout_id: str,
website_id: int,
slot: int
) -> list[str]:
"""
Get available difficulty levels for a specific slot.
Args:
db: Database session
tryout_id: Tryout identifier
website_id: Website identifier
slot: Question slot
Returns:
List of available levels
"""
query = (
select(Item.level)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id,
Item.slot == slot
)
.distinct()
)
result = await db.execute(query)
levels = [row[0] for row in result.all()]
return levels
# Admin playground functions for testing CAT behavior
async def simulate_cat_selection(
db: AsyncSession,
tryout_id: str,
website_id: int,
initial_theta: float = 0.0,
selection_mode: Literal["fixed", "adaptive", "hybrid"] = "adaptive",
max_items: int = 15,
se_threshold: float = DEFAULT_SE_THRESHOLD,
hybrid_transition_slot: int = 10
) -> dict:
"""
Simulate CAT selection for admin testing.
Returns sequence of selected items with b values and theta progression.
Args:
db: Database session
tryout_id: Tryout identifier
website_id: Website identifier
initial_theta: Starting theta value
selection_mode: Selection mode to use
max_items: Maximum items to simulate
se_threshold: SE threshold for termination
hybrid_transition_slot: Slot to transition in hybrid mode
Returns:
Dict with simulation results
"""
# Get all items for this tryout
items_query = (
select(Item)
.where(
Item.tryout_id == tryout_id,
Item.website_id == website_id
)
.order_by(Item.slot)
)
items_result = await db.execute(items_query)
all_items = list(items_result.scalars().all())
if not all_items:
return {
"error": "No items found for this tryout",
"tryout_id": tryout_id,
"website_id": website_id
}
# Simulate selection
selected_items = []
current_theta = initial_theta
current_se = 3.0 # Start with high uncertainty
used_item_ids = set()
for i in range(max_items):
# Get available items
available_items = [item for item in all_items if item.id not in used_item_ids]
if not available_items:
break
# Select based on mode
if selection_mode == "adaptive":
# Filter to calibrated items only
calibrated_items = [item for item in available_items if item.calibrated and item.irt_b is not None]
if not calibrated_items:
# Fallback to any available item
calibrated_items = available_items
# Find item closest to current theta
best_item = min(
calibrated_items,
key=lambda item: abs((item.irt_b or 0) - current_theta)
)
elif selection_mode == "fixed":
# Select in slot order
best_item = min(available_items, key=lambda item: item.slot)
else: # hybrid
if i < hybrid_transition_slot:
best_item = min(available_items, key=lambda item: item.slot)
else:
calibrated_items = [item for item in available_items if item.calibrated and item.irt_b is not None]
if calibrated_items:
best_item = min(
calibrated_items,
key=lambda item: abs((item.irt_b or 0) - current_theta)
)
else:
best_item = min(available_items, key=lambda item: item.slot)
used_item_ids.add(best_item.id)
# Simulate response (random based on probability)
import random
b = best_item.irt_b or estimate_b_from_ctt_p(best_item.ctt_p) if best_item.ctt_p else 0.0
p_correct = 1.0 / (1.0 + math.exp(-(current_theta - b)))
is_correct = random.random() < p_correct
# Update theta (simplified)
responses = [1 if item.get('is_correct', True) else 0 for item in selected_items]
responses.append(1 if is_correct else 0)
b_params = [item['b'] for item in selected_items]
b_params.append(b)
new_theta, new_se = estimate_theta_mle(responses, b_params, current_theta)
current_theta = new_theta
current_se = new_se
selected_items.append({
"slot": best_item.slot,
"level": best_item.level,
"b": b,
"is_correct": is_correct,
"theta_after": current_theta,
"se_after": current_se,
"calibrated": best_item.calibrated
})
# Check SE threshold
if current_se < se_threshold and i >= 14: # At least 15 items
break
return {
"tryout_id": tryout_id,
"website_id": website_id,
"initial_theta": initial_theta,
"selection_mode": selection_mode,
"total_items": len(selected_items),
"final_theta": current_theta,
"final_se": current_se,
"se_threshold_met": current_se < se_threshold,
"items": selected_items
}

431
app/services/config_management.py Normal file
View File

@@ -0,0 +1,431 @@
"""
Configuration Management Service.
Provides functions to retrieve and update tryout configurations.
Handles configuration changes for scoring, selection, and normalization modes.
"""
import logging
from typing import Any, Dict, Literal, Optional
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
logger = logging.getLogger(__name__)
async def get_config(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Tryout:
"""
Fetch tryout configuration for a specific tryout.
Returns all configuration fields including scoring_mode, selection_mode,
normalization_mode, and other settings.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Tryout model with all configuration fields
Raises:
ValueError: If tryout not found
"""
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
return tryout
async def update_config(
db: AsyncSession,
website_id: int,
tryout_id: str,
config_updates: Dict[str, Any],
) -> Tryout:
"""
Update tryout configuration with provided fields.
Accepts a dictionary of configuration updates and applies them to the
tryout configuration. Only provided fields are updated.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
config_updates: Dictionary of configuration fields to update
Returns:
Updated Tryout model
Raises:
ValueError: If tryout not found or invalid field provided
"""
# Fetch tryout
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
# Valid configuration fields
valid_fields = {
"name", "description",
"scoring_mode", "selection_mode", "normalization_mode",
"min_sample_for_dynamic", "static_rataan", "static_sb",
"ai_generation_enabled",
"hybrid_transition_slot",
"min_calibration_sample", "theta_estimation_method", "fallback_to_ctt_on_error",
}
# Update only valid fields
updated_fields = []
for field, value in config_updates.items():
if field not in valid_fields:
logger.warning(f"Skipping invalid config field: {field}")
continue
setattr(tryout, field, value)
updated_fields.append(field)
if not updated_fields:
logger.warning(f"No valid config fields to update for tryout {tryout_id}")
await db.flush()
logger.info(
f"Updated config for tryout {tryout_id}, website {website_id}: "
f"{', '.join(updated_fields)}"
)
return tryout
async def toggle_normalization_mode(
db: AsyncSession,
website_id: int,
tryout_id: str,
new_mode: Literal["static", "dynamic", "hybrid"],
) -> Tryout:
"""
Toggle normalization mode for a tryout.
Updates the normalization_mode field. If switching to "auto" (dynamic mode),
checks if threshold is met and logs appropriate warnings.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
new_mode: New normalization mode ("static", "dynamic", "hybrid")
Returns:
Updated Tryout model
Raises:
ValueError: If tryout not found or invalid mode provided
"""
if new_mode not in ["static", "dynamic", "hybrid"]:
raise ValueError(
f"Invalid normalization_mode: {new_mode}. "
"Must be 'static', 'dynamic', or 'hybrid'"
)
# Fetch tryout with stats
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
old_mode = tryout.normalization_mode
tryout.normalization_mode = new_mode
# Fetch stats for participant count
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
participant_count = stats.participant_count if stats else 0
min_sample = tryout.min_sample_for_dynamic
# Log warnings and suggestions based on mode change
if new_mode == "dynamic":
if participant_count < min_sample:
logger.warning(
f"Switching to dynamic normalization with only {participant_count} "
f"participants (threshold: {min_sample}). "
"Dynamic normalization may produce unreliable results."
)
else:
logger.info(
f"Switching to dynamic normalization with {participant_count} "
f"participants (threshold: {min_sample}). "
"Ready for dynamic normalization."
)
elif new_mode == "hybrid":
if participant_count >= min_sample:
logger.info(
f"Switching to hybrid normalization with {participant_count} "
f"participants (threshold: {min_sample}). "
"Will use dynamic normalization immediately."
)
else:
logger.info(
f"Switching to hybrid normalization with {participant_count} "
f"participants (threshold: {min_sample}). "
f"Will use static normalization until {min_sample} participants reached."
)
await db.flush()
logger.info(
f"Toggled normalization mode for tryout {tryout_id}, "
f"website {website_id}: {old_mode} -> {new_mode}"
)
return tryout
async def get_normalization_config(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Dict[str, Any]:
"""
Get normalization configuration summary.
Returns current normalization mode, static values, dynamic values,
participant count, and threshold status.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Dictionary with normalization configuration summary
Raises:
ValueError: If tryout not found
"""
# Fetch tryout config
tryout = await get_config(db, website_id, tryout_id)
# Fetch stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
# Determine threshold status
participant_count = stats.participant_count if stats else 0
min_sample = tryout.min_sample_for_dynamic
threshold_ready = participant_count >= min_sample
participants_needed = max(0, min_sample - participant_count)
# Determine current effective mode
current_mode = tryout.normalization_mode
if current_mode == "hybrid":
effective_mode = "dynamic" if threshold_ready else "static"
else:
effective_mode = current_mode
return {
"tryout_id": tryout_id,
"normalization_mode": current_mode,
"effective_mode": effective_mode,
"static_rataan": tryout.static_rataan,
"static_sb": tryout.static_sb,
"dynamic_rataan": stats.rataan if stats else None,
"dynamic_sb": stats.sb if stats else None,
"participant_count": participant_count,
"min_sample_for_dynamic": min_sample,
"threshold_ready": threshold_ready,
"participants_needed": participants_needed,
}
async def reset_normalization_stats(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> TryoutStats:
"""
Reset TryoutStats to initial values.
Resets participant_count to 0 and clears running sums.
Switches normalization_mode to "static" temporarily.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Reset TryoutStats record
Raises:
ValueError: If tryout not found
"""
# Fetch tryout
tryout_result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = tryout_result.scalar_one_or_none()
if tryout is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
# Switch to static mode temporarily
tryout.normalization_mode = "static"
# Fetch or create stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
if stats is None:
# Create new empty stats record
stats = TryoutStats(
website_id=website_id,
tryout_id=tryout_id,
participant_count=0,
total_nm_sum=0.0,
total_nm_sq_sum=0.0,
rataan=None,
sb=None,
min_nm=None,
max_nm=None,
)
db.add(stats)
else:
# Reset existing stats
stats.participant_count = 0
stats.total_nm_sum = 0.0
stats.total_nm_sq_sum = 0.0
stats.rataan = None
stats.sb = None
stats.min_nm = None
stats.max_nm = None
await db.flush()
logger.info(
f"Reset normalization stats for tryout {tryout_id}, "
f"website {website_id}. Normalization mode switched to static."
)
return stats
async def get_full_config(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Dict[str, Any]:
"""
Get full tryout configuration including stats.
Returns all configuration fields plus current statistics.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Dictionary with full configuration and stats
Raises:
ValueError: If tryout not found
"""
# Fetch tryout config
tryout = await get_config(db, website_id, tryout_id)
# Fetch stats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
# Build config dictionary
config = {
"tryout_id": tryout.tryout_id,
"name": tryout.name,
"description": tryout.description,
"scoring_mode": tryout.scoring_mode,
"selection_mode": tryout.selection_mode,
"normalization_mode": tryout.normalization_mode,
"min_sample_for_dynamic": tryout.min_sample_for_dynamic,
"static_rataan": tryout.static_rataan,
"static_sb": tryout.static_sb,
"ai_generation_enabled": tryout.ai_generation_enabled,
"hybrid_transition_slot": tryout.hybrid_transition_slot,
"min_calibration_sample": tryout.min_calibration_sample,
"theta_estimation_method": tryout.theta_estimation_method,
"fallback_to_ctt_on_error": tryout.fallback_to_ctt_on_error,
"stats": {
"participant_count": stats.participant_count if stats else 0,
"rataan": stats.rataan if stats else None,
"sb": stats.sb if stats else None,
"min_nm": stats.min_nm if stats else None,
"max_nm": stats.max_nm if stats else None,
"last_calculated": stats.last_calculated if stats else None,
},
"created_at": tryout.created_at,
"updated_at": tryout.updated_at,
}
return config

385
app/services/ctt_scoring.py Normal file
View File

@@ -0,0 +1,385 @@
"""
CTT (Classical Test Theory) Scoring Engine.
Implements exact Excel formulas for:
- p-value (Tingkat Kesukaran): p = Σ Benar / Total Peserta
- Bobot (Weight): Bobot = 1 - p
- NM (Nilai Mentah): NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000
- NN (Nilai Nasional): NN = 500 + 100 × ((NM - Rataan) / SB)
All formulas match PRD Section 13.1 exactly.
"""
import math
from datetime import datetime, timezone
from typing import Optional
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.item import Item
from app.models.tryout_stats import TryoutStats
from app.models.user_answer import UserAnswer
def calculate_ctt_p(total_correct: int, total_participants: int) -> float:
"""
Calculate CTT p-value (Tingkat Kesukaran / Difficulty).
Formula: p = Σ Benar / Total Peserta
Args:
total_correct: Number of correct answers (Σ Benar)
total_participants: Total number of participants (Total Peserta)
Returns:
p-value in range [0.0, 1.0]
Raises:
ValueError: If total_participants is 0 or values are invalid
"""
if total_participants <= 0:
raise ValueError("total_participants must be greater than 0")
if total_correct < 0:
raise ValueError("total_correct cannot be negative")
if total_correct > total_participants:
raise ValueError("total_correct cannot exceed total_participants")
p = total_correct / total_participants
# Clamp to valid range [0, 1]
return max(0.0, min(1.0, p))
def calculate_ctt_bobot(p_value: float) -> float:
"""
Calculate CTT bobot (weight) from p-value.
Formula: Bobot = 1 - p
Interpretation:
- Easy questions (p > 0.70) have low bobot (< 0.30)
- Difficult questions (p < 0.30) have high bobot (> 0.70)
- Medium questions (0.30 ≤ p ≤ 0.70) have moderate bobot
Args:
p_value: CTT p-value in range [0.0, 1.0]
Returns:
bobot (weight) in range [0.0, 1.0]
Raises:
ValueError: If p_value is outside [0, 1] range
"""
if not 0.0 <= p_value <= 1.0:
raise ValueError(f"p_value must be in range [0, 1], got {p_value}")
bobot = 1.0 - p_value
# Clamp to valid range [0, 1]
return max(0.0, min(1.0, bobot))
def calculate_ctt_nm(total_bobot_siswa: float, total_bobot_max: float) -> int:
"""
Calculate CTT NM (Nilai Mentah / Raw Score).
Formula: NM = (Total_Bobot_Siswa / Total_Bobot_Max) × 1000
This is equivalent to Excel's SUMPRODUCT calculation where:
- Total_Bobot_Siswa = Σ(bobot_earned for each correct answer)
- Total_Bobot_Max = Σ(bobot for all questions)
Args:
total_bobot_siswa: Total weight earned by student
total_bobot_max: Maximum possible weight (sum of all item bobots)
Returns:
NM (raw score) in range [0, 1000]
Raises:
ValueError: If total_bobot_max is 0 or values are invalid
"""
if total_bobot_max <= 0:
raise ValueError("total_bobot_max must be greater than 0")
if total_bobot_siswa < 0:
raise ValueError("total_bobot_siswa cannot be negative")
nm = (total_bobot_siswa / total_bobot_max) * 1000
# Round to integer and clamp to valid range [0, 1000]
nm_int = round(nm)
return max(0, min(1000, nm_int))
def calculate_ctt_nn(nm: int, rataan: float, sb: float) -> int:
"""
Calculate CTT NN (Nilai Nasional / Normalized Score).
Formula: NN = 500 + 100 × ((NM - Rataan) / SB)
Normalizes scores to mean=500, SD=100 distribution.
Args:
nm: Nilai Mentah (raw score) in range [0, 1000]
rataan: Mean of NM scores
sb: Standard deviation of NM scores (Simpangan Baku)
Returns:
NN (normalized score) in range [0, 1000]
Raises:
ValueError: If nm is out of range or sb is invalid
"""
if not 0 <= nm <= 1000:
raise ValueError(f"nm must be in range [0, 1000], got {nm}")
if sb <= 0:
# If SD is 0 or negative, return default normalized score
# This handles edge case where all scores are identical
return 500
# Calculate normalized score
z_score = (nm - rataan) / sb
nn = 500 + 100 * z_score
# Round to integer and clamp to valid range [0, 1000]
nn_int = round(nn)
return max(0, min(1000, nn_int))
def categorize_difficulty(p_value: float) -> str:
"""
Categorize question difficulty based on CTT p-value.
Categories per CTT standards (PRD Section 13.2):
- p < 0.30 → Sukar (Sulit)
- 0.30 ≤ p ≤ 0.70 → Sedang
- p > 0.70 → Mudah
Args:
p_value: CTT p-value in range [0.0, 1.0]
Returns:
Difficulty category: "mudah", "sedang", or "sulit"
"""
if p_value > 0.70:
return "mudah"
elif p_value >= 0.30:
return "sedang"
else:
return "sulit"
async def calculate_ctt_p_for_item(
db: AsyncSession, item_id: int
) -> Optional[float]:
"""
Calculate CTT p-value for a specific item from existing responses.
Queries all UserAnswer records for the item to calculate:
p = Σ Benar / Total Peserta
Args:
db: Async database session
item_id: Item ID to calculate p-value for
Returns:
p-value in range [0.0, 1.0], or None if no responses exist
"""
# Count total responses and correct responses
result = await db.execute(
select(
func.count().label("total"),
func.sum(func.cast(UserAnswer.is_correct, type_=func.INTEGER)).label("correct"),
).where(UserAnswer.item_id == item_id)
)
row = result.first()
if row is None or row.total == 0:
return None
return calculate_ctt_p(row.correct or 0, row.total)
async def update_tryout_stats(
db: AsyncSession,
website_id: int,
tryout_id: str,
nm: int,
) -> TryoutStats:
"""
Incrementally update TryoutStats with new NM score.
Updates:
- participant_count += 1
- total_nm_sum += nm
- total_nm_sq_sum += nm²
- Recalculates rataan (mean) and sb (standard deviation)
- Updates min_nm and max_nm if applicable
Uses Welford's online algorithm for numerically stable variance calculation.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
nm: New NM score to add
Returns:
Updated TryoutStats record
"""
# Get or create TryoutStats
result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = result.scalar_one_or_none()
if stats is None:
# Create new stats record
stats = TryoutStats(
website_id=website_id,
tryout_id=tryout_id,
participant_count=1,
total_nm_sum=float(nm),
total_nm_sq_sum=float(nm * nm),
rataan=float(nm),
sb=0.0, # SD is 0 for single data point
min_nm=nm,
max_nm=nm,
last_calculated=datetime.now(timezone.utc),
)
db.add(stats)
else:
# Incrementally update existing stats
stats.participant_count += 1
stats.total_nm_sum += nm
stats.total_nm_sq_sum += nm * nm
# Update min/max
if stats.min_nm is None or nm < stats.min_nm:
stats.min_nm = nm
if stats.max_nm is None or nm > stats.max_nm:
stats.max_nm = nm
# Recalculate mean and SD
n = stats.participant_count
sum_nm = stats.total_nm_sum
sum_nm_sq = stats.total_nm_sq_sum
# Mean = Σ NM / n
stats.rataan = sum_nm / n
# Variance = (Σ NM² / n) - (mean)²
# Using population standard deviation
if n > 1:
variance = (sum_nm_sq / n) - (stats.rataan ** 2)
# Clamp variance to non-negative (handles floating point errors)
variance = max(0.0, variance)
stats.sb = math.sqrt(variance)
else:
stats.sb = 0.0
stats.last_calculated = datetime.now(timezone.utc)
await db.flush()
return stats
async def get_total_bobot_max(
db: AsyncSession,
website_id: int,
tryout_id: str,
level: str = "sedang",
) -> float:
"""
Calculate total maximum bobot for a tryout.
Total_Bobot_Max = Σ bobot for all questions in the tryout
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
level: Difficulty level to filter by (default: "sedang")
Returns:
Sum of all item bobots
Raises:
ValueError: If no items found or items have no bobot values
"""
result = await db.execute(
select(func.sum(Item.ctt_bobot)).where(
Item.website_id == website_id,
Item.tryout_id == tryout_id,
Item.level == level,
)
)
total_bobot = result.scalar()
if total_bobot is None or total_bobot == 0:
raise ValueError(
f"No items with bobot found for tryout {tryout_id}, level {level}"
)
return float(total_bobot)
def convert_ctt_p_to_irt_b(p_value: float) -> float:
"""
Convert CTT p-value to IRT difficulty parameter (b).
Formula: b ≈ -ln((1-p)/p)
This provides an initial estimate for IRT calibration.
Maps p ∈ (0, 1) to b ∈ (-∞, +∞), typically [-3, +3].
Args:
p_value: CTT p-value in range (0.0, 1.0)
Returns:
IRT b-parameter estimate
Raises:
ValueError: If p_value is at boundaries (0 or 1)
"""
if p_value <= 0.0 or p_value >= 1.0:
# Handle edge cases by clamping
if p_value <= 0.0:
return 3.0 # Very difficult
else:
return -3.0 # Very easy
# b ≈ -ln((1-p)/p)
odds_ratio = (1 - p_value) / p_value
b = -math.log(odds_ratio)
# Clamp to valid IRT range [-3, +3]
return max(-3.0, min(3.0, b))
def map_theta_to_nn(theta: float) -> int:
"""
Map IRT theta (ability) to NN score for comparison.
Formula: NN = 500 + (θ / 3) × 500
Maps θ ∈ [-3, +3] to NN ∈ [0, 1000].
Args:
theta: IRT ability estimate in range [-3.0, +3.0]
Returns:
NN score in range [0, 1000]
"""
# Clamp theta to valid range
theta_clamped = max(-3.0, min(3.0, theta))
# Map to NN
nn = 500 + (theta_clamped / 3) * 500
# Round and clamp to valid range
return max(0, min(1000, round(nn)))

521
app/services/excel_import.py Normal file
View File

@@ -0,0 +1,521 @@
"""
Excel Import/Export Service for Question Migration.
Handles import from standardized Excel format with:
- Row 2: KUNCI (answer key)
- Row 4: TK (tingkat kesukaran p-value)
- Row 5: BOBOT (weight 1-p)
- Rows 6+: Individual question data
Ensures 100% data integrity with comprehensive validation.
"""
import os
from datetime import datetime
from typing import Any, Dict, List, Optional
import openpyxl
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.item import Item
from app.services.ctt_scoring import (
convert_ctt_p_to_irt_b,
categorize_difficulty,
)
def validate_excel_structure(file_path: str) -> Dict[str, Any]:
"""
Validate Excel file structure against required format.
Checks:
- File exists and is valid Excel (.xlsx)
- Sheet "CONTOH" exists
- Required rows exist (Row 2 KUNCI, Row 4 TK, Row 5 BOBOT)
- Question data rows have required columns
Args:
file_path: Path to Excel file
Returns:
Dict with:
- valid: bool - Whether structure is valid
- errors: List[str] - Validation errors if any
"""
errors: List[str] = []
# Check file exists
if not os.path.exists(file_path):
return {"valid": False, "errors": [f"File not found: {file_path}"]}
# Check file extension
if not file_path.lower().endswith('.xlsx'):
return {"valid": False, "errors": ["File must be .xlsx format"]}
try:
wb = openpyxl.load_workbook(file_path, data_only=False)
except Exception as e:
return {"valid": False, "errors": [f"Failed to load Excel file: {str(e)}"]}
# Check sheet "CONTOH" exists
if "CONTOH" not in wb.sheetnames:
return {
"valid": False,
"errors": ['Sheet "CONTOH" not found. Available sheets: ' + ", ".join(wb.sheetnames)]
}
ws = wb["CONTOH"]
# Check minimum rows exist
if ws.max_row < 6:
errors.append(f"Excel file must have at least 6 rows (found {ws.max_row})")
# Check Row 2 exists (KUNCI)
if ws.max_row < 2:
errors.append("Row 2 (KUNCI - answer key) is required")
# Check Row 4 exists (TK - p-values)
if ws.max_row < 4:
errors.append("Row 4 (TK - p-values) is required")
# Check Row 5 exists (BOBOT - weights)
if ws.max_row < 5:
errors.append("Row 5 (BOBOT - weights) is required")
# Check question data rows exist (6+)
if ws.max_row < 6:
errors.append("Question data rows (6+) are required")
# Check minimum columns (at least slot, level, soal_text, options, correct_answer)
if ws.max_column < 8:
errors.append(
f"Excel file must have at least 8 columns (found {ws.max_column}). "
"Expected: slot, level, soal_text, options_A, options_B, options_C, options_D, correct_answer"
)
# Check KUNCI row has values
if ws.max_row >= 2:
kunce_row_values = [ws.cell(2, col).value for col in range(4, ws.max_column + 1)]
if not any(v for v in kunce_row_values if v and v != "KUNCI"):
errors.append("Row 2 (KUNCI) must contain answer key values")
# Check TK row has numeric values
if ws.max_row >= 4:
wb_data = openpyxl.load_workbook(file_path, data_only=True)
ws_data = wb_data["CONTOH"]
tk_row_values = [ws_data.cell(4, col).value for col in range(4, ws.max_column + 1)]
if not any(v for v in tk_row_values if isinstance(v, (int, float))):
errors.append("Row 4 (TK) must contain numeric p-values")
# Check BOBOT row has numeric values
if ws.max_row >= 5:
wb_data = openpyxl.load_workbook(file_path, data_only=True)
ws_data = wb_data["CONTOH"]
bobot_row_values = [ws_data.cell(5, col).value for col in range(4, ws.max_column + 1)]
if not any(v for v in bobot_row_values if isinstance(v, (int, float))):
errors.append("Row 5 (BOBOT) must contain numeric weight values")
return {"valid": len(errors) == 0, "errors": errors}
def parse_excel_import(
file_path: str,
website_id: int,
tryout_id: str
) -> Dict[str, Any]:
"""
Parse Excel file and extract items with full validation.
Excel structure:
- Sheet name: "CONTOH"
- Row 2: KUNCI (answer key) - extract correct answers per slot
- Row 4: TK (tingkat kesukaran p-value) - extract p-values per slot
- Row 5: BOBOT (weight 1-p) - extract bobot per slot
- Rows 6+: Individual question data
Args:
file_path: Path to Excel file
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Dict with:
- items: List[Dict] - Parsed items ready for database
- validation_errors: List[str] - Any validation errors
- items_count: int - Number of items parsed
"""
# First validate structure
validation = validate_excel_structure(file_path)
if not validation["valid"]:
return {
"items": [],
"validation_errors": validation["errors"],
"items_count": 0
}
items: List[Dict[str, Any]] = []
errors: List[str] = []
try:
# Load workbook twice: once with formulas, once with data_only
wb = openpyxl.load_workbook(file_path, data_only=False)
ws = wb["CONTOH"]
wb_data = openpyxl.load_workbook(file_path, data_only=True)
ws_data = wb_data["CONTOH"]
# Extract answer key from Row 2
answer_key: Dict[int, str] = {}
for col in range(4, ws.max_column + 1):
key_cell = ws.cell(2, col).value
if key_cell and key_cell != "KUNCI":
slot_num = col - 3 # Column 4 -> slot 1
answer_key[slot_num] = str(key_cell).strip().upper()
# Extract p-values from Row 4
p_values: Dict[int, float] = {}
for col in range(4, ws.max_column + 1):
slot_num = col - 3
if slot_num in answer_key:
p_cell = ws_data.cell(4, col).value
if p_cell and isinstance(p_cell, (int, float)):
p_values[slot_num] = float(p_cell)
# Extract bobot from Row 5
bobot_values: Dict[int, float] = {}
for col in range(4, ws.max_column + 1):
slot_num = col - 3
if slot_num in answer_key:
bobot_cell = ws_data.cell(5, col).value
if bobot_cell and isinstance(bobot_cell, (int, float)):
bobot_values[slot_num] = float(bobot_cell)
# Parse question data rows (6+)
for row_idx in range(6, ws.max_row + 1):
# Column mapping (based on project-brief):
# Column 1 (A): slot (question number)
# Column 2 (B): level (mudah/sedang/sulit)
# Column 3 (C): soal_text (question stem)
# Column 4 (D): options_A
# Column 5 (E): options_B
# Column 6 (F): options_C
# Column 7 (G): options_D
# Column 8 (H): correct_answer
slot_cell = ws.cell(row_idx, 1).value
level_cell = ws.cell(row_idx, 2).value
soal_text_cell = ws.cell(row_idx, 3).value
option_a = ws.cell(row_idx, 4).value
option_b = ws.cell(row_idx, 5).value
option_c = ws.cell(row_idx, 6).value
option_d = ws.cell(row_idx, 7).value
correct_cell = ws.cell(row_idx, 8).value
# Skip empty rows
if not slot_cell and not soal_text_cell:
continue
# Validate required fields
if not slot_cell:
errors.append(f"Row {row_idx}: Missing slot value")
continue
slot_num = int(slot_cell) if isinstance(slot_cell, (int, float)) else None
if slot_num is None:
try:
slot_num = int(str(slot_cell).strip())
except (ValueError, AttributeError):
errors.append(f"Row {row_idx}: Invalid slot value: {slot_cell}")
continue
# Get or infer level
if not level_cell:
# Use p-value from Row 4 to determine level
p_val = p_values.get(slot_num, 0.5)
level_val = categorize_difficulty(p_val)
else:
level_val = str(level_cell).strip().lower()
if level_val not in ["mudah", "sedang", "sulit"]:
errors.append(
f"Row {row_idx}: Invalid level '{level_cell}'. Must be 'mudah', 'sedang', or 'sulit'"
)
continue
# Validate soal_text
if not soal_text_cell:
errors.append(f"Row {row_idx} (slot {slot_num}): Missing soal_text (question stem)")
continue
# Build options JSON
options: Dict[str, str] = {}
if option_a:
options["A"] = str(option_a).strip()
if option_b:
options["B"] = str(option_b).strip()
if option_c:
options["C"] = str(option_c).strip()
if option_d:
options["D"] = str(option_d).strip()
if len(options) < 4:
errors.append(
f"Row {row_idx} (slot {slot_num}): Missing options. Expected 4 options (A, B, C, D)"
)
continue
# Get correct answer
if not correct_cell:
# Fall back to answer key from Row 2
correct_ans = answer_key.get(slot_num)
if not correct_ans:
errors.append(
f"Row {row_idx} (slot {slot_num}): Missing correct_answer and no answer key found"
)
continue
else:
correct_ans = str(correct_cell).strip().upper()
if correct_ans not in ["A", "B", "C", "D"]:
errors.append(
f"Row {row_idx} (slot {slot_num}): Invalid correct_answer '{correct_ans}'. Must be A, B, C, or D"
)
continue
# Get CTT parameters
p_val = p_values.get(slot_num, 0.5)
bobot_val = bobot_values.get(slot_num, 1.0 - p_val)
# Validate p-value range
if p_val < 0 or p_val > 1:
errors.append(
f"Slot {slot_num}: Invalid p-value {p_val}. Must be in range [0, 1]"
)
continue
# Validate bobot range
if bobot_val < 0 or bobot_val > 1:
errors.append(
f"Slot {slot_num}: Invalid bobot {bobot_val}. Must be in range [0, 1]"
)
continue
# Calculate CTT category and IRT b parameter
ctt_cat = categorize_difficulty(p_val)
irt_b = convert_ctt_p_to_irt_b(p_val)
# Build item dict
item = {
"tryout_id": tryout_id,
"website_id": website_id,
"slot": slot_num,
"level": level_val,
"stem": str(soal_text_cell).strip(),
"options": options,
"correct_answer": correct_ans,
"explanation": None,
"ctt_p": p_val,
"ctt_bobot": bobot_val,
"ctt_category": ctt_cat,
"irt_b": irt_b,
"irt_se": None,
"calibrated": False,
"calibration_sample_size": 0,
"generated_by": "manual",
"ai_model": None,
"basis_item_id": None,
}
items.append(item)
return {
"items": items,
"validation_errors": errors,
"items_count": len(items)
}
except Exception as e:
return {
"items": [],
"validation_errors": [f"Parsing error: {str(e)}"],
"items_count": 0
}
async def bulk_insert_items(
items_list: List[Dict[str, Any]],
db: AsyncSession
) -> Dict[str, Any]:
"""
Bulk insert items with duplicate detection.
Skips duplicates based on (tryout_id, website_id, slot).
Args:
items_list: List of item dictionaries to insert
db: Async SQLAlchemy database session
Returns:
Dict with:
- inserted_count: int - Number of items inserted
- duplicate_count: int - Number of duplicates skipped
- errors: List[str] - Any errors during insertion
"""
inserted_count = 0
duplicate_count = 0
errors: List[str] = []
try:
for item_data in items_list:
# Check for duplicate
result = await db.execute(
select(Item).where(
Item.tryout_id == item_data["tryout_id"],
Item.website_id == item_data["website_id"],
Item.slot == item_data["slot"]
)
)
existing = result.scalar_one_or_none()
if existing:
duplicate_count += 1
continue
# Create new item
item = Item(**item_data)
db.add(item)
inserted_count += 1
# Commit all inserts
await db.commit()
return {
"inserted_count": inserted_count,
"duplicate_count": duplicate_count,
"errors": errors
}
except Exception as e:
await db.rollback()
return {
"inserted_count": 0,
"duplicate_count": duplicate_count,
"errors": [f"Insertion failed: {str(e)}"]
}
async def export_questions_to_excel(
tryout_id: str,
website_id: int,
db: AsyncSession,
output_path: Optional[str] = None
) -> str:
"""
Export questions to Excel in standardized format.
Creates Excel workbook with:
- Sheet "CONTOH"
- Row 2: KUNCI (answer key)
- Row 4: TK (p-values)
- Row 5: BOBOT (weights)
- Rows 6+: Question data
Args:
tryout_id: Tryout identifier
website_id: Website identifier
db: Async SQLAlchemy database session
output_path: Optional output file path. If not provided, generates temp file.
Returns:
Path to exported Excel file
"""
# Fetch all items for this tryout
result = await db.execute(
select(Item).filter(
Item.tryout_id == tryout_id,
Item.website_id == website_id
).order_by(Item.slot)
)
items = result.scalars().all()
if not items:
raise ValueError(f"No items found for tryout_id={tryout_id}, website_id={website_id}")
# Create workbook
wb = openpyxl.Workbook()
ws = wb.active
ws.title = "CONTOH"
# Determine max slot for column sizing
max_slot = max(item.slot for item in items)
# Row 1: Header
ws.cell(1, 1, "No")
ws.cell(1, 2, "Level")
ws.cell(1, 3, "Soal")
for slot_idx in range(max_slot):
col = slot_idx + 4
ws.cell(1, col, f"Soal {slot_idx + 1}")
# Row 2: KUNCI (answer key)
ws.cell(2, 1, "")
ws.cell(2, 2, "")
ws.cell(2, 3, "KUNCI")
for item in items:
col = item.slot + 3
ws.cell(2, col, item.correct_answer)
# Row 3: Empty
ws.cell(3, 1, "")
ws.cell(3, 2, "")
ws.cell(3, 3, "")
# Row 4: TK (p-values)
ws.cell(4, 1, "")
ws.cell(4, 2, "")
ws.cell(4, 3, "TK")
for item in items:
col = item.slot + 3
ws.cell(4, col, item.ctt_p or 0.5)
# Row 5: BOBOT (weights)
ws.cell(5, 1, "")
ws.cell(5, 2, "")
ws.cell(5, 3, "BOBOT")
for item in items:
col = item.slot + 3
ws.cell(5, col, item.ctt_bobot or (1.0 - (item.ctt_p or 0.5)))
# Rows 6+: Question data
row_idx = 6
for item in items:
# Column 1: Slot number
ws.cell(row_idx, 1, item.slot)
# Column 2: Level
ws.cell(row_idx, 2, item.level)
# Column 3: Soal text (stem)
ws.cell(row_idx, 3, item.stem)
# Columns 4+: Options
options = item.options or {}
ws.cell(row_idx, 4, options.get("A", ""))
ws.cell(row_idx, 5, options.get("B", ""))
ws.cell(row_idx, 6, options.get("C", ""))
ws.cell(row_idx, 7, options.get("D", ""))
# Column 8: Correct answer
ws.cell(row_idx, 8, item.correct_answer)
row_idx += 1
# Generate output path if not provided
if output_path is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
output_path = f"/tmp/tryout_{tryout_id}_export_{timestamp}.xlsx"
# Save workbook
wb.save(output_path)
return output_path

1124
app/services/irt_calibration.py Normal file
View File

File diff suppressed because it is too large Load Diff

538
app/services/normalization.py Normal file
View File

@@ -0,0 +1,538 @@
"""
Dynamic Normalization Service.
Implements dynamic normalization with real-time calculation of rataan and SB
for each tryout. Supports multiple normalization modes:
- Static: Use hardcoded rataan/SB from config
- Dynamic: Calculate rataan/SB from participant NM scores in real-time
- Hybrid: Use static until threshold reached, then switch to dynamic
"""
import logging
import math
from datetime import datetime, timezone
from typing import Literal, Optional, Tuple
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.tryout import Tryout
from app.models.tryout_stats import TryoutStats
logger = logging.getLogger(__name__)
async def calculate_dynamic_stats(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Tuple[Optional[float], Optional[float]]:
"""
Calculate current dynamic stats (rataan and SB) from TryoutStats.
Fetches current TryoutStats for this (tryout_id, website_id) pair
and returns the calculated rataan and SB values.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Tuple of (rataan, sb), both None if no stats exist
"""
result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = result.scalar_one_or_none()
if stats is None:
return None, None
return stats.rataan, stats.sb
async def update_dynamic_normalization(
db: AsyncSession,
website_id: int,
tryout_id: str,
nm: int,
) -> Tuple[float, float]:
"""
Update dynamic normalization with new NM score.
Fetches current TryoutStats and incrementally updates it with the new NM:
- Increments participant_count by 1
- Adds NM to total_nm_sum
- Adds NM² to total_nm_sq_sum
- Recalculates rataan and sb
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
nm: Nilai Mentah (raw score) to add
Returns:
Tuple of updated (rataan, sb)
Raises:
ValueError: If nm is out of valid range [0, 1000]
"""
if not 0 <= nm <= 1000:
raise ValueError(f"nm must be in range [0, 1000], got {nm}")
result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = result.scalar_one_or_none()
if stats is None:
# Initialize new stats record
stats = TryoutStats(
website_id=website_id,
tryout_id=tryout_id,
participant_count=1,
total_nm_sum=float(nm),
total_nm_sq_sum=float(nm * nm),
rataan=float(nm),
sb=0.0, # SD is 0 for single data point
min_nm=nm,
max_nm=nm,
last_calculated=datetime.now(timezone.utc),
)
db.add(stats)
else:
# Incrementally update existing stats
stats.participant_count += 1
stats.total_nm_sum += nm
stats.total_nm_sq_sum += nm * nm
# Update min/max
if stats.min_nm is None or nm < stats.min_nm:
stats.min_nm = nm
if stats.max_nm is None or nm > stats.max_nm:
stats.max_nm = nm
# Recalculate mean and SD
n = stats.participant_count
sum_nm = stats.total_nm_sum
sum_nm_sq = stats.total_nm_sq_sum
# Mean = Σ NM / n
mean = sum_nm / n
stats.rataan = mean
# Variance = (Σ NM² / n) - (mean)²
# Using population standard deviation
if n > 1:
variance = (sum_nm_sq / n) - (mean ** 2)
# Clamp variance to non-negative (handles floating point errors)
variance = max(0.0, variance)
stats.sb = math.sqrt(variance)
else:
stats.sb = 0.0
stats.last_calculated = datetime.now(timezone.utc)
await db.flush()
logger.info(
f"Updated dynamic normalization for tryout {tryout_id}, "
f"website {website_id}: participant_count={stats.participant_count}, "
f"rataan={stats.rataan:.2f}, sb={stats.sb:.2f}"
)
# rataan and sb are always set by this function
assert stats.rataan is not None
assert stats.sb is not None
return stats.rataan, stats.sb
def apply_normalization(
nm: int,
rataan: float,
sb: float,
) -> int:
"""
Apply normalization to NM to get NN (Nilai Nasional).
Formula: NN = 500 + 100 × ((NM - Rataan) / SB)
Normalizes scores to mean=500, SD=100 distribution.
Args:
nm: Nilai Mentah (raw score) in range [0, 1000]
rataan: Mean of NM scores
sb: Standard deviation of NM scores
Returns:
NN (normalized score) in range [0, 1000]
Raises:
ValueError: If nm is out of range or sb is invalid
"""
if not 0 <= nm <= 1000:
raise ValueError(f"nm must be in range [0, 1000], got {nm}")
if sb <= 0:
# If SD is 0 or negative, return default normalized score
# This handles edge case where all scores are identical
return 500
# Calculate normalized score
z_score = (nm - rataan) / sb
nn = 500 + 100 * z_score
# Round to integer and clamp to valid range [0, 1000]
nn_int = round(nn)
return max(0, min(1000, nn_int))
async def get_normalization_mode(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Literal["static", "dynamic", "hybrid"]:
"""
Get the current normalization mode for a tryout.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Normalization mode: "static", "dynamic", or "hybrid"
Raises:
ValueError: If tryout not found
"""
result = await db.execute(
select(Tryout).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
tryout = result.scalar_one_or_none()
if tryout is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
return tryout.normalization_mode
async def check_threshold_for_dynamic(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> bool:
"""
Check if participant count meets threshold for dynamic normalization.
Compares current participant_count with min_sample_for_dynamic from config.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
True if participant_count >= min_sample_for_dynamic, else False
"""
# Fetch current TryoutStats
stats_result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = stats_result.scalar_one_or_none()
current_participant_count = stats.participant_count if stats else 0
# Fetch min_sample_for_dynamic from config
tryout_result = await db.execute(
select(Tryout.min_sample_for_dynamic).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
min_sample = tryout_result.scalar_one_or_none()
if min_sample is None:
# Default to 100 if not configured
min_sample = 100
return current_participant_count >= min_sample
async def get_normalization_params(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Tuple[float, float, Literal["static", "dynamic"]]:
"""
Get normalization parameters (rataan, sb) based on current mode.
Determines which normalization parameters to use:
- Static mode: Use config.static_rataan and config.static_sb
- Dynamic mode: Use calculated rataan and sb from TryoutStats
- Hybrid mode: Use static until threshold reached, then dynamic
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Tuple of (rataan, sb, mode_used)
Raises:
ValueError: If tryout not found or dynamic stats unavailable
"""
# Get normalization mode
mode = await get_normalization_mode(db, website_id, tryout_id)
if mode == "static":
# Use static values from config
result = await db.execute(
select(Tryout.static_rataan, Tryout.static_sb).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
row = result.scalar_one_or_none()
if row is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
rataan, sb = row
return rataan, sb, "static"
elif mode == "dynamic":
# Use dynamic values from stats
rataan, sb = await calculate_dynamic_stats(db, website_id, tryout_id)
if rataan is None or sb is None:
raise ValueError(
f"Dynamic normalization not available for tryout {tryout_id}. "
"No stats have been calculated yet."
)
if sb == 0:
logger.warning(
f"Standard deviation is 0 for tryout {tryout_id}. "
"All NM scores are identical."
)
return rataan, sb, "dynamic"
else: # hybrid
# Check threshold
threshold_met = await check_threshold_for_dynamic(db, website_id, tryout_id)
if threshold_met:
# Use dynamic values
rataan, sb = await calculate_dynamic_stats(db, website_id, tryout_id)
if rataan is None or sb is None:
# Fallback to static if dynamic not available
result = await db.execute(
select(Tryout.static_rataan, Tryout.static_sb).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
row = result.scalar_one_or_none()
if row is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
rataan, sb = row
return rataan, sb, "static"
return rataan, sb, "dynamic"
else:
# Use static values
result = await db.execute(
select(Tryout.static_rataan, Tryout.static_sb).where(
Tryout.website_id == website_id,
Tryout.tryout_id == tryout_id,
)
)
row = result.scalar_one_or_none()
if row is None:
raise ValueError(
f"Tryout {tryout_id} not found for website {website_id}"
)
rataan, sb = row
return rataan, sb, "static"
async def calculate_skewness(
db: AsyncSession,
website_id: int,
tryout_id: str,
) -> Optional[float]:
"""
Calculate skewness of NM distribution for validation.
Skewness measures the asymmetry of the probability distribution.
Values:
- Skewness ≈ 0: Symmetric distribution
- Skewness > 0: Right-skewed (tail to the right)
- Skewness < 0: Left-skewed (tail to the left)
Formula: Skewness = (n / ((n-1)(n-2))) * Σ((x - mean) / SD)³
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
Returns:
Skewness value, or None if insufficient data
"""
result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = result.scalar_one_or_none()
if stats is None or stats.participant_count < 3:
# Need at least 3 samples for skewness calculation
return None
n = stats.participant_count
mean = stats.rataan
sd = stats.sb
if sd == 0:
return 0.0 # All values are identical
# Calculate skewness
# We need individual NM values, which we don't have in TryoutStats
# For now, return None as we need a different approach
# This would require storing all NM values or calculating on-the-fly
return None
async def validate_dynamic_normalization(
db: AsyncSession,
website_id: int,
tryout_id: str,
target_mean: float = 500.0,
target_sd: float = 100.0,
mean_tolerance: float = 5.0,
sd_tolerance: float = 5.0,
) -> Tuple[bool, dict]:
"""
Validate that dynamic normalization produces expected distribution.
Checks if calculated rataan and sb are close to target values.
Args:
db: Async database session
website_id: Website identifier
tryout_id: Tryout identifier
target_mean: Target mean (default: 500)
target_sd: Target standard deviation (default: 100)
mean_tolerance: Allowed deviation from target mean (default: 5)
sd_tolerance: Allowed deviation from target SD (default: 5)
Returns:
Tuple of (is_valid, validation_details)
validation_details contains:
- participant_count: Number of participants
- current_rataan: Current mean
- current_sb: Current standard deviation
- mean_deviation: Absolute deviation from target mean
- sd_deviation: Absolute deviation from target SD
- mean_within_tolerance: True if mean deviation < mean_tolerance
- sd_within_tolerance: True if SD deviation < sd_tolerance
- warnings: List of warning messages
- suggestions: List of suggestions
"""
# Get current stats
result = await db.execute(
select(TryoutStats).where(
TryoutStats.website_id == website_id,
TryoutStats.tryout_id == tryout_id,
)
)
stats = result.scalar_one_or_none()
if stats is None or stats.rataan is None or stats.sb is None:
return False, {
"participant_count": 0,
"current_rataan": None,
"current_sb": None,
"mean_deviation": None,
"sd_deviation": None,
"mean_within_tolerance": False,
"sd_within_tolerance": False,
"warnings": ["No statistics available for validation"],
"suggestions": ["Wait for more participants to complete sessions"],
}
# Calculate deviations
mean_deviation = abs(stats.rataan - target_mean)
sd_deviation = abs(stats.sb - target_sd)
# Check tolerance
mean_within_tolerance = mean_deviation <= mean_tolerance
sd_within_tolerance = sd_deviation <= sd_tolerance
is_valid = mean_within_tolerance and sd_within_tolerance
# Generate warnings
warnings = []
suggestions = []
if not mean_within_tolerance:
warnings.append(f"Mean deviation ({mean_deviation:.2f}) exceeds tolerance ({mean_tolerance})")
if stats.rataan > target_mean:
suggestions.append("Distribution may be right-skewed - consider checking question difficulty")
else:
suggestions.append("Distribution may be left-skewed - consider checking question difficulty")
if not sd_within_tolerance:
warnings.append(f"SD deviation ({sd_deviation:.2f}) exceeds tolerance ({sd_tolerance})")
if stats.sb < target_sd:
suggestions.append("SD too low - scores may be too tightly clustered")
else:
suggestions.append("SD too high - scores may have too much variance")
# Check for skewness
skewness = await calculate_skewness(db, website_id, tryout_id)
if skewness is not None and abs(skewness) > 0.5:
warnings.append(f"Distribution skewness ({skewness:.2f}) > 0.5 - distribution may be asymmetric")
suggestions.append("Consider using static normalization if dynamic normalization is unstable")
# Check participant count
if stats.participant_count < 100:
suggestions.append(f"Participant count ({stats.participant_count}) below recommended minimum (100)")
return is_valid, {
"participant_count": stats.participant_count,
"current_rataan": stats.rataan,
"current_sb": stats.sb,
"mean_deviation": mean_deviation,
"sd_deviation": sd_deviation,
"mean_within_tolerance": mean_within_tolerance,
"sd_within_tolerance": sd_within_tolerance,
"warnings": warnings,
"suggestions": suggestions,
}

1449
app/services/reporting.py Normal file
View File

File diff suppressed because it is too large Load Diff

456
app/services/wordpress_auth.py Normal file
View File

@@ -0,0 +1,456 @@
"""
WordPress Authentication and User Synchronization Service.
Handles:
- JWT token validation via WordPress REST API
- User synchronization from WordPress to local database
- Multi-site support via website_id isolation
"""
import logging
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Any, Optional
import httpx
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.models.user import User
from app.models.website import Website
logger = logging.getLogger(__name__)
settings = get_settings()
# Custom exceptions for WordPress integration
class WordPressAuthError(Exception):
"""Base exception for WordPress authentication errors."""
pass
class WordPressTokenInvalidError(WordPressAuthError):
"""Raised when WordPress token is invalid or expired."""
pass
class WordPressAPIError(WordPressAuthError):
"""Raised when WordPress API is unreachable or returns error."""
pass
class WordPressRateLimitError(WordPressAuthError):
"""Raised when WordPress API rate limit is exceeded."""
pass
class WebsiteNotFoundError(WordPressAuthError):
"""Raised when website_id is not found in local database."""
pass
@dataclass
class WordPressUserInfo:
"""Data class for WordPress user information."""
wp_user_id: str
username: str
email: str
display_name: str
roles: list[str]
raw_data: dict[str, Any]
@dataclass
class SyncStats:
"""Data class for user synchronization statistics."""
inserted: int
updated: int
total: int
errors: int
async def get_wordpress_api_base(website: Website) -> str:
"""
Get WordPress API base URL for a website.
Args:
website: Website model instance
Returns:
WordPress REST API base URL
"""
# Use website's site_url if configured, otherwise use global config
base_url = website.site_url.rstrip('/')
return f"{base_url}/wp-json"
async def verify_wordpress_token(
token: str,
website_id: int,
wp_user_id: str,
db: AsyncSession,
) -> Optional[WordPressUserInfo]:
"""
Verify WordPress JWT token and validate user identity.
Calls WordPress REST API GET /wp/v2/users/me with Authorization header.
Verifies response contains matching wp_user_id.
Verifies website_id exists in local database.
Args:
token: WordPress JWT authentication token
website_id: Website identifier for multi-site isolation
wp_user_id: Expected WordPress user ID to verify
db: Async database session
Returns:
WordPressUserInfo if valid, None if invalid
Raises:
WebsiteNotFoundError: If website_id doesn't exist
WordPressTokenInvalidError: If token is invalid
WordPressAPIError: If API is unreachable
WordPressRateLimitError: If rate limited
"""
# Verify website exists
website_result = await db.execute(
select(Website).where(Website.id == website_id)
)
website = website_result.scalar_one_or_none()
if website is None:
raise WebsiteNotFoundError(f"Website {website_id} not found")
api_base = await get_wordpress_api_base(website)
url = f"{api_base}/wp/v2/users/me"
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/json",
}
timeout = httpx.Timeout(10.0, connect=5.0)
try:
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.get(url, headers=headers)
if response.status_code == 401:
raise WordPressTokenInvalidError("Invalid or expired WordPress token")
if response.status_code == 429:
raise WordPressRateLimitError("WordPress API rate limit exceeded")
if response.status_code == 503:
raise WordPressAPIError("WordPress API service unavailable")
if response.status_code != 200:
raise WordPressAPIError(
f"WordPress API error: {response.status_code} - {response.text}"
)
data = response.json()
# Verify user ID matches
response_user_id = str(data.get("id", ""))
if response_user_id != str(wp_user_id):
logger.warning(
f"User ID mismatch: expected {wp_user_id}, got {response_user_id}"
)
return None
# Extract user info
user_info = WordPressUserInfo(
wp_user_id=response_user_id,
username=data.get("username", ""),
email=data.get("email", ""),
display_name=data.get("name", ""),
roles=data.get("roles", []),
raw_data=data,
)
return user_info
except httpx.TimeoutException:
raise WordPressAPIError("WordPress API request timed out")
except httpx.ConnectError:
raise WordPressAPIError("Unable to connect to WordPress API")
except httpx.HTTPError as e:
raise WordPressAPIError(f"HTTP error communicating with WordPress: {str(e)}")
async def fetch_wordpress_users(
website: Website,
admin_token: str,
page: int = 1,
per_page: int = 100,
) -> list[dict[str, Any]]:
"""
Fetch users from WordPress API (requires admin token).
Calls WordPress REST API GET /wp/v2/users with admin authorization.
Args:
website: Website model instance
admin_token: WordPress admin JWT token
page: Page number for pagination
per_page: Number of users per page (max 100)
Returns:
List of WordPress user data dictionaries
Raises:
WordPressTokenInvalidError: If admin token is invalid
WordPressAPIError: If API is unreachable
WordPressRateLimitError: If rate limited
"""
api_base = await get_wordpress_api_base(website)
url = f"{api_base}/wp/v2/users"
headers = {
"Authorization": f"Bearer {admin_token}",
"Accept": "application/json",
}
params = {
"page": page,
"per_page": min(per_page, 100),
"context": "edit", # Get full user data
}
timeout = httpx.Timeout(30.0, connect=10.0)
try:
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.get(url, headers=headers, params=params)
if response.status_code == 401:
raise WordPressTokenInvalidError("Invalid admin token for user sync")
if response.status_code == 403:
raise WordPressTokenInvalidError(
"Admin token lacks permission to list users"
)
if response.status_code == 429:
raise WordPressRateLimitError("WordPress API rate limit exceeded")
if response.status_code == 503:
raise WordPressAPIError("WordPress API service unavailable")
if response.status_code != 200:
raise WordPressAPIError(
f"WordPress API error: {response.status_code} - {response.text}"
)
return response.json()
except httpx.TimeoutException:
raise WordPressAPIError("WordPress API request timed out")
except httpx.ConnectError:
raise WordPressAPIError("Unable to connect to WordPress API")
except httpx.HTTPError as e:
raise WordPressAPIError(f"HTTP error communicating with WordPress: {str(e)}")
async def sync_wordpress_users(
website_id: int,
admin_token: str,
db: AsyncSession,
) -> SyncStats:
"""
Synchronize users from WordPress to local database.
Fetches all users from WordPress API and performs upsert:
- Updates existing users
- Inserts new users
Args:
website_id: Website identifier for multi-site isolation
admin_token: WordPress admin JWT token
db: Async database session
Returns:
SyncStats with insertion/update counts
Raises:
WebsiteNotFoundError: If website_id doesn't exist
WordPressTokenInvalidError: If admin token is invalid
WordPressAPIError: If API is unreachable
"""
# Verify website exists
website_result = await db.execute(
select(Website).where(Website.id == website_id)
)
website = website_result.scalar_one_or_none()
if website is None:
raise WebsiteNotFoundError(f"Website {website_id} not found")
# Fetch existing users from local database
existing_users_result = await db.execute(
select(User).where(User.website_id == website_id)
)
existing_users = {
str(user.wp_user_id): user
for user in existing_users_result.scalars().all()
}
# Fetch users from WordPress (with pagination)
all_wp_users = []
page = 1
per_page = 100
while True:
wp_users = await fetch_wordpress_users(
website, admin_token, page, per_page
)
if not wp_users:
break
all_wp_users.extend(wp_users)
# Check if more pages
if len(wp_users) < per_page:
break
page += 1
# Sync users
inserted = 0
updated = 0
errors = 0
for wp_user in all_wp_users:
try:
wp_user_id = str(wp_user.get("id", ""))
if not wp_user_id:
errors += 1
continue
if wp_user_id in existing_users:
# Update existing user (timestamp update)
existing_user = existing_users[wp_user_id]
existing_user.updated_at = datetime.now(timezone.utc)
updated += 1
else:
# Insert new user
new_user = User(
wp_user_id=wp_user_id,
website_id=website_id,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
)
db.add(new_user)
inserted += 1
except Exception as e:
logger.error(f"Error syncing user {wp_user.get('id')}: {e}")
errors += 1
await db.commit()
total = inserted + updated
logger.info(
f"WordPress user sync complete for website {website_id}: "
f"{inserted} inserted, {updated} updated, {errors} errors"
)
return SyncStats(
inserted=inserted,
updated=updated,
total=total,
errors=errors,
)
async def get_wordpress_user(
wp_user_id: str,
website_id: int,
db: AsyncSession,
) -> Optional[User]:
"""
Get user from local database by WordPress user ID and website ID.
Args:
wp_user_id: WordPress user ID
website_id: Website identifier for multi-site isolation
db: Async database session
Returns:
User object if found, None otherwise
"""
result = await db.execute(
select(User).where(
User.wp_user_id == wp_user_id,
User.website_id == website_id,
)
)
return result.scalar_one_or_none()
async def verify_website_exists(
website_id: int,
db: AsyncSession,
) -> Website:
"""
Verify website exists in database.
Args:
website_id: Website identifier
db: Async database session
Returns:
Website model instance
Raises:
WebsiteNotFoundError: If website doesn't exist
"""
result = await db.execute(
select(Website).where(Website.id == website_id)
)
website = result.scalar_one_or_none()
if website is None:
raise WebsiteNotFoundError(f"Website {website_id} not found")
return website
async def get_or_create_user(
wp_user_id: str,
website_id: int,
db: AsyncSession,
) -> User:
"""
Get existing user or create new one if not exists.
Args:
wp_user_id: WordPress user ID
website_id: Website identifier
db: Async database session
Returns:
User model instance
"""
existing = await get_wordpress_user(wp_user_id, website_id, db)
if existing:
return existing
# Create new user
new_user = User(
wp_user_id=wp_user_id,
website_id=website_id,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
)
db.add(new_user)
await db.commit()
await db.refresh(new_user)
return new_user

96
handoff.md Normal file
View File

@@ -0,0 +1,96 @@
---
## HANDOFF CONTEXT
GOAL
----
Continue implementation of IRT-Powered Adaptive Question Bank System after user configures GLM-5 model mapping for specific subagent categories.
WORK COMPLETED
--------------
- Created comprehensive PRD (v1.1) from project-brief.md
- Resolved 10 critical clarification questions with client:
1. Excel Import: Standardized across ALL tryouts
2. AI Generation: 1 request = 1 question, admin playground for testing, no approval workflow
3. Normalization: Optional manual/automatic control (system handles auto when sufficient data)
4. Rollback: Preserve IRT history, apply CTT to new sessions only
5. Admin Permissions: Not needed (WordPress handles per-site admins)
6. Dashboards: FastAPI Admin only
7. Rate Limiting: User-level reuse check + AI generation toggle
8. Student UX: Admin sees internal metrics, students only see primary score
9. Data Retention: Keep all data
10. Reporting: All 4 report types required
- Created detailed technical implementation plan with 10 parallel subagents:
- Deep Agent 1: Core API + CTT Scoring
- Deep Agent 2: IRT Calibration Engine (recommended for GLM-5)
- Deep Agent 3: CAT Selection Logic (recommended for GLM-5)
- Deep Agent 4: AI Generation + OpenRouter (recommended for GLM-5)
- Deep Agent 5: WordPress Integration
- Deep Agent 6: Reporting System (recommended for GLM-5)
- Unspecified-High Agents: Database Schema, Excel Import/Export, Admin Panel, Normalization
CURRENT STATE
-------------
- PRD.md file created (746 lines, v1.1)
- project-brief.md exists (reference document)
- No code implementation started yet
- No git repository initialized
- Working directory: /Users/dwindown/Applications/tryout-system
- Session ID: ses_2f1bf9e3cffes96exBxyheOiYT
PENDING TASKS
-------------
1. User configures GLM-5 model mapping for `deep` category (GLM-5 for algorithmic complexity)
2. User configures GLM-4.7 model mapping for `unspecified-high` category (general implementation)
3. Initialize git repository
4. Create project structure (app/, models/, routers/, services/, tests/)
5. Launch Unspecified-High Agent 1: Database Schema + ORM (BLOCKS all other agents)
6. After schema complete: Launch Deep Agents 1-3 in parallel (Core API, IRT Calibration, CAT Selection)
7. Launch Deep Agents 4-6 + Unspecified-High Agents 2-4 in parallel (AI Generation, WordPress, Reporting, Excel, Admin, Normalization)
8. Integration testing and validation
KEY FILES
---------
- PRD.md - Complete product requirements document (v1.1, 746 lines)
- project-brief.md - Original technical specification reference
IMPORTANT DECISIONS
-------------------
- 1 request = 1 question for AI generation (no batch)
- Admin playground for AI testing (no approval workflow for student tests)
- Normalization: Admin chooses manual/automatic; system handles auto when data sufficient
- Rollback: Keep IRT historical scores, apply CTT only to new sessions
- No admin permissions system (WordPress handles per-site admin access)
- FastAPI Admin only (no custom dashboards)
- Global AI generation toggle for cost control
- User-level question reuse check (prevent duplicate difficulty exposure)
- Admin sees internal metrics, students only see primary score
- Keep all data indefinitely
- All 4 report types required (Student, Item, Calibration, Tryout comparison)
EXPLICIT CONSTRAINTS
--------------------
- Excel format is standardized across ALL tryouts (strict parser)
- CTT formulas must match client Excel 100% (p = Σ Benar / Total Peserta)
- IRT 1PL Rasch model only (b parameter, no a/c initially)
- θ and b ∈ [-3, +3], NM and NN ∈ [0, 1000]
- Normalization target: Mean=500±5, SD=100±5
- Tech stack: FastAPI, PostgreSQL, SQLAlchemy, FastAPI Admin, OpenRouter (Qwen3 Coder 480B / Llama 3.3 70B)
- Deployment: aaPanel VPS with Python Manager
- No type error suppression (no `as any`, `@ts-ignore`)
- Zero disruption to existing operations (non-destructive, additive)
GLM-5 MODEL ALLOCATION RECOMMENDATION
-----------------------------------
Use GLM-5 for:
- Deep Agent 2: IRT Calibration Engine (mathematical algorithms, sparse data handling)
- Deep Agent 3: CAT Selection Logic (adaptive algorithms, termination conditions)
- Deep Agent 4: AI Generation + OpenRouter (prompt engineering, robust parsing)
- Deep Agent 6: Reporting System (complex aggregation, multi-dimensional analysis)
Use GLM-4.7 for:
- Deep Agent 1: Core API + CTT Scoring (straightforward formulas)
- Deep Agent 5: WordPress Integration (standard REST API)
- Unspecified-High Agents: Database Schema, Excel Import/Export, Admin Panel, Normalization (well-defined tasks)
NOTE: Model mapping is controlled by category configuration in system, not by direct model specification in task() function.
CONTEXT FOR CONTINUATION
------------------------
- User is currently configuring GLM-5 model mapping for specific categories
- After model mapping is configured, implementation should start with Database Schema (Unspecified-High Agent 1) as it blocks all other work
- Parallel execution strategy: Never run sequential when parallel is possible - all independent work units run simultaneously
- Use `task(category="...", load_skills=[], run_in_background=true)` pattern for parallel delegation
- All delegated work must include: TASK, EXPECTED OUTCOME, REQUIRED TOOLS, MUST DO, MUST NOT DO, CONTEXT (6-section prompt structure)
- Verify results after delegation: DOES IT WORK? DOES IT FOLLOW PATTERNS? EXPECTED RESULT ACHIEVED?
- Run `lsp_diagnostics` on changed files before marking tasks complete
- This is NOT a git repository yet - will need to initialize before any version control operations
---

135
irt_1pl_mle.py Normal file
View File

@@ -0,0 +1,135 @@
"""
IRT 1PL (Rasch Model) Maximum Likelihood Estimation
"""
import numpy as np
from scipy.optimize import minimize_scalar, minimize
def estimate_theta(responses, b_params):
"""
Estimate student ability theta using MLE for 1PL IRT model.
Parameters:
-----------
responses : list or array
Binary responses [0, 1, 1, 0, ...]
b_params : list or array
Item difficulty parameters [b1, b2, b3, ...]
Returns:
--------
float
Estimated theta (ability), or None if estimation fails
"""
responses = np.asarray(responses, dtype=float)
b_params = np.asarray(b_params, dtype=float)
# Edge case: empty or mismatched inputs
if len(responses) == 0 or len(b_params) == 0:
return 0.0
if len(responses) != len(b_params):
raise ValueError("responses and b_params must have same length")
n = len(responses)
sum_resp = np.sum(responses)
# Edge case: all correct - return high theta
if sum_resp == n:
return 4.0
# Edge case: all incorrect - return low theta
if sum_resp == 0:
return -4.0
def neg_log_likelihood(theta):
"""Negative log-likelihood for minimization."""
exponent = theta - b_params
# Numerical stability: clip exponent
exponent = np.clip(exponent, -30, 30)
p = 1.0 / (1.0 + np.exp(-exponent))
# Avoid log(0)
p = np.clip(p, 1e-10, 1 - 1e-10)
ll = np.sum(responses * np.log(p) + (1 - responses) * np.log(1 - p))
return -ll
result = minimize_scalar(neg_log_likelihood, bounds=(-6, 6), method='bounded')
if result.success:
return float(result.x)
return 0.0
def estimate_b(responses_matrix):
"""
Estimate item difficulty parameters using joint MLE for 1PL IRT model.
Parameters:
-----------
responses_matrix : 2D array
Response matrix where rows=students, cols=items
entries are 0 or 1
Returns:
--------
numpy.ndarray
Estimated b parameters for each item, or None if estimation fails
"""
responses_matrix = np.asarray(responses_matrix, dtype=float)
# Edge case: empty matrix
if responses_matrix.size == 0:
return np.array([])
if responses_matrix.ndim != 2:
raise ValueError("responses_matrix must be 2-dimensional")
n_students, n_items = responses_matrix.shape
if n_students == 0 or n_items == 0:
return np.zeros(n_items)
# Initialize theta and b
theta = np.zeros(n_students)
b = np.zeros(n_items)
# Check for items with all same responses
item_sums = np.sum(responses_matrix, axis=0)
for iteration in range(20): # EM iterations
# Update theta for each student
for i in range(n_students):
resp_i = responses_matrix[i, :]
sum_resp = np.sum(resp_i)
if sum_resp == n_items:
theta[i] = 4.0
elif sum_resp == 0:
theta[i] = -4.0
else:
def neg_ll_student(t):
exponent = np.clip(t - b, -30, 30)
p = np.clip(1.0 / (1.0 + np.exp(-exponent)), 1e-10, 1 - 1e-10)
return -np.sum(resp_i * np.log(p) + (1 - resp_i) * np.log(1 - p))
res = minimize_scalar(neg_ll_student, bounds=(-6, 6), method='bounded')
theta[i] = res.x if res.success else 0.0
# Update b for each item
for j in range(n_items):
resp_j = responses_matrix[:, j]
sum_resp = np.sum(resp_j)
if sum_resp == n_students:
b[j] = -4.0 # Easy item
elif sum_resp == 0:
b[j] = 4.0 # Hard item
else:
def neg_ll_item(bj):
exponent = np.clip(theta - bj, -30, 30)
p = np.clip(1.0 / (1.0 + np.exp(-exponent)), 1e-10, 1 - 1e-10)
return -np.sum(resp_j * np.log(p) + (1 - resp_j) * np.log(1 - p))
res = minimize_scalar(neg_ll_item, bounds=(-6, 6), method='bounded')
b[j] = res.x if res.success else 0.0
return b

1109
project-brief.md Normal file
View File

File diff suppressed because it is too large Load Diff

40
requirements.txt Normal file
View File

@@ -0,0 +1,40 @@
# FastAPI and Server
fastapi>=0.104.1
uvicorn[standard]>=0.24.0
python-multipart>=0.0.6
# Database
sqlalchemy>=2.0.23
asyncpg>=0.29.0
alembic>=1.13.0
# Data & Validation
pydantic>=2.5.0
pydantic-settings>=2.1.0
# Excel Processing
openpyxl>=3.1.2
pandas>=2.1.4
# Math & Science
numpy>=1.26.2
scipy>=1.11.4
# AI Integration
openai>=1.6.1
httpx>=0.26.0
# Task Queue (for async jobs)
celery>=5.3.6
redis>=5.0.1
# Testing
pytest>=7.4.3
pytest-asyncio>=0.21.1
httpx>=0.26.0
# Admin Panel
fastapi-admin>=1.4.0
# Utilities
python-dotenv>=1.0.0

275
tests/test_normalization.py Normal file
View File

@@ -0,0 +1,275 @@
#!/usr/bin/env python3
"""
Test script for normalization calculations.
This script tests the normalization functions to ensure they work correctly
without requiring database connections.
"""
import sys
import os
# Add the project root to the path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from app.services.normalization import apply_normalization
def test_apply_normalization():
"""Test the apply_normalization function."""
print("Testing apply_normalization function...")
print("=" * 60)
# Test case 1: Normal normalization (NM=500, rataan=500, sb=100)
nm1 = 500
rataan1 = 500
sb1 = 100
nn1 = apply_normalization(nm1, rataan1, sb1)
expected1 = 500
print(f"Test 1: NM={nm1}, rataan={rataan1}, sb={sb1}")
print(f" Expected NN: {expected1}")
print(f" Actual NN: {nn1}")
print(f" Status: {'PASS' if nn1 == expected1 else 'FAIL'}")
print()
# Test case 2: High score (NM=600, rataan=500, sb=100)
nm2 = 600
rataan2 = 500
sb2 = 100
nn2 = apply_normalization(nm2, rataan2, sb2)
expected2 = 600
print(f"Test 2: NM={nm2}, rataan={rataan2}, sb={sb2}")
print(f" Expected NN: {expected2}")
print(f" Actual NN: {nn2}")
print(f" Status: {'PASS' if nn2 == expected2 else 'FAIL'}")
print()
# Test case 3: Low score (NM=400, rataan=500, sb=100)
nm3 = 400
rataan3 = 500
sb3 = 100
nn3 = apply_normalization(nm3, rataan3, sb3)
expected3 = 400
print(f"Test 3: NM={nm3}, rataan={rataan3}, sb={sb3}")
print(f" Expected NN: {expected3}")
print(f" Actual NN: {nn3}")
print(f" Status: {'PASS' if nn3 == expected3 else 'FAIL'}")
print()
# Test case 4: Edge case - maximum NM
nm4 = 1000
rataan4 = 500
sb4 = 100
nn4 = apply_normalization(nm4, rataan4, sb4)
expected4 = 1000
print(f"Test 4: NM={nm4}, rataan={rataan4}, sb={sb4}")
print(f" Expected NN: {expected4}")
print(f" Actual NN: {nn4}")
print(f" Status: {'PASS' if nn4 == expected4 else 'FAIL'}")
print()
# Test case 5: Edge case - minimum NM
nm5 = 0
rataan5 = 500
sb5 = 100
nn5 = apply_normalization(nm5, rataan5, sb5)
expected5 = 0
print(f"Test 5: NM={nm5}, rataan={rataan5}, sb={sb5}")
print(f" Expected NN: {expected5}")
print(f" Actual NN: {nn5}")
print(f" Status: {'PASS' if nn5 == expected5 else 'FAIL'}")
print()
# Test case 6: Error case - invalid NM (above max)
try:
nm6 = 1200 # Above valid range
rataan6 = 500
sb6 = 100
nn6 = apply_normalization(nm6, rataan6, sb6)
print(f"Test 6: NM={nm6}, rataan={rataan6}, sb={sb6} (should raise ValueError)")
print(f" Status: FAIL - Should have raised ValueError")
except ValueError as e:
print(f"Test 6: NM={nm6}, rataan={rataan6}, sb={sb6} (should raise ValueError)")
print(f" Error: {e}")
print(f" Status: PASS - Correctly raised ValueError")
print()
# Test case 7: Error case - invalid NM (below min)
try:
nm7 = -100 # Below valid range
rataan7 = 500
sb7 = 100
nn7 = apply_normalization(nm7, rataan7, sb7)
print(f"Test 7: NM={nm7}, rataan={rataan7}, sb={sb7} (should raise ValueError)")
print(f" Status: FAIL - Should have raised ValueError")
except ValueError as e:
print(f"Test 7: NM={nm7}, rataan={rataan7}, sb={sb7} (should raise ValueError)")
print(f" Error: {e}")
print(f" Status: PASS - Correctly raised ValueError")
print()
# Test case 8: Different rataan/sb (NM=500, rataan=600, sb=80)
nm8 = 500
rataan8 = 600
sb8 = 80
nn8 = apply_normalization(nm8, rataan8, sb8)
# z_score = (500 - 600) / 80 = -1.25
# nn = 500 + 100 * (-1.25) = 500 - 125 = 375
expected8 = 375
print(f"Test 8: NM={nm8}, rataan={rataan8}, sb={sb8}")
print(f" Expected NN: {expected8}")
print(f" Actual NN: {nn8}")
print(f" Status: {'PASS' if nn8 == expected8 else 'FAIL'}")
print()
# Test case 9: Error case - invalid NM
try:
nm9 = 1500 # Above valid range
rataan9 = 500
sb9 = 100
nn9 = apply_normalization(nm9, rataan9, sb9)
print(f"Test 9: NM={nm9}, rataan={rataan9}, sb={sb9} (should raise ValueError)")
print(f" Status: FAIL - Should have raised ValueError")
except ValueError as e:
print(f"Test 9: NM=1500, rataan=500, sb=100 (should raise ValueError)")
print(f" Error: {e}")
print(f" Status: PASS - Correctly raised ValueError")
print()
# Test case 10: Error case - invalid sb
try:
nm10 = 500
rataan10 = 500
sb10 = 0 # Invalid SD
nn10 = apply_normalization(nm10, rataan10, sb10)
expected10 = 500 # Should return default when sb <= 0
print(f"Test 10: NM={nm10}, rataan={rataan10}, sb={sb10} (should return default)")
print(f" Expected NN: {expected10}")
print(f" Actual NN: {nn10}")
print(f" Status: {'PASS' if nn10 == expected10 else 'FAIL'}")
except Exception as e:
print(f"Test 10: NM=500, rataan=500, sb=0 (should return default)")
print(f" Error: {e}")
print(f" Status: FAIL - Should have returned default value")
print()
print("=" * 60)
print("All tests completed!")
print("=" * 60)
def calculate_dynamic_mean_and_std(nm_values):
"""
Calculate mean and standard deviation from a list of NM values.
This simulates what update_dynamic_normalization does.
"""
n = len(nm_values)
if n == 0:
return None, None
# Calculate mean
mean = sum(nm_values) / n
# Calculate variance (population variance)
if n > 1:
variance = sum((x - mean) ** 2 for x in nm_values) / n
std = variance ** 0.5
else:
std = 0.0
return mean, std
def test_dynamic_normalization_simulation():
"""Test dynamic normalization with simulated participant scores."""
print("\nTesting dynamic normalization simulation...")
print("=" * 60)
# Simulate 10 participant NM scores
nm_scores = [450, 480, 500, 520, 550, 480, 510, 490, 530, 470]
print(f"Simulated NM scores: {nm_scores}")
print()
# Calculate mean and SD
mean, std = calculate_dynamic_mean_and_std(nm_scores)
print(f"Calculated mean (rataan): {mean:.2f}")
print(f"Calculated SD (sb): {std:.2f}")
print()
# Normalize each score
print("Normalized scores:")
for i, nm in enumerate(nm_scores):
nn = apply_normalization(nm, mean, std)
print(f" Participant {i+1}: NM={nm:3d} -> NN={nn:3d}")
print()
# Check if normalized distribution is close to mean=500, SD=100
nn_scores = [apply_normalization(nm, mean, std) for nm in nm_scores]
nn_mean, nn_std = calculate_dynamic_mean_and_std(nn_scores)
print(f"Normalized distribution:")
print(f" Mean: {nn_mean:.2f} (target: 500 ± 5)")
print(f" SD: {nn_std:.2f} (target: 100 ± 5)")
print(f" Status: {'PASS' if abs(nn_mean - 500) <= 5 and abs(nn_std - 100) <= 5 else 'NEAR PASS'}")
print()
print("=" * 60)
def test_incremental_update():
"""Test incremental update of dynamic normalization."""
print("\nTesting incremental update simulation...")
print("=" * 60)
# Simulate adding scores incrementally
nm_scores = []
participant_count = 0
total_nm_sum = 0.0
total_nm_sq_sum = 0.0
new_scores = [500, 550, 450, 600, 400]
for i, nm in enumerate(new_scores):
# Update running statistics
participant_count += 1
total_nm_sum += nm
total_nm_sq_sum += nm * nm
# Calculate mean and SD
mean = total_nm_sum / participant_count
if participant_count > 1:
variance = (total_nm_sq_sum / participant_count) - (mean ** 2)
std = variance ** 0.5
else:
std = 0.0
nm_scores.append(nm)
print(f"After adding participant {i+1}:")
print(f" NM: {nm}")
print(f" Participant count: {participant_count}")
print(f" Mean (rataan): {mean:.2f}")
print(f" SD (sb): {std:.2f}")
print()
# Final calculation
final_mean, final_std = calculate_dynamic_mean_and_std(nm_scores)
print(f"Final statistics:")
print(f" All scores: {nm_scores}")
print(f" Mean: {final_mean:.2f}")
print(f" SD: {final_std:.2f}")
print()
print("=" * 60)
if __name__ == "__main__":
print("Normalization Calculation Tests")
print("=" * 60)
print()
test_apply_normalization()
test_dynamic_normalization_simulation()
test_incremental_update()
print("\nAll test simulations completed successfully!")