dwindown/dewemoji
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a4d2031117a6f60fc70a949e2434b5f52bb4aef6
dewemoji/rebuild-progress.md

13 KiB
Raw Blame History

NativePHP Rebuild Progress (with dewemoji-live audit)

Confirmed source folders

  • Backend: ../dewemoji-api
  • Live backend reference: ../dewemoji-live-backend (source of truth for current API + community/pro flows)
  • Chrome extension: ../dewemoji-chrome-ext
  • Website (legacy scaffold): ../dewemoji-site
  • Live production reference: ../dewemoji-live (source of truth for parity)

Current baseline

  • New rebuild app: app/ (Laravel + NativePHP Desktop).
  • API v1 routes exist in rebuild: /v1/emojis, /v1/categories, /v1/license/verify.
  • Website routes currently in rebuild: /, /emoji/{slug}, /api-docs, /pricing, /privacy, /terms.
  • Live site has additional behavior and SEO/route details that must be ported before full parity.

Agreed strategy (locked)

  • Build order: backend-first, then frontend integration.
  • Community feature: included in migration scope, but implemented last after core/pro stabilization.
  • Payment/provider mode: start in sandbox, document switch path to live (SANDBOX -> LIVE) in project docs.
  • Database: fresh Laravel-first schema + import scripts from legacy data sources.
  • Metrics endpoints: keep, but internal-only (admin token/IP allowlist), not public.
  • Upgrade policy: migration is parity-first, but we will take safe opportunities to improve architecture, security, and observability.

Phase checklist

Phase 0 - Retrace and documentation

  • Revalidated source folders with corrected names.
  • Rebuilt initial docs against corrected folders.
  • Added live audit from dewemoji-live.
  • Added backend audit from dewemoji-live-backend.

Phase 1 - Foundation

  • Initialize NativePHP app in dewemoji/app.
  • Install NativePHP Desktop scaffolding.
  • Define env/config strategy.
  • Use canonical emoji dataset as baseline.

Phase 2 - API parity (extension first)

  • Implement GET /v1/emojis.
  • Implement GET /v1/categories.
  • Implement POST /v1/license/verify (temporary env-based validation).
  • Support both q and query.
  • Add/verify full response contract parity with live docs (variants, related, trimmed keywords_en, limit behavior by tier).
  • Match live cache/rate semantics baseline (page=1 metering behavior, 401/429 payload shape, ETag/304 behavior).
  • Verify header compatibility baseline: Authorization, X-License-Key, X-Account-Id, X-Dewemoji-Frontend, X-Dewemoji-Tier.
  • Restrict CORS to configured origins (no default *).
  • Add missing live backend routes/contracts now present in production API:
    • /v1/license/activate
    • /v1/license/deactivate
    • /v1/health
    • /v1/metrics and /v1/metrics-lite (internal/admin decision needed: keep, secure, or remove)
  • Reconcile live route mismatch: added /v1/emoji and /v1/emoji/{slug} in rebuild API.
  • Add sandbox/live provider switch documentation and env examples (BILLING_MODE=sandbox|live, keys, callbacks, smoke test flow).

Phase 3 - Website parity from dewemoji-live

  • Core pages exist in rebuild.
  • Add missing pages/routes: /support, /browse, pretty category routes (/{category} and /{category}/{subcategory}).
    • /browse, /{category}, /{category}/{subcategory} implemented in rebuild.
  • Keep URL behavior parity (canonical no-trailing-slash pages, redirect rules, pretty-to-query hydration).
    • no-trailing-slash redirect middleware and canonical link baseline implemented.
    • pretty route hydration wired into homepage initial filters + URL sync.
  • Port homepage behavior parity:
    • API-backed filters (q, category, subcategory), URL sync, load-more pagination.
    • API fallback when scoped search returns 0 (retry on all + hint).
  • Port single emoji page parity:
    • 404 noindex for missing.
    • 410 + X-Robots-Tag: noindex, noarchive for policy-hidden emoji.
    • skin-tone variant logic and optional tone path (/emoji/{slug}/{tone}).
    • related fallback (same subcategory), prev/next navigation.
    • details blocks: aliases, shortcodes, EN/ID keywords, copy interactions.
    • curated blurbs support from data/emoji_descriptions.json.
  • Port legal/support content parity and FAQ schema blocks.

Phase 4 - Pricing and payments

  • Keep pricing structure parity:
    • Free, Pro subscription, Lifetime.
    • Pro: $3/mo and yearly display $27/yr in UI.
    • Lifetime: $69.
  • Preserve live Gumroad links:
    • https://dwindown.gumroad.com/l/dewemoji-pro-subscription
    • https://dwindown.gumroad.com/l/dewemoji-pro-lifetime
  • Keep IDR/Mayar messaging parity (manual-renew note).
  • Implement real license lifecycle (activate/deactivate/verify + max 3 Chrome profiles) in new backend.
  • Implement real license lifecycle baseline in rebuild (verify/activate/deactivate, immutable owner binding behavior, max device cap).
  • Implement provider verification parity:
    • Baseline service layer + env/config wiring + safe HTTP fallback
    • /v1/license/verify contract hardening: provider details + diagnostics (details.gumroad, details.mayar)
    • Gumroad verify API flow (final payload/contract parity with live provider account)
    • Mayar verify API flow (final payload/contract parity with live provider account)
    • gateway mode switch (sandbox vs live)
  • Implement immutable license binding to user + multi-device activation policy parity.

Phase 5 - SEO parity (must not disrupt GSC)

  • Preserve canonical strategy for all pages (including emoji detail + pretty category pages).
  • Add/verify meta + social tags parity: title/description/OG/Twitter + theme color.
  • Port JSON-LD strategy:
    • Global WebSite + SearchAction + Organization.
    • TechArticle on /api-docs.
    • Product + FAQPage on /pricing.
    • FAQPage on /support.
    • CreativeWork + BreadcrumbList on emoji pages.
  • Implement robots.txt parity and dynamic sitemap.xml parity.
  • Ensure sitemap excludes policy-hidden emoji URLs (same filter policy as live).
  • Keep core indexed URLs stable: /, /pricing, /api-docs, /support, /privacy, /terms, /emoji/{slug}.

Phase 6 - Analytics, consent, and compliance

  • Re-implement cookie consent flow before analytics activation.
  • Re-implement GA4 only on allowed production hosts (live uses G-R7FYYRBVJK).
  • Keep privacy/terms statements aligned with live content.

Phase 7 - Data/ops pipelines

  • Port blurb pipeline:
    • jobs/seed_blurbs_from_dataset.php
    • jobs/sync_blurbs.php (NocoDB approved blurbs sync).
  • Define NativePHP/Laravel replacement for live file microcache (cache/emoji/*.html) if still needed for SEO performance.
  • Add rebuild-side commands/jobs for sitemap regeneration and cache warmup.
  • Port backend dataset pipelines from dewemoji-live-backend/jobs:
    • JSON -> SQL import (import_emojis_json_to_sql.php)
    • SQL -> JSON build (build_emojis_json_from_sql.php)
    • Keywords index build (build_keywords_json_from_sql.php)
    • Unicode parity validation (validate_emojis_against_unicode.php)
    • License expiry revocation cron (check_license_expiry.php)

Phase 8 - Community feature migration

  • Port contributor auth flow:
    • magic link token issue/verify (/v1/contrib/auth/request, /v1/contrib/auth/verify)
    • stateless HMAC token strategy or Laravel equivalent.
  • Port contribution flows:
    • suggest keywords (/v1/contrib/suggest)
    • private -> public promotion (/v1/contrib/make-public)
    • list and search (/v1/contrib/list, /v1/contrib/search)
    • voting + pending queue (/v1/contrib/vote, /v1/keywords/pending)
  • Port moderation protections:
    • Turnstile verification
    • AI guard moderation pipeline (OpenRouter + usage caps/cache)
    • Redis/APCu rate limiting (vote/suggest/publish paths)
  • Port auto-moderation behavior:
    • score thresholds (vote_auto_approve_score, vote_auto_reject_score)
    • status/visibility transitions (private, public_pending, public, approved, rejected)
  • Fix live bug during migration: /v1/contrib/search is nested under /v1/keywords/pending block in current controller, so route behavior should be revalidated in rebuild.

Phase 9 - Extension integration and release

  • Point dewemoji-chrome-ext to new API host.
  • Validate free/pro flow end-to-end with real license checks.
  • Run parity QA on tone handling, insert/copy behavior, and API limits.
  • Prepare migration + rollback checklist (DNS/host switch, redirects, monitoring).

Recent implementation update

  • Added new API endpoints in rebuild:
    • GET /v1/emoji
    • GET /v1/emoji/{slug}
    • POST /v1/license/activate
    • POST /v1/license/deactivate
    • GET /v1/health
    • GET /v1/metrics
    • GET /v1/metrics-lite
  • Added internal-protected metrics controller (token or IP allowlist).
  • Added sandbox/live billing mode documentation: billing-sandbox-live.md.
  • Added fresh Laravel migrations for core backend state:
    • licenses
    • license_activations
    • usage_logs
  • Added LicenseVerificationService and wired controllers to use one verification path:
    • sandbox mode
    • live key-list mode
    • baseline Gumroad/Mayar provider calls (+ local stub test keys)
  • Added SEO-safe route/canonical baseline:
    • /browse route
    • pretty category routes (/{category}, /{category}/{subcategory})
    • trailing slash -> canonical path redirect (301)
    • canonical <link> output from layout

Live audit highlights (reference)

  • Live web routes in dewemoji-live/public_html: /, /emoji/{slug}, /browse, /pricing, /api-docs, /support, /privacy, /terms, /sitemap.xml.
  • Rewrite rules and canonicalization live in dewemoji-live/public_html/.htaccess.
  • SEO assets:
    • dewemoji-live/public_html/includes/head.php
    • dewemoji-live/public_html/sitemap.xml.php
    • dewemoji-live/public_html/robots.txt
  • Emoji page implementation reference:
    • dewemoji-live/public_html/emoji.php
    • includes microcache + structured data + policy filtering.
  • Pricing + payment references:
    • dewemoji-live/public_html/pricing.php
    • dewemoji-live/public_html/support.php
    • dewemoji-live/public_html/privacy.php
    • dewemoji-live/public_html/terms.php
  • Blurb data + jobs:
    • dewemoji-live/public_html/data/emoji_descriptions.json
    • dewemoji-live/jobs/sync_blurbs.php
    • dewemoji-live/jobs/seed_blurbs_from_dataset.php

Live backend audit highlights (dewemoji-live-backend)

  • Backend architecture is a custom PHP router (public_html/public/index.php) with controller-per-endpoint files and shared helpers.
  • Main live API surface discovered:
    • /v1/emojis, /v1/categories
    • /v1/license/verify, /v1/license/activate, /v1/license/deactivate
    • /v1/contrib/* community endpoints (suggest/list/vote/auth/make-public/search)
    • /v1/keywords/pending
    • /v1/health, /v1/metrics, /v1/metrics-lite
  • Data mode is hybrid:
    • API reads from JSON dataset (public_html/app/data/emojis.json) for emoji search.
    • Licensing, usage logs, and community data read/write in MySQL.
    • Redis/APCu/in-memory used for runtime rate-limiting fallback chain.
  • Pro logic currently exists in live backend:
    • plan resolution from license key + first-party whitelist headers/origin.
    • pro/free limits for API and contribution quotas.
    • device activation model with max active devices per license/product.
  • Community feature maturity:
    • keyword contribution flow exists with private/public states.
    • voting and auto-approval/rejection thresholds implemented.
    • Turnstile + AI moderation + rate limiting integrated.
    • still has fragile areas that should be normalized in Laravel service layer.
  • Database model inferred from code (must be migrated with proper Laravel migrations):
    • emojis, emoji_aliases, emoji_shortcodes, emoji_usage_examples, emoji_related, emoji_intent_tags, emoji_search_tokens
    • emoji_keywords, keyword_votes, moderation_events
    • users
    • licenses, license_activations, usage_logs
    • ai_guard_logs, ai_provider_usage, ai_lang_cache

Security and configuration migration requirements

  • Current live backend keeps many secrets directly in public_html/config/env.php (DB, Redis, Turnstile, payment providers, OpenRouter).
  • Rebuild must move all secrets to .env, rotate exposed credentials, and remove committed secret values from repo history.
  • Metrics endpoints currently appear open by default; rebuild should protect admin/internal endpoints with auth or network policy.
  • Add internal observability baseline in rebuild:
    • structured request logging
    • protected metrics endpoint(s)
    • deploy healthcheck endpoint

Important note on Gumroad API tracing

  • In dewemoji-live/public_html/helpers/auth.php, Gumroad/Mayar validation is currently a stub (return false), so live verification logic is not fully present in this folder.
  • There is legacy local activation SQL flow in dewemoji-live/public_html/db.php (activate/deactivate/verify + device cap), which should be used only as behavioral reference for rebuild design.