Files
dewedev/public/robots.txt
dwindown 2e67a2bca2 feat: comprehensive SEO optimization and GDPR compliance
- Added Terms of Service and Privacy Policy pages with contact info
- Implemented Google Analytics with Consent Mode v2 for GDPR compliance
- Created sitemap.xml and robots.txt for search engine optimization
- Added dynamic meta tags, Open Graph, and structured data (JSON-LD)
- Implemented GDPR consent banner with TCF 2.2 compatibility
- Enhanced sidebar with category-colored hover states and proper active/inactive styling
- Fixed all ESLint warnings for clean deployment
- Added comprehensive SEO utilities and privacy-first analytics tracking

Ready for production deployment with full legal compliance and SEO optimization.
2025-09-24 00:12:28 +07:00

30 lines
446 B
Plaintext

# Robots.txt for https://dewe.dev
# Generated automatically
User-agent: *
Allow: /
# Sitemap location
Sitemap: https://dewe.dev/sitemap.xml
# Block any future admin or private routes
Disallow: /admin/
Disallow: /api/
Disallow: /.well-known/
# Allow all major search engines
User-agent: Googlebot
Allow: /
User-agent: Bingbot
Allow: /
User-agent: Slurp
Allow: /
User-agent: DuckDuckBot
Allow: /
# Crawl delay for politeness
Crawl-delay: 1