Files
rmtPocketWatcher/PRD.md
2025-12-03 18:00:10 -05:00

381 lines
6.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Product Requirements Document (PRD)
Star Citizen AUEC Price Index Tracker
Unified Modern TypeScript Stack + Electron Desktop Application
1. Product Overview
This product is a cross-platform desktop application (Electron + React/TSX) that displays real-money trading (RMT) prices for Star Citizen AUEC. A modern TypeScript-based backend performs scraping, normalization, time-series storage, and API hosting.
The goal is to generate a reliable AUEC Price Index, similar in spirit to a financial market index, and visualize it in a Bloomberg-style terminal UI that includes tables, charts, and alerts.
Alerts are evaluated client-side, based on data received from the backend.
2. Goals & Non-Goals
Goals
Scrape AUEC prices from:
Eldorado
PlayerAuctions
Normalize them into a USD ↔ AUEC Price Index.
Provide REST + WebSocket API to clients.
Display Bloomberg-like UI using Electron + React.
Allow users to set local alerts (no backend evaluation).
Store long-term historical price data.
Non-Goals
Buying/selling AUEC inside the app.
Mobile or web app (desktop only).
Legal/regulatory RMT analysis.
Server-side alert evaluation.
3. System Architecture
[Scraper Services - TypeScript + Playwright]
|
v
[Backend API - Fastify/NestJS - TypeScript]
- Normalize prices
- Store historical data (PostgreSQL + TimescaleDB)
- Expose REST + WebSocket
|
v
[Electron Desktop App - React/TSX UI]
- Bloomberg-style terminal interface
- Client-side alerts
- Charting & vendor tables
4. Detailed Requirements
4.1 Scraping Module (Backend)
Technology
TypeScript
Playwright for dynamic scraping
Node Scheduler or BullMQ for job scheduling
Deployed as containers (Docker)
Scraping Frequency
Every 5 minutes (configurable per environment)
Scraping Targets
Eldorado AUEC Market
https://www.eldorado.gg/star-citizen-auec/g/141-0-0
PlayerAuctions AUEC Market
https://www.playerauctions.com/star-citizen-auec/
Scraped Data Fields
Vendor name
Listing price (USD)
AUEC amount per listing
Derived:
USD per 1 million AUEC
AUEC per USD
Timestamp
Listing min/max spread (if available)
Scraper Requirements
Headless mode with Playwright Chromium
Randomized UA + viewport
Retry logic (3 attempts)
Automatically detect layout changes (optional heuristics)
Error Handling
Failures logged to DB
Continue schedule regardless of scrape results
4.2 Backend Server (TypeScript)
Technology Stack
Node.js + TypeScript
Fastify or NestJS (devs choice)
WebSockets via native ws or Fastify WS plugin
PostgreSQL + TimescaleDB for time-series storage
Prisma or Drizzle ORM
Backend Responsibilities
Accept scraped data
Compute Price Index
Store time-series data
Serve historical & latest prices
Push WS updates to connected Electron clients
Price Index Formula
Default:
lowest_price = min(USD_per_million_AUEC_from_all_vendors)
All individual listings are stored with seller and platform information for historical tracking and comparison.
REST API Endpoints
GET /prices/latest - Get all current listings with seller info
GET /prices/lowest - Get current lowest price across all vendors
GET /prices/by-seller?seller=&platform= - Get specific seller's price history
GET /prices/by-platform?platform= - Get all listings from a platform
GET /prices/history?from=&to=&seller=&platform= - Historical data with filters
GET /index/history?range=7d|30d|90d|all - Lowest price over time
WebSocket Channels
/ws/index → broadcast each new scrape
/ws/vendors → vendor-level updates
Performance Requirements
Handle 5000+ concurrent WS connections
API latency < 50ms for standard queries
4.3 Database Layer
Database
PostgreSQL (core DB)
TimescaleDB extension for time-series compression & fast queries
Schema
raw_vendor_prices
field type
id uuid
timestamp timestamptz
vendor text (eldorado/playerauctions)
seller_name text
usd_price numeric
auec_amount numeric
usd_per_million numeric
delivery_time text
url text
price_index
field type
timestamp timestamptz
lowest_price numeric
vendor text
seller_name text
scrape_log
field type
timestamp timestamptz
status text (success/failure)
message text
runtime_ms int
4.4 Electron Desktop App (React/TSX)
Technology Stack
Electron 30+
React + TypeScript/TSX
Vite build tooling
TailwindCSS for styling
Recharts, ECharts, or TradingView Charting Library for visuals
Zustand/Recoil for state management
Main Features
Dashboard (Bloomberg-style)
Large real-time Price Index number
Color-coded trend (% change)
Mini sparkline
Vendor Comparison Table
Columns:
Platform (Eldorado/PlayerAuctions)
Seller Name
USD per 1M AUEC
Delivery Time
Last updated
Sortable by price, seller, platform
Filter by platform
Auto-refresh from WebSocket events
Historical Chart
Select by:
Lowest price across all vendors (default)
Specific platform (Eldorado/PlayerAuctions)
Specific seller
Chart types:
Line (price over time)
Area
Multi-line comparison (compare multiple sellers)
Time ranges:
1h, 6h, 24h, 7d, 30d, All
Show all seller prices as separate lines for comparison
Client-Side Alerts
User selects:
Condition:
Price rises above X
Price falls below X
Threshold value
Stored locally (JSON or SQLite)
Trigger rules:
On each WS update, re-evaluate alerts
Trigger:
Windows/macOS/Linux native notification
Sound (toggle in settings)
UI popup/log entry
Settings Panel
Dark/light mode
Alert sounds on/off
Launch on startup
Chart theme
Electron-Specific Requirements
Auto-update (e.g., using electron-updater)
Secure IPC messaging (no remote code eval)
Sandboxed renderer process
Signed binaries (Windows .exe, macOS .dmg/.pkg)
5. Non-Functional Requirements
Security
HTTPS for all API calls
No sensitive user data stored
Sanitized DB inputs
CSRF not needed (desktop app only)
Signed Electron builds
Scalability
Scrapers run in Docker for horizontal scaling
Backend stateless easy to scale behind a load balancer
TimescaleDB handles time-series at scale
Reliability
WebSocket auto-reconnect
Graceful handling if server is offline
Retry logic in scraping pipeline
Performance
Desktop app must remain responsive when charts have 50k+ datapoints
(handled with TimescaleDB downsampling + chart library performance)
Testing
Unit tests: Jest (backend & Electron renderer)
E2E tests: Playwright for UI
Integration tests for API via Supertest or Pact
6. Risks & Mitigation
Risk Mitigation
Vendor site layout changes break scrapers Monitoring + abstraction layers + fallback parsing
Browser automation instability Playwright retries + container isolation
WS disconnects Auto-reconnect logic in Electron
Heavy Electron UI Use Vite + lazy-loaded components + chart virtualization
TimescaleDB complexity Can be optional if scale is small
7. Future Enhancements
Add more RMT vendors
Discord alerts via webhook
Predictive modeling (ML)
Add terminal-like command bar (e.g., /chart index 30d)
Desktop widgets and system tray price ticker