Intial Version
This commit is contained in:
381
PRD.md
Normal file
381
PRD.md
Normal file
@@ -0,0 +1,381 @@
|
||||
Product Requirements Document (PRD)
|
||||
Star Citizen AUEC Price Index Tracker
|
||||
|
||||
Unified Modern TypeScript Stack + Electron Desktop Application
|
||||
|
||||
1. Product Overview
|
||||
|
||||
This product is a cross-platform desktop application (Electron + React/TSX) that displays real-money trading (RMT) prices for Star Citizen AUEC. A modern TypeScript-based backend performs scraping, normalization, time-series storage, and API hosting.
|
||||
|
||||
The goal is to generate a reliable AUEC Price Index, similar in spirit to a financial market index, and visualize it in a Bloomberg-style terminal UI that includes tables, charts, and alerts.
|
||||
|
||||
Alerts are evaluated client-side, based on data received from the backend.
|
||||
|
||||
2. Goals & Non-Goals
|
||||
Goals
|
||||
|
||||
Scrape AUEC prices from:
|
||||
|
||||
Eldorado
|
||||
|
||||
PlayerAuctions
|
||||
|
||||
Normalize them into a USD ↔ AUEC Price Index.
|
||||
|
||||
Provide REST + WebSocket API to clients.
|
||||
|
||||
Display Bloomberg-like UI using Electron + React.
|
||||
|
||||
Allow users to set local alerts (no backend evaluation).
|
||||
|
||||
Store long-term historical price data.
|
||||
|
||||
Non-Goals
|
||||
|
||||
Buying/selling AUEC inside the app.
|
||||
|
||||
Mobile or web app (desktop only).
|
||||
|
||||
Legal/regulatory RMT analysis.
|
||||
|
||||
Server-side alert evaluation.
|
||||
|
||||
3. System Architecture
|
||||
[Scraper Services - TypeScript + Playwright]
|
||||
|
|
||||
v
|
||||
[Backend API - Fastify/NestJS - TypeScript]
|
||||
- Normalize prices
|
||||
- Store historical data (PostgreSQL + TimescaleDB)
|
||||
- Expose REST + WebSocket
|
||||
|
|
||||
v
|
||||
[Electron Desktop App - React/TSX UI]
|
||||
- Bloomberg-style terminal interface
|
||||
- Client-side alerts
|
||||
- Charting & vendor tables
|
||||
|
||||
4. Detailed Requirements
|
||||
4.1 Scraping Module (Backend)
|
||||
Technology
|
||||
|
||||
TypeScript
|
||||
|
||||
Playwright for dynamic scraping
|
||||
|
||||
Node Scheduler or BullMQ for job scheduling
|
||||
|
||||
Deployed as containers (Docker)
|
||||
|
||||
Scraping Frequency
|
||||
|
||||
Every 5 minutes (configurable per environment)
|
||||
|
||||
Scraping Targets
|
||||
|
||||
Eldorado AUEC Market
|
||||
https://www.eldorado.gg/star-citizen-auec/g/141-0-0
|
||||
|
||||
PlayerAuctions AUEC Market
|
||||
https://www.playerauctions.com/star-citizen-auec/
|
||||
|
||||
Scraped Data Fields
|
||||
|
||||
Vendor name
|
||||
|
||||
Listing price (USD)
|
||||
|
||||
AUEC amount per listing
|
||||
|
||||
Derived:
|
||||
|
||||
USD per 1 million AUEC
|
||||
|
||||
AUEC per USD
|
||||
|
||||
Timestamp
|
||||
|
||||
Listing min/max spread (if available)
|
||||
|
||||
Scraper Requirements
|
||||
|
||||
Headless mode with Playwright Chromium
|
||||
|
||||
Randomized UA + viewport
|
||||
|
||||
Retry logic (3 attempts)
|
||||
|
||||
Automatically detect layout changes (optional heuristics)
|
||||
|
||||
Error Handling
|
||||
|
||||
Failures logged to DB
|
||||
|
||||
Continue schedule regardless of scrape results
|
||||
|
||||
4.2 Backend Server (TypeScript)
|
||||
Technology Stack
|
||||
|
||||
Node.js + TypeScript
|
||||
|
||||
Fastify or NestJS (dev’s choice)
|
||||
|
||||
WebSockets via native ws or Fastify WS plugin
|
||||
|
||||
PostgreSQL + TimescaleDB for time-series storage
|
||||
|
||||
Prisma or Drizzle ORM
|
||||
|
||||
Backend Responsibilities
|
||||
|
||||
Accept scraped data
|
||||
|
||||
Compute Price Index
|
||||
|
||||
Store time-series data
|
||||
|
||||
Serve historical & latest prices
|
||||
|
||||
Push WS updates to connected Electron clients
|
||||
|
||||
Price Index Formula
|
||||
|
||||
Default:
|
||||
|
||||
lowest_price = min(USD_per_million_AUEC_from_all_vendors)
|
||||
|
||||
|
||||
All individual listings are stored with seller and platform information for historical tracking and comparison.
|
||||
|
||||
REST API Endpoints
|
||||
|
||||
GET /prices/latest - Get all current listings with seller info
|
||||
|
||||
GET /prices/lowest - Get current lowest price across all vendors
|
||||
|
||||
GET /prices/by-seller?seller=&platform= - Get specific seller's price history
|
||||
|
||||
GET /prices/by-platform?platform= - Get all listings from a platform
|
||||
|
||||
GET /prices/history?from=&to=&seller=&platform= - Historical data with filters
|
||||
|
||||
GET /index/history?range=7d|30d|90d|all - Lowest price over time
|
||||
|
||||
WebSocket Channels
|
||||
|
||||
/ws/index → broadcast each new scrape
|
||||
|
||||
/ws/vendors → vendor-level updates
|
||||
|
||||
Performance Requirements
|
||||
|
||||
Handle 5000+ concurrent WS connections
|
||||
|
||||
API latency < 50ms for standard queries
|
||||
|
||||
4.3 Database Layer
|
||||
Database
|
||||
|
||||
PostgreSQL (core DB)
|
||||
|
||||
TimescaleDB extension for time-series compression & fast queries
|
||||
|
||||
Schema
|
||||
raw_vendor_prices
|
||||
field type
|
||||
id uuid
|
||||
timestamp timestamptz
|
||||
vendor text (eldorado/playerauctions)
|
||||
seller_name text
|
||||
usd_price numeric
|
||||
auec_amount numeric
|
||||
usd_per_million numeric
|
||||
delivery_time text
|
||||
url text
|
||||
price_index
|
||||
field type
|
||||
timestamp timestamptz
|
||||
lowest_price numeric
|
||||
vendor text
|
||||
seller_name text
|
||||
scrape_log
|
||||
field type
|
||||
timestamp timestamptz
|
||||
status text (success/failure)
|
||||
message text
|
||||
runtime_ms int
|
||||
|
||||
4.4 Electron Desktop App (React/TSX)
|
||||
Technology Stack
|
||||
|
||||
Electron 30+
|
||||
|
||||
React + TypeScript/TSX
|
||||
|
||||
Vite build tooling
|
||||
|
||||
TailwindCSS for styling
|
||||
|
||||
Recharts, ECharts, or TradingView Charting Library for visuals
|
||||
|
||||
Zustand/Recoil for state management
|
||||
|
||||
Main Features
|
||||
Dashboard (Bloomberg-style)
|
||||
|
||||
Large real-time Price Index number
|
||||
|
||||
Color-coded trend (% change)
|
||||
|
||||
Mini sparkline
|
||||
|
||||
Vendor Comparison Table
|
||||
|
||||
Columns:
|
||||
|
||||
Platform (Eldorado/PlayerAuctions)
|
||||
|
||||
Seller Name
|
||||
|
||||
USD per 1M AUEC
|
||||
|
||||
Delivery Time
|
||||
|
||||
Last updated
|
||||
|
||||
Sortable by price, seller, platform
|
||||
|
||||
Filter by platform
|
||||
|
||||
Auto-refresh from WebSocket events
|
||||
|
||||
Historical Chart
|
||||
|
||||
Select by:
|
||||
|
||||
Lowest price across all vendors (default)
|
||||
|
||||
Specific platform (Eldorado/PlayerAuctions)
|
||||
|
||||
Specific seller
|
||||
|
||||
Chart types:
|
||||
|
||||
Line (price over time)
|
||||
|
||||
Area
|
||||
|
||||
Multi-line comparison (compare multiple sellers)
|
||||
|
||||
Time ranges:
|
||||
|
||||
1h, 6h, 24h, 7d, 30d, All
|
||||
|
||||
Show all seller prices as separate lines for comparison
|
||||
|
||||
Client-Side Alerts
|
||||
|
||||
User selects:
|
||||
|
||||
Condition:
|
||||
|
||||
Price rises above X
|
||||
|
||||
Price falls below X
|
||||
|
||||
Threshold value
|
||||
|
||||
Stored locally (JSON or SQLite)
|
||||
|
||||
Trigger rules:
|
||||
|
||||
On each WS update, re-evaluate alerts
|
||||
|
||||
Trigger:
|
||||
|
||||
Windows/macOS/Linux native notification
|
||||
|
||||
Sound (toggle in settings)
|
||||
|
||||
UI popup/log entry
|
||||
|
||||
Settings Panel
|
||||
|
||||
Dark/light mode
|
||||
|
||||
Alert sounds on/off
|
||||
|
||||
Launch on startup
|
||||
|
||||
Chart theme
|
||||
|
||||
Electron-Specific Requirements
|
||||
|
||||
Auto-update (e.g., using electron-updater)
|
||||
|
||||
Secure IPC messaging (no remote code eval)
|
||||
|
||||
Sandboxed renderer process
|
||||
|
||||
Signed binaries (Windows .exe, macOS .dmg/.pkg)
|
||||
|
||||
5. Non-Functional Requirements
|
||||
Security
|
||||
|
||||
HTTPS for all API calls
|
||||
|
||||
No sensitive user data stored
|
||||
|
||||
Sanitized DB inputs
|
||||
|
||||
CSRF not needed (desktop app only)
|
||||
|
||||
Signed Electron builds
|
||||
|
||||
Scalability
|
||||
|
||||
Scrapers run in Docker for horizontal scaling
|
||||
|
||||
Backend stateless → easy to scale behind a load balancer
|
||||
|
||||
TimescaleDB handles time-series at scale
|
||||
|
||||
Reliability
|
||||
|
||||
WebSocket auto-reconnect
|
||||
|
||||
Graceful handling if server is offline
|
||||
|
||||
Retry logic in scraping pipeline
|
||||
|
||||
Performance
|
||||
|
||||
Desktop app must remain responsive when charts have 50k+ datapoints
|
||||
(handled with TimescaleDB downsampling + chart library performance)
|
||||
|
||||
Testing
|
||||
|
||||
Unit tests: Jest (backend & Electron renderer)
|
||||
|
||||
E2E tests: Playwright for UI
|
||||
|
||||
Integration tests for API via Supertest or Pact
|
||||
|
||||
6. Risks & Mitigation
|
||||
Risk Mitigation
|
||||
Vendor site layout changes break scrapers Monitoring + abstraction layers + fallback parsing
|
||||
Browser automation instability Playwright retries + container isolation
|
||||
WS disconnects Auto-reconnect logic in Electron
|
||||
Heavy Electron UI Use Vite + lazy-loaded components + chart virtualization
|
||||
TimescaleDB complexity Can be optional if scale is small
|
||||
7. Future Enhancements
|
||||
|
||||
Add more RMT vendors
|
||||
|
||||
Discord alerts via webhook
|
||||
|
||||
Predictive modeling (ML)
|
||||
|
||||
Add terminal-like command bar (e.g., /chart index 30d)
|
||||
|
||||
Desktop widgets and system tray “price ticker”
|
||||
Reference in New Issue
Block a user