rmtPocketWatcher/backend

rmtPocketWatcher Backend

Backend scraping and API service for rmtPocketWatcher.

Features

• Playwright-based web scraping for Eldorado and PlayerAuctions
• Automatic retry logic (3 attempts with exponential backoff)
• Scheduled scraping every 5 minutes (configurable)
• PostgreSQL + Prisma for data storage
• REST API + WebSocket for real-time updates
• Tracks all seller listings over time

Setup

# Install dependencies
npm install

# Install Playwright browsers
npx playwright install chromium

# Setup environment
cp .env.example .env
# Edit .env with your PostgreSQL connection string

# Generate Prisma client
npm run db:generate

# Run database migrations
npm run db:migrate

Quick Start with Docker Compose

The easiest way to run the entire stack:

# From the project root
docker-compose up -d

# View logs
docker-compose logs -f backend

# Stop everything
docker-compose down

This will:

• Start PostgreSQL database
• Run database migrations
• Start the backend API server
• Begin scraping every 5 minutes

Local Development Setup

If you want to run locally without Docker:

# Start PostgreSQL with Docker
docker-compose up -d postgres

# Install dependencies
cd backend
npm install

# Setup environment
cp .env.example .env
# Edit .env if needed

# Generate Prisma client and push schema
npm run db:generate
npm run db:push

# Install Playwright browsers
npx playwright install chromium

Usage

# Development mode with hot reload
npm run dev

# Manual scrape test (no database)
npm run scrape

# Build for production
npm run build

# Run production build
npm start

# Database management
npm run db:studio    # Open Prisma Studio
npm run db:push      # Push schema changes without migration

API Endpoints

Prices

• GET /api/prices/latest - Get all current listings
• GET /api/prices/lowest - Get current lowest price
• GET /api/prices/by-seller?seller=&platform= - Get seller's price history
• GET /api/prices/by-platform?platform= - Get all listings from a platform
• GET /api/prices/history?from=&to=&seller=&platform= - Historical data

Index

• GET /api/index/history?range=7d|30d|90d|all - Lowest price over time
• GET /api/index/ws - WebSocket for real-time updates

Health

• GET /health - Server health check

Architecture

• src/scrapers/ - Playwright scraping modules
• src/api/ - Fastify REST + WebSocket API
• src/database/ - Prisma client and repository
• prisma/ - Database schema and migrations