Intial Version
This commit is contained in:
120
backend/README.md
Normal file
120
backend/README.md
Normal file
@@ -0,0 +1,120 @@
|
||||
# rmtPocketWatcher Backend
|
||||
|
||||
Backend scraping and API service for rmtPocketWatcher.
|
||||
|
||||
## Features
|
||||
|
||||
- Playwright-based web scraping for Eldorado and PlayerAuctions
|
||||
- Automatic retry logic (3 attempts with exponential backoff)
|
||||
- Scheduled scraping every 5 minutes (configurable)
|
||||
- PostgreSQL + Prisma for data storage
|
||||
- REST API + WebSocket for real-time updates
|
||||
- Tracks all seller listings over time
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Install Playwright browsers
|
||||
npx playwright install chromium
|
||||
|
||||
# Setup environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your PostgreSQL connection string
|
||||
|
||||
# Generate Prisma client
|
||||
npm run db:generate
|
||||
|
||||
# Run database migrations
|
||||
npm run db:migrate
|
||||
```
|
||||
|
||||
## Quick Start with Docker Compose
|
||||
|
||||
The easiest way to run the entire stack:
|
||||
|
||||
```bash
|
||||
# From the project root
|
||||
docker-compose up -d
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f backend
|
||||
|
||||
# Stop everything
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
This will:
|
||||
- Start PostgreSQL database
|
||||
- Run database migrations
|
||||
- Start the backend API server
|
||||
- Begin scraping every 5 minutes
|
||||
|
||||
## Local Development Setup
|
||||
|
||||
If you want to run locally without Docker:
|
||||
|
||||
```bash
|
||||
# Start PostgreSQL with Docker
|
||||
docker-compose up -d postgres
|
||||
|
||||
# Install dependencies
|
||||
cd backend
|
||||
npm install
|
||||
|
||||
# Setup environment
|
||||
cp .env.example .env
|
||||
# Edit .env if needed
|
||||
|
||||
# Generate Prisma client and push schema
|
||||
npm run db:generate
|
||||
npm run db:push
|
||||
|
||||
# Install Playwright browsers
|
||||
npx playwright install chromium
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Development mode with hot reload
|
||||
npm run dev
|
||||
|
||||
# Manual scrape test (no database)
|
||||
npm run scrape
|
||||
|
||||
# Build for production
|
||||
npm run build
|
||||
|
||||
# Run production build
|
||||
npm start
|
||||
|
||||
# Database management
|
||||
npm run db:studio # Open Prisma Studio
|
||||
npm run db:push # Push schema changes without migration
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Prices
|
||||
- `GET /api/prices/latest` - Get all current listings
|
||||
- `GET /api/prices/lowest` - Get current lowest price
|
||||
- `GET /api/prices/by-seller?seller=&platform=` - Get seller's price history
|
||||
- `GET /api/prices/by-platform?platform=` - Get all listings from a platform
|
||||
- `GET /api/prices/history?from=&to=&seller=&platform=` - Historical data
|
||||
|
||||
### Index
|
||||
- `GET /api/index/history?range=7d|30d|90d|all` - Lowest price over time
|
||||
- `GET /api/index/ws` - WebSocket for real-time updates
|
||||
|
||||
### Health
|
||||
- `GET /health` - Server health check
|
||||
|
||||
## Architecture
|
||||
|
||||
- `src/scrapers/` - Playwright scraping modules
|
||||
- `src/api/` - Fastify REST + WebSocket API
|
||||
- `src/database/` - Prisma client and repository
|
||||
- `prisma/` - Database schema and migrations
|
||||
Reference in New Issue
Block a user