import 'dotenv/config'; import server from './api/server'; import { ScraperScheduler } from './scrapers/scheduler'; import { PriceRepository } from './database/repository'; import { broadcastPriceUpdate } from './api/routes/websocket'; const PORT = parseInt(process.env.PORT || '3000', 10); const HOST = process.env.HOST || '0.0.0.0'; const SCRAPE_INTERVAL = parseInt(process.env.SCRAPE_INTERVAL_MINUTES || '5', 10); const repository = new PriceRepository(); const scheduler = new ScraperScheduler(SCRAPE_INTERVAL); // Handle scrape results scheduler.onScrapeComplete(async (results) => { const startTime = Date.now(); try { // Save all listings const allListings = results.flatMap(r => r.listings); if (allListings.length > 0) { await repository.saveListings(allListings); // Find and save lowest price const lowestListing = allListings.reduce((min, listing) => listing.pricePerMillion < min.pricePerMillion ? listing : min ); await repository.savePriceIndex( lowestListing.pricePerMillion, lowestListing.vendor, lowestListing.seller ); // Broadcast update to WebSocket clients const latestPrices = await repository.getLatestPrices(); broadcastPriceUpdate({ timestamp: new Date(), lowestPrice: lowestListing.pricePerMillion, platform: lowestListing.vendor, sellerName: lowestListing.seller, allPrices: latestPrices.map((p: any) => ({ id: p.id, platform: p.vendor, sellerName: p.sellerName, pricePerMillion: Number(p.usdPerMillion), timestamp: p.timestamp, url: p.url })) }); console.log(`✓ Broadcasted price update to ${latestPrices.length} listings`); } // Log successful scrape const runtimeMs = Date.now() - startTime; await repository.logScrape('success', `Saved ${allListings.length} listings`, runtimeMs); console.log(`✓ Saved ${allListings.length} listings to database`); } catch (error) { const runtimeMs = Date.now() - startTime; const message = error instanceof Error ? error.message : 'Unknown error'; await repository.logScrape('failure', message, runtimeMs); console.error('✗ Failed to save listings:', message); } }); // Start server const start = async () => { try { await server.listen({ port: PORT, host: HOST }); console.log(`Server listening on ${HOST}:${PORT}`); // Start scraper scheduler.start(); console.log(`Scraper started (interval: ${SCRAPE_INTERVAL} minutes)`); } catch (err) { server.log.error(err); process.exit(1); } }; // Graceful shutdown process.on('SIGINT', async () => { console.log('\nShutting down gracefully...'); await scheduler.close(); await server.close(); process.exit(0); }); start();