Files
Train-ID/README.md
Hudson Riggs 338bdf838b
All checks were successful
Deploy to Server / deploy (push) Successful in 47s
Working Ebay Scraping
2025-11-03 16:42:52 -05:00

5.2 KiB

Train-ID

Simple Node/Express app to identify and inventory model trains from photos using OpenAI Vision, store results in MySQL, and export PDFs/XLSX.

Requirements

  • Node.js 20+ and npm
  • MySQL 8+
  • OpenAI API key

Environment variables

Create a .env in the project root on the server:

PORT=3000
# OpenAI
OPENAI_API_KEY=sk-...
# or alternative variable name also supported by code:
# openapi_key=sk-...

# MySQL connection
db_ip=127.0.0.1
db_port=3306
db_user=trainid
db_pass=changeme
db_name=trainid

# eBay API (optional - falls back to scraping if not configured)
# Get credentials from https://developer.ebay.com/my/keys
# Create an app and use the Client ID and Client Secret
EBAY_CLIENT_ID=YourAppId...
EBAY_CLIENT_SECRET=YourClientSecret...
EBAY_SANDBOX=false  # Set to 'true' to use eBay sandbox environment

Local development

npm install
npm run build
npm start
# or for auto-reload on TypeScript build changes:
npm run dev

Visit http://localhost:3000.

API overview

  • POST /api/upload with multipart/form-data field image → analyzes photo, inserts record
  • GET /api/items → list items (optional ?q= to search model/SKU)
  • GET /api/items/:id/pdf → generate/download PDF for one item
  • GET /api/export/xlsx → download XLSX export of inventory with embedded thumbnails
  • DELETE /api/items/:id → delete one
  • DELETE /api/items → wipe all
  • GET /api/debug/ebay-prices?sku=... → debug eBay price lookup (shows API and scraping attempts)
  • POST /api/prices/update → update cached prices for all SKUs from eBay
  • GET /api/price-report → get price report with item values

eBay Price Checking

The system uses eBay's Browse API to search for items by SKU. Important limitations:

  • Browse API only searches active listings, not sold/completed ones
  • For sold/completed listings, the system falls back to web scraping
  • In sandbox mode, test data is limited - you may see 0 results even if the API is working correctly
  • The API method: GET /buy/browse/v1/item_summary/search with query parameter q (SKU)

Debian 13 (Trixie) LXC install

These steps assume a fresh Debian 13 LXC and deployment directory /opt/Train-ID with a system user deployuser that has passwordless sudo for service management.

  1. Base packages and Node.js
sudo apt update
sudo apt install -y curl ca-certificates gnupg build-essential pkg-config
# Node 20 using Nodesource or Debian repo (choose one). Nodesource example:
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs
node -v && npm -v
  1. MySQL server (or connect to external MySQL)
sudo apt install -y mariadb-server mariadb-client
sudo systemctl enable --now mariadb
sudo mysql -e "CREATE DATABASE IF NOT EXISTS trainid CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
sudo mysql -e "CREATE USER IF NOT EXISTS 'trainid'@'%' IDENTIFIED BY 'changeme';"
sudo mysql -e "GRANT ALL PRIVILEGES ON trainid.* TO 'trainid'@'%'; FLUSH PRIVILEGES;"

If using external MySQL, skip install and set db_ip etc. in .env.

  1. App checkout and build
sudo mkdir -p /opt/Train-ID
sudo chown $USER:$USER /opt/Train-ID
git clone https://git.hudsonriggs.systems/HRiggs/Train-ID.git /opt/Train-ID
cd /opt/Train-ID
cp .env.example .env || true  # if you keep a template in the future
# create .env as per above
npm ci || npm install
npm run build
  1. Systemd service Create /etc/systemd/system/train-id.service:
[Unit]
Description=Train-ID API
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/Train-ID
EnvironmentFile=/opt/Train-ID/.env
ExecStart=/usr/bin/node /opt/Train-ID/dist/server.js
Restart=always
RestartSec=5
User=www-data
Group=www-data

[Install]
WantedBy=multi-user.target

Then enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now train-id
sudo systemctl status train-id --no-pager
  1. Reverse proxy (optional, for port 80/443) Example Nginx site (/etc/nginx/sites-available/train-id):
server {
  listen 80;
  server_name _;
  location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

Enable and reload:

sudo apt install -y nginx
sudo ln -s /etc/nginx/sites-available/train-id /etc/nginx/sites-enabled/train-id
sudo nginx -t && sudo systemctl reload nginx

CI/CD deploy

The repo includes .gitea/workflows/deploy.yaml which, on push to main, SSHes to 192.168.30.114, pulls latest, installs dependencies, builds, and restarts the train-id service. Set DEPLOY_KEY in repository secrets to a private SSH key authorized for deployuser@192.168.30.114.

If your service name or directory differ, update SERVICE/APP_DIR in the workflow accordingly.

apt update
apt install -y sudo

SYSCTL_PATH=$(command -v systemctl)  # usually /bin/systemctl or /usr/bin/systemctl
cat >/etc/sudoers.d/train-id <<EOF
deployuser ALL=(root) NOPASSWD: ${SYSCTL_PATH} restart train-id, ${SYSCTL_PATH} restart train-id.service
EOF
chmod 440 /etc/sudoers.d/train-id
visudo -cf /etc/sudoers.d/train-id

sudo -n systemctl restart train-id