Documentation

PQAP system documentation and architecture

PQAP Deployment Guide

Table of Contents

  1. Development Workflow
  2. Deployment Architecture
  3. Staging Environment
  4. Syncing to EC2
  5. Restarting Services
  6. Rollback Procedures
  7. Configuration Management
  8. Monitoring and Logs
  9. Backup and Restore
  10. Troubleshooting

Development Workflow

Overview

PQAP uses a local-first development workflow:

┌─────────────────┐      sync       ┌─────────────────┐
│   Local Dev     │ ──────────────▶ │   EC2 (Prod)    │
│   (MacBook)     │                 │  54.85.161.228  │
└─────────────────┘                 └─────────────────┘
   Development                        Running System
   Testing                            Trading
   Code changes                       Data storage

Key Principle: EC2 is the single source of truth for: - Running PQAP 24/7 - Production data (databases, state files) - Live trading execution

Local is for: - Writing code - Testing changes - Development experimentation

Typical Workflow

  1. Make changes locally ```bash # Edit code in src/ vim src/strategies/my_strategy/strategy.py

# Test locally python -m src.main configs/dev.yaml ```

  1. Sync to EC2 bash python scripts/pqap_manager.py sync

  2. Restart EC2 bash python scripts/pqap_manager.py stop ec2 python scripts/pqap_manager.py start ec2

  3. Monitor bash python scripts/pqap_manager.py logs ec2

Deployment Architecture

Environment Separation

Environment Location Purpose Config File
Development Local MacBook Code changes, testing configs/dev.yaml
Staging Local or EC2 Pre-production validation configs/staging.yaml
Production EC2 (54.85.161.228) Live trading, 24/7 operation configs/prod.yaml

What Gets Synced

Synced (code and config): - src/ - All Python source code - configs/ - Configuration files - scripts/ - Utility scripts - docs/ - Documentation - requirements.txt - Dependencies

NOT synced (data and environment): - data/*.db - Databases (EC2 has its own) - .env - Credentials (different per environment) - venv/ - Virtual environment - *.log - Log files - models/*.pkl - ML models (sync manually if needed) - data/paper_trading_state.json - Paper trading state

EC2 Data (Production Data)

These files live ONLY on EC2 and are backed up to S3:

File Purpose Backup Schedule
data/market_history.db Historical price snapshots Daily 2am UTC
data/pqap_dev.db Trades, positions, signals Daily 2am UTC
data/paper_trading_state.json Current portfolio state Daily 2am UTC
models/price_prediction_v1.pkl Trained ML model Weekly

Staging Environment

Overview

The staging environment is a pre-production testing environment that bridges the gap between development and production. It provides a safe, realistic testing ground for validating changes before they go live.

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Development   │ ──▶ │     Staging     │ ──▶ │   Production    │
│   (dev.yaml)    │     │  (staging.yaml) │     │   (prod.yaml)   │
└─────────────────┘     └─────────────────┘     └─────────────────┘
   $100 capital            $1000 capital           $5000 capital
   Loose limits            Prod-like limits        Prod limits
   3 strategies            5 strategies            3 strategies
   Experimentation         Validation              Live trading

When to Use Staging

Use staging when:

  1. Testing new strategies - Validate strategy behavior with realistic capital/limits before prod
  2. Testing configuration changes - Verify risk limit changes work as expected
  3. Pre-deployment validation - Run staging for 24-48 hours before promoting to prod
  4. Testing multiple strategies together - Staging enables all strategies simultaneously
  5. Debugging production issues - Reproduce issues in a safe environment
  6. Training new team members - Provide realistic environment without production risk

Do NOT use staging for: - Quick code tests (use dev instead) - Actual trading (use prod instead) - Long-term data collection (use prod instead)

How Staging Differs from Dev/Prod

Aspect Development Staging Production
Capital $100 $1,000 $5,000
Max per market $5,000 $500 $500
Max per strategy $2,000 $400 $2,000
Daily loss limit $500 $100 $250
Max drawdown 10% 10% 10%
Debug logging Yes Yes No
Strategies enabled 3 5 3
Private key None None Required
Database pqap_dev.db pqap_staging.db pqap.db

Key Staging Characteristics

  1. Paper Trading Mode: No POLYMARKET_PRIVATE_KEY configured, so no real orders execute
  2. Conservative Risk Limits: Matches production limits to catch issues before they matter
  3. All Strategies Enabled: Tests strategy interactions and resource contention
  4. Separate Database: Uses data/pqap_staging.db to avoid polluting dev/prod data
  5. Debug Logging: Enabled for troubleshooting

How to Deploy to Staging

Local Staging

Run staging on your local machine:

# Start PQAP with staging config
python -m src.main configs/staging.yaml

# Access staging dashboard
open http://localhost:8080

EC2 Staging

For longer validation runs, deploy staging to EC2 (alongside production):

# 1. SSH to EC2
ssh tailwindtech-ec2

# 2. Stop production (if needed)
pqap-ctl stop

# 3. Start staging
cd ~/pqap
source venv/bin/activate
python -m src.main configs/staging.yaml

# 4. Or run staging on different port (to run alongside prod)
python -m src.main configs/staging.yaml --port 8081

Staging Validation Checklist

Before promoting to production, verify:

  • [ ] All enabled strategies initialize without errors
  • [ ] Signals are being generated appropriately
  • [ ] Risk limits are enforced correctly
  • [ ] No memory leaks over 24+ hour run
  • [ ] Telegram alerts working (if configured)
  • [ ] Dashboard displays correct data
  • [ ] Paper trades execute at expected frequency

Staging Environment Configuration

Create .env.staging for staging-specific credentials:

# Staging .env file
POLYMARKET_API_KEY=your_api_key
POLYMARKET_API_SECRET=your_api_secret
# NO PRIVATE KEY - paper trading only

# Telegram (use staging bot or same as prod)
TELEGRAM_BOT_TOKEN=your_staging_bot_token
TELEGRAM_CHAT_ID=your_chat_id

Promoting from Staging to Production

After successful staging validation:

# 1. Stop staging
Ctrl+C  # or pkill -f staging.yaml

# 2. Review staging logs for any issues
grep ERROR pqap.log

# 3. Sync to EC2 if not already synced
python scripts/pqap_manager.py sync

# 4. Start production
python scripts/pqap_manager.py start ec2

# 5. Monitor production startup
python scripts/pqap_manager.py logs ec2

Local Development Setup

Prerequisites

  • Python 3.11 or later
  • SSH access to EC2 instance
  • Git (optional but recommended)

Initial Setup

  1. Clone repository (if using git) bash git clone <repository-url> cd polymarket

  2. Create virtual environment bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install -r requirements.txt

  3. Configure local environment

Create .env file: ```bash # Polymarket credentials (can be empty for monitoring-only mode) POLYMARKET_API_KEY= POLYMARKET_API_SECRET= POLYMARKET_PRIVATE_KEY=

# Telegram (use test bot for local) TELEGRAM_BOT_TOKEN=your_test_bot_token TELEGRAM_CHAT_ID=your_chat_id ```

  1. Configure SSH access to EC2

Add to ~/.ssh/config: Host tailwindtech-ec2 HostName 54.85.161.228 User ubuntu IdentityFile ~/.ssh/tailwindtech-ec2.pem

  1. Test local run bash python -m src.main configs/dev.yaml

  2. Access local dashboard

Open browser to http://localhost:8080

Development Best Practices

  1. Always test locally first bash python -m src.main configs/dev.yaml

Verify: - No syntax errors - Strategies load correctly - Dashboard accessible - Telegram alerts working (if configured)

  1. Use small capital in dev config yaml # configs/dev.yaml initial_capital: 100 # Small for testing

  2. Enable only strategies you're testing ```yaml enabled_strategies:

    • my_new_strategy_v1 # Only enable what you're working on ```

  3. Check logs for errors bash tail -f pqap.log

Syncing to EC2

Using pqap_manager.py

The pqap_manager.py script handles code synchronization.

Basic Sync

python scripts/pqap_manager.py sync

What it does: - Copies src/, configs/, scripts/, docs/ to EC2 - Preserves EC2 data files (databases, state) - Shows diff of changed files - Fast (only transfers changed files)

Full Deployment (with dependencies)

python scripts/pqap_manager.py deploy ec2

What it does: - Syncs code (like sync) - Installs/updates Python dependencies - Restarts PQAP service - Runs health check

When to use: - After updating requirements.txt - First-time deployment - Major version upgrades

Manual Sync (Alternative)

If pqap_manager.py is unavailable:

# Sync source code
rsync -avz --exclude='data' --exclude='venv' --exclude='.env' \
  src/ tailwindtech-ec2:~/pqap/src/

# Sync configs
rsync -avz configs/ tailwindtech-ec2:~/pqap/configs/

# Sync scripts
rsync -avz scripts/ tailwindtech-ec2:~/pqap/scripts/

Verification

After syncing, verify changes took effect:

# Check file timestamps
ssh tailwindtech-ec2 'ls -la ~/pqap/src/strategies/my_strategy/'

# Check file contents
ssh tailwindtech-ec2 'head -20 ~/pqap/src/strategies/my_strategy/strategy.py'

Restarting Services

PQAP on EC2 can be managed three ways: 1. pqap-ctl (recommended) - Simple wrapper script 2. systemd - Linux service manager 3. pqap_manager.py - Remote control from local machine

Method 1: pqap-ctl (On EC2)

# SSH to EC2
ssh tailwindtech-ec2

# Start PQAP
pqap-ctl start

# Stop PQAP
pqap-ctl stop

# Restart PQAP
pqap-ctl restart

# Check status
pqap-ctl status

# View logs
pqap-ctl logs

Method 2: systemd (On EC2)

# SSH to EC2
ssh tailwindtech-ec2

# Start service
sudo systemctl start pqap

# Stop service
sudo systemctl stop pqap

# Restart service
sudo systemctl restart pqap

# Check status
sudo systemctl status pqap

# Enable auto-start on boot
sudo systemctl enable pqap

# Disable auto-start
sudo systemctl disable pqap

# View logs
journalctl -u pqap -f

Method 3: pqap_manager.py (From Local)

# Start EC2 PQAP
python scripts/pqap_manager.py start ec2

# Stop EC2 PQAP
python scripts/pqap_manager.py stop ec2

# Check status
python scripts/pqap_manager.py status

# View logs
python scripts/pqap_manager.py logs ec2

Typical Deployment Sequence

After making code changes:

# 1. Stop running instance
python scripts/pqap_manager.py stop ec2

# 2. Sync code
python scripts/pqap_manager.py sync

# 3. Start with new code
python scripts/pqap_manager.py start ec2

# 4. Monitor startup
python scripts/pqap_manager.py logs ec2

# 5. Verify health
curl http://54.85.161.228:8080/health

Quick Restart (No Code Changes)

If you just need to restart the service:

# From local
python scripts/pqap_manager.py stop ec2 && \
  sleep 3 && \
  python scripts/pqap_manager.py start ec2

# Or on EC2
ssh tailwindtech-ec2 'pqap-ctl restart'

Rollback Procedures

Quick Rollback (Last Working Version)

If deployment breaks:

  1. Stop the broken deployment bash python scripts/pqap_manager.py stop ec2

  2. Restore from git (if using git) bash git log --oneline -5 # Find last working commit git checkout <commit-hash> python scripts/pqap_manager.py sync python scripts/pqap_manager.py start ec2

  3. Or restore from local backup bash # Restore from your local version cd /path/to/backup rsync -avz src/ tailwindtech-ec2:~/pqap/src/ ssh tailwindtech-ec2 'pqap-ctl restart'

Database Rollback

If database schema changes caused issues:

  1. Stop PQAP bash ssh tailwindtech-ec2 'pqap-ctl stop'

  2. Restore database from backup bash ssh tailwindtech-ec2 '~/pqap/scripts/restore_from_s3.sh'

  3. Restart PQAP bash ssh tailwindtech-ec2 'pqap-ctl start'

Configuration Rollback

If config changes caused issues:

  1. Edit config directly on EC2 bash ssh tailwindtech-ec2 vim ~/pqap/configs/prod.yaml # Revert problematic changes

  2. Or sync old config from local ```bash # Restore configs/prod.yaml to previous version locally git checkout HEAD~1 configs/prod.yaml

# Sync to EC2 rsync -avz configs/prod.yaml tailwindtech-ec2:~/pqap/configs/

# Restart ssh tailwindtech-ec2 'pqap-ctl restart' ```

Emergency Stop

If PQAP is misbehaving:

# Kill all Python processes running PQAP
ssh tailwindtech-ec2 'pkill -f "python -m src.main"'

# Or force kill
ssh tailwindtech-ec2 'pkill -9 -f "python -m src.main"'

Then investigate logs before restarting.

Configuration Management

Environment-Specific Configs

PQAP uses YAML config files for different environments:

configs/dev.yaml (local development):

env: dev
debug: true
initial_capital: 100  # Small for testing

enabled_strategies:
  - dual_arb_v1

risk_limits:
  max_capital_per_market: 50
  daily_loss_limit: 25

configs/staging.yaml (pre-production validation):

env: staging
debug: true
initial_capital: 1000  # Between dev and prod

enabled_strategies:
  - dual_arb_v1
  - momentum_v1
  - value_bet_v1
  - market_maker_v1
  - mean_reversion_v1

risk_limits:
  max_capital_per_market: 500  # Matches prod
  max_capital_per_strategy: 400
  daily_loss_limit: 100

configs/prod.yaml (EC2 production):

env: prod
debug: false
initial_capital: 5000  # Real capital

enabled_strategies:
  - dual_arb_v1
  - momentum_v1
  - value_bet_v1

risk_limits:
  max_capital_per_market: 500
  daily_loss_limit: 250

Environment Variables (.env)

Each environment has its own .env file:

Local .env:

# Test credentials or empty
POLYMARKET_API_KEY=
POLYMARKET_PRIVATE_KEY=

# Test Telegram bot
TELEGRAM_BOT_TOKEN=test_bot_token
TELEGRAM_CHAT_ID=test_chat_id

EC2 .env (at ~/pqap/.env):

# Production credentials
POLYMARKET_API_KEY=prod_key
POLYMARKET_PRIVATE_KEY=prod_private_key

# Production Telegram bot
TELEGRAM_BOT_TOKEN=prod_bot_token
TELEGRAM_CHAT_ID=prod_chat_id

IMPORTANT: .env files are NEVER synced. Each environment maintains its own.

Changing Configuration

Strategy Parameters

  1. Edit config file locally bash vim configs/prod.yaml

  2. Update strategy parameters yaml strategies: mean_reversion_v1: parameters: entry_zscore: "2.0" # Changed from 1.5 exit_zscore: "0.5"

  3. Sync and restart bash python scripts/pqap_manager.py sync python scripts/pqap_manager.py stop ec2 python scripts/pqap_manager.py start ec2

Enabling/Disabling Strategies

  1. Edit enabled_strategies list ```yaml enabled_strategies:

    • dual_arb_v1
    • mean_reversion_v1 # - momentum_v1 # Disabled ```

  2. Sync and restart bash python scripts/pqap_manager.py sync ssh tailwindtech-ec2 'pqap-ctl restart'

Risk Limits

  1. Edit risk_limits section yaml risk_limits: max_capital_per_market: 250 # Increased from 200 daily_loss_limit: 150 # Increased from 100

  2. Sync and restart - risk limits load at startup

Configuration Validation

Before deploying config changes:

  1. Test locally bash python -m src.main configs/dev.yaml # Check for config errors in logs

  2. Validate YAML syntax bash python -c "import yaml; yaml.safe_load(open('configs/prod.yaml'))"

  3. Check strategy validation

  4. Startup logs show if strategies fail validation
  5. Invalid configs prevent strategy from starting

Monitoring and Logs

Access Points

Service Local EC2 Production
Dashboard http://localhost:8080 http://54.85.161.228:8080
HTTPS Dashboard N/A https://pqap.tailwindtech.ai
Health Check http://localhost:8080/health http://54.85.161.228:8080/health

Health Checks

HTTP Health Endpoint

# Local
curl http://localhost:8080/health

# EC2
curl http://54.85.161.228:8080/health

Response:

{
  "status": "ok",
  "uptime_seconds": 3600,
  "active_strategies": 3,
  "markets_tracked": 50
}

API Status

curl http://54.85.161.228:8080/api/status

Detailed status including environment, capital, strategy states.

Log Monitoring

Real-time Logs (EC2)

# Using pqap_manager (from local)
python scripts/pqap_manager.py logs ec2

# Using SSH
ssh tailwindtech-ec2 'tail -f ~/pqap/pqap.log'

# Using systemd
ssh tailwindtech-ec2 'journalctl -u pqap -f'

Log Files

EC2 Locations: - Main log: ~/pqap/pqap.log - systemd journal: journalctl -u pqap

Local: - Logs print to console (stdout)

Log Levels

PQAP uses Python logging: - INFO: Normal operations (startup, signals, trades) - WARNING: Unexpected but recoverable events - ERROR: Problems that need attention - CRITICAL: Severe errors (kill switch, etc.)

Filtering Logs

# On EC2
ssh tailwindtech-ec2

# Errors only
grep ERROR ~/pqap/pqap.log

# Specific strategy
grep "mean_reversion_v1" ~/pqap/pqap.log

# Trade executions
grep "Trade executed" ~/pqap/pqap.log

# Last 100 lines
tail -100 ~/pqap/pqap.log

# Follow with grep filter
tail -f ~/pqap/pqap.log | grep Signal

Telegram Monitoring

PQAP sends hourly status updates via Telegram:

⏰ PQAP Hourly Update
Time: 2024-12-24 15:00 UTC

📈 Portfolio:
  P&L: +$12.45 (+1.25%)
  Capital: $987.55
  Positions: 5
  Trades: 34

📊 Data Collection:
  Markets: 50
  Snapshots: 12,345
  Anomalies: 8

⚠️ Errors: 0

Alert Types: - Hourly status updates - Trade executions - Risk alerts (limit breaches) - Error alerts (critical failures) - Strategy auto-disable notifications

Backup and Restore

Automated Backups (EC2)

EC2 runs daily backups to S3 via cron (2am UTC):

# Crontab entry (on EC2)
0 2 * * * ~/pqap/scripts/backup_to_s3.sh

Backup Script (scripts/backup_to_s3.sh):

#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
S3_BUCKET="s3://pqap-backups"

# Backup databases
aws s3 cp ~/pqap/data/market_history.db \
  "$S3_BUCKET/daily/market_history_$DATE.db"

aws s3 cp ~/pqap/data/pqap_dev.db \
  "$S3_BUCKET/daily/pqap_dev_$DATE.db"

# Backup paper trading state
aws s3 cp ~/pqap/data/paper_trading_state.json \
  "$S3_BUCKET/daily/paper_trading_state_$DATE.json"

# Keep last 7 daily backups, 4 weekly backups
# Cleanup handled by S3 lifecycle policy

Manual Backup

From EC2

# SSH to EC2
ssh tailwindtech-ec2

# Create backup directory
mkdir -p ~/backups/$(date +%Y%m%d)

# Backup databases
cp ~/pqap/data/*.db ~/backups/$(date +%Y%m%d)/

# Backup state files
cp ~/pqap/data/*.json ~/backups/$(date +%Y%m%d)/

To Local Machine

# Download databases
scp tailwindtech-ec2:~/pqap/data/market_history.db ./backup/
scp tailwindtech-ec2:~/pqap/data/pqap_dev.db ./backup/

# Download paper trading state
scp tailwindtech-ec2:~/pqap/data/paper_trading_state.json ./backup/

Restore Procedures

Restore from S3

# SSH to EC2
ssh tailwindtech-ec2

# Stop PQAP
pqap-ctl stop

# Run restore script
~/pqap/scripts/restore_from_s3.sh

# Start PQAP
pqap-ctl start

Restore from Local Backup

# Stop EC2 PQAP
python scripts/pqap_manager.py stop ec2

# Upload backup databases
scp ./backup/market_history.db tailwindtech-ec2:~/pqap/data/
scp ./backup/pqap_dev.db tailwindtech-ec2:~/pqap/data/

# Start EC2 PQAP
python scripts/pqap_manager.py start ec2

Restore Specific Database Table

# SSH to EC2
ssh tailwindtech-ec2

# Restore specific table from backup
sqlite3 ~/pqap/data/pqap_dev.db <<EOF
.output /tmp/signals_backup.sql
.dump signals
.quit
EOF

# ... perform restore operations

Troubleshooting

Common Issues

1. PQAP Won't Start

Symptoms: Service fails to start, exits immediately

Diagnosis:

# Check logs
ssh tailwindtech-ec2 'tail -50 ~/pqap/pqap.log'

# Check systemd status
ssh tailwindtech-ec2 'systemctl status pqap'

# Check if port in use
ssh tailwindtech-ec2 'ss -tlnp | grep 8080'

Common Causes: - Port 8080 already in use - Missing dependencies - Invalid configuration - Database locked

Solutions:

# Kill old process
ssh tailwindtech-ec2 'pkill -f "python -m src.main"'

# Reinstall dependencies
ssh tailwindtech-ec2 'cd ~/pqap && source venv/bin/activate && pip install -r requirements.txt'

# Check config syntax
ssh tailwindtech-ec2 'python -c "import yaml; yaml.safe_load(open(\\"~/pqap/configs/prod.yaml\\"))"'

# Unlock database
ssh tailwindtech-ec2 'fuser -k ~/pqap/data/pqap_dev.db'

2. Code Changes Not Taking Effect

Symptoms: After sync and restart, old behavior persists

Diagnosis:

# Verify file sync
ssh tailwindtech-ec2 'ls -la ~/pqap/src/strategies/my_strategy/'

# Check file contents
ssh tailwindtech-ec2 'head -20 ~/pqap/src/strategies/my_strategy/strategy.py'

# Check running process
ssh tailwindtech-ec2 'ps aux | grep pqap'

Solutions:

# Force sync
python scripts/pqap_manager.py sync

# Hard restart (kill and start)
ssh tailwindtech-ec2 'pkill -9 -f "python -m src.main"'
sleep 3
ssh tailwindtech-ec2 'pqap-ctl start'

# Clear Python cache
ssh tailwindtech-ec2 'find ~/pqap -type d -name __pycache__ -exec rm -rf {} +'

3. No Signals Generated

Symptoms: Strategies running but no signals appear

Diagnosis:

# Check strategy status
curl http://54.85.161.228:8080/api/strategies

# Check market data
curl http://54.85.161.228:8080/api/markets

# Check logs for strategy activity
ssh tailwindtech-ec2 'grep "on_orderbook_update" ~/pqap/pqap.log | tail -20'

Common Causes: - Markets don't meet strategy criteria - Strategy thresholds too strict - No market data flowing

Solutions:

# Lower strategy thresholds in config
vim configs/prod.yaml
# Adjust min_volume, min_edge, etc.

# Verify market data is updating
ssh tailwindtech-ec2 'tail -f ~/pqap/pqap.log | grep "Refreshed prices"'

4. Database Issues

Symptoms: Database locked, corruption errors

Diagnosis:

# Check database integrity
ssh tailwindtech-ec2 'sqlite3 ~/pqap/data/pqap_dev.db "PRAGMA integrity_check"'

# Check for locks
ssh tailwindtech-ec2 'fuser ~/pqap/data/pqap_dev.db'

Solutions:

# Kill processes locking database
ssh tailwindtech-ec2 'fuser -k ~/pqap/data/pqap_dev.db'

# Restore from backup if corrupted
ssh tailwindtech-ec2 '~/pqap/scripts/restore_from_s3.sh'

5. High CPU/Memory Usage

Symptoms: EC2 instance slow, system resources exhausted

Diagnosis:

# Check resource usage
ssh tailwindtech-ec2 'top -b -n 1 | head -20'

# Check PQAP process
ssh tailwindtech-ec2 'ps aux | grep pqap'

# Check database size
ssh tailwindtech-ec2 'du -h ~/pqap/data/*.db'

Solutions:

# Reduce market polling frequency (edit src/main.py)
# Change polling interval from 10s to 30s

# Reduce number of markets tracked (edit src/main.py)
# Change from 50 to 20 markets

# Clean old database records
ssh tailwindtech-ec2 'sqlite3 ~/pqap/data/market_history.db "DELETE FROM market_snapshots WHERE timestamp < datetime(\\"now\\", \\"-90 days\\")"'

6. Telegram Alerts Not Sending

Symptoms: No alerts received

Diagnosis:

# Check Telegram config
ssh tailwindtech-ec2 'grep TELEGRAM ~/pqap/.env'

# Check logs for Telegram errors
ssh tailwindtech-ec2 'grep -i telegram ~/pqap/pqap.log | tail -20'

Solutions:

# Verify bot token and chat ID
# Test manually:
curl -X POST "https://api.telegram.org/bot<TOKEN>/sendMessage" \
  -d "chat_id=<CHAT_ID>&text=Test message"

# Restart PQAP to reinitialize Telegram client
ssh tailwindtech-ec2 'pqap-ctl restart'

Getting Support

When reporting issues, provide:

  1. Error messages from logs
  2. Steps to reproduce
  3. Configuration files (sanitize credentials)
  4. Environment (local vs EC2, which config)
  5. Recent changes (code, config, dependencies)

Emergency Contacts

  • System Administrator: [contact info]
  • On-call Engineer: [contact info]
  • Telegram Alert Bot: @YourPQAPBot

Summary Commands Reference

Local Development

# Start PQAP locally (development)
python -m src.main configs/dev.yaml

# Start PQAP locally (staging)
python -m src.main configs/staging.yaml

# Access dashboard
open http://localhost:8080

Remote Management (from local)

# Sync code to EC2
python scripts/pqap_manager.py sync

# Full deployment
python scripts/pqap_manager.py deploy ec2

# Start/stop/status
python scripts/pqap_manager.py start ec2
python scripts/pqap_manager.py stop ec2
python scripts/pqap_manager.py status

# View logs
python scripts/pqap_manager.py logs ec2

EC2 Management (on EC2)

# SSH to EC2
ssh tailwindtech-ec2

# Service control (pqap-ctl)
pqap-ctl start
pqap-ctl stop
pqap-ctl restart
pqap-ctl status
pqap-ctl logs

# Service control (systemd)
sudo systemctl start pqap
sudo systemctl stop pqap
sudo systemctl restart pqap
sudo systemctl status pqap

# View logs
tail -f ~/pqap/pqap.log
journalctl -u pqap -f

Health Checks

# Local
curl http://localhost:8080/health

# EC2
curl http://54.85.161.228:8080/health

Backup/Restore

# Backup (on EC2)
~/pqap/scripts/backup_to_s3.sh

# Restore (on EC2)
~/pqap/scripts/restore_from_s3.sh

System Overview

Polymarket API

Market data source

↓

Data Collector

Every 5 minutes

↓

SQLite Database

Price history + trades

↓

Strategy Engine

Signal generation

ML Model

XGBoost (72% acc)

↓

Execution Engine

Paper trading

↓

Dashboard

You are here!

Telegram

Alerts & updates

Trading Strategies

Each strategy looks for different market inefficiencies:

Dual Arbitrage Active

Finds when YES + NO prices don't add to 100%. Risk-free profit.

Mean Reversion Active

Buys when price drops too far from average, sells when it recovers.

Market Maker Active

Places bid/ask orders to capture the spread.

Time Arbitrage Active

Exploits predictable price patterns at certain hours.

ML Prediction Active

Uses machine learning to predict 6-hour price direction.

Value Betting Disabled

Finds underpriced outcomes based on implied probability.

Data Storage (Single Source of Truth)

All data lives on EC2. Local machines are for development only. The EC2 instance is the authoritative source for all market data, trades, and positions.
Database Purpose Location
market_history.db Price snapshots every 5 minutes (8.2 MB) EC2 (primary)
pqap_staging.db Trades, positions, P&L history EC2 (primary)
paper_trading_state.json Current portfolio state EC2 (primary)

Environment Architecture

EC2 (Production)

  • Runs 24/7
  • All databases live here
  • Executes all trades
  • Single source of truth

Local (Development)

  • For code changes only
  • Syncs code to EC2
  • No production data
  • Can be turned off

Environment Details

Component Details
Dashboard URL https://pqap.tailwindtech.ai
Server AWS EC2 (us-east-1)
SSL Let's Encrypt via Traefik
Mode Paper Trading (simulated)

How It Works (Simple Version)

1. Data Collection: Every 5 minutes, we fetch prices from Polymarket for 50 markets and save them to our database.

2. Analysis: Our strategies analyze this data looking for patterns - like prices that moved too far from normal, or markets where the math doesn't add up.

3. Signals: When a strategy finds an opportunity, it generates a "signal" - a recommendation to buy or sell.

4. Execution: The execution engine takes these signals and simulates trades (paper trading). Eventually, this will place real orders.

5. Monitoring: This dashboard shows you what's happening. Telegram sends alerts for important events.