Documentation
Quickstart GuideInstallationAuthenticationBasic Usage
AuthenticationEndpointsError CodesRate LimitsWebhooks
AI EngineStealth ModeProxy ManagementCustom Rules
PythonNode.jscURLRubyGoPHP
E-Commerce ScrapingLead GenerationPrice MonitoringSEO Data ExtractionSocial Media
Security Best PracticesLegal ComplianceData Privacy

API Endpoints

Complete reference for all ScrapeHub API endpoints

Base URL

https://api.scrapehub.io/api/v1

All endpoints are relative to this base URL.

Authentication

All API requests require authentication via API key in the header:

Terminal
X-API-KEY: sk_live_xxxx_449x

🚀 Core Scraping API

Goal-based scraping with AI-powered data extraction

POST /jobs

Create a new scraping job with natural language goal or direct URL.

Request Body

ParameterTypeRequiredDescription
goalstringNo*Natural language description of what to scrape
urlstringNo*Direct URL to scrape (skips intent engine)
output_formatstringNoOutput format: json | csv | xlsx (default: json)
max_resultsintegerNoMaximum records to return (1-1000, default: 10)
min_resultsintegerNoMinimum target results (1-500)
use_smart_cascadebooleanNoEnable intelligent multi-method cascade (default: false)
fieldsarrayNoSpecific fields to extract (e.g., ["name", "phone", "email"])
webhook_urlstringNoURL to receive results when job completes

* Either goal or url is required

Terminal
curl -X POST https://api.scrapehub.io/api/v1/jobs \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -H "Content-Type: application/json" \
  -d '{
    "goal": "Find dentists in New York City with contact details",
    "output_format": "json",
    "max_results": 50,
    "fields": ["name", "phone", "email", "address"]
  }'

Response

{
  "job_id": "job_abc123xyz456",
  "status": "queued",
  "goal": "Find dentists in New York City with contact details",
  "output_format": "json",
  "progress": 0,
  "records_found": 0,
  "sources_total": 0,
  "sources_completed": 0,
  "created_at": "2026-01-27T10:30:00Z",
  "download_url": null
}

GET /jobs/:job_id

Get the status, progress, and results of a scraping job.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/jobs/job_abc123xyz456 \
  -H "X-API-KEY: sk_live_xxxx_449x"

Response

{
  "job_id": "job_abc123xyz456",
  "status": "completed",
  "goal": "Find dentists in New York City with contact details",
  "output_format": "json",
  "progress": 100,
  "records_found": 47,
  "sources_total": 3,
  "sources_completed": 3,
  "plan_summary": {
    "search_queries": ["dentists NYC", "dental clinics New York"],
    "sources": ["Google Maps", "Yelp", "Yellow Pages"]
  },
  "created_at": "2026-01-27T10:30:00Z",
  "completed_at": "2026-01-27T10:35:20Z",
  "download_url": "/api/v1/jobs/job_abc123xyz456/download?format=json"
}

Status values: queued, planning, running, completed, failed

GET /jobs/:job_id/download

Download job results in various formats.

Query Parameters

formatjson, csv, xlsx (optional, uses job's output_format if not specified)
Terminal
# Download as CSV
curl -X GET "https://api.scrapehub.io/api/v1/jobs/job_abc123xyz456/download?format=csv" \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -o results.csv

# Download as JSON
curl -X GET "https://api.scrapehub.io/api/v1/jobs/job_abc123xyz456/download?format=json" \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -o results.json

# Download as Excel
curl -X GET "https://api.scrapehub.io/api/v1/jobs/job_abc123xyz456/download?format=xlsx" \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -o results.xlsx

GET /jobs

List all scraping jobs for your account.

Terminal
curl -X GET "https://api.scrapehub.io/api/v1/jobs" \
  -H "X-API-KEY: sk_live_xxxx_449x"

Response

[
  {
    "job_id": "job_abc123xyz456",
    "status": "completed",
    "goal": "Find dentists in New York City",
    "records_found": 47,
    "created_at": "2026-01-27T10:30:00Z"
  },
  {
    "job_id": "job_def789uvw012",
    "status": "running",
    "goal": "Scrape product prices from example.com",
    "records_found": 12,
    "created_at": "2026-01-27T11:15:00Z"
  }
]

DELETE /jobs/:job_id

Cancel a running job or delete a completed job.

Terminal
curl -X DELETE https://api.scrapehub.io/api/v1/jobs/job_abc123xyz456 \
  -H "X-API-KEY: sk_live_xxxx_449x"

Response

{
  "message": "Job cancelled/deleted successfully"
}

🔑 API Keys Management

Manage your API keys programmatically

GET /api-keys

List all API keys for your account.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/api-keys \
  -H "X-API-KEY: sk_live_xxxx_449x"

POST /api-keys

Create a new API key.

Terminal
curl -X POST https://api.scrapehub.io/api/v1/api-keys \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production API Key",
    "scopes": ["jobs:read", "jobs:write"]
  }'

DELETE /api-keys/:key_id

Revoke an API key.

Terminal
curl -X DELETE https://api.scrapehub.io/api/v1/api-keys/key_xyz123 \
  -H "X-API-KEY: sk_live_xxxx_449x"

📊 Data Pools

Manage your scraped data pools for continuous monitoring

GET /pools

List all data pools for your account.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/pools \
  -H "X-API-KEY: sk_live_xxxx_449x"

GET /pools/:pool_id/records

Get all records in a specific data pool.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/pools/pool_abc123/records \
  -H "X-API-KEY: sk_live_xxxx_449x"

GET /pools/:pool_id/download

Download full pool data as CSV or JSON.

Terminal
curl -X GET "https://api.scrapehub.io/api/v1/pools/pool_abc123/download?format=csv" \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -o pool_data.csv

⏰ Scheduled Jobs

Schedule recurring scraping jobs with cron-like syntax

GET /scheduled-jobs

List all scheduled jobs.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/scheduled-jobs \
  -H "X-API-KEY: sk_live_xxxx_449x"

POST /scheduled-jobs

Create a new scheduled scraping job.

Terminal
curl -X POST https://api.scrapehub.io/api/v1/scheduled-jobs \
  -H "X-API-KEY: sk_live_xxxx_449x" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Daily Price Monitoring",
    "goal": "Track prices on competitor website",
    "schedule": "0 9 * * *",
    "pool_id": "pool_abc123",
    "enabled": true
  }'

💰 Credits & Billing

Monitor your account credits and usage

GET /credits/balance

Get your current credit balance.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/credits/balance \
  -H "X-API-KEY: sk_live_xxxx_449x"

Response

{
  "balance": 1250,
  "plan": "growth",
  "monthly_limit": 10000,
  "used_this_month": 847
}

GET /credits/transactions

View your credit transaction history.

Terminal
curl -X GET https://api.scrapehub.io/api/v1/credits/transactions \
  -H "X-API-KEY: sk_live_xxxx_449x"

📚 Additional Resources

  • Templates: GET /templates - Browse pre-built scraping templates
  • Stats: GET /stats/dashboard - View your usage statistics
  • Agent: POST /agent/run - Run AI agent for complex extraction
  • Authentication: POST /auth/login, POST /auth/register

Need Help?

Check out our Quickstart Guide or explore SDK integrations for your preferred language.