Errors

In this guide, we will talk about what happens when something goes wrong while you work with the Supacrawler API. Understanding error responses helps you build robust integrations and debug issues quickly.

You can tell if your request was successful by checking the status code when receiving an API response. If a response comes back unsuccessful, you can use the error code and message to figure out what has gone wrong.

Most errors are client-side issues like invalid parameters or authentication problems. Check your request format and API key before reaching out for support.

Status codes

Here is a list of the different categories of status codes returned by the Supacrawler API:

Name
2xx
Description
A 2xx status code indicates a successful response.
Name
4xx
Description
A 4xx status code indicates a client error — this means there's an issue with your request.
Name
5xx
Description
A 5xx status code indicates a server error — these are rare but can happen during high load or maintenance.

Error response format

All Supacrawler API error responses follow a consistent format. The response will include a success: false field along with error details to help you understand what went wrong.

Error Properties

Name
success
Type
boolean
Description
Always false for error responses.
Name
error
Type
string
Description
A human-readable error message describing what went wrong.
Name
code
Type
string
Description
A machine-readable error code for programmatic handling (optional).
Name
details
Type
object
Description
Additional error details when available (optional).

Error response example

{
  "success": false,
  "error": "Invalid or missing API key",
  "code": "UNAUTHORIZED"
}

Common error codes

Authentication Errors

Name
401 Unauthorized
Description
UNAUTHORIZED - Invalid or missing API key. Check your Authorization header.
Name
403 Forbidden
Description
FORBIDDEN - API key doesn't have permission for this resource or plan limits exceeded.

Authentication error

{
  "success": false,
  "error": "Invalid or missing API key",
  "code": "UNAUTHORIZED"
}

Request Errors

Name
400 Bad Request
Description
INVALID_REQUEST - Invalid parameters or request format.
Name
404 Not Found
Description
NOT_FOUND - Requested resource (like a job ID) doesn't exist.
Name
422 Unprocessable Entity
Description
VALIDATION_ERROR - Request format is correct but contains invalid data.

Validation error

{
  "success": false,
  "error": "URL is required",
  "code": "VALIDATION_ERROR",
  "details": {
    "field": "url",
    "message": "URL parameter cannot be empty"
  }
}

Rate Limiting

Name
429 Too Many Requests
Description
RATE_LIMITED - You've exceeded your plan's rate limits.

Rate limit error

{
  "success": false,
  "error": "Rate limit exceeded. Limit: 100 requests per hour",
  "code": "RATE_LIMITED",
  "details": {
    "limit": 100,
    "window": "hour",
    "reset_at": "2024-01-15T11:00:00Z"
  }
}

Server Errors

Name
500 Internal Server Error
Description
INTERNAL_ERROR - Something went wrong on our end.
Name
502 Bad Gateway
Description
SERVICE_UNAVAILABLE - Temporary service unavailability.
Name
503 Service Unavailable
Description
MAINTENANCE - Scheduled maintenance in progress.

Server error

{
  "success": false,
  "error": "Internal server error. Please try again later.",
  "code": "INTERNAL_ERROR"
}

Endpoint-specific errors

Scrape API Errors

Name
URL_UNREACHABLE
Description
The target URL couldn't be accessed (timeout, DNS error, etc.).
Name
BLOCKED_CONTENT
Description
The target website is blocking scraping attempts.
Name
INVALID_FORMAT
Description
Requested format is not supported.

Scrape error

{
  "success": false,
  "url": "https://blocked-site.com",
  "error": "Unable to access URL: connection timeout",
  "code": "URL_UNREACHABLE",
  "metadata": {
    "status_code": 0
  }
}

Jobs API Errors

Name
JOB_NOT_FOUND
Description
The requested job ID doesn't exist or has expired.
Name
JOB_FAILED
Description
The job encountered an error during processing.
Name
INVALID_JOB_TYPE
Description
Unsupported job type specified.

Job error

{
  "success": false,
  "jobId": "invalid-job-id",
  "type": "crawl",
  "status": "failed",
  "error": "Job not found or expired",
  "code": "JOB_NOT_FOUND"
}

Screenshots API Errors

Name
SCREENSHOT_FAILED
Description
Unable to capture screenshot (page error, timeout, etc.).
Name
INVALID_DEVICE
Description
Unsupported device type specified.
Name
INVALID_DIMENSIONS
Description
Custom viewport dimensions are invalid.

Screenshot error

{
  "success": false,
  "jobId": "screenshot-job-id",
  "type": "screenshot",
  "status": "failed",
  "error": "Page failed to load within timeout period",
  "code": "SCREENSHOT_FAILED"
}

Error handling best practices

Retry Logic

Implement exponential backoff for temporary errors:

Retry with backoff

async function makeRequestWithRetry(url, options, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await fetch(url, options)

      if (response.ok) {
        return await response.json()
      }

      // Don't retry client errors (4xx)
      if (response.status >= 400 && response.status < 500) {
        throw new Error(`Client error: ${response.status}`)
      }

      // Retry server errors (5xx) with backoff
      if (attempt < maxRetries - 1) {
        const delay = Math.pow(2, attempt) * 1000 // 1s, 2s, 4s
        await new Promise((resolve) => setTimeout(resolve, delay))
      }
    } catch (error) {
      if (attempt === maxRetries - 1) throw error
    }
  }
}

Error Code Handling

Handle specific error codes programmatically:

Error handling in Python

import requests
import time

def handle_api_error(response):
    data = response.json()
    error_code = data.get('code')

    if error_code == 'RATE_LIMITED':
        # Wait and retry
        reset_time = data.get('details', {}).get('reset_at')
        print(f"Rate limited. Waiting until {reset_time}")
        return 'retry'
    elif error_code == 'UNAUTHORIZED':
        print("Invalid API key. Check your credentials.")
        return 'abort'
    elif error_code == 'URL_UNREACHABLE':
        print(f"URL unreachable: {data.get('error')}")
        return 'skip'
    else:
        print(f"Unexpected error: {data.get('error')}")
        return 'abort'

# Usage
response = requests.get(url, headers=headers)
if not response.ok:
    action = handle_api_error(response)
    # Handle based on action: retry, abort, skip

Monitoring and Logging

Log errors for debugging and monitoring:

Error logging

function logApiError(error, context) {
  console.error('Supacrawler API Error:', {
    timestamp: new Date().toISOString(),
    error: error.message,
    code: error.code,
    context: context,
    url: context.url,
    statusCode: error.statusCode,
  })

  // Send to monitoring service
  // analytics.track('api_error', { ... });
}