Crawlsight API

Powerful web data extraction API for developers and businesses

Features

🔄

Real-time Web Data Extraction

Extract structured data from any website in real-time with high accuracy.

📋

Custom Data Schemas

Define custom data schemas to extract exactly what you need.

📦

Bulk Processing

Process thousands of URLs in a single API call.

⚡

JavaScript Rendering

Fully render JavaScript-heavy websites before extraction.

🔒

Proxy Management

Automatic rotation of residential and datacenter proxies.

🧠

AI-assisted Extraction

Our AI helps identify and extract unstructured data automatically.

Try the API

Select an Endpoint

Extract Data

POST/api/v1/extract

Extract data from a specific URL using custom selectors.

Parameters

Name	Type	Required	Description
url	string	Yes	Target URL to extract data from
selectors	object	Yes	CSS selectors mapping to extract
render_js	boolean	No	Whether to render JavaScript before extraction

Request

Response

Code Samples

Python

import requests

# Example for Extract Data endpoint
url = "https://api.crawlsight.com/api/v1/extract"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",  # Replace with your API key
    "Content-Type": "application/json"
}

payload = {
    "url": "https://example.com",
    "selectors": {
        "title": "h1",
        "content": "p"
    },
    "render_js": True
}

try:
    response = requests.post(url, json=payload, headers=headers)
    response.raise_for_status()
    data = response.json()
    print("Extracted data:", data)
except requests.exceptions.RequestException as e:
    print("API request failed:", e)