Skip to main content
ScrapeGraphAI logo
  • Category: web scraping & browser
  • Auth: API_KEY
  • Composio-managed OAuth available? N/A
  • Tools: 27
  • Triggers: 0
  • Slug: SCRAPEGRAPH_AI
  • Version: 20261101_00

Tools

Slug: SCRAPEGRAPH_AI_CONVERT_WEBPAGE_TO_MARKDOWN_V2Tool to convert any webpage into clean, well-formatted Markdown with full parameter control. Use when you need advanced options like stealth mode, custom headers, or webhook notifications. Supports all Markdownify API parameters.

Input Parameters

boolean
If true, return mock test data instead of actual conversion. Useful for testing without consuming credits. Default: false.
array
Browser interaction steps to perform before extracting content (e.g., clicking buttons, filling forms, scrolling).
boolean
Whether to return streaming response. Default: false.
object
Optional headers to send with the request, including cookies and user agent. Use to customize request behavior or handle authentication.
boolean
Enable stealth mode to bypass bot protection using advanced anti-detection techniques. Adds +4 credits to the request cost. Default: false.
integer
The number of milliseconds to wait before scraping the website. Useful for pages that load content dynamically. Default: 3000ms.
boolean
Include ScrapeGraphAI branding in the response. Default: false.
string
Webhook URL to send the job result to when processing completes. Enables async notification of completion.
string
required
The URL of the webpage to convert to Markdown. Must be a valid HTTP/HTTPS URL.
string
The country code to use for the scrape (e.g., ‘US’, ‘GB’, ‘FR’). Determines the geographic location for the request.
boolean
Enable rendering of heavy JavaScript. Use for Single Page Applications (SPAs) that require full JavaScript execution. Default: false.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GENERATE_SCHEMAGenerate or modify a JSON schema based on a search query for structured data extraction. Use when you need a schema template for scraping specific data fields.

Input Parameters

string
required
The user’s search query describing what data structure to generate. Be specific about the fields and structure you need (e.g., ‘Extract product name, price, and availability from an e-commerce product page’).
object
Optional existing JSON schema to modify or extend. If provided, the API will refine this schema based on the user_prompt.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_AGENTIC_SCRAPER_HISTORYRetrieve paginated history of agentic scraper jobs. Use to view past scraping requests, their status, and results.

Input Parameters

integer
Page number for pagination. Use to navigate through pages of history results.
integer
Number of history records to return per page. Maximum is typically 100.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_CRAWLER_HISTORYRetrieve the history of crawler jobs for your account. Returns paginated list of past crawler requests with their status, results, and metadata.

Input Parameters

integer
Page number for pagination. Must be a positive integer.
integer
Number of crawler history records to return per page. Must be between 1 and 100.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_CREDITSRetrieve credit information for your ScrapeGraphAI account, including remaining credits and usage. Use it to check credit availability before scraping operations.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_ENDPOINT_SUGGESTIONSTool to get AI-powered suggestions for creating scraping endpoints. Use when you need to identify what data can be extracted from a website and how to structure the scraping logic.

Input Parameters

string
required
Natural language description of what data you want to extract. Be specific about the type of information needed (e.g., ‘product listings’, ‘user profiles’, ‘news articles’).
string
required
The website URL to analyze for scraping opportunities. Must be a valid URL with https:// or http:// protocol.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_LIVE_SESSION_URLTool to get a URL for a live browser session. Use when you need to interact with a webpage in real-time through a controlled browser environment.

Input Parameters

string
required
The URL of the webpage to open in the live browser session. Must be a valid HTTP/HTTPS URL.
integer
Timeout for the live session in seconds. Default is 300 seconds (5 minutes).

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_MARKDOWNIFY_HISTORYTool to retrieve the history of markdownify webpage-to-Markdown conversion jobs. Use when you need to view past markdownify requests and their statuses.

Input Parameters

integer
Page number for pagination. Defaults to 1.
integer
Number of records to return per page. Defaults to 10.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_SCRAPE_HISTORYRetrieve the history of scrape jobs from your ScrapeGraphAI account. Use this to check the status of past scrapes, view results, and track credit usage.

Input Parameters

integer
Page number for pagination. Starts at 1.
integer
Number of scrape requests to return per page. Maximum depends on API limits.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_SEARCHSCRAPER_HISTORYGet the history of searchscraper jobs with pagination support. Use this to retrieve past searchscraper requests, their status, and results.

Input Parameters

integer
Page number for pagination. Must be 1 or greater.
integer
Number of records per page. Must be between 1 and 100.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_SITEMAP_HISTORYTool to retrieve the history of sitemap extraction jobs. Use when you need to view past sitemap extraction requests, their status, and results.

Input Parameters

integer
Page number for pagination. Defaults to 1.
integer
Number of items per page. Defaults to 10.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_SMARTSCRAPER_HISTORYTool to retrieve the history of smartscraper jobs. Use when you need to view past scraping requests and their results.

Input Parameters

integer
Page number for pagination (starts at 1). Use to navigate through multiple pages of history.
integer
Number of records to return per page (1-100). Default is 10.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_USAGE_TIMELINETool to retrieve usage timeline statistics for your ScrapeGraphAI account. Use when you need to visualize or analyze service usage patterns over time.

Input Parameters

string ("7" | "14" | "30" | "all")
Time range for usage timeline data. Choose ‘7’ for last 7 days, ‘14’ for last 14 days, ‘30’ for last 30 days, or ‘all’ for complete history.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_GET_WEBHOOK_LOGSRetrieve webhook delivery logs for a crawler job. A 404 response is returned as an empty log list with a zero count.

Input Parameters

string
required
The unique identifier of the crawler job to retrieve webhook logs for. Obtained from starting a crawler job.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_LIST_SCHEDULED_JOBSRetrieve a paginated list of all scheduled scraping jobs for your account. Use this action to view and manage your scheduled jobs, including their configuration, cron schedules, and active status. Supports filtering by service type and active status.

Input Parameters

integer
Page number for pagination. Must be 1 or greater.
string
Filter jobs by active status. Use ‘true’ to show only active jobs, ‘false’ for inactive jobs, or omit to show all jobs.
integer
Number of jobs to return per page. Must be between 1 and 100.
string
Filter jobs by service type (e.g., ‘smartscraper’, ‘markdownify’, ‘searchscraper’). If not provided, returns jobs of all service types.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_MARKDOWNIFY_STATUSCheck the status and retrieve results of a Markdownify webpage-to-Markdown conversion job. Use this action to poll for the status of an async Markdownify request started via SCRAPEGRAPH_AI_MARKDOWNIFY. Note: The ScrapeGraph AI API typically returns completed results synchronously, so this status endpoint is primarily useful for long-running conversions of large or complex webpages.

Input Parameters

string
required
The unique identifier (UUID) of the Markdownify request returned by the SCRAPEGRAPH_AI_MARKDOWNIFY action

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SAVE_ENDPOINTTool to save custom scraping endpoint configurations to ScrapeGraphAI. Use when you need to create reusable scraping endpoints with specific parameters and extraction logic.

Input Parameters

array
required
List of endpoint configurations to save. Each configuration defines a custom scraping endpoint with its parameters and extraction logic.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SEARCH_SCRAPERPerform web searches and scrape results, with AI-powered structured extraction or raw Markdown content. JS-rendered pages may yield incomplete content.

Input Parameters

boolean
If true, returns a mock response for testing without consuming credits. Default is false.
string
required
The search query describing what you want to find on the web.
string
ISO 639-1 language code for the search (e.g., ‘en’).
integer
Number of websites to search (3-20). Each website incurs credit costs. Default is 3. Insufficient credits returns a 402 error; verify balance before setting values above 3.
boolean
If true (default), uses AI to extract structured data (10 credits/page). If false, returns raw markdown content (2 credits/page).

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SEARCH_SCRAPER_STATUSCheck the status and results of an asynchronous SearchScraper job.

Input Parameters

string
required
The unique identifier (UUID) of the SearchScraper request obtained from the SearchScraper action response.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SMART_CRAWLER_STATUSCheck the status and retrieve results of a SmartCrawler web crawling job. Use this action to poll for completion and get the extracted content from a previously started SmartCrawler job. Returns the job status, crawled URLs, page content in markdown/HTML format, and LLM extraction results (if enabled). Implement a polling timeout (e.g., max retries or elapsed time cap) to avoid indefinite loops when waiting for long-running jobs.

Input Parameters

string
required
The unique identifier of the SmartCrawler task to check status for. Obtained from starting a SmartCrawler job.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SMART_SCRAPER_STARTExtract structured JSON from one URL, HTML document, or Markdown document using ScrapeGraphAI v2. The v2 Extract endpoint is synchronous and returns the completed result directly.

Input Parameters

object
Cookies to include in the request for authentication or session management.
object
Custom HTTP headers for the scraping request (e.g., User-Agent, Accept-Language). Used when fetching website_url.
boolean
Enable anti-bot detection techniques. Adds 5 credits per request. Use for sites with bot protection.
string
required
Natural language description of what information to extract from the webpage. Be specific about the data you want (e.g., ‘Extract the product name, price, and description’).
string
Full URL of the webpage to scrape (must include https://). Required if website_html and website_markdown are not provided.
string
Raw HTML content to scrape (max 2MB). Use this if you already have the page HTML. Required if website_url and website_markdown are not provided.
object
JSON Schema defining the structure of extracted data. Helps ensure consistent, structured output.
boolean
Enable enhanced JavaScript rendering for Single Page Applications (SPAs) and heavy JS sites. May increase processing time.
string
Raw Markdown content to scrape (max 2MB). Use this if you already have the page content as Markdown. Required if website_url and website_html are not provided.
integer
Number of scroll iterations for infinite scroll pages (0-100). Use for pages that load content on scroll.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SMART_SCRAPER_STATUSRetrieve the v2 History record for a SmartScraper Extract request ID. Extract calls are synchronous, so this action is primarily useful for looking up a previous result.

Input Parameters

string
required
The unique identifier (UUID) of the SmartScraper request returned by the Start SmartScraper action

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_START_SMART_CRAWLERStart a multi-page SmartCrawler job for AI-powered structured data extraction or markdown conversion. Returns a task ID for checking progress and retrieving results.

Input Parameters

string
required
The starting URL for the crawl. Must be a valid HTTP or HTTPS URL.
integer
Maximum crawl depth - how many link levels to follow from the start URL. Default is 1.
object
Crawl rules for filtering URLs during the crawl process.
string
Instructions for what data to extract during the crawl. Required when extraction_mode=true. Be specific about the information you want extracted.
object
JSON Schema object defining the structure of extracted data. Helps ensure consistent, structured output in extraction mode.
integer
Maximum number of links to crawl per depth level. If not specified (null), unlimited breadth is allowed. Ignored when sitemap=true.
boolean
Whether to use sitemap.xml for URL discovery instead of link following. When enabled, ignores breadth parameter. Default is false.
boolean
Enable stealth mode to bypass bot detection and anti-scraping measures. Adds +4 credits per page. Use for sites with bot protection. Default is false.
integer
Maximum total number of pages to crawl across all depth levels. Default is 10.
integer
Number of pages to process in each batch during the crawl. Higher values may speed up large crawls. Default is 1.
string
Webhook URL to receive the job completion notification. The result will be POSTed to this URL when the crawl finishes.
boolean
Whether to cache the website content for faster subsequent crawls. Default is false.
boolean
When true, enables AI-powered extraction using LLM (default, 10 credits/page). When false, enables markdown conversion mode (NO AI processing, 2 credits/page, 80% cheaper).
boolean
Enable enhanced JavaScript rendering for Single Page Applications (SPAs) and sites with heavy JavaScript. May increase processing time. Default is false.
boolean
Whether to restrict crawling to only pages on the same domain as the starting URL. Default is true.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SUBMIT_FEEDBACKSubmit feedback and ratings for completed ScrapeGraphAI requests.

Input Parameters

integer
required
Star rating from 0 (lowest) to 5 (highest)
string
required
UUID of the request/session this feedback is for
string
Optional comments about the request

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_SUBMIT_PRODUCT_FEEDBACKSubmit product feedback for ScrapeGraphAI. Use to provide ratings, comments, suggestions, and other feedback about the product itself.

Input Parameters

string
Your name
string
Your email address
string
Any issues you’ve encountered
integer
Overall rating from 1 (lowest) to 5 (highest)
string
Your company name
string
What you disliked about the product
array
Your use cases for the product
string
What you liked most about the product
boolean
Whether the setup process was easy
boolean
Whether you consent to be contacted about your feedback
string
required
UUID of the feedback submission
string
Preferred method of contact
string
How you discovered ScrapeGraphAI
integer
How likely you are to recommend this product (0-10 scale)
string
How frequently you use the product
string
Features you’d like to see added
string
Your suggestions for product improvements

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_TOONIFYConvert a JSON object to TOON (Token-Oriented Object Notation) for compact LLM processing while maintaining data structure.

Input Parameters

object
The JSON data to convert to TOON (Token-Oriented Object Notation) format. Can be any valid JSON object or array. TOON format reduces token usage by 30-60% compared to JSON while maintaining structure and readability. If not provided, returns empty response.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPEGRAPH_AI_VALIDATE_API_KEYCheck whether your ScrapeGraphAI API key is valid and authorized for use before making other API calls.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not