> ## Documentation Index
> Fetch the complete documentation index at: https://composio-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# WebScraping.AI

> WebScraping.AI provides an API for web scraping with features like Chrome JS rendering, rotating proxies, and HTML parsing.

![WebScraping.AI logo](https://mintcdn.com/composio-preview/DMB8UrgJmE7r-ZQ5/images/toolkits/webscraping_ai.svg?fit=max\&auto=format\&n=DMB8UrgJmE7r-ZQ5\&q=85\&s=7d50557a87d55e2540fcd7e0310244b6)

* **Category:** ai web scraping
* **Auth:** API\_KEY
* **Composio-managed OAuth available?** N/A
* **Tools:** 7
* **Triggers:** 0
* **Slug:** `WEBSCRAPING_AI`
* **Version:** 20261101\_00

## Tools

> [accordion] Get account usage and quota
>
> **Slug:** `WEBSCRAPING_AI_ACCOUNT_INFO`
>
> Tool to retrieve account API call quota and usage. Use when checking remaining requests and subscription details.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Ask Question About Web Page
>
> **Slug:** `WEBSCRAPING_AI_ASK_QUESTION`
>
> Get an LLM answer to a question about a web page for AI-powered analysis or extraction. Page retrieval supports proxies and optional JavaScript rendering.
>
> #### Input Parameters
>
> **js** `boolean`
>
> Execute on-page JavaScript using a headless browser. Default: true.
>
> **url** `string` *required*
>
> URL of the target page to analyze.
>
> **proxy** `string ("datacenter" | "residential")`
>
> Proxy type to use for the request.
>
> **device** `string ("desktop" | "mobile" | "tablet")`
>
> Device emulation type. Default: desktop.
>
> **format** `string ("json" | "text")`
>
> Response format. Default: json.
>
> **country** `string`
>
> Proxy country code. Default: us.
>
> **headers** `object`
>
> HTTP headers to pass to the target page.
>
> **timeout** `integer`
>
> Maximum web page retrieval time in milliseconds. Default: 10000, Max: 30000.
>
> **question** `string` *required*
>
> Question or instructions to ask the LLM model about the target page.
>
> **wait\_for** `string`
>
> CSS selector to wait for before returning page content.
>
> **js\_script** `string`
>
> Custom JavaScript code to execute on the target page.
>
> **js\_timeout** `integer`
>
> Maximum JavaScript rendering time in milliseconds. Default: 2000, Max: 20000.
>
> **custom\_proxy** `string`
>
> Custom proxy URL in `http://user:password@host:port` format.
>
> **error\_on\_404** `boolean`
>
> Return error on 404 HTTP status on the target page. Default: false.
>
> **error\_on\_redirect** `boolean`
>
> Return error on redirect on the target page. Default: false.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Extract Fields with AI
>
> **Slug:** `WEBSCRAPING_AI_EXTRACT_FIELDS`
>
> Tool to extract structured data fields from a web page using AI. Returns extracted fields as JSON. Uses proxies and Chromium JavaScript rendering for page retrieval and processing.
>
> #### Input Parameters
>
> **url** `string` *required*
>
> The target website URL to scrape and extract structured data from.
>
> **fields** `object` *required*
>
> Object describing fields to extract from the page. Keys are field names, values are descriptions of what to extract. Example: \{'title': 'Main product title', 'price': 'Current product price'}
>
> **unknown** `string`
>
> Optional parameter for handling unknown or additional configuration.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Rendered HTML
>
> **Slug:** `WEBSCRAPING_AI_GET_RENDERED_HTML`
>
> Tool to retrieve fully rendered HTML of a webpage. Use when JS-generated content must be included.
>
> #### Input Parameters
>
> **js** `string`
>
> Base64-encoded JavaScript to execute after rendering. Must be Base64-encoded; plain script strings are ignored.
>
> **url** `string` *required*
>
> The target URL to render and fetch HTML.
>
> **wait** `integer`
>
> Wait time before capture, in milliseconds. Increase for deferred JS execution or slow-loading pages to ensure complete content capture.
>
> **device** `string ("desktop" | "mobile")`
>
> Browser device mode to simulate.
>
> **locale** `string`
>
> Browser locale (RFC5646 code).
>
> **cookies** `string`
>
> Cookies in 'key1=value1; key2=value2;' format.
>
> **headers** `object`
>
> Extra HTTP headers as JSON object.
>
> **referer** `string`
>
> Referer header value.
>
> **timeout** `integer`
>
> Request timeout, in milliseconds.
>
> **proxy\_type** `string ("datacenter" | "residential")`
>
> Proxy type to use for the request.
>
> **user\_agent** `string`
>
> Custom User-Agent string.
>
> **disable\_images** `boolean`
>
> Whether to disable image loading.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Selected HTML
>
> **Slug:** `WEBSCRAPING_AI_GET_SELECTED_HTML`
>
> Tool to extract HTML from specific page elements using CSS selectors. Use when you need HTML from a particular section rather than the entire page.
>
> #### Input Parameters
>
> **js** `boolean`
>
> Execute JavaScript via headless browser. Default: true.
>
> **url** `string` *required*
>
> URL of the target page to scrape.
>
> **proxy** `string ("datacenter" | "residential")`
>
> Proxy type options.
>
> **device** `string ("desktop" | "mobile" | "tablet")`
>
> Device type options.
>
> **country** `string`
>
> Proxy country code (2-letter ISO code). Default: us.
>
> **headers** `object`
>
> HTTP headers to pass to the target page as a JSON object.
>
> **timeout** `integer`
>
> Maximum retrieval time in milliseconds. Default: 10000, max: 30000.
>
> **selector** `string`
>
> CSS selector to extract specific page elements. If not provided, returns the entire page HTML.
>
> **wait\_for** `string`
>
> CSS selector to wait for before returning content.
>
> **js\_script** `string`
>
> Custom JavaScript code to execute on the page before extraction.
>
> **js\_timeout** `integer`
>
> JavaScript rendering time in milliseconds. Default: 2000, max: 20000.
>
> **custom\_proxy** `string`
>
> Custom proxy URL to use instead of built-in proxies.
>
> **error\_on\_404** `boolean`
>
> Return error when target page responds with 404 status. Default: false.
>
> **error\_on\_redirect** `boolean`
>
> Return error when target page redirects. Default: false.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Selected Multiple Elements
>
> **Slug:** `WEBSCRAPING_AI_GET_SELECTED_MULTIPLE`
>
> Extract HTML from multiple areas of a web page using its URL and CSS selectors, without parsing the page yourself. Optional settings control JavaScript rendering, retrieval timeouts, and the proxy type and country.
>
> #### Input Parameters
>
> **js** `boolean`
>
> Execute on-page JavaScript using a headless browser. True by default.
>
> **url** `string` *required*
>
> The target URL to scrape.
>
> **proxy** `string ("datacenter" | "residential")`
>
> Proxy type options.
>
> **country** `string`
>
> Country of the proxy to use. US by default.
>
> **timeout** `integer`
>
> Maximum web page retrieval time in milliseconds. Default is 15000, maximum is 30000.
>
> **selectors** `array`
>
> Multiple CSS selectors to extract. If null, returns whole page HTML.
>
> **js\_timeout** `integer`
>
> Maximum JavaScript rendering time in milliseconds. Default is 2000.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Text
>
> **Slug:** `WEBSCRAPING_AI_GET_TEXT`
>
> Retrieve text content from a specified web page. Non-JSON responses are returned as raw text.
>
> #### Input Parameters
>
> **url** `string` *required*
>
> The target URL to scrape text from.
>
> **proxy** `string ("us" | "eu")`
>
> Proxy region to use for the request (e.g., 'us' or 'eu').
>
> **locale** `string`
>
> Browser locale/language (e.g., 'en-US').
>
> **session** `string`
>
> Session ID for preserving cookies across multiple calls.
>
> **timeout** `integer`
>
> Request timeout in seconds (must be >= 1).
>
> **render\_js** `boolean`
>
> Whether to render JavaScript on the page before extracting text.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not
