Skip to main content
Scrapingant logo
  • Category: ai web scraping
  • Auth: API_KEY
  • Composio-managed OAuth available? N/A
  • Tools: 9
  • Triggers: 0
  • Slug: SCRAPINGANT
  • Version: 20261101_00

Tools

Slug: SCRAPINGANT_EXTRACT_CONTENT_AS_MARKDOWNThis tool extracts content from a given URL and converts it into Markdown format. It is particularly useful for preparing text for Language Learning Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It supports GET, POST, PUT, and DELETE methods.

Input Parameters

string
required
The URL of the web page to scrape and convert to Markdown.
string ("get" | "post" | "put" | "delete")
HTTP method to use for the request.
boolean
Enables the use of a headless browser for scraping. Default is true.
string
Cookies to include with the request.
string
Base64-encoded JavaScript to execute on the page after it loads.
string
Specifies the type of proxy to use.
string
Specifies the country for the proxy (e.g., US, GB).
array
List of resource types to block (e.g., image, script, stylesheet, font, media, websocket, other).
string
CSS selector to wait for before returning the result.
boolean
Returns the raw HTML as received from the server, without JavaScript rendering. Default is false.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_EXTRACT_DATA_WITH_AIThis tool allows you to extract structured data from a web page using ScrapingAnt’s AI-powered extraction capabilities. You provide a URL and an AI query (prompt) describing what data you want to extract, and the tool returns the extracted data in a structured format. It supports additional parameters for browser rendering, proxies, and cookies to handle dynamic content and localization.

Input Parameters

string
required
The URL of the page to extract data from.
string
Cookies to use for the request. (e.g. cookie1=value1; cookie2=value2)
string
Proxy type to use for the request. (datacenter, residential)
boolean
Return text content of the page. (default: false)
string
Proxy country to use for the request. (e.g. US, GB, DE)
boolean
Enable browser rendering. (default: true)
string
Wait for a specific selector to appear on the page before extracting data.
string
required
A free-form text describing the data you want to extract.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_GET_API_CREDITS_USAGERetrieves the current API credit usage status for the authenticated ScrapingAnt account, including subscription plan credits and remaining credits for the billing period. Use it to monitor consumption against the plan’s credit limit.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_GET_V1_USAGE[DEPRECATED - Use v2] Tool to get the current subscription status and API credits usage information. This is the legacy v1 endpoint which is no longer actively maintained.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_SCRAPE_WEB_PAGEThis tool scrapes a web page using the ScrapingAnt API. It fetches the HTML content of the specified URL. Users can customize the scraping behavior by enabling a headless browser, using proxies, waiting for specific elements, executing JavaScript, passing cookies, and blocking certain resources.

Input Parameters

string
required
URL of the web page to scrape.
boolean
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
string
Cookies to pass with the scraping request.
string
Base64 encoded JavaScript snippet to execute on the page. Requires headless browser.
string ("datacenter" | "residential")
Specifies the type of proxy to use.
string
Specifies the country for the proxy.
array
List of resource types to block. Requires headless browser.
string
CSS selector to wait for before returning the result. Requires headless browser.
boolean
Enable to return the raw HTML from the server without JavaScript rendering. Requires headless browser. Defaults to False.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_SCRAPE_WEBPAGE_POSTTool to perform a POST request through ScrapingAnt’s proxy to scrape a webpage. Use when you need to scrape pages that require POST method, such as form submissions or APIs that only accept POST requests. Data is forwarded transparently to the target web page.

Input Parameters

string
required
URL of the target web page to scrape using POST request.
boolean
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
string
Cookies to pass with the scraping request to the target site. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
object
POST data to send to the target web page. This will be forwarded transparently to the target. Useful for form submissions and APIs requiring POST requests.
string
Base64 encoded JavaScript snippet to execute once the page is loaded. Requires browser=True.
string ("datacenter" | "residential")
Specifies the type of proxy to use for the request. Defaults to datacenter.
string
Specifies the country for the proxy. If not specified, a random country will be used.
array
List of resource types to block. Prevents cloud browser from loading specified resource types. Requires browser=True.
string
CSS selector to wait for before returning the result. Requires browser=True.
boolean
Enable to return the raw HTML from the server without JavaScript rendering. Requires browser=True. Defaults to False.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_SCRAPE_WEBPAGE_PUTTool to perform a PUT request through ScrapingAnt’s proxy to scrape a webpage that requires PUT method. Use when the target webpage requires PUT method for data submission. Data is forwarded transparently to the target web page.

Input Parameters

string
required
URL of the web page to scrape using PUT method.
boolean
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
string
Cookies to pass with the scraping request to the target site. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
string
Base64 encoded JavaScript snippet to execute on the page once loaded. Requires headless browser.
string ("datacenter" | "residential")
Specifies the type of proxy to use. Defaults to datacenter.
string
Content-Type header to use for the PUT request. Will be sent as Ant-Content-Type header.
string
Request body data to send with the PUT request. This data will be forwarded transparently to the target web page.
string
Specifies the country for the proxy. If not specified, request will be made from a random country.
array
List of resource types to block from loading. Requires headless browser.
string
CSS selector to wait for before returning the result. Requires headless browser.
boolean
Enable to return the raw HTML from the server without JavaScript rendering. Requires headless browser. Defaults to False.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_SCRAPE_WEBPAGE_V1_POST[DEPRECATED - Use v2] Tool to scrape a webpage using POST method with ScrapingAnt’s v1 API. Returns JSON with content, cookies, and status_code. This is the legacy v1 endpoint which is no longer actively maintained. Use the v2 endpoints for new implementations.

Input Parameters

string
required
URL of the web page to scrape.
boolean
Enables using headless browser for scraping. Default: true. If false, JavaScript will not be rendered.
string
Cookie data to include with the scraping request. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
string
Base64 encoded JavaScript snippet to run once the page is loaded. Requires browser=true.
string ("datacenter" | "residential")
Proxy classification to use. Default is datacenter. Options: datacenter or residential.
boolean
Enables returning text only content from the page. Default: false. If true, returns plain text instead of HTML.
string
Geographic location for proxy requests. If not specified, a random country will be used.
string
CSS selector of the element our service will wait for before returning the result. Requires browser=true.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: SCRAPINGANT_SCRAPE_WITH_EXTENDED_JSON_OUTPUTScrapes a web page and returns comprehensive data including HTML content, plain text, cookies, HTTP headers, XHR/Fetch requests, and iframe content. This tool uses ScrapingAnt’s extended endpoint which provides much richer data than standard scraping: - Full HTML and extracted plain text content - All cookies and HTTP response headers from the target page - Captured XHR/Fetch API requests made by the page (useful for finding hidden APIs) - Content from embedded iframes Best used when you need more than just the HTML - such as analyzing cookies, headers, or JavaScript API calls made by a page. For simple HTML scraping, consider using the basic scrape tool instead for lower API credit usage.

Input Parameters

string
required
The full URL of the web page to scrape (must include protocol, e.g., https://).
boolean
Enable or disable headless browser rendering. When True (default), JavaScript is executed and dynamic content is loaded. Set to False for faster scraping of static pages.
string
Custom cookies to send with the request. Format: ‘name1=value1; name2=value2’.
integer
Maximum time in seconds to wait for the page to load. Must be between 5 and 60. Default is 60 seconds.
string
Type of proxy to use: ‘datacenter’ (faster, cheaper) or ‘residential’ (better for anti-bot sites). Default is ‘datacenter’.
string
Two-letter country code (ISO 3166-1 alpha-2) for geographic proxy location (e.g., ‘US’, ‘GB’, ‘DE’).
string
CSS selector to wait for before returning the page content. Useful for pages with dynamic content that loads after initial page load.
boolean
When True, returns the raw HTML from the server without JavaScript rendering. Useful for faster scraping when JS execution is not needed.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not