- Category: ai web scraping
- Auth: API_KEY
- Composio-managed OAuth available? N/A
- Tools: 9
- Triggers: 0
- Slug:
SCRAPINGANT - Version: 20261101_00
Tools
Extract Content as Markdown
Extract Content as Markdown
Slug:
SCRAPINGANT_EXTRACT_CONTENT_AS_MARKDOWNThis tool extracts content from a given URL and converts it into Markdown format. It is particularly useful for preparing text for Language Learning Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It supports GET, POST, PUT, and DELETE methods.Input Parameters
List of resource types to block (e.g., image, script, stylesheet, font, media, websocket, other).
Returns the raw HTML as received from the server, without JavaScript rendering. Default is false.
Output
Extract Data with AI
Extract Data with AI
Slug:
SCRAPINGANT_EXTRACT_DATA_WITH_AIThis tool allows you to extract structured data from a web page using ScrapingAnt’s AI-powered extraction capabilities. You provide a URL and an AI query (prompt) describing what data you want to extract, and the tool returns the extracted data in a structured format. It supports additional parameters for browser rendering, proxies, and cookies to handle dynamic content and localization.Input Parameters
Output
Get API Credits Usage
Get API Credits Usage
Get V1 Usage (Deprecated)
Get V1 Usage (Deprecated)
Scrape Web Page
Scrape Web Page
Slug:
SCRAPINGANT_SCRAPE_WEB_PAGEThis tool scrapes a web page using the ScrapingAnt API. It fetches the HTML content of the specified URL. Users can customize the scraping behavior by enabling a headless browser, using proxies, waiting for specific elements, executing JavaScript, passing cookies, and blocking certain resources.Input Parameters
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
Enable to return the raw HTML from the server without JavaScript rendering. Requires headless browser. Defaults to False.
Output
Scrape Webpage via POST
Scrape Webpage via POST
Slug:
SCRAPINGANT_SCRAPE_WEBPAGE_POSTTool to perform a POST request through ScrapingAnt’s proxy to scrape a webpage. Use when you need to scrape pages that require POST method, such as form submissions or APIs that only accept POST requests. Data is forwarded transparently to the target web page.Input Parameters
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
Cookies to pass with the scraping request to the target site. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
POST data to send to the target web page. This will be forwarded transparently to the target. Useful for form submissions and APIs requiring POST requests.
Specifies the type of proxy to use for the request. Defaults to datacenter.
List of resource types to block. Prevents cloud browser from loading specified resource types. Requires browser=True.
Enable to return the raw HTML from the server without JavaScript rendering. Requires browser=True. Defaults to False.
Output
Scrape Webpage with PUT
Scrape Webpage with PUT
Slug:
SCRAPINGANT_SCRAPE_WEBPAGE_PUTTool to perform a PUT request through ScrapingAnt’s proxy to scrape a webpage that requires PUT method. Use when the target webpage requires PUT method for data submission. Data is forwarded transparently to the target web page.Input Parameters
Enable to use a headless browser for scraping. Defaults to True. If False, JavaScript will not be rendered.
Cookies to pass with the scraping request to the target site. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
Base64 encoded JavaScript snippet to execute on the page once loaded. Requires headless browser.
Request body data to send with the PUT request. This data will be forwarded transparently to the target web page.
Specifies the country for the proxy. If not specified, request will be made from a random country.
Enable to return the raw HTML from the server without JavaScript rendering. Requires headless browser. Defaults to False.
Output
Scrape Webpage (v1 POST - Deprecated)
Scrape Webpage (v1 POST - Deprecated)
Slug:
SCRAPINGANT_SCRAPE_WEBPAGE_V1_POST[DEPRECATED - Use v2] Tool to scrape a webpage using POST method with ScrapingAnt’s v1 API. Returns JSON with content, cookies, and status_code. This is the legacy v1 endpoint which is no longer actively maintained. Use the v2 endpoints for new implementations.Input Parameters
Enables using headless browser for scraping. Default: true. If false, JavaScript will not be rendered.
Cookie data to include with the scraping request. Format: cookie_name1=cookie_value1;cookie_name2=cookie_value2
Proxy classification to use. Default is datacenter. Options: datacenter or residential.
Enables returning text only content from the page. Default: false. If true, returns plain text instead of HTML.
CSS selector of the element our service will wait for before returning the result. Requires browser=true.
Output
Scrape with Extended JSON Output
Scrape with Extended JSON Output
Slug:
SCRAPINGANT_SCRAPE_WITH_EXTENDED_JSON_OUTPUTScrapes a web page and returns comprehensive data including HTML content, plain text, cookies, HTTP headers, XHR/Fetch requests, and iframe content. This tool uses ScrapingAnt’s extended endpoint which provides much richer data than standard scraping: - Full HTML and extracted plain text content - All cookies and HTTP response headers from the target page - Captured XHR/Fetch API requests made by the page (useful for finding hidden APIs) - Content from embedded iframes Best used when you need more than just the HTML - such as analyzing cookies, headers, or JavaScript API calls made by a page. For simple HTML scraping, consider using the basic scrape tool instead for lower API credit usage.Input Parameters
Enable or disable headless browser rendering. When True (default), JavaScript is executed and dynamic content is loaded. Set to False for faster scraping of static pages.
Maximum time in seconds to wait for the page to load. Must be between 5 and 60. Default is 60 seconds.
Type of proxy to use: ‘datacenter’ (faster, cheaper) or ‘residential’ (better for anti-bot sites). Default is ‘datacenter’.
Two-letter country code (ISO 3166-1 alpha-2) for geographic proxy location (e.g., ‘US’, ‘GB’, ‘DE’).
CSS selector to wait for before returning the page content. Useful for pages with dynamic content that loads after initial page load.
When True, returns the raw HTML from the server without JavaScript rendering. Useful for faster scraping when JS execution is not needed.