> ## Documentation Index
> Fetch the complete documentation index at: https://composio-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawlbase

> Crawlbase provides APIs for crawling web pages, managing crawler queues, storing crawl results, and monitoring account usage.

![Crawlbase logo](https://mintcdn.com/composio-preview/NPf8agM6UrvZD-Iu/images/toolkits/crawlbase.svg?fit=max\&auto=format\&n=NPf8agM6UrvZD-Iu\&q=85\&s=be879b6b7cb4457d5002200d76125130)

* **Category:** web scraping & browser
* **Auth:** API\_KEY
* **Composio-managed OAuth available?** N/A
* **Tools:** 8
* **Triggers:** 0
* **Slug:** `CRAWLBASE`
* **Version:** 20261030\_00

## Tools

> [accordion] Crawl URL
>
> **Slug:** `CRAWLBASE_CRAWL_URL`
>
> Fetch a public HTTP or HTTPS URL through Crawlbase using the connected Normal token and return its content with separate target and Crawlbase statuses.
>
> #### Input Parameters
>
> **url** `string` *required*
>
> Public HTTP or HTTPS URL to crawl.
>
> **store** `boolean`
>
> Store the crawl result in Crawlbase Cloud Storage when true.
>
> **device** `string ("desktop" | "tablet" | "mobile")`
>
> Device profile used for the request.
>
> **cookies** `string`
>
> Optional cookie string to send to the target.
>
> **country** `string`
>
> Optional country code for geolocated crawling.
>
> **scraper** `string`
>
> Optional Crawlbase scraper name for structured extraction.
>
> **autoparse** `boolean`
>
> Ask Crawlbase to automatically parse supported pages.
>
> **user\_agent** `string`
>
> Optional User-Agent value to send to the target.
>
> **tor\_network** `boolean`
>
> Route through the Tor network when true.
>
> **output\_format** `string ("html" | "json" | "markdown")`
>
> Content representation to request. Markdown uses Crawlbase's md format.
>
> **cookie\_session** `string`
>
> Optional Crawlbase cookie-session identifier, at most 32 characters.
>
> **request\_headers** `object`
>
> Optional headers to send to the target URL.
>
> **custom\_success\_codes** `array`
>
> Additional target HTTP status codes Crawlbase should treat as successful.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Delete Stored Pages
>
> **Slug:** `CRAWLBASE_DELETE_STORED_PAGES`
>
> Irreversibly delete explicitly selected pages from Crawlbase Cloud Storage by request ID and return each page's deletion outcome.
>
> #### Input Parameters
>
> **rids** `array` *required*
>
> One to 100 unique request IDs of stored pages to delete irreversibly.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Generate User Agents
>
> **Slug:** `CRAWLBASE_GENERATE_USER_AGENTS`
>
> Generate one to ten realistic randomized User-Agent strings for a selected device class.
>
> #### Input Parameters
>
> **size** `integer`
>
> Number of User-Agent strings to return, from 1 to 10.
>
> **device** `string ("desktop" | "mobile")`
>
> Device class for generated User-Agent strings.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Crawling Usage
>
> **Slug:** `CRAWLBASE_GET_CRAWLING_USAGE`
>
> Return current Crawling API usage and optionally previous-month statistics for the connected Crawlbase account.
>
> #### Input Parameters
>
> **include\_previous\_month** `boolean`
>
> Include previous-month crawling usage statistics when available.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Storage Count
>
> **Slug:** `CRAWLBASE_GET_STORAGE_COUNT`
>
> Return the number of pages currently held in Crawlbase Cloud Storage.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Stored Page
>
> **Slug:** `CRAWLBASE_GET_STORED_PAGE`
>
> Retrieve the latest Crawlbase Cloud Storage page by exactly one request ID or original URL.
>
> #### Input Parameters
>
> **rid** `string`
>
> Stored request ID. Provide exactly one of rid or url.
>
> **url** `string`
>
> Original HTTP or HTTPS URL. Provide exactly one of url or rid.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] Get Stored Pages
>
> **Slug:** `CRAWLBASE_GET_STORED_PAGES`
>
> Retrieve up to 100 Crawlbase Cloud Storage pages by request ID without deleting them. Returns decoded page content and identifies requested IDs that were not found.
>
> #### Input Parameters
>
> **rids** `array` *required*
>
> One to 100 unique request IDs for stored pages to retrieve.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not

> [accordion] List Stored Page IDs
>
> **Slug:** `CRAWLBASE_LIST_STORED_PAGE_IDS`
>
> Return one page of Crawlbase Cloud Storage request IDs and a short-lived continuation cursor.
>
> #### Input Parameters
>
> **limit** `integer`
>
> Maximum number of stored request IDs to return, from 1 to 1000.
>
> **order** `string ("asc" | "desc")`
>
> Order stored request IDs from oldest to newest (asc) or newest to oldest (desc).
>
> **next\_cursor** `string`
>
> Short-lived cursor from the previous response. Omit for the first page and use a returned cursor within about 15 seconds.
>
> #### Output
>
> **data** `string` *required*
>
> Data from the action execution
>
> **error** `string`
>
> Error if any occurred during the execution of the action
>
> **successful** `boolean` *required*
>
> Whether or not the action execution was successful or not
