Skip to main content
Diffbot logo
  • Category: web scraping & browser
  • Auth: API_KEY
  • Composio-managed OAuth available? N/A
  • Tools: 35
  • Triggers: 0
  • Slug: DIFFBOT
  • Version: 20261101_00

Tools

Slug: DIFFBOT_COMBINE_ENTITY_PROFILESCombine entity information into an enhanced view using the Diffbot Knowledge Graph, matching identifying attributes such as name, email, employer, or URL. The explicit entity type parameter supports Person.

Input Parameters

string
IP address of the entity to enhance. Can be used with types Person and Organization.
array
Origin or homepage URI(s) of entity to enhance. Can be used with types Person and Organization.
array
Name(s) of the entity to enhance. Can be used with types Person and Organization. Multiple names can be provided for better matching.
string ("Person")
Valid entity types for the combine API.
string
Email address of the entity to enhance. Can be used only with type Person.
string
Phone number of the entity to enhance. Can be used with types Person and Organization.
string
Job title of the entity to enhance. Can be used only with type Person.
string
Semi-colon separated path filter to include specific fields in response JSON. Use dot notation (e.g., ‘skills.name’) or JsonPath expression (e.g., ‘$.name;$.locations.country.name’).
string
School or educational institution of the entity to enhance. Can be used only with type Person.
When true, Diffbot will attempt to search the web for origins and merge relevant results with what’s found in the Knowledge Graph.
boolean
When true, Diffbot will attempt to recrawl all origins of the identified entity and reconstruct the entity from refreshed data.
string
User-defined ID for correlation and tracking purposes.
string
Employer of the entity to enhance. Can be used only with type Person.
string (" " | "extended")
JSON mode options for response formatting.
string
Location of the entity to enhance. Can be used with types Person and Organization.
number
Enhance similarity threshold score (0.0 to 1.0). Higher values require stronger match confidence.
string
Semi-colon separated path filter to exclude specific fields from response JSON. Use dot notation or JsonPath expression.
boolean
When true, returns non-canonical facts in addition to canonical ones.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_CREATE_BULKTool to submit a bulk extract job to process multiple URLs with Extract APIs. Use when you need to process many URLs asynchronously using any Extract API. The job will process URLs in the background and provide downloadable results.

Input Parameters

string
required
Job name for identification. Must be unique.
string
required
URLs to process. Can be a single URL or comma-separated list of URLs.
string
required
Extract API endpoint URL to use for processing (e.g., ‘https://api.diffbot.com/v3/article’). The token will be added automatically.
string
Email address to notify when the job completes.
string
Webhook URL to POST to when the job completes.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_CREATE_CUSTOM_APITool to create or update the parameters and ruleset of a Custom API. Use this when you need to define custom extraction rules for specific websites that require tailored parsing logic beyond standard Diffbot APIs. Allows defining URL patterns, CSS selectors, extraction rules, and preprocessing filters to extract structured data from websites with unique layouts.

Input Parameters

string
required
The specific API being targeted. Always precede the API name with “/api/” as in “/api/article” (except for “all”)
array
An array of strings that can be added manually. The API automatically adds a note specifying when the API was last updated
array
An array of objects that defines a set of rules for the specific urlPattern-api combination
string
A URL that can be used to check that the rule still works as intended. This is the page that will load automatically when editing the ruleset in the Dashboard UI
string
Used to disable proxies (when they have been set globally), by applying the value “none”
array
An array of CSS selector strings that should be omitted from the DOM before extraction occurs
string
required
A regex pattern that defines the URLs for which the ruleset will be applied
string
Rendering options for the page (e.g., ‘mobile’, ‘desktop’)
object
X-Forward headers configuration.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_CREATE_KG_BULK_ENHANCESubmit an asynchronous bulk enhance job for Person or Organization records using entity descriptions or Knowledge Graph IDs. Returns a job ID for tracking the job and retrieving its enriched results.

Input Parameters

string
Human-readable name for the bulk job to help identify it later.
integer
Maximum number of results to return per entity. Default is 1.
boolean
If true, Diffbot will search the web for additional origins and merge results. Default is false.
boolean
If true, Diffbot will recrawl all origins and reconstruct entities from fresh data. Default is false.
array
required
List of entities to enhance. Each entity can be a Person, Organization, or a reference by ID. Minimum 1 entity required.
string (" " | "extended")
JSON mode options for enhance results.
number
Similarity threshold for matching entities (0.0 to 1.0). Higher values require closer matches.
string
Webhook URL to receive notifications when the job completes.
boolean
If true, returns non-canonical facts in results. Default is false.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_DELETE_CUSTOM_APIDelete a Custom API definition and its extraction rules for a specified base API and URL pattern.

Input Parameters

string
required
Base API of the custom API to delete. Always precede the API name with ‘/api/’ (e.g., ‘/api/article’)
string
required
URL pattern (regex) of the custom API to delete. This defines which URLs the custom API applies to.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_DELETE_KG_ENHANCE_BULKJOBTool to delete an Enhance Bulkjob. Removes the bulk job and its results from the system. Use when cleaning up completed or failed jobs.

Input Parameters

string
required
Enhance Bulkjob ID to delete (e.g., ‘B-a6a72339-3af7’)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_DOWNLOAD_BULK_RESULTSDownload bulk enhance job results via POST, with filtering and export options for JSON, JSONL, CSV, XLS and XLSX. An HTTP 201 response is reported as executing but may also indicate a stopped job.

Input Parameters

integer
Starting index for pagination (offset). Should be used together with ‘size’ parameter.
integer
Return first n results. Use this to limit the number of results returned.
integer
Maximum number of results to return. Should be specified with ‘from’ parameter for pagination.
integer
Seconds to wait for bulkjob results to export. Results will continue to export in the background. Use 0 to only trigger an export without waiting.
string
Semi-colon separated path filter to filter response json. You can use simple dot notation like ‘skills.name’ or JsonPath expressions like ‘$.name;$.locations.country.name’.
string ("json" | "jsonl" | "csv" | "xls" | "xlsx")
Export format options for bulk results.
string
required
Enhance Bulkjob ID (e.g., ‘B-89cfc3b2-e744’). This is the unique identifier of the bulk job whose results you want to download.
string
File name of the export file. Specify a custom filename for the exported results.
string
The spec defines the columns to export. This is applicable for csv, xls and xlsx formats. Simple spec looks like ‘name;summary’. For complex specs including list handling, see Diffbot documentation.
boolean
Prefixes the enhance query parameters to the CSV export result. Only applicable for CSV exports.
boolean
Return only records that have a match. Use this to filter out records without matches.
string
Semi-colon separated path filter to exclude data from response json. You can use simple dot notation or JsonPath expressions.
string
Separator for multi-value fields when exporting columnar results (csv, xls, xlsx formats).

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_ENHANCE_ENTITYEnrich a person or organization with comprehensive data from the Diffbot Knowledge Graph. Provide identifiers like name, email, employer, or URL and receive detailed entity information including employment history, education, location, skills, and more. Use when you need to gather all publicly available knowledge about a specific person or organization from billions of web pages.

Input Parameters

string
DiffbotId of entity to enhance. Can be used with types Person and Organization. If you know the exact entity ID, this is the most precise identifier
string
IP address of the entity to enhance. Can be used with types Person and Organization
array
Origin or homepage URI of entity to enhance. Can be used with types Person and Organization. Provide multiple URLs associated with the entity
array
Name of the entity to enhance. Can be used with types Person and Organization. Provide multiple name variations to increase match accuracy
integer
Maximum number of results to return (default=1). Set higher to get multiple potential matches
string ("Person" | "Organization")
Type of entity to enhance.
array
Email address(es) of the entity to enhance. Can be used only with type Person. Provide multiple emails if available
string
Phone number of the entity to enhance. Can be used with types Person and Organization
string
Job title of the entity to enhance. Can be used only with type Person
string
Semi-colon separated path filter to include only specific fields in response JSON. Use dot notation (e.g., ‘skills.name’) or JsonPath expressions (e.g., ‘$.name;$.locations.country.name’). Reduces response size
string
School or educational institution of the entity to enhance. Can be used only with type Person
boolean
If true, Diffbot will attempt to search the web for origins for the search query and merge relevant results with what’s found in the KG (default=false). Useful when entity might not be in the KG yet
boolean
If true, Diffbot will attempt to recrawl all origins of the identified entity and reconstruct the entity from refreshed data (default=false). This provides the most up-to-date information but takes longer
string
User-defined ID for correlation and tracking purposes. Will be returned in the response for request matching
string
Employer of the entity to enhance. Can be used only with type Person. Helps identify the correct person when names are common
string (" " | "extended")
JSON mode for response formatting.
string
Location of the entity to enhance. Can be used with types Person and Organization. Can be city, state, country, or full address
number
Enhance similarity threshold (0.0 to 1.0). Only return matches with similarity score above this threshold. Higher values return fewer but more confident matches
string
Description of the entity to enhance. Can be used with types Person and Organization. Helps identify the correct entity
string
Semi-colon separated path filter to exclude specific fields from response JSON. Use dot notation or JsonPath expressions. Useful to remove verbose fields
boolean
If true, returns non-canonical facts in addition to canonical facts (default=false). Non-canonical facts are alternative values found across multiple sources

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_EXTRACT_JOBTool to extract structured job posting data from job listing pages. Returns job title, company, location, salary, requirements, skills, and other job-related information. Use when you need to parse and structure data from job postings.

Input Parameters

string
required
Target URL of the job listing page to extract structured data from
string
IP address of a custom proxy that will be used to fetch the target page. Leave empty to use default proxy
string
Comma-separated list of optional fields to be returned from any fully-extracted pages (e.g. ‘querystring,links’). Valid values: links, extlinks, meta, querystring, breadcrumb
integer
Maximum time in milliseconds to wait for the retrieval/fetch of content from the requested URL. Default is 30000 (30 seconds)
string
Set to ‘default’ to use Diffbot’s datacenter proxy for this request. Set to ‘none’ to instruct Extract to not use proxies
string
Authentication parameters for the custom proxy specified in the proxy parameter (format: username:password)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_EXTRACT_LISTTool to extract structured data from list-style pages like news indexes, product listings, and directory pages. Returns an array of items with their titles, links, and descriptions. Use when you need to extract multiple items from a page organized as a list or index.

Input Parameters

string
required
Target URL to extract list data from. Must be a valid URL starting with http or https.
string
Specify an IP address of a custom proxy that will be used to fetch the target page.
string
Comma-separated list of optional fields to be returned from any fully-extracted pages (e.g., ‘links,meta,querystring’). Valid values: links, extlinks, meta, querystring, breadcrumb.
integer
Sets a value in milliseconds to wait for the retrieval/fetch of content from the requested URL. The default timeout for the third-party response is 30 seconds (30000).
string
Set to ‘default’ to use Diffbot’s datacenter proxy for this request. ‘none’ will instruct Extract to not use proxies, even if proxies have been enabled for this particular URL globally.
string
Used to specify the authentication parameters that will be used with a custom proxy specified in the &proxy parameter.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_ACCOUNTRetrieve Diffbot account details, including subscription plan credits, account status, account holder information, and daily usage history. Use this to monitor API usage or review account metadata.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_ANALYZEAutomatically analyze a web page to determine its type and extract structured data. When the response contains extracted objects, returns the first object; useful for URLs whose page type is unknown.

Input Parameters

string
required
The full URL of the page to analyze, including http:// or https://
string
Restrict extraction to a specific page type. Options: article, product, discussion, image, video, list, event. If not specified, all types are considered.
string
Comma-separated list of additional fields to include or limit output fields. Options include: links, extlinks, meta, querystring, breadcrumb, or specific field names to limit output.
integer
Maximum time (in milliseconds) to wait for API response. Default is 30000ms (30 seconds). Maximum is 300000ms (5 minutes).
string
API to use if page type cannot be determined. Options: article, product, discussion, image, video.
boolean
Set to false to disable automatic extraction of comments/discussions from the page. Default is true.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_ARTICLETool to extract information from articles, including authors, publication dates, and images. Use when you need structured metadata from a web article URL.

Input Parameters

string
required
Full URL of the web page to analyze, must start with http or https
string
Extraction mode override (defaults to ‘article’)
boolean
Whether to include statistics like word count
string
List of specific fields to include in the response. If provided, only these fields are returned.
string
Paging token for multi-page articles (returned in previous response)
integer
Maximum time in milliseconds to wait for page rendering
boolean
Whether to include discussion/comment data in the response

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_BULK_DATATool to download extracted results from a completed bulk job. Use after a bulk job has finished processing to retrieve the data. Supports JSON and CSV formats.

Input Parameters

integer
Limit results to the N most recently processed URLs. Useful for testing or sampling large result sets.
string
required
Name of the bulk job whose results you want to download. Must match the job name specified when the job was created.
string ("data" | "urls")
Type of data to retrieve.
string ("json" | "csv")
Output format for bulk job data.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_BULK_JOB_STATUSTool to poll the status of a specific Diffbot Knowledge Graph Enhance bulk job. Use when you need to check the progress, completion status, or details of a bulk enhancement job.

Input Parameters

string
required
Enhance Bulkjob ID to poll status for

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_BULK_RESULTSTool to download the results of a completed Enhance Bulkjob. Returns enriched records from the bulk job. Use after a bulk enhance job has completed processing.

Input Parameters

integer
Starting index for pagination (use with size parameter)
integer
Return first n results
integer
Maximum number of results to return. Should be specified with from parameter. Use -1 for all results.
integer
Seconds to wait for bulkjob results to export. Results will continue to export in the background. Use 0 to only trigger an export without waiting.
string
Semi-colon separated path filter to filter response json. Use dot notation like ‘skills.name’ or JsonPath expressions like ‘$.name;$.locations.country.name’.
string ("json" | "jsonl" | "csv" | "xls" | "xlsx")
Export format options.
string
required
Enhance Bulkjob ID to retrieve results for
string
File name of the export file
string
Defines the columns to export for csv, xls and xlsx formats. Use semi-colon separated field paths like ‘name;summary’ or JsonPath expressions. See Diffbot documentation for advanced syntax.
string ("true" | "false")
Enum for exportquery parameter.
string ("true" | "false")
Enum for onlyMatches parameter.
string
Semi-colon separated path filter to exclude data from response json. Use dot notation or JsonPath expressions.
string
Separator for multi-value fields when exporting columnar results

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_BULK_SINGLE_RESULTTool to download the result of a single job within a Diffbot bulk enhance job. Returns enriched entity data for a specific input record by its index. Use after a bulk enhance job has completed to retrieve individual results without downloading the entire dataset.

Input Parameters

integer
required
Job index within the bulkjob (0-based index of the input record)
string
required
Enhance Bulkjob ID (e.g., ‘B-1ff60452-8421’)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_CRAWL_DATADownload extracted results from a completed crawl job, optionally limiting the number of results. JSON results are returned as objects; CSV content is returned as text in a single object.

Input Parameters

integer
Maximum number of results to return. Omit to return all available results.
string
required
Name of the crawl job to retrieve data from. Must be an existing crawl job created via Start Crawl.
string ("json" | "csv")
Output format for crawl data.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_DISCUSSIONExtract structured discussion threads from web pages, including forums, comment sections and product reviews. Results may include posts, authors, timestamps and parent relationships; by default the request uses one page, with an option to concatenate more pages.

Input Parameters

string
required
The URL of the discussion page to process (e.g., forum thread, Reddit discussion, product review page, or comment section).
string
Comma-separated list of additional fields to include in the response (e.g., ‘sentiment’, ‘links’, ‘meta’, ‘breadcrumb’).
integer
Maximum time in milliseconds to wait for the response. Default is 30000 (30 seconds).
string
Maximum number of pages to concatenate. Default is 1 (no concatenation). Set to ‘all’ to retrieve all pages in the thread. Each page counts as a separate API call.
boolean
Whether to disable full page rendering. Set to True for faster responses with potentially lower extraction quality. Default is False.
boolean
Whether to extract comments/reviews. Set to False to disable comment extraction for faster response times. Default is True.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_EVENTTool to extract event details from web pages. Use when you need structured event data such as venue, date, and description.

Input Parameters

string
required
URL of the event page to analyze
string
Comma-separated list of fields to return, e.g., title,date,location
boolean
Enable automatic paging of results
integer
Maximum timeout in milliseconds for the API call

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_IMAGETool to extract detailed information about images, including dimensions and recognition data. Use after confirming the image URL is publicly accessible.

Input Parameters

string
required
Publicly-accessible URL of the image to analyze
string
Comma-separated list or array of specific fields to include in response, e.g., ‘naturalWidth’,‘captions’
boolean
Whether to include paging information for multi-image responses
integer
Maximum time to wait for API response, in milliseconds

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_KG_COVERAGE_REPORT_BY_IDDownload Knowledge Graph coverage report by report ID. Returns detailed CSV coverage statistics showing field presence across query results. Use this after generating a coverage report from a DQL query to retrieve the statistical breakdown of field coverage.

Input Parameters

string
required
Report ID to retrieve. Format: C-<hash> (e.g., ‘C-a0b39dad-68bf’). Reports may expire quickly, so use immediately after generation.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_PRODUCTTool to extract product information such as specifications, prices, availability, and reviews. Use when you need structured product data including specs, pricing, and reviews.

Input Parameters

string
required
URL of the product page to analyze
string
Extraction mode override (defaults to ‘product’)
array
List of fields to return, e.g., title,offerPrice,images
boolean
Enable automatic paging of results
integer
Maximum timeout in milliseconds for the API call
boolean
Include discussions/comments in the response

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_GET_VIDEOTool to extract information from videos, including titles, descriptions, and embedded HTML. Use when you need structured video metadata from any web page.

Input Parameters

string
required
Full URL of the web page to analyze for embedded videos, must start with http or https
string
Extraction mode override (e.g., ‘auto’)
string
Comma-separated list or array of optional fields to include in the response (e.g., ‘links’, ‘meta’, ‘querystring’, ‘breadcrumb’). Standard fields are always returned.
boolean
Whether to return all detected results in one call (may increase runtime)
integer
Maximum time in milliseconds to wait for extraction
string
Name of the JSONP callback function (if using JSONP)
boolean
Whether to try an alternate extraction method if the primary fails
boolean
Include user discussion data (comments) if available

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_LIST_BULK_JOBSTool to list all Bulk jobs associated with a specific token. Use after authenticating to retrieve statuses of all jobs for the account.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_LIST_BULK_JOBS_STATUS_FOR_TOKENTool to get the status of all bulk enhance jobs for a token. Returns list of all bulk jobs associated with your API token. Use when you need to monitor or retrieve the status of multiple bulk jobs at once.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_LIST_CUSTOM_APISTool to retrieve all Custom APIs and their extraction rules currently defined on your Diffbot token. Use when you need to list, review, or audit custom API configurations for your account.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_MANAGE_CRAWLManages Diffbot crawl jobs: pause, restart, delete, or view status. Returns list of all active crawl jobs when called without parameters. Use ‘name’ parameter with action flags (pause=1, restart=1, delete=1) to control specific jobs.

Input Parameters

string
Unique identifier of the crawl job to manage. Omit to list all active crawl jobs.
integer
Set to 1 to pause the specified crawl job. Requires ‘name’ parameter.
integer
Set to 1 to delete the specified crawl job. Requires ‘name’ parameter.
integer
Set to 1 to restart the specified crawl job. Requires ‘name’ parameter.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_RESOLVE_LOST_IDTool to resolve lost IDs in the Knowledge Graph. Use when you need to map a lost identifier to its canonical counterpart for data consistency.

Input Parameters

string
The type of object (e.g., ‘article’, ‘product’). If omitted, Diffbot will attempt to infer.
string
required
The lost ID which needs to be resolved to a canonical ID.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_SEARCHSearch the Diffbot Knowledge Graph using DQL (Diffbot Query Language). Query billions of entities including organizations, people, articles, products, and more. Use structured queries to filter by type, fields, and relationships.

Input Parameters

string
Comma-separated list of custom crawl collections to search (default=“all”). Only used when query_type=“crawl”.
integer
Maximum number of results to return (default=50). Use -1 to return all results. Constraint: from+size ≤ 10,000 for facet queries.
string
required
DQL (Diffbot Query Language) query to search the Knowledge Graph. Use structured syntax to filter entities and documents.
string
Path filter to include a specific field in the response JSON, using dot notation or JsonPath. Only a single path is accepted by the Diffbot API.
string
Output format. Only “json” is supported by this action. Non-JSON formats (jsonl, csv, xls, xlsx) cannot be processed and will be ignored.
integer
Starting index for pagination (API param ‘from’; default=0). Constraint: from+size ≤ 10,000 for facet queries.
string
JSON mode: “extended” (includes origin info) or “id” (returns diffbotIds only).
string
Type of query: “query” (default, structured DQL), “text” (free-text search), “queryTextFallback” (tries DQL first, falls back to text), or “crawl” (search custom crawl collections).
string
Path filter to exclude a specific field from the response JSON. Only a single path is accepted by the Diffbot API.
boolean
Include non-canonical facts in results (default=false).

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_SEARCH_CRAWL_DATATool to query crawl job collections using DQL (Diffbot Query Language). Use when you need to search extracted data from completed crawl or bulk jobs by collection name.

Input Parameters

string
required
Name of the collection (Crawl or Bulk job name) to search
integer
Number of results to return per page (default varies by API)
string
required
Search query string using Diffbot Query Language (DQL). Supports operators like type:Article, sortby:date, etc.
integer
Pagination offset - number of results to skip (default=0)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_START_BULKStart an asynchronous Bulk Extract job for a list of URLs using a specified Extract API. The response identifies the matching job by name, or the most recently created job if no name match is found.

Input Parameters

string
required
Unique job name for identification. Required by the Diffbot API.
array
required
List of page URLs to process. URLs should be separated by whitespace in the API call.
string
required
Diffbot Extract API endpoint to use (e.g., ‘https://api.diffbot.com/v3/article’). Do NOT include the token - it will be added automatically.
integer
Maximum number of URLs to crawl.
string
Email to notify when job completes.
integer
Maximum number of URLs to process.
string
Webhook URL to POST on job completion.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_START_CRAWLInitiates a Diffbot crawl job that spiders a website starting from seed URLs and processes discovered pages with a specified Extract API. The crawler follows links within the domain, collects structured data (articles, products, etc.), and stores results for download. Use this to systematically extract data from entire websites or sections. Requires Diffbot Plus plan or higher.

Input Parameters

string
required
Unique identifier for the crawl job. Used to manage and retrieve the crawl.
array
required
List of seed URLs where crawling will begin. URLs will be URL-encoded automatically.
string
required
Full Diffbot Extract API endpoint URL to process crawled pages. Examples: ‘https://api.diffbot.com/v3/article’ for articles, ‘https://api.diffbot.com/v3/product’ for products, ‘https://api.diffbot.com/v3/analyze’ for automatic type detection.
number
Number of days between automatic crawl repeats. Use 7.0 for weekly, 1.0 for daily. Omit for one-time crawl.
number
Delay in seconds between requests to the same IP address. Default is 0.25 seconds.
integer
Maximum number of pages to crawl/spider. Default is 100,000. Use -1 for unlimited.
integer
Whether to respect robots.txt directives. 1 = obey (default), 0 = ignore.
string
Email address to notify when the crawl completes.
integer
Maximum number of pages to process with the Extract API. Default is 100,000. Use -1 for unlimited.
object
Custom HTTP headers to include in crawl requests (e.g., {‘User-Agent’: ‘MyBot/1.0’}).
string
Webhook URL to POST to when the crawl completes.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_STOP_BULK_JOBTool to pause (stop) a running Bulk job. Pausing halts further processing of URLs while preserving existing progress. To resume, use the appropriate resume action. Specify the exact job name (case-sensitive) as provided when the job was created.

Input Parameters

string
required
Name of the Bulk job to pause/stop (as defined when creating the job)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: DIFFBOT_STOP_KG_BULK_JOB_BY_IDTool to stop an active Knowledge Graph Enhance bulk job by its ID. Halts processing of a running KG bulk job immediately. Use when you need to stop a specific KG bulk job using its bulkjobId.

Input Parameters

string
required
The unique identifier of the Knowledge Graph Enhance bulk job to stop (e.g., ‘B-1ff60452-8421’)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not