Skip to main content
Honeyhive logo
  • Category: artificial intelligence
  • Auth: API_KEY
  • Composio-managed OAuth available? N/A
  • Tools: 42
  • Triggers: 0
  • Slug: HONEYHIVE
  • Version: 20261101_00

Tools

Slug: HONEYHIVE_ADD_DATAPOINTS_TO_DATASETTool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.

Input Parameters

array
required
List of JSON objects representing datapoints to add
object
required
Mapping of data fields to inputs, ground truth, and history
string
required
Project name associated with the dataset
string
required
Dataset identifier to which datapoints will be added

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_COMPARE_RUNSTool to retrieve experiment comparison between two evaluation runs. Use when you need to analyze the differences in metrics, datapoints, and events between two runs.

Input Parameters

string
Optional filters to apply (JSON string or array of filter objects)
string
required
New experiment run ID to compare (UUIDv4)
string
required
Old experiment run ID to compare against (UUIDv4)
string
required
Project ID to scope the comparison
string ("average" | "min" | "max" | "median" | "p95" | "p99" | "p90" | "sum" | "count")
Aggregation function to apply to metrics. Defaults to ‘average’ if not specified

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_COMPARE_RUNS_EVENTSTool to compare events between two experiment runs side-by-side. Use when analyzing differences in model behavior, performance metrics, or outputs between evaluation runs. Returns matched event pairs with their respective data from both runs for comparison.

Input Parameters

integer
Page number for pagination (default 1)
integer
Maximum number of event pairs to return (default 1000)
string
Optional additional filter criteria as JSON string or query object to narrow comparison results
string
required
First experiment run ID (UUIDv4) to compare. Get this from Start Evaluation Run or list runs endpoint.
string
required
Second experiment run ID (UUIDv4) to compare. Get this from Start Evaluation Run or list runs endpoint.
string
Optional filter to only compare events with this specific name
string
Optional filter to only compare events of this type

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_BATCH_DATAPOINTSTool to create multiple datapoints in a single batch operation. Use when you need to bulk-import events into a dataset or create many datapoints at once. Supports filtering by date range, event IDs, or custom criteria. Efficient for migrating large numbers of events to evaluation datasets.

Input Parameters

array
List of event IDs to convert into datapoints. Use to import events as datapoints.
string
Filter criteria to select events or datapoints. Can be an object or array depending on the filtering logic.
object
Mapping configuration for datapoint fields
object
Date range filter for batch datapoint creation
boolean
If true, selects all matching items based on filters. Use to create datapoints from all events in a date range.
object
State flags to control batch operation behavior. Keys represent state attributes.
string
required
Unique identifier of the target dataset where datapoints will be created. Required.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_BATCH_MODEL_EVENTSTool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.

Input Parameters

array
required
Array of model event objects to create

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_BATCH_TOOL_EVENTSTool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.

Input Parameters

array
required
Array of tool event objects to create
boolean
If true, all events in the batch are associated with the same session. Defaults to false.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_CONFIGURATIONCreates a new configuration in HoneyHive for managing LLM or pipeline settings. Use this to define reusable configurations with specific models, prompts, and parameters that can be deployed across different environments (dev, staging, prod). Configurations enable version control and environment-specific management of your AI application settings.

Input Parameters

array
List of environments where this configuration should be active. If not specified, the configuration will not be tied to any specific environment.
string
required
Name of the configuration. Must be unique within the project and no longer than 200 characters.
array
List of tags for categorizing and filtering configurations.
string ("LLM" | "pipeline")
Type of configuration.
string
required
Provider name such as ‘openai’, ‘anthropic’, ‘cohere’, etc.
object
required
Configuration parameters including model, call_type, and optional settings.
object
Additional custom properties for tracking metadata about the configuration.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_DATAPOINTTool to create a new datapoint with input-output pairs. Use when you need to add a single datapoint with inputs, ground truth, conversation history, and metadata.

Input Parameters

object
Input data for the datapoint as key-value pairs (e.g., {‘prompt’: ‘What is AI?’, ‘context’: ’…’})
array
Conversation history as a list of message objects with ‘role’ and ‘content’ fields
object
Additional metadata for the datapoint (e.g., token counts, custom fields)
object
Expected output/ground truth as key-value pairs (e.g., {‘response’: ‘Paris’})
string
ID of the event this datapoint is created from, if any
array
List of dataset IDs to link this datapoint to

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_DATASETTool to create a dataset. Use when you need to initialize a new dataset within a project.

Input Parameters

string
required
Name of the dataset
string ("evaluation" | "fine-tuning")
What the dataset is used for
boolean
Flag indicating if the dataset is saved
string
required
Name of the project associated with this dataset
object
Any helpful metadata to track for the dataset
array
List of unique datapoint IDs to include in the dataset
string
A description for the dataset
array
List of unique evaluation run IDs to associate with the dataset
string ("event" | "session")
Type of data pipeline

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_EVENTTool to create a new event in HoneyHive to track execution of different parts of your application. Use when you need to log a model call, tool execution, or chain step. Events can be grouped into sessions and nested hierarchically using parent_id and children_ids.

Input Parameters

object
required
Event data object containing all event properties

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_METRICTool to create a new metric in HoneyHive. Use when you need to define how to evaluate model outputs, whether through code (PYTHON), AI evaluation (LLM), human review (HUMAN), or combining multiple metrics (COMPOSITE). Important: LLM metrics require both model_provider and model_name to be specified.

Input Parameters

string
required
Name of the metric. Must be unique within the project and descriptive of what is being measured.
string ("PYTHON" | "LLM" | "HUMAN" | "COMPOSITE")
required
Type of metric evaluation. PYTHON: code-based, LLM: AI-evaluated, HUMAN: manually evaluated, COMPOSITE: combination of other metrics.
integer
Scale for rating-type metrics (e.g., 1-5 scale, 1-10 scale)
object
Filter conditions to apply before computing the metric
string
required
Evaluation criteria that defines what this metric measures and how it should be assessed. Required for all metric types.
object
Threshold settings for determining when a metric passes or fails.
array
List of categories with scores for categorical metrics. Required when return_type is ‘categorical’.
string
Specific model name for LLM-evaluated metrics (e.g., ‘gpt-4’, ‘claude-3-opus’). Required when type is LLM.
string
Detailed description of what the metric measures and when to use it
string ("float" | "boolean" | "string" | "categorical")
Data type of the metric’s return value
array
List of child metrics with weights for composite metrics. Required when type is ‘COMPOSITE’.
string
Model provider for LLM-evaluated metrics (e.g., ‘openai’, ‘anthropic’). Required when type is LLM.
boolean
Whether to automatically compute this metric on production events
boolean
Whether ground truth data is required to compute this metric
number
Percentage of events to sample for this metric (0-100)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_MODEL_EVENTTool to create a new model event to log LLM call data. Use when you need to track a single model interaction including messages, responses, usage, and metadata.

Input Parameters

object
required
Model event object containing LLM call data

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_CREATE_TOOLCreates a new tool definition in a HoneyHive project. Use this to register functions or plugins that can be invoked and tracked within HoneyHive. Tools are defined with a JSON Schema for their parameters, allowing HoneyHive to validate inputs and track tool usage in your AI workflows. Tool names must be unique within a project.

Input Parameters

string
required
Unique name for the tool within the project. Must contain only alphanumeric characters and underscores (no special characters or unicode).
string
required
Project name or ID where this tool will be registered. Must match an existing HoneyHive project.
string ("function" | "tool")
required
Type of the tool. Use ‘function’ for callable functions or ‘tool’ for plugins/integrations.
object
required
JSON Schema object defining the tool’s input parameters. Use standard JSON Schema format with ‘type’, ‘properties’, and optionally ‘required’ fields.
string
Human-readable description explaining what the tool does and when to use it.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_DELETE_DATAPOINTDelete a HoneyHive datapoint by its ID. The datapoint may instead be marked as soft-deleted rather than removed.

Input Parameters

string
required
Unique identifier of the datapoint to delete. Can be obtained from retrieve_datapoints or add_datapoints_to_dataset responses.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_DELETE_DATASETDelete a HoneyHive dataset by its ID. The dataset may instead be marked as soft-deleted rather than removed, with its description changed.

Input Parameters

string
required
Unique identifier of the dataset to delete

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_END_EVALUATION_RUNTool to update an evaluation run’s status and metadata. Use to mark a run as completed after finishing evaluations, or update run properties like name, metadata, configuration, and associated event/datapoint IDs.

Input Parameters

string
Display name for the evaluation run
string
required
Unique identifier of the evaluation run to update
string ("pending" | "completed")
required
Status of the evaluation run. Set to ‘completed’ to finalize the run, or ‘pending’ to keep it open for further updates
object
Arbitrary metadata fields to attach to the run
array
List of session/event UUIDs to associate with this run
string
The UUID of the dataset this run is associated with
object
Configuration parameters used in this run
array
List of datapoint UUIDs to associate with this run

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_CONFIGURATIONSTool to retrieve a list of configurations. Use when you need to fetch all configurations for a specific project before making changes.

Input Parameters

string ("dev" | "staging" | "prod")
Environment to filter by. Allowed values: dev, staging, prod.
string
The name of the configuration to filter by, e.g. ‘v0’.
string
required
Project name for configuration like ‘Example Project’.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_DATASETSRetrieve datasets from HoneyHive for a specified project. Use this tool when you need to: - List all datasets within a project - Find datasets by type (evaluation or fine-tuning) - Retrieve a specific dataset by its ID Returns dataset details including name, description, datapoints count, type, and timestamps.

Input Parameters

string ("evaluation" | "fine-tuning")
Optional filter by dataset type. Use ‘evaluation’ for evaluation datasets or ‘fine-tuning’ for fine-tuning datasets.
string
required
Project ID to filter datasets. Obtain project IDs using the Get Projects action.
string
Optional unique dataset ID to retrieve a specific dataset. When provided, returns only the matching dataset.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_EVENTSTool to query events with filters and projections from HoneyHive. Use this action when you need to retrieve events with lightweight filtering (limit 1000 results). For bulk exports or more complex queries, use the Retrieve Events action instead. Supports filtering by date range, event properties, and field projections.

Input Parameters

integer
Page number for pagination. Default is 1
integer
Maximum number of results to return. Default is 1000
string
Array of filter objects as JSON string. Each filter should have ‘field’, ‘operator’, and ‘value’ properties (e.g., ’[{“field”:“event_type”,“operator”:“is”,“value”:“completion”}]’)
string
Date range filter as ISO string or JSON object with $gte/$lte fields (e.g., ’{“$gte”:“2023-01-01T00:00:00Z”,“$lte”:“2023-12-31T23:59:59Z”}’)
string
Whether to ignore ordering in the results. Set to ‘true’ or ‘false’ as string
string
Fields to include in response as JSON array string (e.g., ’[“event_id”,“event_type”,“metadata.cost”]’). If not specified, all fields are returned
string
Filter events by evaluation ID

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_EVENTS_BY_SESSION_IDTool to retrieve the complete tree of nested events for a specific session. Use when you need to analyze all events (model calls, tool calls, chains) that occurred within a session, including their hierarchical relationships, inputs, outputs, metrics, and costs. Returns a tree structure with recursive children.

Input Parameters

string
required
Session ID (UUIDv4 format) to retrieve all events for. This should be the session_id returned from start_session or retrieved from other event queries.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_EVENTS_CHARTTool to retrieve charting and analytics data for events over time. Use when you need aggregated metrics (duration, cost, token usage) grouped by time buckets or fields. Supports percentile analysis (p50, p95, p99) for latency monitoring and custom filters for targeted analytics.

Input Parameters

string ("minute" | "minutes" | "1m" | "hour" | "hours" | "1h" | "day" | "days" | "1d" | "week" | "weeks" | "1w" | "month" | "months" | "1M")
Time bucket options for aggregating chart data.
string
Metric to aggregate for charting (default ‘duration’). Common metrics: duration, metadata.cost, metadata.tokens
string
Array of filter objects as JSON string to narrow results by field, operator, and value
string
Field to group chart data by (e.g., event_type, event_name, metadata.model)
string
Date range filter as ISO string or JSON object with $gte/$lte fields for start/end timestamps
string ("avg" | "average" | "mean" | "p50" | "p75" | "p90" | "p95" | "p99" | "count" | "sum" | "min" | "max" | "median")
Aggregation function options for chart metrics.
string
Filter chart data to a specific evaluation run by evaluation ID
string
Filter to include only experiment events (set to ‘true’ or ‘1’ to enable)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_METRICSRetrieves all metrics associated with a HoneyHive project. Returns a list of metrics including their configuration (name, type, description, thresholds, evaluator details) and metadata (creation/update timestamps, sampling settings). Use this tool when you need to: - List all metrics configured for a project - Get metric IDs for updating metrics via HONEYHIVE_UPDATE_METRIC - Understand what evaluations are set up for a project Prerequisites: Obtain a valid project_name using HONEYHIVE_GET_PROJECTS first.

Input Parameters

string
required
The name of the project to retrieve metrics for. This is a required filter that scopes which metrics are returned. Use HONEYHIVE_GET_PROJECTS to find valid project names.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_PROJECTSRetrieve HoneyHive projects in the account, optionally filtering by a case-insensitive name substring. Use the results to discover projects and their IDs for other API calls.

Input Parameters

string
Optional filter to return only projects whose name contains this string (case-insensitive)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_RUNTool to get details of an evaluation run by its UUID. Use when you need to check the status, configuration, results, or metadata of a specific evaluation run.

Input Parameters

string
required
UUID of the evaluation run to retrieve

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_RUN_METRICSTool to get event metrics for an experiment run. Use when you need to retrieve metrics computed on events within a specific experiment run. Returns an array of event objects with their associated metrics, which can be filtered by date range or custom filters.

Input Parameters

string
required
Experiment run ID (UUIDv4) to retrieve metrics for
string
Optional filters to apply as JSON string or array of filter objects (e.g., ’[{“field”: “event_type”, “operator”: “is”, “value”: “completion”}]’)
string
Date range filter as JSON string (e.g., ’{“$gte”: “2023-01-01T00:00:00Z”, “$lte”: “2023-12-31T23:59:59Z”}‘)

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_RUNSTool to retrieve a list of evaluation runs from HoneyHive. Use when you need to: - List all evaluation runs for analysis - Find runs by status, name, or dataset - Get specific runs by their IDs - Paginate through large sets of evaluation runs Returns evaluation details including status, results, configuration, and timestamps.

Input Parameters

string
Filter by run name. Returns runs whose name contains this string (case-sensitive).
integer
Page number for pagination (default: 1)
integer
Number of results per page (default: 20)
string ("pending" | "completed" | "failed" | "cancelled" | "running")
Enumeration of possible evaluation run statuses.
array
List of specific run IDs to fetch. When provided, only returns the specified runs.
string ("created_at" | "updated_at" | "name" | "status")
Enumeration of fields that can be used for sorting runs.
string
Filter by dataset ID. Get dataset IDs from the Get Datasets action.
string
Filter by date range. Format depends on API implementation (e.g., ISO date range or relative time).
string ("asc" | "desc")
Enumeration of sort order options.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_RUNS_SCHEMATool to retrieve the schema for experiment runs in HoneyHive. Use when you need to understand available fields, datasets, and mappings for experiment runs.

Input Parameters

string
Filter by date range
string
Filter by evaluation/run ID

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_GET_SESSIONRetrieve a complete session tree by session ID from HoneyHive. Use this tool to fetch the full session hierarchy including all nested events (model calls, tool calls, chains) with their inputs, outputs, durations, and metadata. Returns a recursive tree structure with aggregated metrics. Prerequisites: You need a valid session ID from HONEYHIVE_START_SESSION or HONEYHIVE_RETRIEVE_EVENTS.

Input Parameters

string
required
Session ID (UUIDv4) to retrieve. Obtain this from HONEYHIVE_START_SESSION or HONEYHIVE_RETRIEVE_EVENTS.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_LIST_TOOLSList registered HoneyHive tools, optionally filtered by project and paginated. Returned entries must have an ID and name; entries missing either are omitted.

Input Parameters

integer
Page number for paginated results
integer
Maximum number of tools to return per page
string
Project name (task) to filter tools by. If not provided, returns tools from all projects.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_RETRIEVE_DATAPOINTRetrieve a specific datapoint by its ID from HoneyHive. Use this tool when you need the full details of a single datapoint, including its inputs, ground truth, conversation history, linked datasets, and metadata. Prerequisites: You need a valid datapoint ID. Get datapoint IDs from: - HONEYHIVE_RETRIEVE_DATAPOINTS (list datapoints by project/dataset) - HONEYHIVE_ADD_DATAPOINTS_TO_DATASET (returns IDs of newly created datapoints)

Input Parameters

string
required
The unique datapoint ID to retrieve. Obtain this from HONEYHIVE_RETRIEVE_DATAPOINTS or HONEYHIVE_ADD_DATAPOINTS_TO_DATASET.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_RETRIEVE_DATAPOINTSRetrieve datapoints from a HoneyHive project. Use this tool to fetch evaluation datapoints containing inputs, ground truth, and metadata. Supports filtering by specific datapoint IDs or dataset name. Commonly used to: - Review existing test cases before running evaluations - Export datapoints for analysis - Verify datapoint contents after adding them to a dataset

Input Parameters

string
required
Name or ID of the HoneyHive project to retrieve datapoints from. Required parameter.
string
Filter by dataset name. When provided, returns only datapoints belonging to the specified dataset.
array
Specific datapoint IDs to retrieve. When provided, only datapoints with matching IDs are returned. Omit to retrieve all datapoints.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_RETRIEVE_EVENTSRetrieve and export events from a HoneyHive project. Use this tool to query traced events (model calls, tool calls, sessions, chains) with optional filters by event_type, metadata, feedback scores, or date range. Returns events with their inputs, outputs, duration, and metrics. Supports pagination for large result sets (max 7500 per page).

Input Parameters

integer
Page number for pagination (default 1)
integer
Limit number of events returned (default 1000; max 7500)
array
Array of filter objects to narrow results. Pass empty list to retrieve all events.
string
required
Name of the project to query
object
Date range filter for events, using ISO 8601 timestamps.
array
Fields to include in the response

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_RETRIEVE_EXPERIMENT_RESULTTool to retrieve the result of a specific experiment run. Use when you need the status, metrics, and datapoint-level details of a completed experiment.

Input Parameters

string
required
UUID of the experiment run to retrieve results for
string
required
Project identifier (ID or name) that the experiment run belongs to
string ("average" | "min" | "max" | "median" | "p95" | "p99" | "p90" | "sum" | "count")
Aggregation function for metrics calculation. Defaults to ‘average’ if not specified

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_START_EVALUATION_RUNCreates a new evaluation run to group and track multiple session events for analysis. Use this action when you want to: - Compare model performance across multiple sessions - Create evaluation batches for quality assurance - Link existing events to datasets for structured evaluation Prerequisites: - Get project ID using Get Projects action - Get event IDs from Start Session or Retrieve Events actions - (Optional) Get dataset ID from Get Datasets action

Input Parameters

string
required
Human-readable name for identifying this evaluation run in the HoneyHive dashboard
string ("pending" | "completed")
Initial status of the run. Use ‘pending’ for runs that will receive more data, ‘completed’ for finished runs.
string
required
Project ID to associate this evaluation run with. Get this from the Get Projects action.
object
Optional key-value metadata to attach to this run (e.g., environment, version)
array
required
List of session/event UUIDs to include in this evaluation. Get these from Start Session or Retrieve Events actions.
string
Optional dataset ID to link with this run. Get this from the Get Datasets action.
object
Optional configuration object for the run (e.g., model settings, parameters)
array
Optional list of specific datapoint IDs from the linked dataset to evaluate

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_START_SESSIONStart a new HoneyHive session for tracing and observability. Use this tool to initiate a tracking session that groups together related model, tool, and chain events. Returns a session_id that should be used to link subsequent events to this session. Common use cases: - Start tracing a user conversation - Begin logging an LLM pipeline execution - Initialize observability for a batch processing job

Input Parameters

object
required
Session configuration object containing all session properties

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_CONFIGURATIONTool to update an existing HoneyHive configuration. Use when you need to modify a configuration’s name, provider, model parameters, environments, or other settings. You must provide the configuration ID (obtainable via Get Configurations action) and the name field. All other fields are optional and will only update if provided.

Input Parameters

string
required
Configuration ID to update, e.g., ‘6638187d505c6812e4043f24’
array
List of environments where this configuration should be active
string
required
Name of the configuration
array
Tags to categorize the configuration
string ("LLM" | "pipeline")
Type of configuration.
string
Provider name, e.g., ‘openai’, ‘anthropic’, ‘google’
object
Parameters for the configuration.
object
Additional user-defined properties for the configuration

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_DATAPOINTUpdate an existing datapoint by ID. Use this to modify any combination of inputs, ground_truth, history, metadata, linked_datasets, or linked_evals for a datapoint. Requires a valid datapoint ID obtained from retrieve_datapoints or add_datapoints_to_dataset.

Input Parameters

string
required
Datapoint ID to update (string)
object
Arbitrary JSON object containing the inputs for the datapoint
array
Conversation history associated with the datapoint
object
Additional metadata for the datapoint
object
Expected output JSON object for the datapoint
array
IDs of evaluations where the datapoint is included
array
IDs of datasets that include the datapoint

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_DATASETTool to update an existing dataset. Use when you need to modify a dataset’s details (name, description, datapoints, linked evaluations, or metadata) after confirming its ID.

Input Parameters

string
Updated name for the dataset
object
Arbitrary metadata to attach to the dataset
array
Full list of datapoint IDs to associate with the dataset
string
required
Unique identifier of the dataset to update
string
Updated description for the dataset
array
List of evaluation run IDs to link to this dataset

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_EVENTUpdate an existing HoneyHive event by ID. Use to attach feedback, metrics, metadata, outputs, config, user properties, or update duration on events created via start_session or batch event creation. At least one optional field must be provided alongside the event_id.

Input Parameters

object
Configuration to apply/update
object
Metrics to record/update
object
Outputs to set/update for the event
number
Duration of the event in seconds
string
required
UUID of the event to update. Can be obtained from retrieve_events or start_session actions.
object
Feedback payload to attach/update
object
Additional metadata to set/update for the event
object
Custom user properties to set/update

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_METRICTool to update an existing metric. Use when you need to modify a metric’s properties after creation. Ensure you retrieve the metric first to verify its current state.

Input Parameters

string
Updated name of the metric
string ("custom" | "model" | "human")
Type of the metric
string
Updated evaluator prompt for the metric
string
Criteria for human-evaluated metrics
string
required
Unique identifier of the metric to update
boolean
Expected pass condition for boolean metrics
object
Threshold settings for numeric metrics.
string
Name of the event the metric is computed on
string ("model" | "tool" | "chain" | "session")
Type of event the metric is computed on
string
Short description of what the metric does
string ("boolean" | "float" | "string")
Expected return type of the metric
string
Updated code block for the metric
boolean
Whether to compute this metric automatically on production events
boolean
Whether a ground truth value is required to compute this metric

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_PROJECTUpdates an existing HoneyHive project’s name or description. Use this action to modify project metadata after creation. You must provide the project_id and at least one field to update (name or description). To find project IDs, use the Get Projects action first.

Input Parameters

string
The new name for the project. If not provided, the existing name is preserved.
string
required
The unique identifier of the project to update. Can be obtained from the Get Projects action.
string
The new description for the project. If not provided, the existing description is preserved.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not
Slug: HONEYHIVE_UPDATE_TOOLTool to update an existing tool in HoneyHive. Use when you need to modify a tool’s name, description, parameters, or type after confirming its ID. At least one optional field must be provided alongside the required tool ID.

Input Parameters

string
required
Unique identifier of the tool to update. Must match an existing tool ID in HoneyHive.
string
Updated name for the tool. Must contain only alphanumeric characters and underscores (no special characters or unicode).
string ("function" | "tool")
Type of tool in HoneyHive.
object
Updated JSON Schema object defining the tool’s input parameters. Use standard JSON Schema format with ‘type’, ‘properties’, and optionally ‘required’ fields.
string
Updated human-readable description explaining what the tool does and when to use it.

Output

string
required
Data from the action execution
string
Error if any occurred during the execution of the action
boolean
required
Whether or not the action execution was successful or not