llm

Grok LLM service implementation using OpenAI-compatible interface.

This module provides a service for interacting with Grok’s API through an OpenAI-compatible interface, including specialized token usage tracking and context aggregation functionality.

class pipecat.services.xai.llm.GrokLLMSettings(model: str | None | _NotGiven = <factory>, extra: dict[str, Any]=<factory>, system_instruction: str | None | _NotGiven = <factory>, temperature: float | None | _NotGiven | NotGiven = <factory>, max_tokens: int | None | _NotGiven | NotGiven = <factory>, top_p: float | None | _NotGiven | NotGiven = <factory>, top_k: int | None | _NotGiven = <factory>, frequency_penalty: float | None | _NotGiven | NotGiven = <factory>, presence_penalty: float | None | _NotGiven | NotGiven = <factory>, seed: int | None | _NotGiven | NotGiven = <factory>, filter_incomplete_user_turns: bool | None | _NotGiven = <factory>, user_turn_completion_config: UserTurnCompletionConfig | None | _NotGiven = <factory>, max_completion_tokens: int | _NotGiven | NotGiven = <factory>)[source]

Bases: OpenAILLMSettings

Settings for GrokLLMService.

class pipecat.services.xai.llm.GrokLLMService(*, api_key: str, base_url: str = 'https://api.x.ai/v1', model: str | None = None, settings: GrokLLMSettings | None = None, **kwargs)[source]

Bases: OpenAILLMService

A service for interacting with Grok’s API using the OpenAI-compatible interface.

This service extends OpenAILLMService to connect to Grok’s API endpoint while maintaining full compatibility with OpenAI’s interface and functionality. Includes specialized token usage tracking that accumulates metrics during processing and reports final totals.

Settings

alias of GrokLLMSettings

__init__(*, api_key: str, base_url: str = 'https://api.x.ai/v1', model: str | None = None, settings: GrokLLMSettings | None = None, **kwargs)[source]

Initialize the GrokLLMService with API key and model.

Parameters:

  • api_key – The API key for accessing Grok’s API.

  • base_url – The base URL for Grok API. Defaults to “https://api.x.ai/v1”.

  • model –

    The model identifier to use. Defaults to “grok-3”.

    Deprecated since version 0.0.105: Use settings=GrokLLMService.Settings(model=...) instead.

  • settings – Runtime-updatable settings. When provided alongside deprecated parameters, settings values take precedence.

  • **kwargs – Additional keyword arguments passed to OpenAILLMService.

create_client(api_key=None, base_url=None, **kwargs)[source]

Create OpenAI-compatible client for Grok API endpoint.

Parameters:

  • api_key – The API key to use. If None, uses instance default.

  • base_url – The base URL to use. If None, uses instance default.

  • **kwargs – Additional arguments passed to client creation.

Returns:

The configured client instance for Grok API.

async start_llm_usage_metrics(tokens: LLMTokenUsage)[source]

Accumulate token usage metrics during processing.

This method intercepts the incremental token updates from Grok’s API and accumulates them instead of passing each update to the metrics system. The final accumulated totals are reported at the end of processing.

Parameters:

tokens – The token usage metrics for the current chunk of processing, containing prompt_tokens, completion_tokens, and optional cached/reasoning tokens.