llm

Together.ai LLM service implementation using OpenAI-compatible interface.

class pipecat.services.together.llm.TogetherLLMSettings(model: str | None | _NotGiven = <factory>, extra: dict[str, Any]=<factory>, system_instruction: str | None | _NotGiven = <factory>, temperature: float | None | _NotGiven | NotGiven = <factory>, max_tokens: int | None | _NotGiven | NotGiven = <factory>, top_p: float | None | _NotGiven | NotGiven = <factory>, top_k: int | None | _NotGiven = <factory>, frequency_penalty: float | None | _NotGiven | NotGiven = <factory>, presence_penalty: float | None | _NotGiven | NotGiven = <factory>, seed: int | None | _NotGiven | NotGiven = <factory>, filter_incomplete_user_turns: bool | None | _NotGiven = <factory>, user_turn_completion_config: UserTurnCompletionConfig | None | _NotGiven = <factory>, max_completion_tokens: int | _NotGiven | NotGiven = <factory>)[source]

Bases: OpenAILLMSettings

Settings for TogetherLLMService.

class pipecat.services.together.llm.TogetherLLMService(*, api_key: str, base_url: str = 'https://api.together.xyz/v1', model: str | None = None, settings: TogetherLLMSettings | None = None, **kwargs)[source]

Bases: OpenAILLMService

A service for interacting with Together.ai’s API using the OpenAI-compatible interface.

This service extends OpenAILLMService to connect to Together.ai’s API endpoint while maintaining full compatibility with OpenAI’s interface and functionality.

supports_developer_role = False

Whether this service’s API supports the “developer” message role.

OpenAI’s native API supports it, but some OpenAI-compatible services (e.g. Cerebras) do not. Subclasses that don’t support it should set this to False, which causes the adapter to convert “developer” messages to “user” messages before sending them to the API.

Settings

alias of TogetherLLMSettings

__init__(*, api_key: str, base_url: str = 'https://api.together.xyz/v1', model: str | None = None, settings: TogetherLLMSettings | None = None, **kwargs)[source]

Initialize Together.ai LLM service.

Parameters:

  • api_key – The API key for accessing Together.ai’s API.

  • base_url – The base URL for Together.ai API. Defaults to “https://api.together.xyz/v1”.

  • model –

    The model identifier to use. Defaults to “openai/gpt-oss-20b”.

    Deprecated since version 0.0.105: Use settings=TogetherLLMService.Settings(model=...) instead.

  • settings – Runtime-updatable settings. When provided alongside deprecated parameters, settings values take precedence.

  • **kwargs – Additional keyword arguments passed to OpenAILLMService.

create_client(api_key=None, base_url=None, **kwargs)[source]

Create OpenAI-compatible client for Together.ai API endpoint.

Parameters:

  • api_key – The API key to use for the client. If None, uses instance api_key.

  • base_url – The base URL for the API. If None, uses instance base_url.

  • **kwargs – Additional keyword arguments passed to the parent create_client method.

Returns:

An OpenAI-compatible client configured for Together.ai’s API.