llm

OLLama LLM service implementation for Pipecat AI framework.

class pipecat.services.ollama.llm.OllamaLLMSettings(model: str | None | _NotGiven = <factory>, extra: dict[str, Any]=<factory>, system_instruction: str | None | _NotGiven = <factory>, temperature: float | None | _NotGiven | NotGiven = <factory>, max_tokens: int | None | _NotGiven | NotGiven = <factory>, top_p: float | None | _NotGiven | NotGiven = <factory>, top_k: int | None | _NotGiven = <factory>, frequency_penalty: float | None | _NotGiven | NotGiven = <factory>, presence_penalty: float | None | _NotGiven | NotGiven = <factory>, seed: int | None | _NotGiven | NotGiven = <factory>, filter_incomplete_user_turns: bool | None | _NotGiven = <factory>, user_turn_completion_config: UserTurnCompletionConfig | None | _NotGiven = <factory>, max_completion_tokens: int | _NotGiven | NotGiven = <factory>)[source]

Bases: OpenAILLMSettings

Settings for OLLamaLLMService.

class pipecat.services.ollama.llm.OLLamaLLMService(*, model: str | None = None, base_url: str = 'http://localhost:11434/v1', settings: OllamaLLMSettings | None = None, **kwargs)[source]

Bases: OpenAILLMService

OLLama LLM service that provides local language model capabilities.

This service extends OpenAILLMService to work with locally hosted OLLama models, providing a compatible interface for running large language models locally.

supports_developer_role = False

Whether this service’s API supports the “developer” message role.

OpenAI’s native API supports it, but some OpenAI-compatible services (e.g. Cerebras) do not. Subclasses that don’t support it should set this to False, which causes the adapter to convert “developer” messages to “user” messages before sending them to the API.

Settings

alias of OllamaLLMSettings

__init__(*, model: str | None = None, base_url: str = 'http://localhost:11434/v1', settings: OllamaLLMSettings | None = None, **kwargs)[source]

Initialize OLLama LLM service.

Parameters:

  • model –

    The OLLama model to use. Defaults to “llama2”.

    Deprecated since version 0.0.105: Use settings=OLLamaLLMService.Settings(model=...) instead.

  • base_url – The base URL for the OLLama API endpoint. Defaults to “http://localhost:11434/v1”.

  • settings – Runtime-updatable settings. When provided alongside deprecated parameters, settings values take precedence.

  • **kwargs – Additional keyword arguments passed to OpenAILLMService.

create_client(base_url=None, **kwargs)[source]

Create OpenAI-compatible client for Ollama.

Parameters:

  • base_url – The base URL for the API. If None, uses instance base_url.

  • **kwargs – Additional keyword arguments passed to the parent create_client method.

Returns:

An OpenAI-compatible client configured for Ollama.