transport

LiveKit transport implementation for Pipecat.

This module provides comprehensive LiveKit real-time communication integration including audio streaming, data messaging, participant management, and room event handling for conversational AI applications.

class pipecat.transports.livekit.transport.LiveKitOutputTransportMessageFrame(message: Any, participant_id: str | None = None)[source]

Bases: OutputTransportMessageFrame

Frame for transport messages in LiveKit rooms.

Parameters:: participant_id – Optional ID of the participant this message is for/from.

participant_id: str | None = None

class pipecat.transports.livekit.transport.LiveKitOutputTransportMessageUrgentFrame(message: Any, participant_id: str | None = None)[source]

Bases: OutputTransportMessageUrgentFrame

Frame for urgent transport messages in LiveKit rooms.

Parameters:: participant_id – Optional ID of the participant this message is for/from.

participant_id: str | None = None

class pipecat.transports.livekit.transport.LiveKitParams(*, audio_out_enabled: bool = False, audio_out_sample_rate: int | None = None, audio_out_channels: int = 1, audio_out_bitrate: int = 96000, audio_out_10ms_chunks: int = 4, audio_out_mixer: Mapping[str | None, ~pipecat.audio.mixers.base_audio_mixer.BaseAudioMixer] | None=None, audio_out_destinations: list[str] = <factory>, audio_out_end_silence_secs: int = 2, audio_out_auto_silence: bool = True, audio_in_enabled: bool = False, audio_in_sample_rate: int | None = None, audio_in_channels: int = 1, audio_in_filter: BaseAudioFilter | None = None, audio_in_stream_on_start: bool = True, audio_in_passthrough: bool = True, video_in_enabled: bool = False, video_out_enabled: bool = False, video_out_is_live: bool = False, video_out_width: int = 1024, video_out_height: int = 768, video_out_bitrate: int | None = None, video_out_framerate: int = 30, video_out_color_format: str = 'RGB', video_out_codec: str | None = None, video_out_destinations: list[str] = <factory>)[source]

Bases: TransportParams

Configuration parameters for LiveKit transport.

Inherits all parameters from TransportParams without additional configuration.

class pipecat.transports.livekit.transport.LiveKitCallbacks(*, on_connected: Callable[[], Awaitable[None]], on_disconnected: Callable[[], Awaitable[None]], on_before_disconnect: Callable[[], Awaitable[None]], on_participant_connected: Callable[[str], Awaitable[None]], on_participant_disconnected: Callable[[str], Awaitable[None]], on_audio_track_subscribed: Callable[[str], Awaitable[None]], on_audio_track_unsubscribed: Callable[[str], Awaitable[None]], on_video_track_subscribed: Callable[[str], Awaitable[None]], on_video_track_unsubscribed: Callable[[str], Awaitable[None]], on_data_received: Callable[[bytes, str], Awaitable[None]], on_first_participant_joined: Callable[[str], Awaitable[None]])[source]

Bases: BaseModel

Callback handlers for LiveKit events.

Parameters:

on_connected – Called when connected to the LiveKit room.
on_disconnected – Called when disconnected from the LiveKit room.
on_participant_connected – Called when a participant joins the room.
on_participant_disconnected – Called when a participant leaves the room.
on_audio_track_subscribed – Called when an audio track is subscribed.
on_audio_track_unsubscribed – Called when an audio track is unsubscribed.
on_data_received – Called when data is received from a participant.
on_first_participant_joined – Called when the first participant joins.

on_connected: Callable[[], Awaitable[None]]

on_disconnected: Callable[[], Awaitable[None]]

on_before_disconnect: Callable[[], Awaitable[None]]

on_participant_connected: Callable[[str], Awaitable[None]]

on_participant_disconnected: Callable[[str], Awaitable[None]]

on_audio_track_subscribed: Callable[[str], Awaitable[None]]

on_audio_track_unsubscribed: Callable[[str], Awaitable[None]]

on_video_track_subscribed: Callable[[str], Awaitable[None]]

on_video_track_unsubscribed: Callable[[str], Awaitable[None]]

on_data_received: Callable[[bytes, str], Awaitable[None]]

on_first_participant_joined: Callable[[str], Awaitable[None]]

class pipecat.transports.livekit.transport.LiveKitTransportClient(url: str, token: str, room_name: str, params: LiveKitParams, callbacks: LiveKitCallbacks, transport_name: str)[source]

Bases: object

Core client for interacting with LiveKit rooms.

Manages the connection to LiveKit rooms and handles all low-level API interactions including room management, audio streaming, data messaging, and event handling.

__init__(url: str, token: str, room_name: str, params: LiveKitParams, callbacks: LiveKitCallbacks, transport_name: str)[source]

Initialize the LiveKit transport client.

Parameters:

url – LiveKit server URL to connect to.
token – Authentication token for the room.
room_name – Name of the LiveKit room to join.
params – Configuration parameters for the transport.
callbacks – Event callback handlers.
transport_name – Name identifier for the transport.

property participant_id: str

Get the participant ID for this client.

Returns:: The participant ID assigned by LiveKit.

property room: Room

Get the LiveKit room instance.

Returns:: The LiveKit room object.
Raises:: Exception – If room object is not available.

async setup(setup: FrameProcessorSetup)[source]

Setup the client with task manager and room initialization.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup client resources.

async start(frame: StartFrame)[source]

Start the client and initialize audio components.

Parameters:: frame – The start frame containing initialization parameters.

async connect()[source]: Connect to the LiveKit room with retry logic.

async disconnect()[source]: Disconnect from the LiveKit room.

async send_data(data: bytes, participant_id: str | None = None)[source]

Send data to participants in the room.

Parameters:

data – The data bytes to send.
participant_id – Optional specific participant to send to.

async send_dtmf(digit: str)[source]

Send DTMF tone to the room.

Parameters:: digit – The DTMF digit to send (0-9, *, #).

async publish_audio(audio_frame: AudioFrame) → bool[source]

Publish an audio frame to the room.

Parameters:: audio_frame – The LiveKit audio frame to publish.

get_participants() → list[str][source]

Get list of participant IDs in the room.

Returns:: List of participant IDs.

async get_participant_metadata(participant_id: str) → dict[source]

Get metadata for a specific participant.

Parameters:: participant_id – ID of the participant to get metadata for.
Returns:: Dictionary containing participant metadata.

async set_participant_metadata(metadata: str)[source]

Set metadata for the local participant.

Parameters:: metadata – Metadata string to set.

async mute_participant(participant_id: str)[source]

Mute a specific participant’s audio tracks.

Parameters:: participant_id – ID of the participant to mute.

async unmute_participant(participant_id: str)[source]

Unmute a specific participant’s audio tracks.

Parameters:: participant_id – ID of the participant to unmute.

async get_next_audio_frame()[source]: Get the next audio frame from the queue.

async get_next_video_frame()[source]: Get the next video frame from the queue.

class pipecat.transports.livekit.transport.LiveKitInputTransport(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Bases: BaseInputTransport

Handles incoming media streams and events from LiveKit rooms.

Processes incoming audio streams from room participants and forwards them as Pipecat frames, including audio resampling and VAD integration.

__init__(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Initialize the LiveKit input transport.

Parameters:

transport – The parent transport instance.
client – LiveKitTransportClient instance.
params – Configuration parameters.
**kwargs – Additional arguments passed to parent class.

async start(frame: StartFrame)[source]

Start the input transport and connect to LiveKit room.

Parameters:: frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the input transport and disconnect from LiveKit room.

Parameters:: frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the input transport and disconnect from LiveKit room.

Parameters:: frame – The cancel frame signaling immediate cancellation.

async setup(setup: FrameProcessorSetup)[source]

Setup the input transport with shared client setup.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup input transport and shared resources.

async push_app_message(message: Any, sender: str)[source]

Push an application message as an urgent transport frame.

Parameters:

message – The message data to send.
sender – ID of the message sender.

class pipecat.transports.livekit.transport.LiveKitOutputTransport(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Bases: BaseOutputTransport

Handles outgoing media streams and events to LiveKit rooms.

Manages sending audio frames and data messages to LiveKit room participants, including audio format conversion for LiveKit compatibility.

__init__(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Initialize the LiveKit output transport.

Parameters:

transport – The parent transport instance.
client – LiveKitTransportClient instance.
params – Configuration parameters.
**kwargs – Additional arguments passed to parent class.

async start(frame: StartFrame)[source]

Start the output transport and connect to LiveKit room.

Parameters:: frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the output transport and disconnect from LiveKit room.

Parameters:: frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the output transport and disconnect from LiveKit room.

Parameters:: frame – The cancel frame signaling immediate cancellation.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process frames, clearing the LiveKit AudioSource buffer on interruption.

When an InterruptionFrame arrives, any audio already submitted to the LiveKit AudioSource (but not yet played out) is cleared immediately so the bot stops speaking without delay.

Parameters:

frame – The frame to process.
direction – The direction of frame flow in the pipeline.

async setup(setup: FrameProcessorSetup)[source]

Setup the output transport with shared client setup.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup output transport and shared resources.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a transport message to participants.

Parameters:: frame – The transport message frame to send.

async write_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Write an audio frame to the LiveKit room.

Parameters:: frame – The audio frame to write.
Returns:: True if the audio frame was written successfully, False otherwise.

class pipecat.transports.livekit.transport.LiveKitTransport(url: str, token: str, room_name: str, params: LiveKitParams | None = None, input_name: str | None = None, output_name: str | None = None)[source]

Bases: BaseTransport

Transport implementation for LiveKit real-time communication.

Provides comprehensive LiveKit integration including audio streaming, data messaging, participant management, and room event handling for conversational AI applications.

Event handlers available:

on_connected: Called when the bot connects to the room.
on_disconnected: Called when the bot disconnects from the room.
on_before_disconnect: [sync] Called just before the bot disconnects.
on_call_state_updated: Called when the call state changes. Args: (state: str)
on_first_participant_joined: Called when the first participant joins. Args: (participant_id: str)
on_participant_connected: Called when a participant connects. Args: (participant_id: str)
on_participant_disconnected: Called when a participant disconnects. Args: (participant_id: str)
on_participant_left: Called when a participant leaves. Args: (participant_id: str, reason: str)
on_audio_track_subscribed: Called when an audio track is subscribed. Args: (participant_id: str)
on_audio_track_unsubscribed: Called when an audio track is unsubscribed. Args: (participant_id: str)
on_video_track_subscribed: Called when a video track is subscribed. Args: (participant_id: str)
on_video_track_unsubscribed: Called when a video track is unsubscribed. Args: (participant_id: str)
on_data_received: Called when data is received from a participant. Args: (data: bytes, participant_id: str)

Example:

@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
    await task.queue_frame(TTSSpeakFrame("Hello!"))

@transport.event_handler("on_participant_disconnected")
async def on_participant_disconnected(transport, participant_id):
    await task.queue_frame(EndFrame())

__init__(url: str, token: str, room_name: str, params: LiveKitParams | None = None, input_name: str | None = None, output_name: str | None = None)[source]

Initialize the LiveKit transport.

Parameters:

url – LiveKit server URL to connect to.
token – Authentication token for the room.
room_name – Name of the LiveKit room to join.
params – Configuration parameters for the transport.
input_name – Optional name for the input transport.
output_name – Optional name for the output transport.

input() → LiveKitInputTransport[source]

Get the input transport for receiving media and events.

Returns:: The LiveKit input transport instance.

output() → LiveKitOutputTransport[source]

Get the output transport for sending media and events.

Returns:: The LiveKit output transport instance.

property participant_id: str

Get the participant ID for this transport.

Returns:: The participant ID assigned by LiveKit.

async send_audio(frame: OutputAudioRawFrame)[source]

Send an audio frame to the LiveKit room.

Parameters:: frame – The audio frame to send.

get_participants() → list[str][source]

Get list of participant IDs in the room.

Returns:: List of participant IDs.

async get_participant_metadata(participant_id: str) → dict[source]

Get metadata for a specific participant.

Parameters:: participant_id – ID of the participant to get metadata for.
Returns:: Dictionary containing participant metadata.

async set_metadata(metadata: str)[source]

Set metadata for the local participant.

Parameters:: metadata – Metadata string to set.

async mute_participant(participant_id: str)[source]

Mute a specific participant’s audio tracks.

Parameters:: participant_id – ID of the participant to mute.

async unmute_participant(participant_id: str)[source]

Unmute a specific participant’s audio tracks.

Parameters:: participant_id – ID of the participant to unmute.

async send_message(message: str, participant_id: str | None = None)[source]

Send a message to participants in the room.

Parameters:

message – The message string to send.
participant_id – Optional specific participant to send to.

async send_message_urgent(message: str, participant_id: str | None = None)[source]

Send an urgent message to participants in the room.

Parameters:

message – The urgent message string to send.
participant_id – Optional specific participant to send to.

async on_room_event(event)[source]

Handle room events.

Parameters:: event – The room event to handle.

async on_participant_event(event)[source]

Handle participant events.

Parameters:: event – The participant event to handle.

async on_track_event(event)[source]

Handle track events.

Parameters:: event – The track event to handle.