transport

LemonSlice transport for Pipecat.

This module adds LemonSlice avatars to Daily rooms, enabling real-time voice conversations with synchronized avatars.

class pipecat.transports.lemonslice.transport.LemonSliceNewSessionRequest(*, agent_image_url: str | None = None, agent_id: str | None = None, agent_prompt: str | None = None, idle_timeout: int | None = None, daily_room_url: str | None = None, daily_token: str | None = None, lemonslice_properties: dict | None = None, api_url: str | None = None, **extra_data: Any)[source]

Bases: BaseModel

Request model for creating a new LemonSlice session.

Parameters:

  • agent_image_url – URL to an agent image. Provide either agent_id or agent_image_url.

  • agent_id – ID of a LemonSlice agent. Provide either agent_id or agent_image_url.

  • agent_prompt – A high-level system prompt that subtly influences the avatar’s movements, expressions, and emotional demeanor.

  • idle_timeout – Idle timeout in seconds.

  • daily_room_url – Daily room URL to use for the session.

  • daily_token – Daily token for authenticating with the room.

  • lemonslice_properties – Additional connection properties to pass to the session.

  • api_url – Override the LemonSlice API URL.

agent_image_url: str | None
agent_id: str | None
agent_prompt: str | None
idle_timeout: int | None
daily_room_url: str | None
daily_token: str | None
lemonslice_properties: dict | None
api_url: str | None
class pipecat.transports.lemonslice.transport.LemonSliceCallbacks(*, on_participant_joined: Callable[[Mapping[str, Any]], Awaitable[None]], on_participant_left: Callable[[Mapping[str, Any], str], Awaitable[None]])[source]

Bases: BaseModel

Callback handlers for LemonSlice events.

Parameters:

  • on_participant_joined – Called when a participant joins the conversation.

  • on_participant_left – Called when a participant leaves the conversation.

on_participant_joined: Callable[[Mapping[str, Any]], Awaitable[None]]
on_participant_left: Callable[[Mapping[str, Any], str], Awaitable[None]]
class pipecat.transports.lemonslice.transport.LemonSliceParams(*, audio_out_enabled: bool = True, audio_out_sample_rate: int | None = None, audio_out_channels: int = 1, audio_out_bitrate: int = 96000, audio_out_10ms_chunks: int = 4, audio_out_mixer: Mapping[str | None, ~pipecat.audio.mixers.base_audio_mixer.BaseAudioMixer] | None=None, audio_out_destinations: list[str] = <factory>, audio_out_end_silence_secs: int = 2, audio_out_auto_silence: bool = True, audio_in_enabled: bool = True, audio_in_sample_rate: int | None = None, audio_in_channels: int = 1, audio_in_filter: BaseAudioFilter | None = None, audio_in_stream_on_start: bool = True, audio_in_passthrough: bool = True, video_in_enabled: bool = False, video_out_enabled: bool = False, video_out_is_live: bool = False, video_out_width: int = 1024, video_out_height: int = 768, video_out_bitrate: int | None = None, video_out_framerate: int = 30, video_out_color_format: str = 'RGB', video_out_codec: str | None = None, video_out_destinations: list[str] = <factory>, api_url: str = 'https://api.daily.co/v1', api_key: str = '', audio_in_user_tracks: bool = True, camera_out_enabled: bool = True, camera_out_send_settings: dict[str, ~typing.Any] | None=None, custom_audio_track_params: Mapping[str, ~pipecat.transports.daily.transport.DailyCustomAudioTrackParams] | None=None, custom_video_track_params: Mapping[str, ~pipecat.transports.daily.transport.DailyCustomVideoTrackParams] | None=None, dialin_settings: DailyDialinSettings | None = None, microphone_out_enabled: bool = False, transcription_enabled: bool = False, transcription_settings: DailyTranscriptionSettings = DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}))[source]

Bases: DailyParams

Configuration parameters for the LemonSlice transport.

Parameters:

  • audio_in_enabled – Whether to enable audio input from participants.

  • audio_out_enabled – Whether to enable audio output to participants.

  • microphone_out_enabled – Whether to enable microphone output track.

audio_in_enabled: bool
audio_out_enabled: bool
microphone_out_enabled: bool
class pipecat.transports.lemonslice.transport.LemonSliceTransportClient(*, bot_name: str, params: LemonSliceParams = LemonSliceParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True})), callbacks: LemonSliceCallbacks, api_key: str, session_request: LemonSliceNewSessionRequest | None = None, session: ClientSession)[source]

Bases: object

Transport client that integrates Pipecat with the LemonSlice platform.

A transport client that integrates a Pipecat Bot with the LemonSlice platform by managing conversation sessions using the LemonSlice API.

This client uses LemonSliceApi to interact with the LemonSlice backend. LemonSlice either provides a room URL where the avatar is already present, or adds the LemonSlice avatar to a Daily room the user supplies.

__init__(*, bot_name: str, params: LemonSliceParams = LemonSliceParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True})), callbacks: LemonSliceCallbacks, api_key: str, session_request: LemonSliceNewSessionRequest | None = None, session: ClientSession) → None[source]

Initialize the LemonSlice transport client.

Parameters:

  • bot_name – The name of the Pipecat bot instance.

  • params – Optional parameters for LemonSlice operation.

  • callbacks – Callback handlers for LemonSlice-related events.

  • api_key – API key for authenticating with LemonSlice API.

  • session_request – Optional session creation parameters. If not provided, a default agent will be used.

  • session – The aiohttp session for making async HTTP requests.

async setup(setup: FrameProcessorSetup)[source]

Setup the client and initialize the conversation.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Cleanup client resources.

async get_bot_name() → str[source]

Get the name of the LemonSlice participant.

Returns:

The name of the LemonSlice participant.

async start(frame: StartFrame)[source]

Start the client and join the room.

Parameters:

frame – The start frame containing initialization parameters.

async stop()[source]

Stop the client and end the conversation.

async capture_participant_video(participant_id: str, callback: Callable, framerate: int = 30, video_source: str = 'camera', color_format: str = 'RGB')[source]

Capture video from a participant.

Parameters:

  • participant_id – ID of the participant to capture video from.

  • callback – Callback function to handle video frames.

  • framerate – Desired framerate for video capture.

  • video_source – Video source to capture from.

  • color_format – Color format for video frames.

async capture_participant_audio(participant_id: str, callback: Callable, audio_source: str = 'microphone', sample_rate: int = 16000, callback_interval_ms: int = 20)[source]

Capture audio from a participant.

Parameters:

  • participant_id – ID of the participant to capture audio from.

  • callback – Callback function to handle audio data.

  • audio_source – Audio source to capture from.

  • sample_rate – Desired sample rate for audio capture.

  • callback_interval_ms – Interval between audio callbacks in milliseconds.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a message to participants.

Parameters:

frame – The message frame to send.

property out_sample_rate: int

Get the output sample rate.

Returns:

The output sample rate in Hz.

property in_sample_rate: int

Get the input sample rate.

Returns:

The input sample rate in Hz.

async send_interrupt_message() → None[source]

Send an interrupt message to the LemonSlice session.

async send_response_started_message() → None[source]

Send a response_started message to the LemonSlice session.

async send_response_finished_message() → None[source]

Send a response_finished message to the LemonSlice session.

async update_subscriptions(participant_settings=None, profile_settings=None)[source]

Update subscription settings for participants.

Parameters:

  • participant_settings – Per-participant subscription settings.

  • profile_settings – Global subscription profile settings.

async write_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Write an audio frame to the transport.

Parameters:

frame – The audio frame to write.

Returns:

True if the audio frame was written successfully, False otherwise.

async register_audio_destination(destination: str)[source]

Register an audio destination for output.

Parameters:

destination – The destination identifier to register.

class pipecat.transports.lemonslice.transport.LemonSliceInputTransport(client: LemonSliceTransportClient, params: TransportParams, **kwargs)[source]

Bases: BaseInputTransport

Input transport for receiving audio and events from LemonSlice.

Handles incoming audio streams from participants and manages audio capture from the Daily room connected to LemonSlice.

__init__(client: LemonSliceTransportClient, params: TransportParams, **kwargs)[source]

Initialize the LemonSlice input transport.

Parameters:

  • client – The LemonSlice transport client instance.

  • params – Transport configuration parameters.

  • **kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the input transport.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Cleanup input transport resources.

async start(frame: StartFrame)[source]

Start the input transport.

Parameters:

frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the input transport.

Parameters:

frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the input transport.

Parameters:

frame – The cancel frame signaling immediate cancellation.

async start_capturing_audio(participant)[source]

Start capturing audio from a participant.

Parameters:

participant – The participant to capture audio from.

class pipecat.transports.lemonslice.transport.LemonSliceOutputTransport(client: LemonSliceTransportClient, params: TransportParams, **kwargs)[source]

Bases: BaseOutputTransport

Output transport for sending audio and events to LemonSlice.

Handles outgoing audio streams to participants and manages the custom audio track expected by the LemonSlice platform.

__init__(client: LemonSliceTransportClient, params: TransportParams, **kwargs)[source]

Initialize the LemonSlice output transport.

Parameters:

  • client – The LemonSlice transport client instance.

  • params – Transport configuration parameters.

  • **kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the output transport.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Cleanup output transport resources.

async start(frame: StartFrame)[source]

Start the output transport.

Parameters:

frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the output transport.

Parameters:

frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the output transport.

Parameters:

frame – The cancel frame signaling immediate cancellation.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a message to participants.

Parameters:

frame – The message frame to send.

async push_frame(frame: Frame, direction: FrameDirection = FrameDirection.DOWNSTREAM)[source]

Push a frame to the next processor in the pipeline.

Parameters:

  • frame – The frame to push.

  • direction – The direction to push the frame.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process frames and handle interruptions.

Parameters:

  • frame – The frame to process.

  • direction – The direction of frame flow in the pipeline.

async write_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Write an audio frame to the LemonSlice transport.

Parameters:

frame – The audio frame to write.

Returns:

True if the audio frame was written successfully, False otherwise.

async register_audio_destination(destination: str)[source]

Register an audio destination.

Parameters:

destination – The destination identifier to register.

class pipecat.transports.lemonslice.transport.LemonSliceTransport(bot_name: str, session: ClientSession, api_key: str, session_request: LemonSliceNewSessionRequest | None = None, params: LemonSliceParams = LemonSliceParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True})), input_name: str | None = None, output_name: str | None = None)[source]

Bases: BaseTransport

Transport implementation to add a LemonSlice avatar to Daily calls.

When used, the Pipecat bot joins the same virtual room as the LemonSlice Avatar and the user. This is achieved by using LemonSliceTransportClient, which initiates the conversation via LemonSliceApi and obtains a room URL that all participants connect to.

Event handlers available:

  • on_client_connected(transport, participant): Participant connected to the session

  • on_client_disconnected(transport, participant): Participant disconnected from the session

  • on_avatar_connected(transport, participant): LemonSlice avatar connected to the session

  • on_avatar_disconnected(transport, participant, reason): LemonSlice avatar disconnected from the session

Example:

@transport.event_handler("on_client_connected")
async def on_client_connected(transport, participant):
    ...
__init__(bot_name: str, session: ClientSession, api_key: str, session_request: LemonSliceNewSessionRequest | None = None, params: LemonSliceParams = LemonSliceParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True})), input_name: str | None = None, output_name: str | None = None)[source]

Initialize the LemonSlice transport.

Parameters:

  • bot_name – The name of the Pipecat bot.

  • session – aiohttp session used for async HTTP requests.

  • api_key – LemonSlice API key for authentication.

  • session_request – Optional session creation parameters. If not provided, a default agent will be used.

  • params – Optional LemonSlice-specific configuration parameters.

  • input_name – Optional name for the input transport.

  • output_name – Optional name for the output transport.

async update_subscriptions(participant_settings=None, profile_settings=None)[source]

Update subscription settings for participants.

Parameters:

  • participant_settings – Per-participant subscription settings.

  • profile_settings – Global subscription profile settings.

input() → FrameProcessor[source]

Get the input transport for receiving media and events.

Returns:

The LemonSlice input transport instance.

output() → FrameProcessor[source]

Get the output transport for sending media and events.

Returns:

The LemonSlice output transport instance.