Pipecat Voice AI Agent Analysis

We deeply admire the Daily.co team. We were lucky enough to get the opportunity to contribute to their Pipecat Voice AI framework. In this post, we describe how to asynchronously send your Pipecat Voice AI calls to Canonical AI.

AI Voice Monitoring

But first, why do you need monitoring and analytics for your Voice AI agent?

We see a lot of Voice AI agents here at Canonical AI. Craftsmanship is what sets great Voice AI agents apart. A Voice AI agent in the wild will encounter scenarios its designer did not expect. The best agents are built by Voice AI developers who are constantly studying their Voice AI’s performance and hewing the rough edges.

With more and more developers swarming to Voice AI, agent quality will be the driving differentiator for closing deals. If you want to craft a great Voice AI product, then use Pipecat for orchestration and Canonical AI for monitoring.

Here is an example of our Call Flows. These give you a visual summary of what your Pipecat agent did yesterday.

Here is an example of our Failed Outcome Analysis. No matter how much you test your agent, no matter how many iterations you’ve made on your prompts and function calls and third-party integrations, some human somewhere is going to break your Voice AI. The above analysis makes it easy to find those rough edges.

You can find more about our analytics and data visualizations here.

Integrating Call Analytics Into Your Pipecat Voice AI

The integration is easy.

The first step is to get a Canonical AI API key. Go to our website, click login on the top right, and authenticate as a first-time user. Once you’re logged in, you’ll see a screen with your API key. Copy it and put it in your .env file. You can always access your API key again by clicking on your profile on the top right corner of our dashboard, then clicking on ‘Setup’.

Next, install the Canonical AI service.

pip install pipecat-ai[canonical]

Finally, import the Canonical AI service and add it to your Pipecat Pipeline.

from pipecat.services.analytics import CanonicalMetricsService
from pipecat.processors.audio import AudioBufferProcessor

async def create_analytics_pipeline():
    # Configure session
    async with aiohttp.ClientSession() as session:
        # Create audio buffer processor
        audio_buffer = AudioBufferProcessor()

        # Configure service
        metrics_service = CanonicalMetricsService(
            aiohttp_session=session,
            audio_buffer_processor=audio_buffer,
            call_id="unique-call-id",
            assistant="assistant-name",
            api_key="your-canonical-api-key",
            assistant_speaks_first=True
        )

        # Use in pipeline
        pipeline = Pipeline([
            transport.input(),
            audio_buffer,          # Capture audio
            stt_service,
            llm_service,
            tts_service,
            metrics_service,       # Process and upload recordings
            transport.output()
        ])

You can find more documentation on the Pipecat and Canonical AI integration here.

Embed Data Visualizations In Your Customer-Facing Dashboard

The biggest problem in Voice AI right now is that Voice AI buyers think the pool is too cold. They’re just dipping their toes in the water. They’ve signed contracts and bought credits from Voice AI builders like you, but they’re slowly using them.

It makes sense from the perspective of the Voice AI buyer. They have their brands and sales funnels on the line. They can’t afford for your Voice AI to mess up.

Again and again, we’ve seen what it takes to get the Voice AI buyer to jump in. Just show them what’s happening in the Voice AI calls. It’s that simple.

And by embedding our analytics and data visualizations into your customer-facing dashboards, it’s easy to help your customers build trust in your Voice AI. You just put your Canonical AI API key and the assistant id in our React component.

import { CallFlowChart, CanonicalProviders } from "@canonicalai/voice";

const apiKey = ; // read the API key from the .env file or something
function App() {
  return (
    <CanonicalProviders apiKey={apiKey}>
      <CallFlowChart
        assistantId="YOUR_ASSISTANT_ID"
        width={1000}
        height={400}
      />
    </CanonicalProviders>
  );
};

You can learn more about embedding components here.

Next Steps

If you're looking for an easy way just to test our platform on our free tier, then you can manually upload a batch of calls with our GUI. Click on sign up, then click on upload calls. But frankly, it’s just as easy to integrate with Pipecat and be done with it.

Adrian and I love the Voice AI space! We love meeting Voice AI developers and learning about the neat things they’re building! If you have any questions about the Pipecat Canonical AI integration, or just want to meet others working in Voice AI, please reach out to us!

Tom and Adrian
January 2025