Build a Multi-Functional Voicebot in Minutes

In this post, we're going to talk about how you, as a developer, can quickly build your own voicebot for a specific use case in the Vapi API.

If we were starting from scratch, the first thing we'd need is a backend infrastructure, i.e., Transcription, Model, and Voice. What would really be nice is if we had a complete speech-to-speech pipeline ready for us to use so we could jump straight to the fun stuff. Maybe something like this one...

Vapi connects you with a number of service providers you can choose from to power your voicebot. For Transcription, we use Deepgram; for Model, OpenAI’s ChatGPT-3.5-turbo and ChatGPT-4-turbo; and for Voice, ElevenLabs, PlayHT, Deepgram, OpenAI, and RimeAI.

Since we already have everything we need at our disposal, let us proceed.

Defining the Purpose

Next, we need to know what the voicebot will be used for, and who it will be used by. This way we know what tasks it will need to perform, and build in functions and customizations accordingly. 

Let’s go with a common use case: an assistant that provides technical support to inbound callers. An additional functionality we will add is the ability for the voicebot to remember by the phone number of a caller where the conversation left off if the call is disconnected.

Setting Up

The final step before we get started is giving you access to Vapi’s API. After signing up, your API key can be found in the Account page > Vapi Keys > Private API Key. When making requests to the API, make sure to include your key in the Authorization header.

const axios = require('axios');
const API_KEY = 'your_api_key';

const res = await axios.get('https://api.vapi.ai' , {
  headers: {
    Authorization: `Bearer ${API_KEY}`
  }
});

console.log(res.data);

Time to Develop

Let’s walk through this together step by step.

1. Create a Voicebot

First, we need to define the voicebot. Access the Vapi API here to get started.

{
  "transcriber":{
    "provider": "deepgram",
    "keywords": ["iPhone", "MacBook", "iPad", "iMac", "Apple Watch", "Apple TV"],
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4",
    "messages": [
      {
          "role": "system",
          "content": "You're a technical support assistant. You're helping a customer troubleshoot their Apple device. You can ask the customer questions, and you can use the following troubleshooting guides to help the customer solve their issue: ..."
      }
    ]
  },
  "forwardingPhoneNumber": "+16054440129",
  "firstMessage": "Hey, I'm an A.I. assistant for Apple. I can help you troubleshoot your Apple device. What's the issue?",
  "recordingEnabled": true,
}

Let's stop for a moment and break down what we’ve just done by component and function:

• transcriber - We’re defining this to ensure the transcriber picks up custom words related to our devices.
• model - We’ve opted for the OpenAI GPT-3.5-turbo model. It’s faster than ChatGPT-4-turbo, and preferable if we don’t need the additional data.
• messages - We’ve defined the voicebot’s instructions for how to run the call.
• forwardingPhoneNumber - We’ve added this so that the voicebot will be provided with the transferCall function to use if the caller asks to be transferred to a person.
• firstMessage - This is the first message the voicebot will say to the caller.
• recordingEnabled - We want to record the call so we can hear the conversation later.

Since the voicebot will remember where a call leaves off, its configuration will change based on the caller. Therefore, we’re going to store the conversation on your server between calls and use the Server URL's assistant request to fetch a new configuration based on the phone number every time there’s a call.

2. Get a Phone Number

Now we need a phone number for inbound calls. You can find more info on obtaining your phone number here.

{
  "id": "c86b5177-5cd8-447f-9013-99e307a8a7bb",
  "orgId": "aa4c36ba-db21-4ce0-9c6e-99e307a8a7bb",
  "number": "+11234567890",
  "createdAt": "2023-09-29T21:44:37.946Z",
  "updatedAt": "2023-12-08T00:57:24.706Z",
}

3. Configure Your Server URL

When someone calls the phone number, the assistant configuration will need to be fetched from the server. Get more information how to do this here.

We'll create an endpoint on the server for Vapi to hit. Then, we’ll add that endpoint URL to the Server URL field in the Account page on the Vapi Dashboard.

4. Enable Saving the Conversation at the End of the Call

We will complete this step using the Server URL's end-of-call-report message.

{
    "message": {
        "type": "end-of-call-report",
        "endedReason": "hangup",
        "call": { Call Object },
        "recordingUrl": "https://vapi-public.s3.amazonaws.com/recordings/1234.wav",
        "summary": "The user mentioned they were having an issue with their iPhone restarting randomly. They restarted their phone, but the issue persisted. They mentioned they were using an iPhone 12 Pro Max. They mentioned they were using iOS 15.",
        "transcript": "Hey, I'm an A.I. assistant for Apple...",
        "messages":[
        {
            "role": "assistant",
            "message": "Hey, I'm an A.I. assistant for Apple. I can help you troubleshoot your Apple device. What's the issue?",
        },
        {
            "role": "user",
            "message": "Yeah I'm having an issue with my iPhone restarting randomly.",
        },
        ...
        ]
    }
}

Save the call.customer.number and summary fields to our database for the next time they call.

5. Get Voicebot Ready to Receive Calls

When the phone number from Step 2 receives a call, Vapi will hit the server’s endpoint with a message like this:

{
    "message": {
        "type": "assistant-request",
        "call": { Call Object },
    }
}

The database will be checked to see if there is a saved conversation for this caller. If so, we’ll need to create another assistant configuration (as in Step 1) to respond with:

{
    "assistant": {
        ...
        "model": {
            "provider": "openai",
            "model": "gpt-4",
            "messages": [
              {
                "role": "system",
                "content": "You're a technical support assistant. Here's where we left off: ..."
              }
            ]
        },
        ...
    }
}

If there is not a saved conversation for the caller, the response will be the assistant configuration from Step 1.

6. Ready to Test!

Give the phone number a call and tell the voicebot you’re having an issue with your iPhone restarting randomly.

Then, hang up and call back. Ask what the issue was. The voicebot should remember where we left off when the call was disconnected. 

This is Vapi's mission: to let you build powerful voicebots in minutes, run a simple test, and deploy. 

To try our voice assistant out for yourself, visit the Vapi dashboard and get $10 worth of minutes on us.