I’m a voice software engineer in Telnyx(https://telnyx.com) and worked for the voice industry for over 10 years. And also I’m one of the Asterisk code contributors.
Making is always fun! But making a Twilio clone was not easy and wasn’t that simple too. Every time, when I trying to add the feature it required a different approach and tech stacks.
Sometimes it was fun, sometimes not. But it was a good experience overall. 🙂
Basic call feature
Basically, the Twilio’s programmable Voice provides RESTful APIs. Using this APIs the customer can control the call easily.
To implement this I implemented the below actions. The action defines call’s behavior. The user can control the call using these actions whatever they want.
Actions:
- Answer: Answer the incoming call.
- Conference join: Join the existed conference.
- Connect: Originate to the other destination(s) and connect to them each other.
- DTMF Receive: Receives the DTMFs for a given duration or numbers.
- DTMF Send: Sends the DTMFs with a given duration and interval.
- Echo: Echoing the call.
- Hangup: Hangup the call.
- Patch: Get the next flow(actions) from the remote.
- Play: Play the linked file.
- Recording Start: Start the recording.
- Recording Stop: Stop the recording.
- Talk: Text to speech.
Outgoing call with flow
The user can make a outgoing call request with flow.
Incoming call handle
The user can handle the incoming call with created flow.
Recording
The user can start and stop the recording whenever they want. The user can see the list of recordings and which recordings are related with the specified call. And the recording file can be downloaded too.
WebRTC
The VoIPBIN supports WebRTC. Using this feature, the user can register their device and can gets the call from the VoIPBIN. The call could be the voice or video and can join to the conference too.
Registration
I wasn’t think about this feature at the beginning. Because I thought this is not main feature for Programmable Voice. But once I implemented WebRTC service, I realized that this feature is needed.
Because I was working for the CTI(Compute Telephony Integrate) industry before. And in the CTI service, it is really important to send the call to the logged in agents. Once the call coming to the company, the welcome message will be played and deliver the call to the agent is the basic call scenario.
I was thinking may I could use the WebRTC feature for CTI agents. But to make an outgoing call to the WebRTC user, I needed to implement this registration feature.
So I added private domain, extension and registration feature.
Webhook event
Sending a call’s status changes to the given destination as a webhook is also important.
System architecture
Let’s talk about how I implemented this service.
Automation
I was alone. So, the automation was not an option at all. It was mandatory.
- Terraform: Used it to create/maintain the cloud instances(compute instance, kubernetes nodes, …). Not only for those instances but also used it to handle the GCP’s network too.
- Ansible: Used it to maintain the Kamailio, RTPEngine and other minor standalone services.
- Kubernetes + Kustomize: Thanks, god! I used this almost everywhere. The kubernetes made this project possible. I use this for all of microservice and Asterisk deployment.
- Gitlab: I really love the Gitlab’s CI/CD feature. It made it possible to focus on the develop itself. Everytime when I deploy something, it deployed exactly the same way all the time. It was smart enough to not doing stupid typo errors what I normally do.
Scalability
This was the one of the things what I want to achieve from this project. I wanted to make this project to having a scalability. To achieve that, I had to re-design the system topology for many times.
- No downtime when it scaling up.
- No downtime when it updating.
- Active-Active high availability.
There are lots of stories about this. I could achieve this feature by using the Kubernetes. But I couldn’t use the k8s for some of the services such as Kamailio, RTPEngine, … Because it is hard to control the incoming/outgoing IP address in the k8s.
I wanted to use the same IP address for incoming/outgoing for those proxies. But most of cloud service didn’t allow that. After many failures, I had to give up this. As you already know, the source/destination IP address in the Telco/VoIP world is so important.
Not only for IP address but also the port number was one of the concerns to stick to the standalone service. – This is what I’m still thinking and looking for the solution all the time. I couldn’t make it this time, but still, keep in mind this.
Monitoring
Monitoring was also one of the thing that I really loved. Because I could actually see what is going on there.
The VoIP world was not an easy to see that what is going there. So, to see the status I needed something special.
- Heplify/Homer: I used this for SIP packet capturing. This service captures the SIP packets every time and provides nice GUI to use it.
- Prometheus/Grafana: It gives a beautiful GUI and flexible ability to use.
Unified Message Bus
I wanted to use the simplified and unified message bus for all of the microservices. It must have below features.
- Request/Response
- Publish / Subscribe
- Message Queuing
- Balanced message distribution
- High availability
There was couple of options for this. Personally, I wanted to use the ZeroMQ for this, because I like that. But the ZeroMQ was not designed for these purposes.
The next option was the Kafka. Actually, I was trying to use the Kafka for this project. I installed the Kafka and did some basic setup to use. And deployed and connected some services.
But the Kafka requires a bit higher CPU/Memory. All of standalone service uses f1-micro type. But This project has limited budget and I couldn’t spend more money for this. Otherwise I could be bankrupt!
So I choose the RabbitMQ. It is powerful and fits for my purposes. And it running well with the limited CPU/Memory!
Stateless
One of the thing that I really considered in the system design was ‘How can I make the program as a stateless?’.
I wanted to make the project scalable. To achieve that, design the system/program as stateless was not an option. Sometimes it was so hard to achieve for some services. So, I had to change the whole system design for many times. That was painful but gave me lots of fun. 😉
Asterisk
One of the main component of this project is the Asterisk(https://asterisk.org/). To make this project, I had to add the some of feature and needed to fix some bug. That brings me to one of the code contributors of the Asterisk. 🙂
Budget
Budget was the biggest concern. I used google’s free credit for many times. But at some point, I had to spend a real money for this. :'(
The average cost is 150 euros per month. And luckily it doesn’t increase that much. 🙂
Testers
I have to mention about my testers. They are big fan of my project. 🙂
Actually, no one knew about my project at the beginning. But after open the SIP service port to the public internet, the fraud calls are came in. That was a huge amount of request. Sometimes I had to spent whole day to just to block them.
But I changed my mind to use them as testers. 🙂 Whenever they come to this service I actually treat them as a normal request and fire the very basic flows. It is really helpful in diagnosing service errors.
Currently, no one uses this service, but I’m getting so many of service requests. 🙂
Why?
One day, someone asked me, why I’m doing this. At the moment, I couldn’t answer that. Because I couldn’t remember that. I started this project a long time ago and didn’t think about that for a long time.
Then after few days later, I just remembered that. Actually, there was no big reason. I just started just for fun.
I was working in this telco industry for over a decade. I’ve seen and handled many legacy telephony infrastructures and software. The legacy code was working, but debugging and adding some new features was always painful.
For example, seeing the hadouken code(in c) was just another ordinary day work. 🙂
So, I was always thinking about something easy way to implement and provide the telco service. Then I saw Twilio and really shocked at how they provide telco services. Then I thought it might be really fun if I implement that service too. So, I just started. 🙂
What’s next
There’re tons of things to do! 🙂 I don’t know what’s the next feature/fix would be, but whatever it is, it will be really exciting and fun!! 🙂
