Plivo elevated failure rate with incoming calls on DID.

Incident Report for Trackdrive

Postmortem

INCIDENT DESCRIPTION

On Nov 8 between 11:44 EST and 12:59 EST incoming calls to Plivo over DID had an elevated failure rate.

PLIVO'S RESOLUTION

As of 12:59 EST Plivo.com has reported that their Upstream Provider has fully restored service to the accidentally deleted numbers and that all functionality should be fully restored at this time. If you have questions about your Plivo.com account, your rep can be contacted on Skype: ankur.passi2

TRACKDRIVE SOLUTION:

There are many moving parts to the TeleCom Industry, and issues can arise at the Upstream from your Providers (Plivo.com on 11/08/2016 had numbers deleted from a table at an Upstream Provider). So while a company you work with may go down, (even if not their own fault), TrackDrive wants you to have the most uptime possible.

TrackDrive will take immediate steps to add in other providers like Twilio and Message360. If you are using one provider who has an issue, you can switch to the other provider’s numbers and continue business with minimal Interruption.

Posted Nov 08, 2016 - 13:19 EST

Resolved

Plivo: I have just got information that normal services have been restored by our Upstream for TF.

Posted Nov 08, 2016 - 12:59 EST

Update

Update from Plivo:

Plivo: We are still not fully restored. I just got off a call from our upstream and they are working on fixing the route.
Plivo: According to our upstream, they are fixing the issue phase by phase.

Posted Nov 08, 2016 - 12:39 EST

Monitoring

Plivo has provided an update:

Our upstream carrier inadvertently removed lots of numbers from our account effectively disabling the routing on those numbers. They are putting them back in now.

We are facing some isolated issues with Toll Free Numbers. The issue is on our inbound route. Plivo is contacting the upstream right away.

Posted Nov 08, 2016 - 11:58 EST

Identified

Plivo is working on resolving this issue.

Trackdrive is going to be adding Twilio as an alternate provider in the next couple days since Plivo has proven unreliable.

Posted Nov 08, 2016 - 11:55 EST

Investigating

Plivo is experiencing an elevated failure rate with incoming calls on DIDs.

We will update as soon as we hear more.

Posted Nov 08, 2016 - 11:44 EST