We have received a Reason for Outage from our 3rd party vendor which facilitates our MS Teams connection.
The outage started on December 6 at 10:10am EST and lasted until 12:25pm EST. During this timeframe, our vendor's affected infrastructure was the US EAST SBC. This SBC became unreachable due to an invalid DNS entry. This caused inbound and outbound calls to fail when trying to come through the US East SBC. This DNS entry was added by mistake during a routine modification to DNS records. This DNS record was then removed to restore service.
In the aftermath of this incident, our 3rd party vendor have implemented new procedures that will require review and approval of all DNS changes by a supervisor before they can be implemented. They have also put in place new monitoring and alerting for service affecting DNS records.
Posted Dec 20, 2023 - 10:15 EST
Resolved
A vendor Level365 utilizes for the MS Teams connector experienced a connectivity issue and has restarted a caching service to resolve the issue. They are investigating the root cause at this time. We will follow up with any post-mortem findings.
Posted Dec 06, 2023 - 15:25 EST
Update
We are continuing to monitor for any further issues.
Posted Dec 06, 2023 - 14:54 EST
Monitoring
Impacted calls are now completing properly. We will continue to monitor these services for any further issues.
Posted Dec 06, 2023 - 13:09 EST
Investigating
We are currently investigating reports of call connection issues when using the MS Teams Connector.
Posted Dec 06, 2023 - 11:48 EST
This incident affected: UCaaS (Secondary Services).