Connectivity has remained stable after the upstream hardware changes implemented. We will continue to monitor and follow up with affected customers. Further maintenance works will take place in Brisbane to improve resiliency of the network.
Posted Nov 04, 2019 - 23:35 ACDT
Update
An upstream vendor hardware replacement has been completed and testing performed shows service stability has been restored. We are monitoring carefully but expect service stability to be restored now.
Apologies for the significant disruption to service this evening. Customers with open tickets will be replied to as a priority.
We are working on major upgrades to the Brisbane network presence to reduce reliance on certain components that have proven to have some reliability issues in the past week and will continue those upgrades further as a high priority.
Posted Nov 04, 2019 - 21:36 ACDT
Monitoring
Connectivity is restored; we are monitoring and continuing preparation for moves to an alternate path.
Posted Nov 04, 2019 - 21:31 ACDT
Identified
Unfortunately the MetroConnect service has dropped again. We are continuing to investigate and pursuing emergency provisioning of other capacity between the facilities. Further updates to be posted as soon as information becomes available.
Posted Nov 04, 2019 - 21:13 ACDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 04, 2019 - 20:37 ACDT
Update
An upstream vendor (NextDC) has a hardware technician en route to address what is believed to be an equipment fault affecting our MetroConnect service between datacentres. We are also working with partners in Brisbane to rapidly provision other capacity in case of further issues or no complete resolution.
Apologies to all customers affected and we will continue to keep this page updated as progress is made.
Posted Nov 04, 2019 - 20:09 ACDT
Identified
The MetroConnect service has gone down again. We have determined a workaround solution to protect our network better and are working with a high level of urgency towards a permanent fix.
Posted Nov 04, 2019 - 19:21 ACDT
Monitoring
The service has been restored.
Unfortunately it appears a MetroConnect service between our Brisbane datacentre's has had a brief outage again. As a result of last weeks incident plans are already in progress to condense our Brisbane presence into NextDC B2 and avoid the NextDC B1 facility.
We will endeavour to expedite this process and make extra capacity available to work around weaknesses with the upstream vendor.
Apologies for the inconvenience caused.
Posted Nov 04, 2019 - 18:12 ACDT
Investigating
We are investigating a connectivity issue between two Brisbane datacentres which is causing disruption to Brisbane services. This appears to be related to the incident last week.
Posted Nov 04, 2019 - 18:02 ACDT
This incident affected: Brisbane, Australia (Brisbane Dedicated Servers, Brisbane VPS Services).