Most client services are back online and we are reaching out to customers independently regarding restoration of recovered data from the failed system where possible. A follow up email will go to affected customers within 12 hours with further information and assistance offered.
Posted Apr 18, 2017 - 22:49 ACST
Update
Data recovery options are looking positive for many customers however will require some manual recovery work. We will be in touch with customers who have data recovery options available within 24 hours.
Posted Apr 18, 2017 - 20:54 ACST
Update
Services are now being recreated as new deployments due to no success so far recovering from the failed storage array. Our priority now is to get services running with the same IP and details as initially deployed. Once this is completed we will continue steps to attempt data recovery for those customers who request it.
Posted Apr 18, 2017 - 19:42 ACST
Update
Unfortunately the recovery process is running into problems due to corrupted disk images from the failed/degraded array. If you are affected and have your own independent backups please advise us with a support ticket and your service will be provisioned as a 'fresh' service on a replacement VPS node. You will then get up and running quickly and be able to restore your own backups.
Otherwise we are continuing attempts to recover data from the problematic array and contacting customers individually as we progress.
Posted Apr 18, 2017 - 19:05 ACST
Update
We are testing a recovered service now and will then advise further as to the status and timeframe of further restorations for individual services. Thank you for your patience and apologies for the inconvenience caused.
Posted Apr 18, 2017 - 18:21 ACST
Identified
We have a rescue environment active and are attempting to retrieve data from the underlying degraded array. Further updates will be posted as soon as possible.
Posted Apr 18, 2017 - 17:21 ACST
Update
Unfortunately it appears multiple (3) drives have failed in succession in a 6 drive RAID10 array. We are attempting a recovery process to try and bring the system online in a degraded state and working through the setup carefully. At this stage the underlying OS is not booting due to storage failure. Another method of attempted recovery is in progress now. Further updates to come as soon as information is available.
Posted Apr 18, 2017 - 17:09 ACST
Update
syd8kvm node has a serious storage/array failure issue which we are attempting to diagnose and rectify. Further updates will be provided as soon as more information is available.
Posted Apr 18, 2017 - 16:55 ACST
Investigating
We are currently investigating an issue affecting all VPS services on syd8kvm VPS node. Please stand by for further updates. At this stage all VPS on this node are offline.