The list of disasters, man-made and natural, seems to be growing by the day. The Asia Pacific is reeling from the loss of life and tremendous devastation that took place when large parts of Queensland flooded in December and January, followed by the earthquake in Christchurch in February a catastrophic earthquake, tsunami and nuclear accident in Japan this March.
Life must go on despite the disasters however – resilience and planning for the worst will help your customers’ businesses to survive and to ensure the livelihoods of those who depend on it.
How do you get this crucial message across though? Just over half of Australian companies only develop disaster recovery plans after a catastrophic event; that is, if they survive the disaster, according to industry experts.
CRN spoke to a range of companies involved in disaster recovery to find out how to make businesses appreciate the importance of being prepared.
Christchurch recovers
The Garden City in New Zealand’s south island was still figuring out how to mend the damage from the September quake, which measured 7.1 when it was shaken by another one, on paper less powerful at magnitude 6.3.
Over 180 people are thought to have died in the quake that caused widespread damage to Christchurch. The quake is thought to cost NZ$16 billion in insurance claims and it has made a huge dent in New Zealand’s economy, with BNZ bank analysts expecting GDP growth for 2011 almost to halve because of it.
Even so, BNZ points out that “we shouldn’t assume the city has ground to a halt; this is far from the case”; the bank analysts have conducted phone surveys that say manufacturing companies in Christchurch were mostly operational as of mid-March, with the Port of Lyttelton being back at full capacity.
BNZ’s analyst team says ICT-based business will be more mobile and faster to return to normal operations but those without offsite backup records will find the going harder. On the ground, Steve Davis from the NZ Red Cross says there is “obviously a lot of damage in the central business district”.
While some key locations in the CBD didn’t collapse, they became unstable and people were unable to retrieve records, stock, IT gear and other material, he says. In some cases, buildings had collapsed and organisations would have to relocate elsewhere.It’s not all bad news.
Key IT infrastructure seems to have stood up “pretty well”, Davis says. Phone networks and internet access were still going on many areas albeit with somewhat degraded service. Davis believes cloud computing will have provided some relief for many organisations as the quake struck.
However, he adds that lack of good backup and disaster recovery plans will leave many companies hurting.
Stuart Speers, CEO of database specialist systems integrator Enterprise IT in Auckland, says, “I was astonished to see early video footage from Christchurch with several people carrying servers under their arms walking around dazed in the streets and still covered in dust.
“How critical is that server to them if it’s the second thing they think about after coming out alive!” Speers says.
Next page: Keep the calls coming
While the phone network and internet access infrastructure held up reasonably well in the Christchurch quake, the mains grid didn’t.
An estimated 70 percent of Christchurch lost power and as a consequence people couldn’t use their landlines.That’s despite Telecom New Zealand exchanges and roadside cabinets continuing to run on diesel generators with battery backups, providing power to landlines.
Due to the mains grid being disrupted, the only telephones that worked were old-fashioned analogue ones.
Wireless handsets failed unless running via a UPS, ditto digital switchboards requiring mains power. As offices in Christchurch CBD were unsafe or inaccessible, being able to work from home became important to ensure business continuity.
This was only possible in areas with power outages if analogue handsets were available. Business VoIP specialist M5 Networks says that phone systems are seldom included in disaster recovery plans, unlike data and email. When disaster strikes, small and medium-sized businesses are especially caught out and essentially disappear until communications access is restored.
This can take days or even weeks, however, and the downtime could further worsen the chance of survival for the business.In the event a disaster strikes, M5 Networks offers a disaster recovery plan via systems integrators for small to medium businesses.
During the recent flooding in Brisbane, M5 Networks says their system was put to test. Staff at an insurance claims management company couldn’t get to their office in Brisbane CBD and the organisation’s disaster recovery plan for its phones was activated.
Staffers were able to work remotely for a week, receiving calls from customers.
M5 Networks Australia managing director Matthew Wilson says its VoIP system provides location independence, meaning customers could pick up a handset from the office and plug it into the home DSL connection and continue to work.
Even without a handset, Wilson says voicemail services can be accessed via a mobile phone using DTMF tones.
“These days disaster recovery plans for phones are not really just ‘nice to have’ any more, but ‘must have’ as recent events have shown,” Wilson says.
Next page: A test of resilience
For vendors and their partners specialising in disaster recovery planning and products, the disasters have been a severe test of their solutions.
“The recent floods in Queensland and earthquakes in New Zealand have tested the backup strategies and disaster recovery plans of hundreds if not thousands of companies,” says Karl Sice, general manager, Pacific, for backup specialist Acronis.
What the disasters clearly show, Sice says, is that merely having an onsite backup strategy isn’t sufficient in today’s IT planning.Companies need to have a robust cloud backup solution so that when disasters happen they can still recover the critical information.
Sice says that the recently released Acronis Global Disaster Recovery Index 2011 show businesses around the world want a single backup and recovery solution for physical, virtual and cloud environments.
In Australia, 78 percent of companies wanted this, regardless of their attitudes towards backup and disaster recovery, Sice says. This, he says, has created a golden opportunity for resellers.
“An ideal solution should be able to work across physical, virtual and cloud-based infrastructure through a single platform to reduce the complexity of data protection,” Sice says.
This is especially important for SMEs that don’t have usually have the IT resources of larger organisations. Smart resellers should look for such solutions for their customers, Sice says, as he believes that approach makes it easier for organisations to move data between different platforms.
Plans need to be tested and well-rehearsed, Sice adds. “Prepared companies run mock disaster drills. With the complex physical, virtual and cloud environments, these mock drills can help identify shortfalls in the current disaster recovery methodology and adjustments can be made to plans accordingly to improve recovery efforts before disasters strikes,” he says.
Speers says the recent devastating natural disasters in Australia and New Zealand have really hit home the critical need for IT disaster recovery plans. He notes the important points resellers need to explain to customers.
First, looking at the type of disaster they’re trying to protect their businesses from helps define the solution needed – for instance natural disasters require a different approach to human interference.
Timelines are important to establish too, Speers says. How long can the customer continue without its systems? In a disaster recovery scenario, what are the minimum immediate requirements for the first week, then the second week, and continuing onwards, he says.
“The actual data is the most critical aspect to make available immediately,” Speers says. Customers need the data first, followed by business applications. With data in hand, it can at least be manually accessed, Speers says.
While organisations are willing to spend on establishing back-up and recovery plans, the testing of these isn’t always done, Speers says.
“This is the piece we always see removed due to lack of funds and time.”Echoing Speers’ concern, Capgemini Australia’s disaster recovery test manager Anthony Woods says the recent disasters emphasise the need to test solutions and procedures.
“Organisations tend to plan for the least likely disaster event, such as losing a data centre. A more likely scenario might be part of the data centre that is lost, or just three crucial servers,” says Woods.
Capgemini does end to end testing and assessment of organisations’ disaster recovery plans and gives customers a probability of these succeeding should disaster strike, Woods says.
The consultancy’s director of its testing services group, Shane Parkinson, explains that disaster recovery plans are like insurance policy. “Like life insurance, it’s not something that you want to draw on but if you have to, you want the policy to be in order.
”Woods points out that University of Texas research shows that only six percent of companies that suffer a major data loss survive after a disaster. Unfortunately, a disaster recovery plan may not be enough to guarantee survival after a disaster.
Parkinson and Woods explain that a range of factors must be looked at, such as the staff in charge, if documentation is up to date (usually, it won’t be) and if the systems in question have changed over time. Will the DR recovery tasks actually work as supposed to?
“I’m always surprised when I hear of hard disk failures, which should be easy to recover from,” Parkinson says. They are, except in many cases, the backup procedure for the disk storage system may have been out of action for months at times and there’s nothing to recover from.
Hence, testing becomes vital for validating a business’ disaster recovery solution, and not when a catastrophe occurs.
Parkinson says over-reliance on cloud computing and data storage could also be dangerous. As Christchurch, Brisbane and Japanese businesses found out, having everything in the cloud when your infrastructure is inaccessible – or even disappears – makes recovery tremendously hard.
Furthermore, Parkinson says a balanced view of the IT systems in an entire organisation is required. Traditional disaster recovery has focused on backing up data on a per-system basis.
However, as today’s systems have grown large and more complicated over the years this may not be sufficient to ensure business continuity as data in one system that others depend on may become unsynchronised, not processed or simply disappear during the downtime.
“I dread to think how some large organisations with multiple complex systems will cope based on what I’ve seen of their disaster recovery planning,” Parkinson says.