The gradual evolution of storage continues

If a week is a long time in politics, then five years is even longer in the information technology industry. While the Australian storage industry hasn’t seen the advent of any major disruptive new technologies since 2003, there has been a gradual evolution, a maturation of various technologies over that time that has altered the Australian storage landscape in a major way.

Non-technological factors have also had an impact on the storage landscape. Prices continue to come down while capacity requirements continue to go up, driven by factors such as data intensive applications and electronic retention policies. Indeed, one of the major changes over the past five years has been a changing attitude towards storage, and the legal implications of getting it wrong.

As a result, storage has now become a C-level topic of discussion which in turn is beginning to drive behavior in the Australian market. Key to this has been the introduction of legislation such as Sarbanes-Oxley and Basel II that has required companies to change their approach to storage and data management.

“This in turn has reflected in more holistic data management strategies where information lifecycle management solutions are now being considered in the market, as opposed to point data storage solutions of the past,” said IDC’s program director - Asia Pacific storage research, Simon Piff. “The need to manage, maintain, secure and access data at the right level at the right time has impacted the Australian market recently.”

The Australian disk storage systems market has been influenced by several factors over the past five years, including a continuing decline in storage hardware prices. Disk storage prices continually fell over that time driven partly by competitive pressure and technology advances such as increasing capacity per drive and cheaper, high-capacity SATA and SAS disks for high-performance SCSI and Fibre Channel Disks.

“Increased take-up rates of lower-cost entry-level and midrange disk storage systems by SMEs have also affected average price points,” said Piff. “The net result of all this is that price per gigabyte has fallen rapidly, for example it fell by 21% from 2006 to 2007, decreasing to only US$5.57.”

Demand for storage capacity has been strong over the past five years. This has been driven by data intensive applications such as email, customer relationship management (CRM), enterprise resource planning (ERP) and business intelligence (BI), as well as business continuity and disaster recovery requirements.

The other major factor driving storage capacity increases is the gradual introduction of electronic records retention policies. “Many organizations in the United States have already introduced enhanced retention policies for customer data, employee records, and corporate data such as memos, presentations, and spreadsheets,” said Piff. “IDC observes that this requirement has been introduced to many countries in the Asia Pacific region.

“Capturing, organizing, and storing such data as email, instant messages and blogs can be expensive, but this is necessary when legal issues arise. These newer forms of electronic data are growing at rates of more than 50 percent annually and are located across a wide spectrum of devices. Most large companies will need to design electronic records retention programs, implement far more extensive archival procedures, and provide far more storage capabilities than before.”

There has also been an increasing proportion of disk storage systems being networked over the past five years. In 2003 there was still a lot of direct attached storage (DAS) around, and while it will most likely never go away completely “IDC has observed an increasing awareness of the benefits associated with the networked storage architecture, leading to a gradual increase in SAN and NAS storage deployments,” said Piff. “The line between SAN and NAS is also becoming less distinct as more vendors overcome architectural barriers.”

The last big trend over the past five years has been the gradual maturation of iSCSI from overhyped technology that no one was actually using to a technology starting to attract serious user attention and interest. “IDC has observed that smaller organizations are starting to deploy iSCSI solutions. Due to the ubiquity of IP networks in user organizations, iSCSI potentially offers a cost-effective way to implement storage networks so iSCSI is poised for high growth.

“Most storage vendors are already supporting iSCSI. Although iSCSI has initially been positioned as a low-cost, easy solution rather than a replacement for Fibre Channel SANs in datacenters, it also has the potential to become a technology that can drive prices down further and offer a cost-effective storage solution for many organizations.”

Network Appliance’s Marketing Director - Australia & New Zealand, Roger Mannett, believes there have been three main changes in the last five years – disk density, network speeds and management. “Disk density, or recording density to be correct, has skyrocketed. In 2003 we were probably working the last of the 73 gigabyte - 3.5-inch drives, mainly Fibre Channel attached with perhaps some SCSI approaching end of life. Today we are using Fibre Channel drives at densities up to 300 gigabyte, SATA drives are becoming mainstream and densities of up to 1 terabyte are now a reality.”

Network speeds have also jumped over the past five years. In 2003, Fibre Channel was 1 GB/sec and IP/Ethernet was 100 MB at the desktop to 1 GB at the core. Today, Fibre Channel has increased to 4 GB, Gigabit Ethernet at the desktop and 10GB at the core is a reality with 100 GB coming down the pipeline.

Management of enterprise disk systems has also developed considerably in the past five years. “In 2005, management was by a command line interface where the syntax was complex and exacting,” said Mannett. “Operational staff required extensive training, and the deployment of storage was slow. Today’s management tools allow for a much simpler GUI interface - often down to a web browser, making the operations and provisioning of storage a much simpler task, less prone to errors, and far speedier to deliver.”

StorageCraft regional director Asia Pacific, Greg Wyman, believes the two big trends of the last five years have been virtual tape libraries (VTL’s) and disk-to-disk backup. “When VTLs first came out, there was lots of market excitement. However, it quickly became apparent they didn’t eliminate backup windows - they simply moved the pain point to another area on the network.” VTLs did pave the way for disk-to-disk-to-tape and real time recovery argued Wyman, a technology that has become increasingly popular over the last five years. “Companies of all sizes are migrating to disk-based backup technology and real-time recovery.”
Sun Microsystems storage product manager A/NZ, Steve Stavridis, and storage practice manager A/NZ, Anthony Clarke, cite storage virtualisation, Information Lifecycle Management (ILM), the development of SATA drives and the growing ubiquity of SAN and NAS technology as being the big trends over the past five years.

Storage virtualisation, the process of abstracting logical storage from a pool of physical storage, is a technique that aims to improve low levels of storage resource utilisation and reduce the high costs of storage management by using less hardware. Savings are realised through reduction in floor space usage, decreased power and cooling requirements, maintenance costs and management costs.

“What we have seen over the past five years is that some storage virtualisation solutions add complexity instead of reducing it,” according to Stavridis and Clarke. “So although storage virtualisation has been a major trend over the past five years, it is not a must at any cost.”

“ILM, a business centred way of managing the data-centre based on the value of information (which changes over time) and the underlying cost of storage, was the ‘next big thing’ five years ago. It was meant to be an approach to managing information growth, yet most vendors used a different definition and pointed to a tiered infrastructure which has made the term ILM nearly meaningless.”

SATA (serial ATA) disks have proved to be an expansive disk drive technology which allowed more data to be put online at a lower cost to Fibre Channel. This helped to drive further the adoption of SAN and NAS technology which has been another major trend over the past five years. “Someone said in 2002 that ‘Storage area networks (SANs) are the future of enterprise storage, period. If your company is heading toward, or has already passed, the terabyte mark in storage, it’s a prime candidate for a SAN migration,’ argued Stavridis and Clarke. “This certainly has been the case. SANs and NASs have become the industry standard architectures for consolidating storage.”

Indeed SAN and NAS technology has been commoditised to such an extent that the ‘boxes’ have become unimportant and the focus is on services and the surrounding software. SAN and NAS technology is also extending into the SME and even the SMB, according to Hitachi Data Systems’ chief technologist, Simon Elisha.

“There is a much bigger focus on the breadth of capability from a solution perspective. Five years ago, the latest sized hard drive and newest ‘box’ got the most attention. Today, it is about what storage services can be provided to the business in the fastest time with the greatest efficiency. All storage providers can service customers’ capacity needs, there is much more of a focus around operational efficiency and effectiveness of the solutions. This extends into the relevant software stacks and services components of any solution.

“The SME/SMB space is now demanding capabilities typically associated with enterprise customers at an SME price-point. The capacities required by these customers are growing dramatically. Today, it is not uncommon for an SME customer to have over 100TB of storage and an SMB customer have 5TB+. This introduces new demands around reliability, performance, flexibility and manageability that did not exist five years ago.”