Big Data and Object Storage Spell the End to RAID
According to IDC, the future market growth of Big Data technology and it associated services will be significant:
The Big Data technology and services market represents a fast-growing multibillion-dollar worldwide opportunity. In fact, a recent IDC forecast shows that the Big Data technology and services market will grow at a 26.4% compound annual growth rate to $41.5 billion through 2018, or about six times the growth rate of the overall information technology market.
A decent amount of the IT services growth is likely to come from storage of the data itself. Another IDC prediction has digital content reaching 35,000 Exabytes by 2020. (One Exabyte equals 1 million terabytes.) Ultimately, that means RAID is dead, at least for Big Data storage needs.
As one of our storage partners, Cleversafe, puts it (here’s the video), RAID schemes are based on parity, and if more than two drives fail simultaneously, data becomes unrecoverable. This concern hasn’t been an issue for most data storage needs, but as capacity requirements grow to Big Data levels—hundreds of terabytes and petabytes—this failure scenario not only becomes a possibility, it becomes a reality.
Object storage is one way to overcome the bottlenecks and failures that Big Data storage exposes, especially with unstructured data. With object storage, data is managed by creating a unique identifier for each item stored. Unlike hierarchically stored data, object storage enables the number of stored items to grow beyond the limits of traditional storage systems while still maintaining the integrity and consistency of the data.
The object storage architecture was developed with web-scale infrastructures in mind. It provides flexibility, cost-effectiveness, security, and availability—making it ideally suited for hosting unstructured data. Stay tuned as we continue to explore the details around the industries, benefits, and opportunities with object storage.