Quantum handling large data workflows in demanding big data environments

As organizations continue to generate new types of valuable data, there is an increasing need to retain data and maintain visibility and access to it. However, keeping all that data on high-performance disk is proving cost-prohibitive and difficult to manage. Jim Simon, Vice President of Global Field and Channel Marketing, Quantum shares with VAINDIA of how the company offers an integrated solution set that has proven effective in managing some of the world's most demanding big data environments –
 

Statista estimates that in 2018, datacenter storage capacity stands at 1,450 exabytes that is further growing to a massive 2,300 exabytes by 2021. This is a lot of data and its growing fast. 

Over 50% of the data stored today consists of video and or images. This is expected to continue to grow in the years ahead with advents in video surveillance, corporate video, and analytics.

“Quantum is the global leader in video and rich media storage. We have a long legacy of helping organizations ingest, edit, share, protect, and archive video through our StorNext file system. We are doubling down in this area by addressing distinct vertical markets such as autonomous vehicles, media/entertainment, corporate video, biomedical research, satellite imaging, and more. These are all heavy growth industries that Quantum resellers can profit from,” says Jim Simon, Vice President of Global Field and Channel Marketing, Quantum.

A range of key industries, including life sciences, engineering design, energy and university research, rely on workflows that involve storing, managing, and moving large numbers of files. Quantum helps companies meet this problem head-on by reducing both operating and capital costs associated with managing large amounts of file-based data across complex storage environments.

From an end-user perspective, Quantum recommends that customers audit their data workflows to understand -
•    Where am I generating data?
•    What kind of data is it?  Structured (database) or unstructured?
•    What is the growth rate of data generation?
•    How big is this data (e.g. terabytes, petabytes, exabytes)?
•    Does it need to be shared instantly?
•    Does it need to be protected such as via replication to one or more sites?
•    Does some or all of it need to kept in archive after a certain period of time?
•    Does some or all of it need to be stored in the cloud?

Quantum’s StorNext file system was designed to handle large data workflows. Its origin dates back more than two decades when it was built for satellite imaging management.  With the growth in different verticals including new ones like autonomous vehicles, StorNext is perfect for helping our customers, specifically with-
•    High-performance ingest such as with our new NVMe tier of storage
•    Multi-protocol access such as NAS and SAN
•    Shared access by an unlimited number of users in the same or globally distributed locations
•    Instant replication for data protection
•    Automatic tiering from highest performance/high cost storage such as flash to high performance/lowest cost tape or cloud

Challenges with growing data…

In business storage, data growth gives a host of challenges. This allows companies like Quantum to improve the storage and the security of the data, which are led by an innovation.With new data generating every day and the old ones piling up, three of the most common challenges include -
•    Inability to scale with the volume of data generation in terms of performance or capacity
•    “Islands” of data that are difficult to access
•    Lack of auto-management of data tiering from highest performance/high cost storage such as flash to high performance/lowest cost tape or cloud 

Quantum StorNext addresses each of these three common issues through its StorNext system - 
•    StorNext is infinitely scalable
•    StorNext provides a “single pane of glass” management of any tier of data, even data stored on non-Quantum disk or tape
•    StorNext StorageManager applies a policy-based approach to move data between tiers.  For example, a 100GB video might be in development for two months and then played out once. It will likely never be deleted but it shouldn’t reside on the highest performance storage tier where it was created. It should automatically be migrated to high performance/lowest cost tape or cloud, making room for the next projects.

“Our customers look very different on the outside but their data generation is very similar, namely rich media. We address distinct vertical markets such as autonomous vehicles, media/entertainment, corporate video, biomedical research, satellite imaging, and more and help organizations ingest, edit, share, protect, and archive video through our StorNext file system," sums up Jim.