Is A Shift in Data Management Trend Happening?
Looking at the trend of data referencing from here. We can see that data has been growing phenomenally. In actual fact, looking at the trend, the boom started in 2020 and has not stop ever since.
Source: https://explodingtopics.com/blog/data-generated-per-day
In today infrastructure, we have two major solutions for data management. One is our traditional storage array and the other is HyperConverge Infrastructure (HCI). The latter only started picking up in early 2011 where some of the known names such as pivot3 and the major players after such as Nutanix and VMware vSAN which exist most commonly in most data center infrastructure.
As data volume increases and the the adoption of AI, capacity and performance for data management become a crucial requirement. Taking example from the like on virtualization, you can have all the compute resources for your virtual machine, but if your workload data IO and capacity cannot be serve, the bottleneck still exist.
In the world of AI, the same thing exist, where you can supply the compute (GPU), but if your IO and Network bandwidth cannot keep up with the demand, the results of training of query in production can be a difference in leap and bounds.
Let's take a look at how some of the storage technology such as HCI has evolved. VMware vSAN has always to run HCI with external storage. It all depending on the type of workload which will fit in each.
While at Nutanix, the "No SAN" message started in 2011 has somehow take a step back recently. Nutanix has announced to work with Dell to support external storage, PowerFlex. The first since its "No SAN" messaging. With the recently announcement to work with Pure Storage is definite an assurance that Nutanix is taking back their no SAN messaging.
Just looking at the market trend especially using HCI adoption and transformation, it seems that external storage is still very relevant and is making a come back after so much consolidation into HCI solution whether is it due to performance or capacity, a 2-tier or 3-tier is still much needed to meet the data growth and demand.
The real move to HCI in the past was all due to cost vs performance and enterprise ready features. The cost of running HCI may no longer run as "cheap" or cost effective. Most did not cost in the compute resources needed to run the storage function or controller VMs, the licenses, the complexity of hardware needed (SSD, HDD, Controller and all the firmware on them and those on the servers) that is needed to work with the HCI software. Catering enough server resources to perform storage activities on top of ensuring performance of the workloads can results in higher cost. Extensive monitoring is often needed. Scaling performance and capacity of storage cannot move away from adding more compute resources which often go wasted.
The cost and complexity has trigger some customers to move towards external storage where data management can be keep in control and not tying with compute resources. Also HCI still lack much of what most standalone external storage provides in many enterprise features, functions.
Next looking at HyperConverged. Taking example from VMware with EMC on VCE, and NetApp, VCN followed Dell with to VxRAIL and VxRack. The investment from customer to start with is huge. The R&D cost from all 2/3 companies was not easy to maintain. To add on top of the overhead, a high cost with high complexity where 3 companies to maintain the hardware stack in terms of hardware firmware, software upgrade and patches and the delay the catch up of bug fixes and security vulnerability.
In my past experience, its takes almost a month for such integrated solution to come out with an integrated solution or release. We can see what has happened to VCE, VxRack and VxRail. Even the adoption of VCN hyperconverged is not that great too. Time can be shorten but I do not see any possible way of shortening due to multiple vendors involvement where priority of R&D effort need to share among each of their own solutions and 3rd party partnership.
Either they have disappear or the adoption has been really poor. The change in EMC by Dell and VMware by Broadcom outcome resulted the change in such integrated solution. Can we ensure such incident would not happen and resulting the full stack to be phase out in the future? It just need one change in one of the vendor in the total stack to make the complexity happens. As a customer, what will be the only option is to migrate everything out to 2/3 tier architecture which will be a massive work all together again.
Some consideration to take into mind is how easy is to move out from commodity hardware as replacement such as moving back to a 2 or 3-tier architecture? Will you as a customer be able to find immediate replacement? How quickly can it be done, weeks, months, years?
Comments