Pure Storage empowers enterprise AI with advanced data storage technologies and validated reference architectures for emerging generative AI use cases.
Summary
AI devours data. With award-winning AI-ready infrastructure, an AI data platform, and collaboration with NVIDIA, Pure Storage is delivering solutions and services that enable organizations to manage the high-performance data and compute requirements of enterprise AI.
AI Then and AI Now
They (some wise anonymous folks out there) say that there is a time and place for everything. They also say there is a season for every purpose. I believe that the time, place, and season for artificial intelligence (AI) data platforms have arrived. To see this, look no further than Pure Storage, whose core mission is to “empower innovators by simplifying how people consume and interact with data.”
In the past, it was sufficient to bring order to the randomness of enterprise data collection through applications of technology resources (databases and storage devices) that were aimed primarily at organizing, storing, indexing, and managing enterprise information assets for single purposes or single business units. However, this data was still left mostly unexploited for its maximum potential and enterprise-wide business value.
Also in the past, it was sufficient for business automation to consist primarily of rigid rule-based robotic non-adaptive repetition of processes and fixed tasks, requiring very little (if any) new knowledge input (i.e., live data consumption) or real-time adaptation to changing business conditions.
And also in the past, it was sufficient for AI to be relegated to academic researchers or R&D departments of big organizations who mostly produced research reports or journal papers, and not much else.
Fast-forward to 2024 and we see a totally different landscape: massive data sets feeding dynamic cross-enterprise processes, increasing automation and dynamic adaption of complex multi-step tasks, and ubiquitous value-producing applications of AI. In particular, in the past year, generative AI has played a major role in the explosive development and growth of these transformations within enterprises.
Pure Storage Meets the Demands of Enterprise AI
To support, sustain, and assure the continued success and cost-effectiveness of the enormous data-fueled AI-powered transformations in such a rapidly changing environment, Pure Storage has stepped up their delivery of an incredible array of award-winning AI-ready infrastructure (AIRI//S™) products and services with an AI data platform that provides the fundamental AI environment for enterprise data management (storage, access, orchestration, delivery), hyperscaled AI training, and AI inference on demand (on-prem, in data centers, at edge sites, and in micro edge devices).
One example of Pure Storage’s advantage in meeting AI’s data infrastructure requirements is demonstrated in their DirectFlash® Modules (DFMs), with an estimated lifespan of 10 years and with super-fast flash storage capacity of 75 terabytes (TB) now, to be followed up with a roadmap that is planning for capacities of 150TB, 300TB, and beyond. Another example is Pure Storage’s FlashBlade® which was invented to help companies handle the rapidly increasing amount of unstructured data coming into greater use, as required in the training of multi-modal AI models. One more example is Pure Storage’s development of non-disruptive upgrades (NDUs), a feature of Pure Storage’s architecture that permits upgrades and expansion of the data infrastructure with no impact on data availability or performance, and with no downtime or data migrations.
Pure Storage’s Announcements at GTC 2024
The preceding examples are industry-leading and exemplary, and yet there’s still more. At the NVIDIA GTC 2024 conference, Pure Storage announced so much more! Here are a few more details on some of those announcements.
A Data Platform for AI
Data is the fuel for AI, because AI devours data—finding patterns in data that drive insights, decisions, and action. Ease of data orchestration (ingest, cleaning, transformation, discovery, access, exploration, delivery, training, inference, deployment) is essential for data-devouring AI products and services. A data platform for AI is key to innovation and long-term affordability, scalability, sustainability, and advancement of enterprise AI applications. Anything less than a complete data platform for AI is a deal-breaker for enterprise AI. Pure Storage provides the ideal data platform for AI, as it provides unified storage for structured and unstructured data and provides enterprise data services for Kubernetes, supporting the entire AI data pipeline, because storage matters!
At GTC 2024, Pure demonstrated the features of their data platform for AI, specifically highlighting these benefits and features of the platform: (a) Helps organizations accelerate model training and inference; (b) Improves operational efficiency for AI/IT infrastructure teams, as well as AI/ML developers and engineers; (c) Delivers cost and energy efficiency as an enterprise scales their AI operations; and (d) Provides an AI storage platform that delivers ultimate reliability and is built to handle all future AI storage needs.
Optimizing GenAI Apps with RAG—Pure Storage + NVIDIA for the Win!
One of the most popular techniques associated with generative AI (GenAI) this past year has been retrieval-augmented generation (RAG). RAG is the essential link between two things: (a) the general large language models (LLMs) available in the market, and (b) a specific organization’s local knowledge base. In deep learning applications (including GenAI, LLMs, and computer vision), a data object (e.g., document, image, video, audio clip) is reduced (transformed) to a condensed vector representation using deep neural networks. The knowledge base then becomes the comprehensive collection of these condensed representations of the enterprise business data repositories, stored in vector format in a vector database—Vector DB being another major data technology development finding widespread adoption this past year.
As a consequence of these activities, RAG provides the bespoke use case-specific context to an organization’s proprietary GenAI LLM applications. This contextualization of the GenAI LLM is not only enterprise-specific, local, and customized, but it is also proprietary—maintaining the privacy and security of the GenAI LLM application within the security firewalls and policies of that organization. Additionally, RAG ensures the use of an organization’s most recent data while eliminating the need for constant retraining of the LLMs. Pure Storage has worked with NVIDIA (GPU memory and GPU servers) to boost the speed, accuracy, and on-prem power of such enterprise GenAI LLM applications. Here are some specific documented results:
(a) “NVIDIA GPUs are used for compute and Pure Storage FlashBlade//S provides all-flash enterprise storage for a large vector database and its associated raw data. In a specific case [presented at GTC], the raw data consisted of a large collection of public documents, typical of a public or private document repository used for RAG.”
Pure Storage’s RAG pipeline, in conjunction with NVIDIA GPUs and NVIDIA’s NeMo Retriever collection of GenAI microservices, ensures accuracy, currency, privacy, and relevance of proprietary enterprise LLMs. Time to insight and time to action in AI applications are faster and better with Pure Storage.
OVX Validated Reference Architecture for AI-ready Infrastructures
First question: What is OVX validation? OVX is NVIDIA’s standard validation paradigm for computing systems that combine high-performance GPU acceleration, graphics, and AI with fast, low-latency networking that are used to design and power complex 3D virtual worlds and digital twins that are transforming how businesses design, simulate, and optimize complex systems and processes. In this fantastic emerging realm of breathtaking technological achievements and innovations, Pure Storage has achieved OVX validation of their reference architecture for AI-ready infrastructures. At this stage, OVX validation applies directly to the increasing business demand for GenAI workloads (including RAG, LLMs, knowledge bases, and Vector DB), full-stack ready-to-run enterprise AI infrastructure, and local proprietary custom data + AI compute, storage, and networking solutions. Note: When you see “full-stack,” read “Pure Storage + NVIDIA working together seamlessly.”
Second question: What about technical debt and the cost of “lift and shift” to these new AI-ready architectures? For Pure Storage, OVX validation also certifies that Pure Storage’s AI-ready infrastructure will run on NVIDIA GPUs and on other vendors’ servers, which is a great savings on technical debt for those organizations that operate diverse server farms. OVX validation complements Pure Storage’s certified reference architecture for NVIDIA DGX BasePOD that was announced last year as well as their FlashStack® for AI Cisco Validated Designs announced here.
Since one of the only certainties about the future is its uncertainty, it is a great benefit that Pure Storage Evergreen//One™ provides storage-as-a-service (STaaS) guarantees and enables future-proof growth with non-disruptive upgrades. That means that Pure Storage owns the hardware (“the end user doesn’t pay for it”), but the end user buys a subscription to the storage with the same agility and flexibility of public cloud storage, and with all the security, proprietary protection, and performance of on-prem all-flash sustainable infrastructure. This is Pure Storage’s SLA-guaranteed cloud-like STaaS!
More Pure Storage Announcements at GTC 2024
Pure Storage’s RAG development (described earlier) is accelerating successful AI adoption across vertical industries. Pure Storage is accomplishing this by creating vertical-specific RAGs in collaboration with NVIDIA. First, “Pure Storage has created a financial services RAG solution to summarize and query massive data sets with higher accuracy than off-the-shelf LLMs. Financial services institutions can now gain faster insight using AI to create instant summaries and analysis from various financial documents and other sources.” Pure Storage will soon release additional RAGs for healthcare and the public sector.
Expanded investment in the AI partner ecosystem: Pure Storage is further investing in its AI partner ecosystem with NVIDIA, engaging in new partnerships with independent software vendors (ISVs). Some of these investments are aimed at optimizing GPU utilization through advanced orchestration and scheduling, and others enable machine learning teams to build, evaluate, and govern their model development lifecycle. Additionally, Pure Storage is working closely with numerous AI-focused resellers and service partners to further operationalize joint customer AI deployments.
Looking at AI Now and at What’s Next
As the award-winning leader in AI-ready (and future-ready) data infrastructure, Pure Storage is collaborating with NVIDIA to empower their global customers with a proven framework to manage the high-performance data and compute requirements that these enterprises need to drive successful AI deployments, both now and into the future. Every technical leader, line of business (LOB) leader, VP of Infrastructure for AI, VP of AI/Data Science, and CDO/CTO/CAIO can benefit right now from these technologies and services.
To put all of Pure Storage’s recent accomplishments, products, services, and solutions into a single statement, I would say that Pure Storage’s primary purpose (their North Star) is to guide and accelerate their customers’ adoption of AI through the Pure Storage platform for AI.
To learn more about all of this, make connections, learn new skills, and get ready for what’s next in this rapidly evolving season of AI, be sure to register and attend the Pure//Accelerate® 2024 live event June 18-21, 2024, at Resorts World Las Vegas. The event will have a special track on “Today’s and Tomorrow’s Applications of AI.” Don’t miss it!
Register Now for Pure//Accelerate 2024
Drive your data success at Pure//Accelerate® at Resorts World Las Vegas from June 18-21. This is the premier event to make connections, learn new skills, and get ready for what’s next. Here’s a sneak peek of what to expect:
When I was young, my dad told me about an incident that he experienced at work. He was a US Air Force officer. On that particular day, there was an unannounced (surprise) drill – a simulated national defense emergency. Though it was just a drill, it was still important to get it done right and efficiently. One of his responsibilities was to contact certain high-ranking officers and communicate with them about the situation (in this case, the simulated emergency). He told me that, in one case, he was able to make contact by phone with one of those top officers within 20 seconds of the start of the drill. If that guy had been in the office or in one of the major facilities, then those 20 seconds would not have been surprising. But it turns out that it was that guy’s day off and he was out playing golf. This was the early 1970’s – hence, no mobile phones, no cell towers, and no internet in your pocket.
How was it possible to find this guy in the middle of some golf course and have him on the phone within 20 seconds in that era? From my young person’s perspective, it was a miracle! Or maybe it was magic. As Sir Arthur C. Clarke said, “Any sufficiently advanced technology is indistinguishable from magic.” So, what enabled this “magic” in the early 1970’s? The answer: satellite phones! The high-ranking officer was required to have such a device with him at all times (either in his possession or to have an assistant accompany him nearby with that device available at a moment’s notice).
A connected world—with a digital divide
Now, fast-forward to the current digital world – for most of us today, the expectation seems to be that we all should have easy access to ubiquitous mobile phones, cell towers, high-speed broadband networks, and the internet at all times! However, though that may be the rule for most of us, it is not the norm for everyone! There are plenty of exceptions to the rule, especially in those areas of the world that fall in the gaps of today’s digital service providers: underdeveloped countries, remote regions of developed countries, regions in which natural disasters or national emergencies have lost ties to those digital services (including national defense emergencies), and massive public events that attract literally billions of people in-person and online to one specific geolocation (which definitely cannot be handled by the standard placement and distribution of cell towers).
Those exceptions to the rule are not rare. In fact, they are quite common. It is imperative that something be done about this, to close those digital gaps, to bring the benefits of digital services to all, and to boost the global digital business value chain. If we cannot make these technologies available for everyone, we risk perpetuating a divide between the haves and have-nots.
Promoting connectivity on a global scale
There is a company that is addressing this “digital divide” and this data-intensive digital connectivity imperative on a global scale. SES is getting it done with major innovations and 21st century technological upgrades to critical satellite communications, going far beyond the traditional satellite technology of the past century.
But this isn’t my dad’s satellite phone service. The SES connectivity system supplies digital (video and data) connectivity worldwide to broadcasters, mobile network operators, fixed network operators, digital content providers, internet service providers, and organizations of all sorts (government agencies, businesses, and other institutions). SES satellites deliver high-speed, high-volume, broadband, and effectively ubiquitous digital access for these organizations over nearly the entire planet Earth.
When I say, “nearly the entire planet”, I mean 96% of the population. Over seven and a half billion people around the world are now a fraction of a second from contact anywhere, anytime. I believe that we can all agree that a “fraction of a second” (e.g., 150 milliseconds of satellite latency over large areas) is much better than 20 seconds – in fact, over 100X better than the 1970’s counterparts!
Furthermore, when the situation demands it (such as a major event in a specific location that is attracting a billion+ digital viewers, or a localized natural disaster requiring massive global deployment of services to that spot), these satellites can be programmed to focus their beam and full capabilities on transmitting massive digital content in and out of that tiny area.
The fast, global, ubiquitous digital access that we are describing is nothing like the crackly satellite phone conversations of the past (have you seen this in the old movies?). We are talking about smooth and faultless streaming video, crisp and clean phone calls, information-intensive online meetings, error-free data-sharing, and nearly instantaneous social media accesses and interactions by vast numbers of people (think World Cup Finals, or a live concert by top music artists for a global cause). Let’s not forget the time-critical demands of digital business and digital government that require instant access to data-intensive cloud-based data and data services! That’s a massive digital enterprise requirement for a massive number of organizations, not just an entertainment requirement for the masses.
An in-person visit to SES in Virginia
SES is headquartered in Luxembourg, with facilities around the world, including Manassas Virginia, where I visited a few weeks ago when I met with Nihar Shah, Head of Strategy and Market Intelligence for SES. In addition to a fun wide-ranging fact-filled chat with Nihar, there was a lot more that I learned about SES in that short visit. It was a great pleasure for me (who spent nearly 20 years working at NASA) to see the satellite operations center, the “big box of electronics” that is placed on-site at major events (including sporting events, for my and your favorite sports), and the dedicated SES staff. I learned how SES has deployed and operates one of the world’s largest satellite constellations, including MEO (Medium Earth Orbit) satellites and GEO (Geosynchronous Earth Orbit) satellites – different orbits for different requirements, plus an incredible technology stack that manages the communications hand-offs between the MEO satellites as they fly fast over any given location. What did we say earlier about “any sufficiently advanced technology”? It certainly is not magic when it delivers real global benefits and generates significant business value.
To top off this experience, I was immediately impressed when I walked in the front door of the SES Global Operations Center, not only because of the welcoming staff, but also because I saw the company mission statement front and center as I entered the lobby. I am a huge fan of meaningful, believable, inspirational, and actionable mission statements. SES might easily have one of the best mission statements that I have seen – and that’s not only because they refer to their statement of purpose as their “North Star”, which suits me (as an astronomer) very well. The statement reads: “We do the extraordinary in space, to deliver amazing experiences everywhere on Earth.” (Note: see me and Nihar in the attached photo below, with the SES statement of purpose.)
I look forward to learning more about SES. You too can learn more about SES and their global content connectivity solutions at https://www.ses.com/.
Kirk and Nihar Shah at the SES Global Operations Center
