BLOGS

How Infinidat’s Recent Announcement Can Enhance and Accelerate Your ROI

M.R. Pamidi, Ph. D.| Srini Chari, Ph. D.

Cabot Partners

Nowadays, everyone is talking about Generative AI and its promise to fundamentally transform many industries, from Retail to Healthcare to Manufacturing to Financial Services. The critical role of IT infrastructure should be discussed more in the industry, especially the less glamorous part of the IT stack, namely, Data and Storage.

As organizations gather, process, and store larger datasets from all sources, such as sensors, instruments, log files, and so on, their workloads are becoming more compute- and data-intensive. Traditional High-Performance Computing (HPC) is converging with AI (Figure 1), including Generative AI. It places similar extreme management, performance, and scale demands on IT infrastructures (particularly Storage) to the point where the rapidly growing Generative AI workloads need an HPC infrastructure.

Figure 1: The Convergence of HPC with AI

By using proven HPC storage software and systems for AI deployments, organizations can reduce data costs, consolidate compute and storage siloes, simplify system administration, improve efficiency, and more. But they must also ensure security, affordability, performance, scalability, compliance, flexibility to manage service level agreements (SLAs), support different configurations on-premises and in multi-cloud environments, and more. Infinidat has incredible promise to deliver a significant ROI to its clients in these situations.

About Infinidat: Delivering Innovative High-Value Storage Solutions

We recently met Eric Herzog, Infinidat’s CMO, at the Flash Memory Summit in California. Founded in 2011, Infinidat is an enterprise storage company headquartered in Waltham, MA, and Herzliya, Israel. In addition to Series A investment, it has received a Series B investment of $150 million from TPG and a Series C investment of $95 million from Goldman Sachs—for a total of $325 million to date. The company claims over 25% of Fortune 50 as its customers, over 140 patents filed to date, and a 100% availability guarantee of its storage products. It has a global presence, with offices and enterprise customers in multiple countries, enhancing its reach and market share. Infinidat’s management team, with storage industry veterans, has a proven track record of success. The team is committed to developing innovative solutions that meet the needs of its customers, with regular announcements that help clients drive innovation and value.

Most Recent Infinidat Announcements Continue to Enhance Innovation and Value

On September 19, 2023, the company announced impressive enhancements to its already-strong product line by adding SSA Express SDS (Soft-Defined Storage) and InfiniBox^TM SSA (Figure 2).

Figure 2: Latest Impressive Infinidat Product Announcements

Traditionally, customers looking to expand their storage infrastructure often are forced to forklift upgrades, which are both expensive and unwieldy. Infinidat’s SSA SDS Express avoids these expensive upgrades by offering Up to 320 TB of usable all-flash capacity and supporting over 95% of existing InfiniBox® systems that can be easily configured into new InfiniBox purchases. This solution expands an InfiniBox hybrid array by allowing customers to leverage the InfiniBox flash layer for performance-sensitive workloads, essentially creating a solution akin to embedding an all-flash array inside the InfiniBox. The non-disruptive, free InfuzeOS 7.3 software upgrade of existing InfiniBox systems has zero downtime, reducing CapEx and OpEx with no new array management interface or need for a second vendor. SSA SDS Express also supports InfiniSafe® Cyber Detection software with a scanning option and storage recovery guarantees.

InfiniBox^TM SSA II offers twice the storage capacity in the same data center footprint, supporting up to 6.635 PB of Storage, about 50% less power (Watts) per effective TB, reduced rack space, floor tile, and power and cooling costs. Increased capacity and performance at scale help consolidate more all-flash and legacy hybrid arrays, resulting in lower storage management costs. The InfiniBox SSA has several guaranteed SLAs: the InfiniSafe Cyber Resilience and Recovery guarantee, guaranteed performance, and 100% availability guarantee. This product also provides scale-up storage systems, adding 60% and 80% partially populated configurations to the existing 100% populated option. Thus, customers can start at a low entry point and scale up as needed.

InfuzeOS with Neural Cache, InfiniSafe, InfiniOps, InfiniVerse, comprehensive enterprise data services, InfuzeOS Cloud Edition, autonomous automation, guaranteed performance, and availability, and InfiniSafe Cyber Resilience and Recovery customers can manage clients’ storage IT operations on a “Set It and Forget It” model.

Infinidat has a growing customer base and includes some of the world’s largest and most demanding organizations. The company’s customers appreciate InfiniBox’s performance, scalability, reliability, as well as Infinidat’s commitment to customer support. We believe this announcement will further strengthen its customer base and reduce the total cost of ownership (TCO). However, these enhancements will increase the total value of ownership (TVO) and ROI and lower TCO.

Using a TVO Framework to Quantify How Infinidat’s Recent Announcements Can Improve ROI

The TVO framework (Figure 3) categorizes the interrelated cost/value drivers (circles) for Storage solutions by each quadrant: Costs, Productivity/Quality, Revenue/Profits, and Risks. Along the horizontal axis, the drivers are arranged based on whether they are primarily Technology or Business drivers. Along the vertical axis, drivers are organized based on ease of measurability: Direct or Derived.

Figure 3: The TVO Framework for Storage with Cost and Value Drivers

The cost/value drivers for Storage solutions (a circle whose size is proportional to the potential impact on a client’s Total Value (Benefits – Cost) of Ownership or TVO) are as follows:

1. Total Costs of Ownership (TCO): Typical costs include one-time acquisition costs for the hardware and deployment and annual charges for software, maintenance, and operations. As described earlier, the latest Infinidat products can lower the TCO.
2. Improved Productivity: The TVO model quantifies the value of productivity gains of

administrators, end-users, and the organization. Productivity gains can be significant with the latest Infinidat solutions.

3. Revenue and Profits: Faster time to value and more innovation capabilities for clients spur revenues and improve profits.
4. Risk Mitigation: A streamlined process and improved governance/security lowers system downtime and minimizes cumbersome iterations in rework.

The TVO and ROI will typically increase with Storage solution size, giving clients even better ROI and Payback in the future as they deploy Gen AI solutions that will require them to manage petascale Storage. It will benefit Infinidat and clients to quantify and monitor ROI from Infinidat Storage solutions as they continue scaling up their infrastructure to handle their most compute- and data-intensive workloads.

Published by: M.R. Pamidi

Cabot Partners View on How Google Cloud Embraces the Duality of Generative AI and Enterprise AI

by

Jean S. Bozman, President, Cloud Architects Advisors LLC

and

M.R. Pamidi, Ph. D., Principal Analyst, Cabot Partners

 Google Cloud Next’ 23 was a three-day conference in San Francisco from August 29-31, 2023. The event brought together cloud computing professionals worldwide to learn about the latest Google Cloud Platform (GCP) innovations. Google Cloud is reinventing itself as it engages more closely with enterprise businesses that plan to reinvent their business models with AI. Many consider Generative AI as a user-oriented tool for efficient searches. However, the enterprise focus of GCP was apparent from the start of the three-day event.

Google Cloud CEO Thomas Kurian opened the keynote with significant announcements that may help the tech giant keep up with its peers—Amazon and Microsoft—in the still-evolving cloud market. Generative AI was the central theme at the conference, as AI powers many of the latest advancements and features. Google continues to face acute pressures to increase its AI offerings as competition from its rivals heats up.

Figure 1: Google Cloud Next 2023 Key Announcement

Some of the key announcements (Figure 1) from the event included:

• General availability of Duet AI, a new natural language processing (NLP) technology that can help businesses generate more creative and engaging content.
• Preview of Vertex AI Workbench, a new integrated development environment (IDE) for building and deploying machine learning (ML) models.
• Launch of Cloud TPU v5e, a new machine learning accelerator that can train models up to three times faster than previous generations.
• Availability of A3 Virtual Machines (VMs) based on NVIDIA’s H100 GPUs offers up to 80% better performance for ML workloads than previous generations.
• Introduction of Cross-Cloud Network is a new networking platform that makes connecting and securing applications across multiple clouds easier.
• The Vertex AI ecosystem expands with new partnerships with DocuSign, Box, Canva, and Salesforce.

The event showcased Google’s commitment to innovation in the cloud computing space. It highlighted the company’s latest technologies for building, deploying, and managing applications and its growing ecosystem of partners.

In addition to the keynote presentations and breakout sessions, the event also featured several hands-on labs and workshops. These sessions allowed attendees to learn about GCP technologies and try them out for themselves.

Google emphasized the duality of personal AI and enterprise AI throughout the three-day conference, along with customer examples from some of GCP’s largest customers, including General Motors, Mahindra Motors, and US  Steel. These customers are running end-to-end enterprise applications for mission-critical workloads on Google Cloud infrastructure, as in global communications (GM’s OnStar), retail sales (Mahindra), and manufacturing (US Steel). The cloud company currently has a $32 billion annual run rate, the largest in its history, as enabled by enterprise AI and the capabilities of the Google Cloud infrastructure.[1]

Kurian, who took the helm at Google Cloud in 2019, has deep experience with enterprise infrastructure; he was Oracle Corp.’s President from 1996 to 2018, reporting to Oracle CEO Larry Ellison and, later, to CEOs Mark Hurd and Safra Catz and engaging with Oracle’s enterprise customers worldwide.

Personal GenAI and Enterprise-Wide GenAI

“We’re announcing important new products to help each of you use generative AI,” said Google Cloud CEO Thomas Kurian, “First, to help the world’s leading customers and developers to build AI with our AI-optimized infrastructure and our AI platform for developers, Vertex AI.” “Generative AI (Figure 2) will likely spark greater customer requirements for performance, scalability, and reliability,” Kurian said, relating user-based AI usage to enterprise-wide AI for corporate applications. The new Duet AI product is positioned as an assistant and collaborative software for end-users.

Figure 2: Google’s Generative AI Stack

Google Cloud’s generative AI product for business users, Duet AI, is positioned as a collaborative assistant, while the Vertex AI targets developers – and end-to-end enterprise AI benefits from Google Cloud’s scalability, performance, and reliability, supporting millions of users across 38 geographic regions worldwide. That network of Google Cloud operations reaches the Americas (North America and South America), the European continent, Asia/Pacific, and Australia) – and will likely continue to expand as GCP competes with AWS and Azure for more business across all continents.

“We’re announcing important new products to help each of you use generative AI,” said Google Cloud CEO Thomas Kurian, “to help the world’s leading customers and developers to build AI with our AI-optimized infrastructure and our AI platform for developers, Vertex AI.”

Kurian has deep experience with enterprise infrastructure; his previous work was as Oracle’s President, engaging with Oracle’s enterprise customers with a global presence worldwide. He expects that broader use of generative will lead to faster cloud adoption for transactional and corporate applications that have not yet moved to the cloud.

Google and AI

Google has long been known as an inventor and developer of new AI technologies. Now, with the fall release of ChatGPT from Microsoft and OpenAI, Google Cloud is showing its Vertex AI, Duet AI, and GCP AI infrastructure as a competitive offer to enterprises. In many cases, those enterprises are working to update aging on-premises applications – often with reduced IT staffing as they consolidate their corporate data centers — and to speed the adoption of AI-based technology throughout their business operations (Core to Cloud to Edge) to help their enterprises achieve better business results.

AI will “touch every sector, every industry, every business function and significantly change the way we live and work,” said Alphabet and Google CEO Sundar Pichai, pointing to Google Cloud’s enterprise focus from the start of the three-day conference in San Francisco.

Business executives, Pichai said, “want a partner that’s been on the cutting edge of technology breakthroughs—be it from the desktop to mobile, to the cloud, or now to AI—and a partner who can help navigate and lead them” through the next phase of digital transformation driven by AI.

GCP Investment and Adoption

Google and Google Cloud, Pichai said, have invested heavily “in the tooling, foundation models and infrastructure” to make that happen, starting with specialized TPU and GPU processors and continuing with software that can run customers’ containers and virtual machines on the GCP.

Pichai cited three customers who are leveraging Google’s GenAI software, which was introduced in March: General Motors (GM), which applied conversational AI to its OnStar-connected vehicles; HCA HealthCare, which is working with Google to create Med-PaLM foundation models for scalable AI; and US Steel, which is using Google AI to summarize and visualize instruction manuals.

Below is a detailed look at everything Google announced at its event.

Vertex AI Gets Better

Figure 3: Vertex AI Architecture

About two years ago, Google unveiled Vertex AI, a unified artificial intelligence platform that offers all of Google’s cloud services under one roof. With Vertex AI, you can build ML models or deploy and scale them easily using pre-trained and custom tooling. It was followed by Vertex AI Vision, the machine learning platform as a service (ML PaaS) offering from Google Cloud. Since the general availability of generative AI services based on Vertex AI earlier this year, developers can use several new tools and models, such as the word completion model driven by PaLM 2, the Embeddings API for text, and other foundation models. Google is adding Meta’s Llama 2 and Technology Innovation Institute’s (TII) loyalty-free Falcon 40B, the UAE’s leading large-scale open-source AI model. This makes Google Cloud the only Cloud provider to support first-party, open-source, and third-party models. APIs of these models will be accessible through a new tool in its Cloud Platform named Model Garden.

Duet AI For Google Workspace

At I/O 2023, Google announced “Duet AI” as the branding for generative AI features in Workspace. At that time, its availability was limited to trusted testers by invitation only. However, now Google has made Duet AI for Google Workspace available for all users with a no-cost trial. Google will charge $30 per user for access to Duet, as per CNBC.

Duet AI is a development interface powered by AI that includes code and chat assistance for developers on Google Cloud’s platform. Duet AI covers a range of generative AI tools for Google‘s productivity apps, including Gmail, Drive, Slides, Docs, and more. Google is essentially taking on Microsoft‘s Co-pilot.

Google wants to make Gmail, Docs, Sheets, Slides, and Meet more helpful with the help of generative AI.

At I/O, Gmail got a new feature called “Help me write,” allowing people to use generative AI to send auto-replies to emails and modify them to meet their needs best. Meanwhile, Sheets has a “Help me organize” feature where users can ask for information to be organized for them in the sheet through a simple word prompt. Similarly, in Google Slides, there’s a new “Help me visualize” entry where users can use prompts to get AI-generated images.

Google also announced further Workspace AI integration in Google‘s other core apps, such as Meet and Chat. Within Meet, Google‘s new AI features include taking notes in real-time: When they click “take notes for me,” the app will capture a summary and action items as the meeting goes on. Google will be able to show them a mid-meeting summary so that they can catch up on what happened.

Another new Meet feature allows Duet to “attend” a meeting on your behalf. Users click on the “attend for me” button on a meeting invite, and Google can auto-generate some text about topics the users might want to discuss. Those notes will be viewable to attendees during the meeting so that they can discuss them in real time.

Other new features include dynamic tiles and face detection, giving users their video tile labeled with their name in a meeting room. An automated translated captions feature will detect when another language is spoken and display the translation on-screen. The new automatic translated captions feature supports up to 18 of the world’s languages.

Lastly, Google is integrating Duet AI into Google Chat. Now, users can chat directly with Duet AI and ask questions about their content, get a summary of documents in a space, and catch up on missed conversations. It’s easier to use Google Chat because of the new interface and a new shortcut option. Google also integrates “smart canvas” capabilities, such as smart chips, inside Google Chat. Google Chat now lets users add up to 50,000 members to a space. The change marks a significant increase from the previous limit of 8,000 members. In addition, a new feature called Huddles is coming to Google Chat. With Huddles, instead of jumping out of the conversation into a meeting, the meeting is integrated directly into the chat experience. Google says Huddles will be available in public preview by the end of the year. In the coming weeks, Google will add support for third-party apps to Chat, including products from Zoho, Workday, and Loom.

Gen AI Unicorns and Google Cloud

During the event, Google said 70 percent of Generative AI unicorns—and more than half of all funded AI startups—are Google Cloud customers. This year’s Google Cloud Next ’23 summit illustrated how Google offers the optimized AI-optimized infrastructure to host and run AI models. That is intended to be a selling point for business decision-makers who have hesitated to migrate mission-critical applications to cloud service providers (CSPs).

Our Analysis

When the pandemic began, the pace of cloud migration accelerated. Still, some applications never moved from corporate data centers, worried that cloud service providers could not meet legacy applications’ security, availability, and governance.

The most widely used Cloud Service Providers (CSPs), including AWS, Microsoft Azure, and Google Cloud Platform, benefited from cloud migrations. Still, many more applications have not yet moved to the cloud. IDC has reported that many enterprise customers subscribe to two, three, or more cloud providers, but still, nearly half of overall applications remain in corporate data centers.

That’s why Google Cloud executives and leading customers spoke about Google’s heavy investments in infrastructure in recent years. “Our ultra-scale, highly reliable, AI supercomputing systems combine TPU and GPU accelerators with high-performance AI-optimized storage, scalable networking, offloads, and water-cooling,” Kurian said. The firm’s compilers and software tools optimize AI  models “to deliver the best performance, latency, and overall throughput.”

Kurian cited those customers—and more—including Yahoo!, which is migrating 500 million mailboxes, with nearly 500 petabytes of data, to run on Google Cloud. He specifically cited Google Cloud infrastructure as a differentiator, given its development of the GKE Kubernetes engine, user-focused products for AI—Vertex AI for developers and Duet AI for users—and optimized cloud infrastructure hardware for end-to-end management of AI resources. Beyond that, Google Cloud is building an AI ecosystem to engage with enterprise customers. Now, Google Cloud must reach out to more business decision-makers – including CXOs and finance executives – to convince them that now is the time to move the next wave of business workloads to the cloud, using Google Cloud services to do so.

[1] “Google Cloud Begins Profitability Era: 5 Huge Q2 Earnings Takeaways,” CRN Cloud News, July 26, 2023.

Published by: M.R. Pamidi

Estimating the Ginormous Growth of Genomics Data and Storage

Genomic data science is a field of study that enables researchers to use powerful computational and statistical methods to decode the functional information hidden in DNA sequences. Our ability to sequence DNA has far outpaced our ability to decipher the information it contains, so genomic data science will be a vibrant field of research for many years to come.

Researchers are now generating more genomic data than ever before to understand how the genome functions and affects human health and disease. These data are coming from millions of people in various populations across the world. Much of the data generated from genome sequencing research must be stored, even if over time, older information is discarded. The demand for storage is colossal with increased and new areas of life sciences research.

New areas of life sciences research are driving up data volumes

We identified four major life sciences research areas that are generating large volumes of data. For each area, we estimated the demand for the instruments that are producing this data. We then estimated average data generated per year by each of the instruments. The total storage demand over the study scope period is obtained by multiplying the number of sequencing instruments with the average data generated by each instrument.

Following areas of life sciences were considered in the data generation estimation:

Next-generation sequencing (NGS) is a massively parallel sequencing technology that offers ultra-high throughput, scalability, and speed.

RNA sequencing (RNA-Seq) uses the capabilities of high-throughput sequencing methods to provide insight into the transcriptome of a cell.

Spatial transcriptomics are methods designed for assigning cell types (identified by the mRNA readouts) to their locations in the histological sections. This method can also be used to determine subcellular localization of mRNA molecules.

Cryogenic electron microscopy (cryo-EM) is a cryo microscopy technique applied on samples cooled to cryogenic temperatures. This approach has attracted wide attention as an alternative to X-ray crystallography or NMR spectroscopy for macromolecular structure determination without the need for crystallization.

Single-cell sequencing technologies refer to the sequencing of a single-cell genome or transcriptome, so as to obtain genomic, transcriptome or other multi-omics information to reveal cell population differences and cellular evolutionary relationships.

How to quantify the colossal storage needs?

We made the following assumptions to estimate the storage needs between years 2021 through 2028.

1. We assumed the information provided in the paper[1] “Big Data – astronomical or genomical” published in 2015 as the basis of our calculation. In the year 2015, the authors working with various subject matter experts surmised that in 2015, there were 2500 gene sequencing instruments and the storage need at that time was 9 Petabytes (PBs).
2. Based on their and other research studies, they looked into two possibilities
   1. Storage needs will double every 12 months (conservative case)
   2. Storage needs will double every 7 months (aggressive case).
3. In our estimation exercise, we assumed that the growth rate will be the average of the above two cases.
4. Based on our discussion with experts, we assumed storage needs per RNA, Spatial transcriptomics, Single-cell sequencing instrument will be 4,8, and 100 times that of an NGS sequencing instrument respectively.
5. We worked with a Grand View Research, a market research company to estimate the global volumes of NGS sequencing, RNA sequencing, Spatial transcriptomics, single cell sequencing and Cryo EM instruments for the period between 2021 thru 2028.
6. The volume of each instrument type multiplied by unit storage requirement for each category provided the total storage need. We also assumed that each year, 80% of the data will be retained (i.e.,20% discarded).
7. The following table provides a summary of our estimate.

From the table, we see that the need for storage for sequencing is around 54 Exabytes in 2028 (assuming a retention of 80%). For this exercise, we have made many assumptions re: growth, sequencing technologies, retention, and size based on what we know today. One thing is certain: storage needs for life sciences and genomics will only increase, and it will be ginormous.

[1] https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002195

Published by: Srini Chari

As the world gathers to fight climate change, let’s recognize the critical role of HPC, AI and Cloud Computing

This week in Glasgow, the COP26 summit will bring global leaders together to accelerate action towards combating climate change. This is happening as energy consumption, mostly with polluting fossil fuels, is at an all-time high. This may seem good for the oil, gas, and coal industries, but it isn’t. The grave reality is that the fossil fuel industry is under immense pressure to mitigate climate change and decarbonize since high levels of energy consumption are causing unsustainable levels of C0₂ and other greenhouse gases.

So, the energy industry is undergoing a profound transition from fossil fuels to renewable and clean energy sources. As oil and gas companies decarbonize, they are looking into new and economically viable solutions including potentially becoming carbon-neutral energy companies.  For this they are investing heavily in physical infrastructure and boosting investments in High-Performance Computing (HPC) and artificial intelligence (AI) to innovate and quickly solve critical problems in this transition.

Oil and gas companies have always used HPC (seismic processing and reservoir simulation) and certain forms of AI technology (analytics) for decades to improve decision-making regarding exploration and production, and to reduce investment risks particularly as fossil fuels become harder to extract. Now as oil and gas companies transition, they must be able to handle newer interdisciplinary workloads in Geophysics, Computer-aided engineering (CAE), Life sciences, Combustion/Turbulence models, Weather modeling, Material science, Computational chemistry, Nuclear engineering, and Advanced optimization.

For this oil and gas companies are increasing their investments in HPC, AI and other innovative and agile solutions to handle exploding compute/storage requirements. As the use of AI and HPC continues to grow in the energy industry, cloud computing is making is easier to process spiky workloads and improve overall user experience. So, many oil and gas companies are using hybrid cloud solutions with several deployment options.

The harmful impact of climate change is becoming more severe day by day, threatening the survival of the planet. The entire world is coming together to fight it.  Oil and gas companies are doing their part and transitioning to renewable energy sources. This transition is hard and expensive but is made easier with novel applications and extensions of proven HPC and AI solutions oil and gas companies have used for years. As the world focuses this week on climate change, let’s also recognize the critical role HPC and AI play in solving one of mankind’s most pressing challenges.

You can learn more by reading this Hewlett Packard Enterprise whitepaper that Cabot Partners recently helped create.

Published by: Srini Chari

HPC and AI enable breakthroughs in genomics for better healthcare

Many Life Sciences organizations are using digital technologies to meet the needs and expectations of patients. These technologies help treat and manage diseases in new ways. HPC and AI solutions are at the forefront of this. They are needed to accelerate breakthroughs in large-scale genomics.

Genomics is a sub-discipline of molecular biology that focuses on the storage, function, evolution, mapping, and editing of genomes. It is a vital and growing field because it can improve the lifestyle and outcomes for patients. In the next five years, the economic impact of genomics is estimated to be in the hundreds of billion to a few trillion dollars a year.

Next-generation Sequencing (NGS), Translational and Precision Medicine

High-performance computing (HPC) and Artificial Intelligence (AI) are essential for Genomics primarily because they help speed up NGS which processes and reduces the amount of raw data into a usable format. NGS helps determine the sequence of DNA or RNA to study genetic variations associated with diseases or other biological phenomena.

After this NGS step, it is important to establish the relationship between genotypes to understand the influence individual DNA variances have on disease and medical outcomes. This is Translational medicine. The final step is Precision or Personalized medicine which customizes disease prevention and treatment for an individual based on their genetic makeup, environment, and lifestyle. This last step uses all the data collected using HPC and AI technology to create a personalized approach for the patient.

How HPC and AI help accelerate genomics

Genomics is difficult, but new HPC technology is making this process easier. NGS deals with complex algorithms that require a lot of memory to assemble and solve. HPC and AI solutions help speed up this process and make it cheaper and more accurate.  Using the cloud and new IT solutions, data sequence analysis also becomes easier and can be done at a much larger scale.

Translational medicine requires HPC solutions that can process a lot of data efficiently. This step looks for relationships between a lot of genes, DNA, and diseases and makes it possible to provide personalized treatment for a patient.

HPC and AI are game changers in life sciences. They make genomics easier and faster for healthcare providers, so they can provide highly effective personalized care for their patients. We expect HPC and AI in Life Sciences to continue grow even more rapidly especially in the post-COVID-era by enabling breakthroughs in vaccines, personalized medicine, and healthcare.

You can learn more by reading this Hewlett Packard Enterprise and NVIDIA whitepaper that Cabot Partners recently helped create.

Published by: Srini Chari

The Promise and Peril of RPA

RPA or Robotic Process Automation emulates human activity when interacting with digital software. It automates tedious and mundane business processes. Artificial Intelligence (AI) when integrated with RPA increases business value. AI can directly be used in bots to execute tasks without human intervention. This results in better efficiency, and improved customer and employee experiences.

RPA software revenue is growing rapidly despite economic disruptions caused by the COVID-19 pandemic and is projected to reach $1.89 billion in 2021 with double-digit rates through 2024.

Automating processes with RPA seems like a great solution in theory, but in practice, this isn’t the case. RPA has been successful for some but disappointing for others. While many organizations are relatively happy with their automation investment, most haven’t fully realized the ROI promised by RPA software vendors. For this reason, clients need to carefully evaluate the various RPA vendors before making this strategic investment.

Read this Cabot Partners paper for more details.

Published by: Srini Chari

TVO Analysis of Federated Learning with IBM Cloud Pak for Data

Analytics and AI are profoundly transforming how businesses and governments engage with consumers and citizens. Across many industries, high value transformative use cases in personalized medicine, predictive maintenance, fraud detection, cybersecurity, logistics, customer engagement and more are rapidly emerging. In fact, AI adoption alone has grown an astounding 270% in the last four years and 40% of organizations expect it to be the leading game changer in business[1]. However, for analytics and AI to become an integral part of an organization, numerous deployment challenges with data and infrastructure must be overcome – data volumes (50%), data quality and management (47%) and skills (44%)[2].

In addition, many companies are beginning to use hybrid cloud and multi-cloud computing models to knit together services to reach higher levels of productivity and scale. Today, large organizations leverage almost five clouds on average. About 84% of organizations have a strategy to use multiple clouds[3].

IBM Cloud Pak for Data is an end-to-end Data and AI platform that reduces complexity, increases scalability, accelerates time to value and maximizes ROI with seamless procedures to extend to multiple clouds. While Cloud Pak for Data and can run on any public or private cloud, it is also modular and composable allowing enterprises to embrace just the capabilities that they need on-premises. So, it is truly a hybrid multi-cloud platform.

Recently, IBM announced enhancements to IBM Cloud Pak for Data (Version 3.5). These enhancements can be broadly grouped into 2 key themes:  Cost Reduction and Innovation to drive digital transformation. Customers can drive down costs through automation, consolidated management and an integrated platform. On the innovation front, Accelerated AI, Federated Learning, improved governance & security and an expanded ecosystem are the key focus areas. In this blog, we primarily focus on the value of Federated Learning.

Federated learning (also known as collaborative learning) is a machine learning technique that trains an algorithm across multiple decentralized edge devices or servers with local datasets, without transferring them ( Figure 1). The data stays local and it allows for executing deep learning algorithms while preserving privacy and security.   This approach is different from traditional centralized machine learning techniques where all the local datasets are uploaded to one server and deep learning ML algorithms are executed on this aggregated dataset.   

Figure 1: Comparison of Federated Learning and a Standard Approach

Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus maintaining data privacy, data security, data access rights and access to heterogeneous data. Many industries including defense, telecommunications, IoT, healthcare, manufacturing, retail and others use federated learning and getting significant additional value from their AI/ML initiatives.

For IBM Cloud Pak for Data, this additional value can be quantified using the Cabot Partners Total Value of Framework.

High Level TVO Framework for Federated learning

TVO analysis is an ideal avenue to quantify and compare the value of Federated Learning compared to the standard approach for Machine Learning.  In the TVO analysis, the Total Value (Total Benefits – Total Costs) of IBM Cloud Pak for Data solution with Federated Learning is compared against IBM cloud Pak for Data solution without Federated Learning

The TVO framework (Figure 2) categorizes the interrelated cost/value drivers (circles) for Analytics by each quadrant:  Costs, Productivity, Revenue/Profits and Risks. Along the horizontal axis, the drivers are arranged based on whether they are primarily Technical or Business drivers. Along the vertical axis, drivers are arranged based on ease of measurability: Direct or Derived.

The cost/value drivers for Analytics are depicted as circles whose size is proportional to the potential impact on a client’s Total Value (Benefits – Cost) of Ownership or TVO as follows:

1. Total Costs of Ownership (TCO): Typical costs include: one-time acquisition costs for the hardware and deployment, and annual costs for software, maintenance and operations. For the case without Federated Learning, the costs associated with data transfer to a central repository need to be considered. 

Figure 2: TVO Framework for Federated Learning with Cost/Value Drivers

• Improved Productivity: The TVO model quantifies the value of productivity gains of data scientists, data engineers, applications developers and the organization. It should also consider the value associated with the availability of additional heterogeneous data due to Federated Learning. Federated Learning enables mobile phones to collaboratively learn a shared prediction model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud and the value associated with this innovation need to be considered for applicable cases.   
• Revenue/Profits: Benefit of Federated Learning is access to a large pool of data, resulting in increased machine learning performance, while respecting data ownership and privacy.  Faster time to value with better performance results in greater innovation and better decision-making capabilities which spur growth, revenues and improve profits. 
• Risk Mitigation: Federated Learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing users to address critical issues such as data privacy, data security, data access rights which also allows for improved governance and compliance.  

The above Framework is a simplified pictorial look of TVO analysis. In a rigorous TVO analysis, which is a major offering of Cabot Partners, the elements of the framework are quantified and expressed in easily understandable business terms. In addition, the analysis can be expanded include other innovation features.   

Conclusions

IBM, recently announced enhancements to IBM Cloud Pak for Data (version 3.5). The enhancements focus primarily on cost reduction and Innovation to drive digital transformation. A major element of innovation is Federated Learning. As detailed above, Federated Learning amplifies the value of IBM Cloud Pak for Data through:

• Lower costs – no costs associated with data migration to a central database location   
• Availability of heterogeneous data improves the quality of ML models
• Access to larger pool of data resulting in increased ML performance
• Improved security
• Multiple actors to build a common robust ML model without sharing data, thus allowing to address critical issues such as data privacy and data access rights

[1] https://futureiot.tech/gartner-ai-adoption-growing-despite-skills-shortage/

[2] Ritu Jyoti, “Accelerate and Operationalize AI Deployments Using AI – Optimized Infrastructure”, IDC Technology Spotlight, June 2018  

[3] RightScale® STATE OF THE CLOUD REPORT 2019 from Flexera™

Published by: Ravi Shankar

IBM Storage Simplified for Multi-cloud and AI

A profound digital transformation is underway as High-Performance Computing (HPC) and Analytics converge to Artificial Intelligence/ Machine Learning/Deep Learning (AI/ML/DL). Across every industry, this is accelerating innovation and improving a company’s competitive position, and the quality and effectiveness of its products/services, operations and customer engagement. Consequently, with 2018 revenues of $28.1 billion, the relatively new AI market is rapidly growing at 35.6% annually^[1].

As the volume, velocity and variety of data continue to explode, spending on storage systems and software just for AI initiatives is already almost $5 billion a year and expected to grow rapidly.[2] In addition, many companies are beginning to use hybrid cloud and multi-cloud computing models to knit together services to reach higher levels of productivity and scale. Today, large organizations leverage almost five clouds on average. About 84% of organizations have a strategy to use multiple clouds[3] and 56% plan to increase the use of containers[4]

What’s needed to handle the data explosion challenges are simple, high-performance and affordable storage solutions that work on hybrid multi-cloud environments (Figure 1).

Figure 1: Data Challenges, Storage Requirements and Solutions for HPC, Analytics and AI

Key Storage Requirements

Scalable and affordable: These two attributes don’t always co-exist in enterprise storage. Historically, highly scalable systems have been more expensive on a cost/capacity basis. However, newer architectures allow computing and storage to be integrated more pervasively and cost-effectively throughout the AI workflow.

Intelligent software: This helps with the cumbersome curatorial, data cleansing tasks, and help run and monitor compute and data-intensive workloads efficiently and reliably from the edge to the core to multiple clouds. It also greatly improves the productivity of highly skilled Data Scientists, Data Engineers, Data Architects, Data Stewards and others throughout the AI workflow.

Data integration/gravity: This provides the flexibility to simplify and optimize complex data flows for performance even with data stored in multiple geographic locations and environments. Wherever possible, moving the algorithms to where the data resides can accelerate the AI workflow and eliminate expensive data movement costs especially when reusing the same data iteratively.

Storage Solutions Attributes

Parallel: As clients add more storage capacity (including Network Attached Storage – NAS), they are realizing that the operating costs (including downtime and productivity loss) of integrating, managing, securing and analyzing exploding data volumes are escalating. To reduce these costs, many clients are using high performance scalable storage with parallel file systems which can store data across multiple networked servers. These systems facilitate high-performance access through concurrent, coordinated input/output operations between clients and storage nodes across multiple sites/clouds.

Hybrid: Different data types and stages in an AI workflow have varying performance requirements. The right mix of storage systems and software is needed to meet the simultaneous needs for scalability, performance and affordability, on premises and on the cloud. A hybrid storage architecture combines file and object storage to achieve an optimal balance between performance, archiving, and data governance and protection requirements throughout the workflow.

Software-defined: It is hard to support and unify many siloed storage architectures and optimize data placement to ensure the AI workflow runs smoothly with the best performance from ingest to insights. With no dependencies in the underlying hardware, Software-defined Storage (SDS) provides a single administrative interface and a policy-based approach to aggregate and manage storage resources with data protection and scale out the system across servers. It also provides data-aware intelligence to dynamically adapt to real-time needs and orchestrate IT resources to meet critical service level agreements (SLAs) in parallel, virtual and hybrid multi-cloud environments. SDS is typically platform agnostic and supports the widest range of hardware, AI frameworks, and APIs.

Integrated: A lot of AI innovation is occurring in the cloud. So, regardless of where the data resides, on-premises storage systems with cloud integration will provide the greatest flexibility to leverage cloud-native tools. Since over 80% of clients are expected to use at least two or more public clouds[5], there will be a need for smooth and integrated data flow to/from multisite/multi-cloud environments. This requires more intelligent storage software for metadata management and integrating physically distributed, globally addressable storage systems.

IBM Spectrum Storage provides these attributes and accelerates the journey to AI from ingest to insights.

IBM Spectrum Storage and Announcements – October 27, 2020

IBM Spectrum Storage is a comprehensive SDS portfolio that helps to affordably manage and integrate all types of data in a hybrid, on-premises, and/or multi-cloud environment with parallel features that increase performance and business agility. Already proven in HPC, IBM Spectrum Storage Software comes with licensing options that provide unique differentiation and value at every stage of the AI workflow from ingest to insights

On October 27, 2020, IBM announced new capabilities and enhancements to its storage and modern data protection solutions that are designed to:

• Enrich protection for containers, and expand cloud options for modern data protection, disaster recovery, and data retention
• Expand support for container-native data access on Red Hat OpenShift
• Increase container app flexibility with object storage.

These enhancements are primarily designed to support the rapidly expanding container and Kubernetes ecosystem, including RedHat OpenShift, and to accelerate clients’ journeys to hybrid cloud. This announcement further extends an enterprise’s capabilities to fully adopt containers, Kubernetes, and Red Hat OpenShift as standards across physical, virtual and cloud platforms.

IBM announced the following new capabilities designed to advance its storage for containers offerings:

• The IBM Storage Suite for Cloud Paks is designed to expand support for container-native data access on OpenShift. This suite aims to provide more flexibility for continuous integration and continuous delivery (CI/CD) teams who often need file, object, and block as software-defined storage. This is an enhancement with new Spectrum Scale capabilities.

• Scheduled to be released in 4Q 2020, IBM Spectrum Scale,a leading filesystem for HPC and AI, adds a fully containerized client and run-time operators to provide access to an IBM Spectrum Scale data lake, which could be IBM Elastic Storage systems, or an SDS deployment. In addition, IBM Cloud Object Storage adds support for the open source s3fs file to object storage interface bundled with Red Hat OpenShift

• For clients who are evaluating container support in their existing infrastructure, IBM FlashSystem provides low latency, high performance, and high-availability storage for physical, virtual, or container workloads with broad CSI support. The latest release in 4Q 2020 includes updated Ansible scripts for rapid deployment, enhanced support for storage-class memory, and improvements in snapshot and data efficiency

• IBM Storage has outlined plans for adding integrated storage management in a fully container-native software-defined solution. This solution will be managed by the Kubernetes administrator, and is designed to provide the performance and capacity scalability demanded by AI and ML workloads in a Red Hat OpenShift environment.

• IBM intends to enhance IBM Spectrum Protect Plus, to protect Red Hat OpenShift environments in 4Q 2020. Enhancements include ease of deployment with the ability to deploy IBM Spectrum Protect Plus server as a container using a certified Red Hat OpenShift operator, the ability to protect metadata which provides the ability to recover applications, namespaces, and clusters to a different location, and expanded container-native and container-ready storage support. IBM is also announced the availability of a beta of IBM Spectrum Protect Plus on Microsoft Azure Marketplace

• IBM announced expanded hybrid cloud support for IBM Spectrum Protect, new Gen7 b-type SAN switches and directorswith new IBM SANnav management software, both available in 4Q 2020, and plans for products based on new LTO 9 tape technology designed to enhance cyber resilience, expected in 1H 2021.

Conclusions

As HPC and Analytics grow and converge, clients can continue to leverage these new IBM storage capabilities to overcome the many challenges with deploying and scaling AI across their enterprise. These simple storage solutions – on-premises or on hybrid multi-clouds can accelerate their AI journey from ingest to insights.

[1] https://www.idc.com/getdoc.jsp?containerId=US45334719

[2] http://www.ibm.com/downloads/cas/DRRDZBL2

[3] RightScale® STATE OF THE CLOUD REPORT 2019 from Flexera™

[4]https://www.redhat.com/cms/managed-files/rh-enterprise-open-source-report-detail-f21756-202002-en.pdf

[5] https://www.gartner.com/smarterwithgartner/why-organizations-choose-a-multicloud-strategy/

Published by: Ravi Shankar

Cloudera Introduces Analytic Experiences for Cloudera Data Platform

Cloudera recently announced new enterprise data cloud services on Cloudera Data Platform (CDP): CDP Data Engineering; CDP Operational Database; and CDP Data Visualization. The new services include key capabilities to help data engineers, data analysts, and data scientists collaborate across the entire analytics workflow and work smarter and faster. CDP enterprise data cloud services are purpose-built to enable data specialists to navigate the exponential data growth and siloed data analytics operating across multiple public and private clouds.

Data lifecycle integration enables data engineers, data analysts and data scientists to work on the same data securely and efficiently, no matter where that data may reside or where the analytics run. CDP not only helps to improve individual data specialist productivity, it also helps data teams work better together, through its unique hybrid data architecture that integrates analytic experiences across the data lifecycle and across public and private clouds. Effectively managing and securing data collection, enrichment, analysis, experimentation and analytics visualization is fundamental to navigating the data deluge. The result is data scientists and engineers can collaborate better and more rapidly deliver data-driven use cases. Following are the new enterprise cloud services announcements:

CDP Data Engineering: is a powerful Apache Spark service on Kubernetes and includes key productivity enhancing capabilities typically not available with basic data engineering services. Preparing data for analysis and production use cases across the data lifecycle is critical for transforming data into business value. CDP Data Engineering is a purpose-built data engineering service to accelerate enterprise data pipelines from collection and enrichment to insight, at scale.

CDP Operational Database: is a high-performance NoSQL database service that provides scale and performance for business-critical operational applications. It offers evolutionary schema support to leverage the power of data while preserving flexibility in application design by allowing changes to underlying data models without having to make changes to the application. In addition, it provides Auto-scaling based on the workload utilization of the cluster to optimize infrastructure utilization.

CDP Data Visualization: CDP Data Visualization simplifies the curation of rich, visual dashboards, reports and charts to provide agile analytical insight in the language of business, democratizing access to data and analytics across the organization at scale. It allows Technical teams to rapidly share analysis and machine learning models using drag and drop custom interactive applications. It provides business teams and decision makers the data insights to make trusted, well informed business decisions.

These data cloud services in combination with CDP are purpose-built for data specialists. They deliver rapid, real-time business insights with the enterprise-grade security and governance and will permit Cloudera to continue to be a leader in data science.

Published by: Ravi Shankar