The Real ROI of Enterprise AI | HP, ServiceNow & Accenture

11/27/24 • 41 min

The “ROI of AI” has been marketed as a panacea, a near-magical solution to all business problems.

Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?”

This week on Chain of Thought, Galileo’s CEO, Vikram Chatterji joins Conor Bronsdon to discuss AI's value proposition, from the initial hype to the current search for tangible returns, offering insights into how businesses can identify the right AI use cases to maximize their investment.

Next, we’re joined by a panel of AI experts to discuss the ROI of Enterprise AI, featuring Alex Klug, Head of Product, Data Science & AI at HP; Sriram Palapudi, Sr. Dir, ML Platform Engineering at ServiceNow; and Jay Subrahmonia, Global MD for AI Research & Products at Accenture.

Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line.

Chapters: 00:00 Current State of AI Investments

03:59 Challenges and Solutions in AI Implementation

08:30 Identifying and Prioritizing AI Use Cases

10:53 Ensuring Trust and Explainability in AI

15:29 Measuring ROI and Efficiency Gains

21:10 Panel Discussion Begins

21:54 Trust and Risk Management at HP

23:27 Accenture's Approach to Operationalizing AI

26:06 ServiceNow's Trade-offs and Prioritization

31:17 Measuring the success of AI for customers

36:29 Frameworks and Best Practices

40:57 Conclusion and Final Thoughts

Follow:

Vikram Chatterji: ⁠https://www.linkedin.com/in/vikram-chatterji/ Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Alex Klug: https://www.linkedin.com/in/alex-klug-67ba3655/ Sriram Palapudi: https://www.linkedin.com/in/sriram-palapudi-11294b1/ Jay Subrahmonia: https://www.linkedin.com/in/jayashree-subrahmonia-99963a/

Show notes: Watch all of Productionize 2.0: ⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

The “ROI of AI” has been marketed as a panacea, a near-magical solution to all business problems.

Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?”

Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line.

Chapters: 00:00 Current State of AI Investments

03:59 Challenges and Solutions in AI Implementation

08:30 Identifying and Prioritizing AI Use Cases

10:53 Ensuring Trust and Explainability in AI

15:29 Measuring ROI and Efficiency Gains

21:10 Panel Discussion Begins

21:54 Trust and Risk Management at HP

23:27 Accenture's Approach to Operationalizing AI

26:06 ServiceNow's Trade-offs and Prioritization

31:17 Measuring the success of AI for customers

36:29 Frameworks and Best Practices

40:57 Conclusion and Final Thoughts

Follow:

Vikram Chatterji: ⁠https://www.linkedin.com/in/vikram-chatterji/ Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Show notes: Watch all of Productionize 2.0: ⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

Previous Episode

GenAI Predictions for 2025 | Databricks & Cohere

Will 2025 be the year open-source LLMs catch up with their closed-source rivals? Will an established set of best practices for evaluating AI emerge?

This week on Chain of Thought, we break out the crystal ball and give our biggest AI predictions for 2025. Listen as Sara Hooker, VP of Research at Cohere and Head of Cohere for AI predicts a trend towards smaller, more optimized AI models; Craig Wiley, Senior Director of Product, Mosaic AI at Databricks, dives into the future of multimodal AI; and Galileo’s CEO, Vikram Chatterji, shares his predictions, including the rise of open-source LLMs.

Chapters:

00:00 Introduction

02:01 Vikram's top 3 predictions

06:19 AI and nuclear energy

08:30 Giving power back to the people

13:46 Craig's predictions

20:46 The "era of toolification"

30:38 Sara's predictions

35:07 AI safety

Follow:

Vikram Chatterji: ⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠ Yash Sheth: https://www.linkedin.com/in/yash-sheth-/ Conor Bronsdon: ⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠

Sara Hooker: https://www.linkedin.com/in/sararosehooker/ Craig Wiley: https://www.linkedin.com/in/craigwiley/

Show notes: Watch all of Productionize 2.0: ⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

Next Episode

Practical Lessons for GenAI Evals | Chip Huyen & Vivienne Zhang

As AI agents and multimodal models become more prevalent, understanding how to evaluate GenAI is no longer optional – it's essential.

Generative AI introduces new complexities in assessment compared to traditional software, and this week on Chain of Thought we’re joined by Chip Huyen (Storyteller, Tép Studio), Vivienne Zhang (Senior Product Manager, Generative AI Software, Nvidia) for a discussion on AI evaluation best practices.

Before we hear from our guests, Vikram Chatterji (CEO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) give their takes on the complexities of AI evals and how to overcome them through the use of objective criteria in evaluating open-ended tasks, the role of hallucinations in AI models, and the importance of human-in-the-loop systems.

Afterwards, Chip and Vivienne sit down with Atin Sanyal (Co-Founder & CTO, Galileo) to explore common evaluation approaches, best practices for building frameworks, and implementation lessons. They also discuss the nuances of evaluating AI coding assistants and agentic systems.

Chapters: 00:00 Challenges in Evaluating Generative AI

05:45 Evaluating AI Agents

13:08 Are Hallucinations Bad?

17:12 Human in the Loop Systems

20:49 Panel discussion begins

22:57 Challenges in Evaluating Intelligent Systems

24:37 User Feedback and Iterative Improvement

26:47 Post-Deployment Evaluations and Common Mistakes

28:52 Hallucinations in AI: Definitions and Challenges

34:17 Evaluating AI Coding Assistants

38:15 Agentic Systems: Use Cases and Evaluations

43:00 Trends in AI Models and Hardware