马克·贝尼奥夫：Salesforce 的增长、公益与 AI 的未来

David Sacks (00:00:00): It is the center of the technology world right now. Marc Benioff (00:00:04): It's not what Mark did. It's when he did it. David Sacks (00:00:08): And the king of the cloud is Salesforce. Marc Benioff (00:00:12): Please welcome Mark Bedeall. One of the things that really matters to me is having a positive global impact. Technology is not good or bad. It's what you do with it that matters. In your quest to change the world, don't forget to do something for other people. And that was a moment in time when I said, wow. When I start a company, I'm gonna make sure that philanthropy and giving and generosity and these values are in the culture of the company from day 1. David Sacks (00:00:48): You wanna stay here? Oh, you want the couch? Marc Benioff (00:00:52): So far. You wanna David Sacks (00:00:53): stay on the couch? Where do you wanna sit? I'll give you the couch. I'll stay here. You deserve I'm going. You deserve the couch. Here. You deserve it. The big couch. Okay. So, little little too close. Marc Benioff (00:01:07): It's nice to see you also. David Sacks (00:01:08): I I warned you that I'm I'm not the the interviewer in the group. You but this is you chose me, so I'm honored. Marc Benioff (00:01:16): But you're the nice one. David Sacks (00:01:17): Oh, okay. Thank you. Well, it it it Marc Benioff (00:01:21): Am I right? Is he the nice one? And you're the one that all the women really like. Like, I'll talk to my friends at dinner. They're like, you know, Sachs. What's he like? Like, he's amazing. David Sacks (00:01:33): Let's just say we're honored to have Marc Benioff here and, Truly, who's a visionary in the world of software. And I would say, you know, there's probably a lot of Marc Benioff (00:01:43): And I thank my mother for writing that video, by the way, as well. Mom, thank you for writing that for me. Great. David Sacks (00:01:50): You know, in the in the world of business software in particular, we don't have that many people who you can describe as visionaries, but you consistently have been 1. You really It's true. You got the I think the whole Marc Benioff (00:02:02): We're sitting now on the edge of the couch. Okay. Here we go. Maybe we'll end up on the floor. I don't know what's gonna happen. David Sacks (00:02:10): I'm trying to keep it engaged. Marc Benioff (00:02:11): You're going around a lot. David Sacks (00:02:12): Oh, okay. Alright. Is this how it's gonna be the whole time? Marc Benioff (00:02:19): Way worse. Way worse? Okay. David Sacks (00:02:22): Alright. Let me finish the this little intro here. I forgot where I was. So Marc Benioff (00:02:28): Can I just before we start, you know listen? So I wanna just do something I would not normally do, and this is, like, gonna be a little bit of a thing. But I just have to do a little riff on this. But we just heard some an extraordinary presentation on an extraordinary man. And there's somebody who's amazing that most people don't get to hear of, and we just heard his name quite a few times. His name is Shiny Yamanaka Yamanaka san. He is, based in Kyoto, Japan, but he works halftime at UCSF. And it's amazing what his vision for the world is that he thinks, basically, that we're salamanders, and we're gonna be able to regenerate ourselves. And that's amazing. And so I've been friends with him for maybe for a decade, but I fund his research. And so a lot of these things to watch him have these breakthroughs you heard about the Yamanaka factors. The Yamanaka factors, which are basically this idea that Yamanaka had this breakthrough in Kyoto, you know, basically is hanging out there in his lab, eating the sushi, the whole thing, and then boom. And he goes, if I take these four things, I can take an ordinary skin cell, just any little skin cell, and turn it into a stem cell, which is, like, the the heart of human existence. And he did it, and he was able to repeat it and repeat it and repeat it. And he won the Nobel Prize for it. Pretty cool. And then he and I'm gonna get the pronunciation of this wrong, but he then was able to take that stem cell, put it into your eye if you have demacular generation, and boom, heal the eye because the eye regenerated. Then he worked with a buddy of his in the lab next door, and he took the same thing, took the stem cells, and he turned it in on a cookie sheet. And it looked like it was like a plastic thing on the cookie sheet. It was really cool. Marc Benioff (00:04:28): And then he's, like, took out somebody's cornea that was all screwed up, cut the material out of the cookie sheet, popped it in the eye, and the guy could see. He was like, amazing. Then he's like, listen. This is amazing. I bet I can grow a brain. So he took the stem cells, and he started growing brains called organoids. And he's, like, got a cookie sheet of brains. And I'm, like, really? He's, like, this is amazing. Look at all the brains. And I was, like and then I went and saw him and had lunch with him, and I'm, like, I'm, like, what's happening with the brains? He's like, I stopped the brains. I'm like, why did you stop the brains? I think they can feel the pain. I'm like, David Sacks (00:05:13): oh, scary. Marc Benioff (00:05:15): Then he's like then I said to him, now what are you doing? I'm growing intestines. I'm like, woah. Intestis. That good? He's like, huge idea. I can now grow intestines on the cookie sheet and taking the, you know, stem cells. I've got a whole intestine here, and then, like, he can do like, turn it into a lab lab for all the horrible things that people get in their gut and all these diseases that have never been cured, but now you have a real simulated environment. So he's an incredible person. Anyway David Sacks (00:05:47): Where do you wanna go with that? Marc Benioff (00:05:48): I'm going. I'm going with this. You gotta stay with me. David Sacks (00:05:51): Okay. I'll Marc Benioff (00:05:52): try to help bring the energy up in here. David Sacks (00:05:54): Well, I'll Marc Benioff (00:05:54): listen. Follow. Just hold on. Hold on. Hold on. Hold on. Hold on. Wait. Wait. Wait. This is gonna get good. Okay. So then I'm like, you heard the story. Like, at the end, they said, listen. How do I get these regenerative factors going inside myself? So UCSF just published research based on funding grant that I and others have given them, and they had a breakthrough that the regenerative factor inside your own blood is called 4. And the way you get 4 and I'm not gonna get this exactly right because, you know, I'm I'm in software. I'm not a doctor. So just follow with me. David Sacks (00:06:28): I thought that's what we're gonna talk about today. Marc Benioff (00:06:29): That. I know. But I gotta tell you this because I'm Scott so jacked watching that. One is it was either that or those crazy shots you have backstage. I don't know. David Sacks (00:06:38): But Okay. Marc Benioff (00:06:39): Number 1 is p o 4, you get more regenerative factors in your body, like, what you know, calorie restriction. And if you know David and I, that does not sound very good. 2, working out with weights, also not exactly our top thing. Paribiosis, do you know what that is? So parabiosis kind of came out of research published a decade ago, New York Times and others, which is came from Saul Valada, another person I work with at UCSF, where they took the blood of a young mouse and put it into an old mouse, and then the old mouse got young again. And that was moving the p f 4 into that old mouse. So that's and the 4th thing is clothotherapy, which is a genetic therapy that I don't really understand. Now these four things can start to generate more of these things inside your body. So then I'm, like, getting excited. I'm, like, god. I have these problems. Maybe I can regenerate different parts of myself, whatever. And so I'm talking to my doctor, UCSF, because I'm going through my own serious problem where I'm, like, my left leg is like a half an inch shorter than my right leg, and I'm running on the treadmill, and I'm always ripping my Achilles, ripping, ripping. And all of a sudden, my Achilles looks like it has a donut. And, in fact, I went to UCSF and there was, like, an MRI, you know. Well, how how many of you have had an MRI? Raise your hands so you know what is horrible it is. Anyway, you get in this big machine. They're looking at my Achilles. They're come out. They're all like this. Oh, sorry about this. Really horrible. And I'm like so I kinda took this thought, and I'm, like, talking to my doctor. I'm like, why can't we, like, do use some of the figure out what we can do? So he's like, alright. I'm come back on Wednesday. So I come back on Wednesday at 5 o'clock. You know? I'm in Mission Bay at UCSF, and I'm like, hey, Anthony. Marc Benioff (00:08:35): Where is everybody? We're gonna talk about it. Come into my lab. So I come into the lab. They've got, like, a centrifuge there, all this stuff going on. I'm like, well, this is interesting. He's like, did you work out today? Yes. I'm working out. Do you following the p f 4? I'm doing it. Okay. This is what we're gonna do. It's gonna be very straightforward because we have 2 we have 2 things we can do with you, Mark. Number 1, we can just take your Achilles, and we could bring you into surgery right now. We'll just shave off half your Achilles and then put you in a boot and see where you are in 6 months. I go, doesn't sound great. 2nd idea. What we're gonna do is we're going to we're going to take a scalpel, bought right here. We're gonna cut into your Achilles, like, 20 times and into your ankle. I'm gonna take your butt. I'm gonna spin it. I'm gonna try to find the PF 4 of your plasma. I'm gonna inject it into your Achilles and into your plasma, slice into it with my scalpel. I go, sounds great. There's one problem. I go, what's that? We don't use anesthesia to do that. Why? Because it destabilizes the PRP and the plasma and all the p f 4 and all that. I'm like, let's rock. Let's rock. So he did the whole thing, and then boom. Like, an I'm like a salamander. They groom me a new Achilles right in place. So that thing that you just heard, that shit is real. And, you know, it's pretty awesome. What can what and, it's yeah. David Sacks (00:10:06): I have a question for you. Marc Benioff (00:10:08): So if he can sell 3,000,000,000 into his startup, I should probably start. I'm ready to go. Got the pitch. David Sacks (00:10:14): Did do you ever consider that you missed your calling as a scientific researcher? Marc Benioff (00:10:19): Definitely not. Definitely not. David Sacks (00:10:21): You're happy with the choices you made. Well, no, that's an incredible story. So you are one of the first to actually try using the Yamanaka factors on yourself? Marc Benioff (00:10:29): I wouldn't think I'm one of the first, but I think that it's very real, and it's gonna gonna have a huge impact on our whole on our lives. And I think that we should be supporting these medical researchers. I think it's it's one of the reasons that I've, you know, put almost a $1,000,000,000 into UCSF of philanthropy because I believe in these people who have Absolutely. Yeah. They've dedicated their lives, you know, to basic science and doing and meeting them. They're so inspiring to me. And, like, I just had lunch with Yamanaka and Saul Valada and Anthony Luke and another incredible researcher, Mark Moesser, at my house. And, like, we're talking about the intersection between oncology and regenerative medicine, which is, like, 2 completely different worlds that don't talk to each other. And it's what inspires me that, you know, we can, you know, work with others to kinda get give them the entrepreneurial push to go do something incredible, and these people are just awesome. Each one is amazing. David Sacks (00:11:24): That is it's that is incredible. So let's shift gears and talk about something else. Marc Benioff (00:11:28): Nice coincidence with Beyond Finance. Yeah. David Sacks (00:11:29): No. It's incredible. It's a great story. I know you you're very philanthropic and do a lot with UCSF. So, kudos to you for encouraging that type of research. Let's shift to the to another thing that's something a huge impact on our lives, which is the cloud and software where you were a pioneer. You started Salesforce back in 1999? Marc Benioff (00:11:47): 25 years ago. David Sacks (00:11:48): 25 years ago. And how long have you been a public company for at this point? Marc Benioff (00:11:53): 2004. To 20 years. 20 years. David Sacks (00:11:57): And one of the things I noticed Marc Benioff (00:11:59): I'd rather count 80 earnings calls. David Sacks (00:12:01): Yeah. Well, actually, speaking of earnings, let's here. Let's see if we have this slide. Marc Benioff (00:12:05): Do we have earnings? David Sacks (00:12:06): Well, first Marc Benioff (00:12:07): Oh, yeah. Oh, boy. What's that? What slide is that? David Sacks (00:12:10): This is your stock chart over 25 I think 25 years, 20 years. Yeah. You're almost so you're all Marc Benioff (00:12:16): I guess there's no linear success. Exactly. Right? David Sacks (00:12:18): Yeah. That really good point. Yeah. We had a Marc Benioff (00:12:20): You just keep going. David Sacks (00:12:21): Had a bubble. We had a bubble in Marc Benioff (00:12:22): You keep going. David Sacks (00:12:23): 2021. We had a huge correction in 2020. Marc Benioff (00:12:25): There was? I need to make a note of that. David Sacks (00:12:27): We should talk about that. But you're you're basically back to where you were. This is one of your tweets. Actually It is. Marc Benioff (00:12:31): This is David Sacks (00:12:32): one of the things I appreciate about the way you do earnings calls is you just put out this really simple tweet. And it just and it shows a progression. And if, you know, if you like looking at numbers the way I do and seeing patterns in them, one of the things I noticed a while ago was that if you start at the bottom and work your way to the top, that Salesforce is growing by about 20% a year. And if you look at it over 3 years, that's roughly a double. So every 3 years, Salesforce was doubling. And that means that over a decade, it's growing 10x. And so every decade is basically exponential if you can stick with it long enough. That was one of the patterns I noticed with sales. Marc Benioff (00:13:13): Look. I I think that, you know, that the growth obviously is incredible, the 38,000,000,000, and, obviously, the cash flow is incredible. You know, it's more than Coca Cola did, I think, last quarter, and the margin is incredible. But let me just say, probably the best decision we made, and it's not on the slide, which is the day we started the company, We put 1% of our equity, 1% of our profit, 1% of our product, 1% of all of our employees' time into a 5 zero six one three foundation. Now at the time, it was very easy because we had no employees. We had no equity. We had no profit. We had no so it wasn't very complicated. But that idea, though, really kind of created the foundation of the company because we were able to do now and I think you know the numbers. Right? We're, you know, almost 10,000,000 hours of volunteerism. We've been able to give away almost a 100,000,000,000 in grants. We run almost a 100,000 nonprofits and NGOs for free on our service. And I think it really set the stage that business could be the greatest platform for change when it came for Salesforce. It under it gave it that philanthropic platform. David Sacks (00:14:20): So is there 2,000,000,000 in equity sitting in that 501c3 at this point? Marc Benioff (00:14:25): A lot. Well, there's more. I think there's about a half a 1000000000 in the foundation, and a lot of it's been already given out. And then we give out more every year and every month, every day, whatever. But, like, on Monday, we'll give another $25,000,000 approximately to the San Francisco and Oakland Public Schools. And that is you know, we've given them about a $150,000,000. I mean, it's it's obviously, I went to public schools. It was very important to me. But all my mother was a teacher in the San Francisco Public Schools. But, also, our employees, you know, we have 75,000 employees. Their kids are in the public schools. And so it's a key part of our mantra and our culture that we're trying to support public education. I've adopted public school. I really think that each one of us can needs to focus more on the public education system in the United States. It's something I encourage in, oh, not all my employees, but whenever I do a presentation, I'm like, you know, my public school is, like, a block from my house, Presidio Middle School, and I just went down there and knocked on the door. And they're like, who are you? And I'm like, what do you how can I help you, and what can I do to support you? They need a new playground. They need this. They need that. And, maybe they just need some cement some support, moral support. But, it's been a great thing to really anchor the company in those values, and I think it's an important thing, for every company. David Sacks (00:15:55): So what did you think when you saw that OpenAI started with a nonprofit, not as 1%, but as a 100%, but then it became a for profit? What did you think of that innovation? Marc Benioff (00:16:05): Confusing. I, you know, I mean, 18,000 companies have now followed our one one one model. You can find out about it at pledgeonepercent.org. That other model, I don't really understand. I think we've proven our model. This is important. You know, we came out with 3 models, the cloud model, which you also have been part of that, the subscription model, you've also been part of that, and the philanthropic model, and you've been part of that. And those ideas that we're doing 3 models, that's continues to be the fuel for the company and extremely important. And I think that for a lot of these companies that have followed us that have gone on to scale and have had huge IPOs and whether it was Slack or whether it was at Lazien or whether it was Twilio or whatever, they've had these huge foundations and have had huge impact. And business can be the greatest platform for change, and you can do a lot with your business. And, you know, we're all building great products. Okay. That's great. And we're selling them. That's great too. But we can also do a little more with our business, and we can use it in a positive way and try to move the world maybe a little bit more in the right direction. David Sacks (00:17:15): Okay. So let's talk about the cloud part of that innovation. Where do you think we're at right now? I mean, is it it's is it all AI all the time? How how are you thinking about it? Marc Benioff (00:17:26): We're at the precipice of the greatest moment in the history of enterprise software and of cloud computing. There there's no question. We you know, I had a moment, I would say, more than a decade ago, which I call my kind of AI freak out moment where I really felt I mean, maybe it's you know, obviously, we've all spent how many of you watched minority report? Alright. We saw that movie. And what about War Games? War Games? Anybody remember that? Okay. Her. Yeah. We have all saw these movies. Terminator. Okay. That one's a little scary. But we all seen the movies. And, you know, like, Peter Schwartz who wrote or was a key part of writing a minority report and, also War Games, you know, as our chief futurist at Salesforce. And a decade more than a decade ago, I had this moment where I was like, okay. This is really happening. Here we go. And bought a bunch of companies and put together Einstein, and Einstein has done amazing. You know, it's doing trillion transaction trillion and a half transactions a week, predictive generative. I really thought, okay. This is what's gonna be the moment. But now I'm really convinced that we are now really at the moment, right now, where enterprise software is gonna be completely transformed with artificial intelligence. And we're gonna see it, and, obviously, I'm getting tuned up for Dreamforce, which is gonna be Tuesday of next week. How many of you are coming to Dreamforce? Not enough. Anyway, sad. These aren't my people. David Sacks (00:18:58): I believe Marc Benioff (00:18:58): you now. David Sacks (00:18:59): Well, it's good. Marc Benioff (00:18:59): You Well, they look. But now let me just take, like, an office since you're not gonna be there, let me tell you what's gonna happen. Thanks for being part of my team. Anyway, number 1 is, you know, we're going to you know, we really see a moment right now where we are a 100% focused on one thing and one idea, and I can tell you why that is if you're interested, but it's Agent Force. And Agent Force is the most exciting thing I have ever worked on in my career. It's the culmination, really, of everything that we've done at Salesforce because to make Agent Force really deliver, we had to have all of our customer touch points wired up, which we do. We have to have an amalgamated data cloud because we need the data especially to achieve the AI accuracy and the metadata as well, and we have to have the agents. It's these three layers that are really gonna deliver this next generation capability, and I was just with Disney last night. And Disney has Agent Force. They have the newest version, which we call ATLAS, which is our most accurate, not just model, but we have an extremely unusual technique that we'll talk about. And Atlas delivers for for Disney, for their cast members, which are their employees, through extremely complex problems that it's solving for them, more than 90% accuracy and almost no hallucinations, and in some cases, 95% accuracy and almost no hallucinations. And that idea that we can kind of come in to a very difficult and complex and sophisticated dataset. Now with good now with Disney, if you go to disneystore.com, that's Salesforce. If you go to the Disney parks, do you still go to Disneyland? David Sacks (00:20:41): Sometimes. Yeah. Marc Benioff (00:20:42): K. You ever get a Disney guide? David Sacks (00:20:44): Sometimes. Marc Benioff (00:20:45): Yeah. It's great because you get to cut around the lines and all that. How many of you have done the Disney guides thing? David Sacks (00:20:49): Yeah. Marc Benioff (00:20:50): Oh, we got, like, a lot of poor people here, actually. Sad. Anyway, you should get these Disney guides because they're like, get you around the lines David Sacks (00:20:59): so you Marc Benioff (00:20:59): can do 30 rides a day. And David Sacks (00:21:01): Yeah. Marc Benioff (00:21:02): It's much better than having to wait. Okay. But, anyway, Disney guides run on Salesforce. They're just slightly have Slack too. They've got we do disneystore.com. We have Disney Plus because, you know, the ServiceNow, like, fell over, and we had to, like, replace that inside the Disney plus call center. We have we do the Disney cruises and the Disney real estate, and we have every Disney customer test point all wired up. So the amalgamated dataset the way around Disney is awesome. So when we can take that Disney dataset and then we apply Atlas and Agent Force David Sacks (00:21:37): Okay. So how do Marc Benioff (00:21:37): you define able to deliver a level of accuracy that has been incredible. And I've got a couple more examples I can tell you that are just blowing my mind. I never thought it was really possible. David Sacks (00:21:47): Let let Marc Benioff (00:21:48): But now it really fits. Go ahead. Yes. David Sacks (00:21:51): Well, I just wanted to find Marc Benioff (00:21:52): me to ask you a question also? David Sacks (00:21:55): No. No. No. What do you mean by agent? Because we're starting to hear this term a lot, but I think a lot of people here may not know what that means in the context of AI. Marc Benioff (00:22:05): Did you see the movie The Matrix? David Sacks (00:22:06): Yes. I did. So were we talking about Agent Smith, or what are we talking about? Marc Benioff (00:22:09): Well, we're at some level. I mean, I think, like, I'll give you an example that, you know, we're working with a large medical company not so far away from Keiser. They've got 20,000,000 patients. They have a super complex data set. They have all of the data from Epic. They have the largest Epic customer in the world. And more than 90% of all patient inquiries and scheduling requests and schedule my doctor, schedule my CT scan, my MRI, my this, my that, are being resolved by Agent Force and Atlas. That idea that we can resolve through a autonomous agent, a deep and complex customer interaction is a breakthrough thought. Obviously, we have to do a few things to make it really work for our customers. Number 1 is it's gotta be trusted because our customers try we're running the largest banks, insurance companies, media companies, CPG companies, blah blah blah blah blah in the world. Number 2 is it's gotta be easy for them. It can't be some separate team that they're gonna spin up. It's their existing Salesforce team that's happening within the Salesforce platform. It's gotta be open. It has to have be able to work with and interoperate with other systems. It's gonna have to be multimodal, so it's gonna have to speak to them and have voice and video and do all of those kind of incredible capabilities. And one other key thing, because, evidently, the humans have not gone away. The doctors have not gone away from Kaiser, and the cast members have not gone away from Disney and on and on. So we're gonna have to handshake seamlessly with our apps. So even though we have all these apps and we've wired up all these customer touch points, the agents are autonomously interacting with and building the data and metadata and extending it. And by the end of this month, we'll have more than a 1000 customers on our Agent Force platform. The efficiency and productivity that we've been had with Agent Force is like nothing I have ever seen with any of our customers or technology in the history of software. But there's a second point. It isn't just about this kind of ease of use. It's that that they have the ability to do things that are truly astonishing, and that is also generate revenue. Marc Benioff (00:24:23): So they can go out, and, like, on a day like today, like, it's a 100 and something degrees outside. I don't know if you've been out there. It's pretty hot. And Disneyland may not be as full today as it's gonna be, and they knew that was gonna be true 2 days ago that a heatwave was coming. Disney can proactively go out to their consumers and their customers and say, hey. Come enjoy the heat with us all, you know, Disneyland, and we're gonna give you a special promotion or price or contest or whatever it is to come to Disneyland. So we wanna be able to proactively go out and generate revenue, and we also wanna be able to kinda bring that customer service in. I think last night, I had dinner at in Beverly Hills at the grill. Have you been there? Great. Right? Cream spinach. David Sacks (00:25:05): Well, I just need different vegetables. Want? Marc Benioff (00:25:07): What do you prefer? David Sacks (00:25:09): Well, I like like potatoes. You know? Potatoes? Okay. You know, any kind of potato. Marc Benioff (00:25:14): You like any kind of potato? David Sacks (00:25:15): Baked potato, steak fries, Marc Benioff (00:25:16): all of them. On OpenTable. Right? Anybody here use OpenTable? It's a very weird group. Anyway so I'm using anyway, you can use OpenTable to make restaurant reservations, and there's a 160,000,000 consumers on OpenTable. They're not in this room, but they're somewhere. And they've got also 60,000 restaurants, and they've got a lot of complex issues, you know, in regard you know, I didn't get my table or my food wasn't right. My potato didn't get cooked, whatever it is. These things are gonna get worked out. But, also, all of a sudden, the restaurant's like, oh, look. We're not as full tonight as we wanna be, and we're willing to do, let's go out to our customer base and bring them in, but let's do it through a complex conversation, you know, an empathic conversation as an agent with our customers. I think it's gonna be a rocket ship. David Sacks (00:26:09): Okay. So what so how long will it be until when you call a customer support center, you're talking to an AI that sounds like a human and you can't tell the difference? Are we there yet? Or Marc Benioff (00:26:22): We're there yet. We are already at that point. We already have that live, and we will have that scaled for thousands of customers before the end of, live for with thousands of customers live before the end of this year. And we just I just demoed it. I was just just at a conference and spoke a mile a couple miles away from here at KPMG, and we showed them that exact situation where, you know, through you know, we used to call, you know, this kind of voice response system, whatever, but you would kinda hit a wall pretty quickly with your bot. You know? But these aren't bots. These are not the bots you're looking for. These are like we're really getting to, like, another level capability, and I think that it's pretty impressive. And I think in the example of Disney, you know, Google has some great products. I know Sergei was here yesterday, and they've done a great job with AI, as you know. But in a head to head benchmark of Salesforce's Agent Force against Google's AI, we 2 x them on accuracy. And the reason why is we'll explain it next week. You know, it's a couple of things. Not only is there our next gen models, but it's also new techniques involving next generation retrieval augmented generation, rag techniques that no one has seen before, and it's really incredible what's possible. David Sacks (00:27:41): So you're kicking Google's ass. I'm cool with that. Marc Benioff (00:27:44): Well, they're a partner also, customer, and I love them. But, yeah, it's it's competitive. David Sacks (00:27:48): Just yeah. Let me let me keep building on this. Marc Benioff (00:27:51): We're trying to all make AI a little more accurate and a lot less a little few less hallucinations along the way. David Sacks (00:27:56): Let me give the audience a little update about something we just heard at OpenAI. They just did a a a day where they brought in a relatively small number of investors and kind of gave us all an update on their product roadmap. And it sounds kind of similar because everyone's moving in the same direction. So there are 3 big takeaways. Number 1 was that they said that LLMs would soon be at PhD level reasoning. Right now, it's more like a smart high schooler or college student in terms of the answers. We're going to be at the next level. Shortly behind that is agents, like you're talking about. And then 3rd and closely related is that agents will have the ability to use tools. And a tool can be a website. So if you think about it, now you've got this LLM. It's really smart. It's got you know, it's like a PhD. It you can give it an objective. It will break that objective into a list of tasks, and those tasks can include using other pieces of software. And thanks to things like OpenAI just launched the, audio API, which developers can use. It's in private beta. We have some companies using it. The LLM can now basically pretend to be a human. And, you know, there's it won't be hard to find a piece of software to enable a phone call. So you can imagine telling a a personal assistant agent that and it could be, you know, it could be OpenTable, that, hey. Book me book me a a dinner reservation at the grill. And it could place a phone call on your behalf and actually talk to the grill. It could also go on OpenTable and just use OpenTable and book it. But if for some reason that didn't work, it could literally place a phone call on your behalf and the person picking up at OpenTable wouldn't even know that your agent actually isn't a human, it's an AI. But here's where I think it gets really crazy, is when the phone gets picked up on the other end, that could be an AI too pretending to be a human. So you could have 2 AIs pretending to be humans talking to each other and resolving tasks on your behalf. And I I literally I think that's where it's headed. Marc Benioff (00:29:59): We're definitely moving in this direction, but there's a cautionary tale here. And I think that I'll just tell you the real world experience with my customers and what I'm the problems that I'm trying to solve for them. I I think in the last few years, what we've kind of heard and, you know, some of it has come from OpenAI, but especially from Microsoft, that we're in this Copilot world, and these Copilots have universally failed. The level of accuracy, the spillage of information, the lack of trusted environment, Copilot has been a complete disaster. And that idea that this kind of amount of technology got, you know, released and sold into these very large customers telling them that the all the promise of AI is here, but didn't do it in a trusted way, didn't do it with the level of accuracy, didn't do it with the level of security needed. And one of the things that was interesting, because I was with one of the customers with you trying to do this exact technique that you're talking about, which is a large telecommunications company in Seattle. And what this company did is take a model David Sacks (00:30:58): and try to heard of? Marc Benioff (00:30:59): Nope. Okay. We're gonna just tell you training a model Okay. Retraining a model, building their own model. Mark, we have to have our own models. We're gonna DIY it. We're gonna DIY our AI, and it's gonna be awesome. Then we're gonna write our own agents, and we're gonna do this, we're gonna do that, the other thing. And I'm sitting there, and I'm all going through it and whatever. And I'm and I finally I'm like, now show me your benchmarks and show me all these different pieces. And and, you know, for them, it's a bit of a science project. And I've seen this now with a number of our customers that they're kind of DIY in their AI. And, you know, DIY, I think it's fine if you're like Neil Young and it's homegrown and it's Canada and it's, you know, Ontario, But this is not what you should be doing with your artificial intelligence. David Sacks (00:31:41): But what what are you guys using as your foundation model? Is it LAMA 2? Like, what are you guys using Marc Benioff (00:31:46): for this? A lot of our own models, our own techniques, our own and then we let you bring in the model that you want, but we are all about achieving your accuracy. Because what I've seen with these kind of approaches, especially the one that you just outlined, is that, yeah, you can get maybe 30 or 40% accuracy. You know, in this case, this customer is 25%. You had somebody on the stage yesterday, I won't tell who it is, who's a common friend of both of ours, who tried to take this approach for a large telecommunications company that he owns, and he said he was getting about a 25% accuracy with this homegrown model. And I'm like, why are you doing that? Instead, in our platform, the platform is building the model for you. You're not having to train and retrain your own models. You're building your own models in our platform, and we're gonna deliver much higher levels of accuracy for you, and we're gonna deliver AI. This is the AI that you want. This is this next generation of AI, and I think that we will have to prove that with benchmarks and with bake offs and to show customers because the promise is amazing. But at a very deep level, customers are gonna need, you know, what you and I have done for the last, you know, 20 years of our life, which is build professional enterprise software and deliver it to them in a capability. And in regards to an agent running enterprise software, I mean, you just saw, like, that was the fundamental business model of Adept, which was David Luan's company who you know, and that he built GPT 3, then he left OpenAI to start Adept and this idea to build agents that are gonna drive apps. I'm sure that all of those things are gonna happen. But, again, you have to get to a level of accuracy because everyone in this room and you and I, we've all had this experience. Marc Benioff (00:33:26): We're on these models, and it's like, this is not really more than hallucinations. And that's no good, or as we say here in Los Angeles, no es bueno when it comes to, okay, Kaiser and you're dealing with health care. You know? When you're dealing with health care and you've got a patient and you're reading their medical records, you better be delivering more than 90 or 95 percent accuracy because the 50% accuracy thing is no good. David Sacks (00:33:53): Well, I can see you're ready for Dreamforce. Marc Benioff (00:33:55): I'm trying to get find it. I'm testing material out here a little bit. David Sacks (00:33:58): What do you guys think? Marc Benioff (00:34:00): I'm like David Sacks (00:34:05): Are you are you guys excited for the rise of agents? Yeah? Oh, yeah. It's gonna be a really big deal. I think that everything we've seen so far with LLMs has been, again, about reasoning and and generating. But with agents, the AI is gonna be able to take actions. Yeah. And and they're gonna know how to use tools, which until now has been something only human. Marc Benioff (00:34:25): I gotta tell you a really good story because you're, like, inspiring me around, you know, Steve Jobs had a huge impact on my life, and then I worked at Apple in 1984 when I was I was, in high school and coming into college, and I was in assembly language program. I wrote the first native assembly language, on this, Macintosh, you know, on the 68,000 assembler. And, sitting there in the cubes. And Steve is running it while I get on whatever it is. And thank god, you know, I have this relationship and influenced me so much in my life and then called me in a series of times. And after I started Salesforce, gave me really key advice. Anyway, it was 2010, and he calls me. I was come down here. I need to talk to you. I'm like, shit. What the hell? What did I do this time? So I go down there to his office, and I always bring a few Salesforce employees with me. And I've got some great folks with me. And we're sitting there, and he's like, I'm gonna show you this. And I'm like, alright. Let's go. And he brings out the iPad, and he's got 2 of them. He's got the big one and the small one. And he's like, yeah, Mark. Here it is. You know? But I don't like the small one. I'm only gonna have one size. You know that. I'm like, yes, sir. And he's like, listen. You know, I've been working on this concept for a long time. And, you know, in 2007, I introduced a iPhone. And, you know, I said, thank you for sending me one. I love it. It's great. It's like, but do you know why now we're doing the iPad? I'm like, no. Because I know you had that too in 2000. Oh, yeah. But you know what? This real situation here is with Apple. I'm like, what is it, Steve? He's like, we only have 1 a team here, 1 a team, so we're only focused on one thing at a time. And then he lays out, like, 5 or 6 products on his coffee table, and he goes, and we will never have more products than can fit on my coffee table. And I'm like, wow. That's really awesome. And he's like, I've been focused on 2,007 on the iPhone, and now I'm gonna zero in and I'm only gonna do iPad. One focus at a time. Marc Benioff (00:36:25): Remember that, Mark. That's the way you need to run Salesforce. And I'm like, okay. Is that why you brought me down here? Yes. You may go. And and that's how I feel right now about Agent Force. This is all I am doing. Just try to take our company. You know, we have a great company, 38,000,000,000 revenue, 75,000 employees, 100 of thousands of customers, and one focus, Agent Force. This is the because of what you're saying, this is the moment. This is the greatest opportunity in the history of enterprise software, and it must be executed with absolute acuity and excellence. And that is what I think we all need to do. David Sacks (00:37:09): You know, so so I agree with you. I mean, I think the agents are gonna be huge. And Elon said something kind of similar the other day. He said we was we got him talking about Optimus, you know, his his robot. He just Marc Benioff (00:37:21): heard about the farm animals. I didn't know about the what was was there another part of the presentation? David Sacks (00:37:25): He said, well, he was talking about he was talking about optimists. And, the the thing the thing about The Marc Benioff (00:37:32): jokes are all each one is kinda dying very fast. It's sad. David Sacks (00:37:35): Took me a second to to realize that you were talking about his cock. But now I got it. Okay. No, he was referring to, It's great how Marc Benioff (00:37:46): you bring this humor into, the all in. David Sacks (00:37:49): It's very subtle sometimes. Yeah. So what Elon mentioned that really stuck with me is he said that humanoid robots the creation of these humanoid robots are the biggest economic opportunity in the history of the world. The average person's Marc Benioff (00:38:05): Is he making some of them by any chance? David Sacks (00:38:07): He is. But well, it's kinda like you saying that, agents are the biggest opportunity in the history of enterprise software. Marc Benioff (00:38:13): Let me write that down. Thank you for letting me know that. David Sacks (00:38:17): It's it strikes me that there's something similar here, which is Marc Benioff (00:38:19): Sean is a good salesman. Is that your point? David Sacks (00:38:21): Well, I'm I'm saying there's an analogy here between Marc Benioff (00:38:24): He didn't give you the regenerative pitch. Is that my David Sacks (00:38:28): Well, no. What what here the point is this, is that is that where we're going with AI is it's gonna be able to take real actions. And in the case of Optimus, it's in the physical world, and it's gonna be the brain for these humanoid robots. In the enterprise, it's basically the brain for these agents. I think these things are actually pretty they're on parallel tracks. I wouldn't say they're competing. Marc Benioff (00:38:51): And These are the droids you're looking for. David Sacks (00:38:54): So I think anyway, I think that, I I think you're right about this opportunity. And what I'm saying is I think it's analogous to what Elon is seeing with robotics. Marc Benioff (00:39:02): I think there's no question. And I think that for our customers, they're gonna augment their employees. They're gonna make things lower cost. They're gonna increase their revenues. They're gonna increase their margins. We're gonna take some customers and just turn them into margin machines. And I think that the opportunity for in the enterprise is unbelievable. He's also directly addressing the consumer market, which I think is very exciting. Obviously, he's an expert in that area. And, yeah, we're about to move into this new world of AI, of droids, of all these things. And, you know, it's a bunch of waves of, you know, we're you know, look. Technology is getting lower cost and easier to use. It's a continuum, and we're all riding that continuum. This is extremely important. But, also, what's very important is, especially as we move into this, we all have to think about what are the values that are gonna guide this technology? Because each of us have seen the movies. We all watched the movies. That was the one place where I got the hands to go up. Right? So we know how it can go really wrong. Right? Everybody saw that part of the movie. So what are the values? What's gonna be really important to us? Will it be trust? Is it customer success? Is it innovation? Is it equality? Is it sustainability? What are the values as we kind of guide into the next level of the future? Because those core values that we need to manifest and really focus on, that is, I think, still out there as a major discussion item. David Sacks (00:40:24): It's gotta be figured out, and that is why we're very lucky that you are one of the great visionaries of our industry because you're not just a great entrepreneur and CEO, but you're a great human being. So thank you, Mark. Thank you.