Brainstorm AI London 2024

Dr. Matthias Röder, Chief Executive Officer, Karajan Institute; Co-founder and Managing Partner, The Mindshift; Director, Beethoven X – The AI Project

Brainstorm AI London 2024: How Deutsche Telekom Used AI to Finish Beethoven’s Unfinished Symphony

Photography by Joe Maher/Fortune Brainstorm AI

Conferences

Thank you, Jeremy. Hey, everyone. 200 years ago, a mildly grumpy, middle aged man with wild hair, stepped out of his apartment in Vienna briskly walked down the street, mumbling, humming and singing from time to time standing writing something in his notebook and continuing on his way. Yes, you guessed right. I'm talking about Ludwig Van Beethoven at the time, the most well known composer of music in Europe and almost completely deaf. At this point. Later at home, he would take out the melodies he found during his walk and copy them into a big sketchbook that was lying on his desk. This is where the real work started, where he started to work on the musical problems that the material posed to him. How should I develop this melody? How should I combine it? Which key should it be modulating to? What is the setting and the structure that is good for this type of material. This was the actual work and Beethoven had amassed many of such books and he would go back and forth. This was his creative Bible, the 10th symphony that he was working on at the time that didn't materialize Beethoven died two years later, having left the work unfinished. 195 years later, I got a phone call not by Beethoven, but Mi Michael Schulz from Deutsche Telekom asked me, Matthias, can we finish this 10th symphony of Beethoven with a I, I don't know, Michael. Uh we got to put a team together, we got to start working on it and we got to see how far we can get well do it. Two months later, I was pretty certain that it was possible to do something whether it was the big thing, I don't know. So Beethoven left 45 sketches for 1/10 symphony. Some of which were a little longer, others were shorter. A musicologist named Barry Cooper a couple of years ago used the longer ones to finish one potential version of the 10th symphony and that worked pretty well. But with the short material, you know, no human could ever possibly take that and turn it into something bigger. Why? Because you would naturally put your own ideas, your own style, your own way of composing into the material. I called Ahmed El Gamal from Rutgers University who is a computer scientist specialized on creating artworks with the help of A I and asked him if he wanted to be part of the project. He said yes, but I've never done anything in music. So I sat down with him, taught him everything I knew about the creative process of Beethoven and how composition works. And he said, let me think about it, called me back next week and said, Matthias, I thought about it. It's quite straightforward. Composing music is just like email, autocomplete. Who I, you can imagine that moment. I thought like, ok, well, maybe the wrong person and this is never going to work. But then he told me no, but look, every note is a function of all of the notes that came before. And it, it makes sense when I have a piece of music, the next note is related to what came before. We can actually try it out. Now together, I'll sing you a melody. I'll stop at some point and you the note that I left out. OK. Are you ready? OK. Ta ta, ta, ta, ta, ta ta. Yeah. You see every node is a function of all of the notes that came before. So we set out, we said, well, let's use this technology, the transformer architecture that Google invented in 2017 and uh create a G BT for music that's trained on late 18th century, early 19th century music and then fine tuned with the works of Beethoven. And then we use the sketches that Beethoven wrote down as prompts. And we let the machine continue these ideas. And then a group of musicologists combine the material, go through it, improve on it, give it back to the machine and this way iteratively create one possible version of 1/10 symphony at the time. Most people thought, oh, they are crazy. This was never going to work. Even some Google folks said that to the CEO of Deutsche Telekom. But we were pretty certain that at least, you know, we would create something that would pass as a potential musical work of that time. So I want to play you an example of that tense symphony which starts with one of these sketches that Beethoven had written down. It's basically 1 to 1 what he wrote down in his book. And at some point, the A I will take over and I invite you to play this game for with me where you raise your hand when you think this is the moment where the A I comes in, of course, I'm going to give you um you know, the solution afterwards. But this is one minute of music, close your eyes, don't get distracted by the people around you and just be bold, raise your hand whenever you think this is where Beethoven stops and A I begins was generated by this algorithm. Uh Granted it was then, you know, going through other algorithms as well. We not only produced an A I that could continue the sketches, we produced one that would find the suitable accompaniment, one that would stitch together pieces of A I generated music and then one final A I that would take the output of all of the other AIS and say this note will be played by that instrument. This note by this instrument, an orchestration engine. This is where we really reached the end of the line because that A I wasn't that successful. So we actually hired a human in our team to orchestrate the final version that you just heard being performed by a real orchestra with real humans. This collaboration started in 2019. As I mentioned in 2020 we were supposed to premiere it. Uh but COVID came along and so we had to postpone, we had one additional year to work on it. And in that time frame, I got a second phone call from Deutsche Telekom. Hey Matthias. Uh Robbie Williams has heard about your Beethoven A I and he thinks he would like to compose with Beethoven. Is that possible? I said, well, you know, what can I say? I don't say no if Robbie Williams wants to use an A I that we produce, so we'll find a way to make it happen. Of course, the question was, how could we do that? I flew to London. I explained it to Guy Chambers, the composer that worked very closely with him and Robbie's manager exactly what we were doing. And they said, well, we'll think about it and they called me back the next day. We want to do angels. Can you recompose angels in the style of Beethoven? And I said, I don't know, I really don't know if that's possible. Um The, the problem was, you see, we had this A I that was trained on 18th and 19th century music and that could very well deal with Beethoven's style. But how do we get Robbie Williams in there? This is when the composer on our team WV, I had this amazingly genius idea. He said, well, we'll take a famous melody of Beethoven and we combine it with bits and pieces from Robbie's Angels. And this combination is what we then use as a prompt. And so by prompting the A I with something that had already the main motific uh content of angels in it. It was capable of producing music that combined Robbie williams' Angels with the style of composing of Beethoven. There was one problem we thought, OK, well, our A I is pretty good at creating long stretches of music, but Guy Chambers told us no. Um you have uh four seconds for the intro and you have another 30 seconds for where the solo is and everything has to happen in there. So we would try out different types of uh combinations of music. And finally, we found one that worked very well. We combined uh the Moonlight Sonata with uh with some people are laughing. Yeah, uh with, with angels and I have another excerpt for you that uh is, is a little mash up of what we did with Robbie. You can listen to the whole thing on his newest album. Um And please let's listen to Angels with Beethoven A I. The point of this was really to collaborate and we started out to create this one A I that could, you know, do something very, very specific. And we ended up providing a tool set to one of the great artist of our time to collaborate with. And I think this is the, the move that we see in a lot of what's happening in generative A I and music right now, we see tools that become collaboration partners and that are really um instances of inspiration of transformation of working bodies for composers. And if you ask me what the future of all of this is, then I'd say all of the tools that right now are sort of text to finished song where you basically say, I want to have a track which is 15th century style with green sleeves and heavy metal combines and then you get finished track. This is, this is all nice and very good, but it will be superseded by tools that will be directly integrated into the software where we compose music where we have our inspirational A I tools. Now, I want to leave you with another excerpt this time from the Beethoven 10th symphony. It's actually a work of 25 minutes and I condensed it for you here to, to a couple of minutes for you to get some sense of how the piece sounds when you hear it performed by an orchestra, we can roll the tape. Um This was done a couple of years ago in Hamburg and the Philammon, this is the Beethoven orchestra conducted by Dirk Kaftan. And there's a little surprise there, an instrument that you wouldn't expect in a Beethoven symphony. So enjoy.

Dr. Olivier Oullier, Co-founder, Inclusive Brains; Chairman, Institute for Artificial Intelligence by Biotech Dental Interviewer: Ellie Austin, FORTUNE

Brainstorm AI London 2024: From Sports To The Workplace

Hello again, everyone. Um I hope you are enjoying your lunch. So next up, we have our keynote conversation throughout history. Sports have served as a catalyst for both technological innovation and societal change. We touched on this in our sessions yesterday and the advent of A I is no different. Today's lunchtime speaker has left robotics and brain computer interfaces to enable a quadriplegic individual to drive a formula one car with his mind. It's incredible. You can look it up on youtube. And this summer our speaker will use similar technology to enable severely disabled individuals to carry the Olympic and Paralympic torches will discuss the potential of A I to enhance the lives of those with diverse physical and cognitive abilities and what this might mean for inclusion in the workplace and beyond. So please join me in welcoming Olivier O chairman of the Institute for Artificial Intelligence by Biotech Dental and CEO and co founder of Inclusive Brains. Thank you very much. Hi, Olivia, take a seat. Um Thank you so much for joining us. So you have many titles. You are a neuroscientist, you are an entrepreneur, you are an investor and as I mentioned, you run your own company Inclusive Brains, which you founded in 2022. We have a fantastic co-founder, Paul Barbas with your co-founder. Um Now, the goal of inclusive brains is to build a I's first neural foundation model. Can you tell me what that means? We want to create algorithms and models that enable machines to adapt to who we are and how we feel regardless of our needs, abilities and physicality. Machines should serve us A I is a fantastic assistive tool and we want to bring it to the masses. Paul and I, when we founded inclusive brains had one benchmark in mind is the remote control. The remote control was invented for people with disabilities, for them to interact with technology and then it conquered the world. Billions of people have changed the way they interact with technology. Thanks to the remote control, we want to be inspired by this story to bring A I that we first developed to help people with special abilities and needs and then leverage it for inclusion because true inclusion means serving and assisting everybody with no discrimination whatsoever. And so most A I systems are trained on language, but I know that you train your systems on a range of biological functions, whether it's the clench of a jaw, the clench of a hand or a heartbeat, a brainwave. Can you talk us through how you do that? Yes, it's incredible because look at us, we're interacting and words matter. But there is so much than words in the way we interact and the way we interact with technology. And there are people, people who can't use words. So we're training our models with brainwaves, facial expressions, voice stress what we call prosody, the intonation habits and lots of biological factors so that we can leverage generative A I to better understand and feel how we are and the technology to adapt to who we are, which I think this is the key we have to create interfaces that adapt, that can predict down the road. But first to adapt to who we are and what makes each of us unique. So to adapt to an individual's needs, cognitive or physical. Absolutely. Um I mentioned in the introduction, the project that you're doing this summer with the Olympics and the Paralympics. It's called Project Prometheus, I believe. Why is the Olympic parade, the torch parade, an important platform for you to showcase this work. The entire world is going to watch the Olympics. I'm old enough to have witnessed someone with a jet pack going out of a Coliseum in Los Angeles in 1984. And as a kid that's really sparkled, you know, my interest in technology and amazing things and I think we can inspire a lot of kids to see that the combination of A I with exoskeleton and neuroscience can help people have a con concrete, really concrete applications. I think this is why we want sport and its values to be associated with top technology that we want to bring to the masses. We're going to a demo in a second, the exoskeleton. Um But first of all, I want you to talk us through how this technology originated because I believe it came from a chance encounter you had when you were Stagg at Harvard. And we've got a photo to illustrate this. I believe I'm going to try not to be too emotional because the person you see is my friend, Mark Pollock. He is an explorer, a technologist. He has a superpower. He brings technology and inventors that don't talk to each other in order to turn their inventions into actionable solutions to help the world. What you see is Mark his team and I almost 10 years ago to this day in a basement in UCL A as Mark was using an exoskeleton in order to walk. Mark is paralyzed from the waist down and he cannot see and I was recording his brain waves in order to understand how his brain is processing movement following the accident that happened in his life. And this is really the very beginning 10 years ago of project proteus. The combination of exoskeleton and neuroscience. Let's try it out. I have to ask you a question and for your permission first, would you mind if I use neurotechnology to record your brain activity? Please go ahead and do you mind if we show your brainwaves to everyone in the room. Please go ahead. It has to be done with informed consent of people. That's very respectful. So this is a brain sensor, wireless brain sensor that is monitoring Ellie's electric activity of of her brain. Just sorry to mess with. No, go ahead your hair. I'm not precious about it. You're not perfect. This is more important. Exactly. So and uh let's have a look at Ellie's brainwaves. So the one thing that we need to show first is that we're doing this live. So would you mind moving your head because there are movement sensors. So this is in real time and this is how we train our algorithms. The technical term is multimodal, meaning that we're using different kinds of information to train generative A I brainwaves. We're also using facial expression. Very important. The interface is asking you whether you allow for the camera to process your facial expressions in real time. Yeah. Heartbeat, lots of things I could go on for hours as you can imagine. But this is just to show you that in real time, we can not only monitor what is happening but leverage artificial intelligence to measure in real time the level of stress attention, cognitive load. Hi right now, how are these? So how are these the markers, stress, cognitive load, attention and arousal, how are they linked kind of how do they sit against the baseline of what you would expect? Well, it all depends on the environment in which you are and how you were before. Well, we're human beings. We're a bit stressed even if we are. You and I are quite used to, to be, uh, to be on stage and it changes a lot and the fact that it's jumping if you have to remember, one thing is the fact that how we stress how we are, our personality cannot be captured by a bunch of questions. Once in a lifetime that determines who you are in your organization, it changes from a minute to another, it changes from a day to another, from a person to another. And this is how we can create interfaces that are tailored to our uniqueness. So when you have this information, what do you do with that? We can use it in order to prevent accidents in order to improve how people learn in order to inform what the outcome of a decision making process can be. But it has to be done respecting all the regulations when it comes to data, to privacy and now regulation of artificial intelligence. And we can do things these days in terms of monitoring in real time that we couldn't do in the past generative A I is amazing for that. But once again, we have to go beyond large language models because there is so much that language can capture. We are a lot more than words. Fortunately. So would you like to try the exoskeleton. I'd love to. Now, you've made it very clear to me beforehand that this is not a robotic arm. It's an exos why is that distinction so important? So first, it's designed to help people. So there are two modes we can use it. One is passive help. There are people who can still move but are losing step by step, the ability to use their muscle, they are losing power. So the device is going to assist you when you move. Can you share with our audience? What's your experience is? Yeah, so I'm only very slightly moving my arm, but it feels like there's a superpower behind my joint that is doing all the hard work. So this is also a brain scan that we launched at a fortune event back in 2019 in Guangzhou, China. It's brain sensing earbuds and we can leverage them to do what people would call mind control. In that case, it's control that is done with clenching because the technology can adapt to what you can do. Some people like Ellie can focus and can do real mind control. One of our participants in the Olympic torch relay, she has a cognitive disability and she cannot focus enough. So we have to tailor the interface to measure her facial expressions and also the fact that she's clenching in order to be doing what we're going to do now, which is me by clenching providing you more assistance Oh, wow. Yeah, that's way stronger. And so this could be installed at someone's workstation, for example, if they needed help at work, it could be in their home, all different use cases. Absolutely. And this is the goal is to provide assistance to everyone. I'm going to free you. I appreciate that. Thank you very much. Yeah, my headpiece and so Olivier, how accessible is this technology right now? Both in terms of cost and availability. Let's go. Let's sit down ready. Yeah, it's a few €100 for the brain sensors, a few 1000 for the exoskeleton. And what we do with our technology is we're hardware agnostic. We're developing the A I that can basically be used in any connected device or environment that has sensors because we're training the model with all this information, the brain waves, the facial muscles, heartbeats, generative A I will be able down the road to recreate the signals and we will need less and less wearables. The goal with that is really to have machines to adapt to us and how we feel. There are moments where I'm too distracted for some task, especially if I'm working on a brain surgery. Ken, we a Ken, I'm not a brain surgeon. You do not want me to operate on your brain, but we did work with surgeons, orthopedic surgeons. Absolutely in Abu Dhabi that are specialized in the needs of your favorite football players and we monitored in real time how their brains interact. So two surgeons, it was a world premiere to understand the difference of how they process information when they are performing routine check, routine operations and when they are dealing with the unknown. And what did those results tell you? And how is that going to change the world of surgery? If we throw it ahead, we can optimize the whole day of work. Surgeons like most caregivers, they have operations after operations after and they are just like every one of us at some point, you know, you're mentally tired, it's harder to focus. So we can better time care delivery in order to minimize errors and accidents. Another example is what we're doing in dentistry. A lot of us are afraid of dentists so we can monitor in real time the level of stress of a patient with the ear buds in order to make sure that we wait until the patient calm down in order to perform whatever intervention we have to do and little details like that can change everything in care delivery, but also on a construction site or for knowledge workers. So it really is about enhancing people's experience regardless of their cognitive or physical abilities. This can be integrated into so many different factors of life. That's your ultimate goal for this technology. Yes. As long as we do it in a responsible, fair, ethical and legal way, this is why people have to be informed people what we did is I did not record your data. Your data is is not stored. We can have aggregated data about a group of people without having information about the individual that is anonymized so that we can learn that a group of people works better at certain times of the day or can perform better. The idea is really to leverage this technology for good in order to improve safety, whether we're talking about physical or mental health. In order to improve decision making, avoiding accidents, there are so many operations and accidents today that are happening just because we do not have a good relation with technology, whomever buys a computer, whichever workstation you have in your office, we have to adapt to the work environment. We have to make the effort. It doesn't make sense. It would be just like giving clothes to go on a construction site to someone who is 1.7 m. You would give clothes of someone who is 2 m, the helmet is too big, the shoes are too big. You don't do that when it comes to physical safety, we have to do it about psychological safety as well. We customize this is like haute couture, but recognition and physicality. And on that note, I wouldn't normally comment on your fashion, but you've told me that you're wearing very significant trainers. Can you talk me through these? Yes, because we're talking a lot about high technology A I exoskeleton but low tech can help a lot. This is a partnership between a famous brand, sportswear and cafes. Another brand that employ only people with cognitive disability and these shoes and explain to people who can't see what's going on in your shoes. So my shoes one is yellow, the other one is black and this low tech can help people who have some cognitive disability. Remember which side is which it's just as simple as that. It's about designing environments that are inclusive work environments, everyday life environments. As long as we leverage A I or low tech for good Olivier. Thank you so much for coming. Thank you for bringing your technology and good luck this summer at the Olympics and Paralympics. Thank you very much.

Tara McGuinness, Founder and Executive Director, New Practice Lab Sir Craig Oliver, Global Co-head, Strategy and Reputation, FGS Global  Julius van de Laar, Political Strategist and Communications Consultant, Van De Laar Campaigning Moderator: Alan Murray, FORTUNE

Brainstorm AI London 2024: Combating Deep Fakes And Protecting Public Trust

Tara McGuinness, New Practice Lab, Sir Craig Oliver, FGS Global, Julius van de Laar, Van De Laar Campaigning, Alan Murray, FORTUNE

Presenter: Julius van de Laar, Political Strategist and Communications Consultant, Van De Laar Campaigning 

Brainstorm AI London 2024: Three Ways AI Is Already Impacting Political Campaigns

Thanks for having me. Great. Uh I thought we'll dive right in and talk about A I and political campaigns who's excited about the upcoming US election? Anyone? Yes, I thought so. Right. A couple of people very hyped up. Obviously, a lot of things are happening and uh not just in the run up to the election campaign, but anyone watching the news last night, right. Donald Trump is right right here in the courtroom in New York City. And obviously we've seen these images many times before, right? You've seen this about a year ago running away from the cops, right? So there's another one right there, right? Getting restrained. It's a Donald. Now, of course, these are deep fakes. Let me just put that disclaimer on there and we'll be talking about deep fakes, which is of course the big shiny thing in this election cycle. A lot of people talk about deep fakes. But I thought, you know, as we're getting into this courtroom campaign that is unveiling right now, Donald Trump will not be televised because, you know New York City law says you can't have cameras in the courtroom, unlike we do over here Uh But Donald Trump obviously is creating the images himself. So if you move over to his Instagram feed, you'll see pictures like this right here. Uh Obviously, you guys cannot Conal it with what the message might be, which is just like Jesus Christ. Uh He is in there, uh you know, uh taking it for us. So, um that is of course, not a real image uh is probably created by mid journey or some other uh A I uh tool. Uh But I thought we'll, we'll, we'll get into this campaign. Now. It's two and two days until election day, November 5th. And when we take a look at the polls right now, um New York Times had a poll coming out this last weekend and we see it's a pretty tight race. Trump is still ahead. He's had about four or five points in April. Now, Biden is catching up was probably some of that good spirit from the State of the Union address. But I thought, you know, in a campaign where it's won by 44,000 votes, at least that was the margin in 2020. You know, we can all sort of figure out this is going to be an incredibly tight campaign and it probably will come down to just a few votes in just a few counties in just a few swing states. So A I might actually make the difference in this campaign. We're going to talk about that. In just a minute on a panel. But given the fact that A I politics is already advancing and given the fact that many votes have already been cast this year, especially in the primaries, I thought we'll go back to the beginning of the year when the Democratic primary took place in New Hampshire. Now we all know who won the New Hampshire primary for the Democrats. Who was it, it was the sitting president, of course, but in the lead up to that vote, there were a couple of campaigns taking place and one of them was a robocall. Have you guys ever received a robocall? Many of you? Anyone from the US in, in this room? What state do you live in? You haven't gotten robo calls. So you're probably not in a swing state. But if you're in a swing state, chances are your phone is blowing up many times a day. You know, just a voice coming online telling you, hey, this is Barack Obama or this is Donald Trump or this is surrogate for the campaign. Please make sure you go out and vote. So I thought we're listening to one of the robocalls that was placed right in the lead up to the new Hampshire campaign. Take a listen, what a bunch of malarkey. We know the value of voting democratic on our votes count. It's important that you save your vote for the November election. We'll need your help in electing Democrats up and down the ticket voting. This Tuesday only enables the Republicans in their quest to elect Donald Trump. Again, your vote makes a difference in November, not this Tuesday. So the message is pretty clear, Joe Biden is telling you to stay at home. Right. And you know, there are thoughts about Republicans being worried that Democrats might enter the Republican primary and so that Robocall was placed. Now, did this make a big difference in the vote? No, it didn't. I think it's a nice way of looking at how deep fakes might impact the campaign. We can only imagine what might take place in a close election if you might receive a Robocall one day ahead of or on election day telling you to stay home because there may have been a terrible car accident or bad weather coming up or what have you. So we'll talk about that in a moment, but I've got 36 seconds left. So I thought I'll talk about, you know, how easy it is to create that. I understand we had the CEO from 11 labs yesterday today. And Wire magazine actually reported that the robocall you just heard was created with the tool 1111 laps. And so when you take a look how easy it is, I'm not going to play it through, but you can actually do with the free version, upload a few sound bites from President Biden or President Trump or any of you guys and actually create that deep fake in just a matter of moments. So with that, I think we're going to switch over and talk about not only deep fakes but what's underneath and how it might impact this next election campaign. Thanks so much. Thank you.

Presenter: Dr. Peter Kecskemethy, Co-founder and CEO, Kheiron Medical Technologies

Brainstorm AI London 2024: Boosting Breast Cancer Detection

Thank you very much. It's an absolute privilege to be here. And uh I, I thought about that. Probably everyone here has heard a lot about what A I could do, what word could be, how it could be bad for humanity, how it could be good for humanity. But I'm bringing you slightly different things today. And that is a very good news story. How A I in practice is actually bringing impact right now today. It's not a future, it's not a hypothesis, it's actually what's happening now. And I'm also going to bring you a secret, hopefully that you can take away from today and maybe a bit of motivation. So let's get into that. So my company Kiron has developed an A I called Mia that is supporting doctors in early detection, specifically breast cancer screening at the moment. Uh And that has got quite a bit of news coverage. We were on the front page of the New York Times. We were also named after our results to be one of the seven biggest breakthroughs of 2023 next to CRISPR and Messenger RN A. So in good company and there's a very good reason why that is the case. One of the reasons is we are solving a very hard problem. People may not know that breast cancer screening is, is very, very complicated as a diagnostic task. Uh It's so complicated. Some people say it's like finding a snowflake in a, in a snowstorm and, and and actually the practice of medicine covers that it pretty much reflects it. Uh most countries outside the US, uh two doctors are required to look at every single mammogram. So basically every single image that is done for breast cancer screening. And even if the two doctors, we get at least 20% of the cancers missed, that means at least every fifth cancer gets missed. That's like the best we can do across the globe. And in some places, over 70% of the cancers get missed. This is, this is the best we are doing in screening right now as humanity. And you can see a couple of examples of like a hard case on the left and a hard case on the, on the right, on the left, the cancer was missed and on the right, a lot of the times the doctors look at all the little tiny nodules and they're thinking, OK, which one of these is potentially a cancer? Which one is not, it's like extremely hard. Some people call it magic sometimes. So OK, what can, what can A I do? I'm going to bring you talk, you through this particular case, this is an actual case where a woman went for breast cancer screening regularly. So did the best she could do uh multiple years. Uh and she died of breast cancer and she died of breast cancer because the cancer was not picked at the last screening. It was not picked up two years before and it was also not picked up four years before every single time missed by two doctors. OK. This is, this is reality. And what can, what can the A I do? The A I, we ran the many thousands, hundreds of thousands of cases. What the A I could do is find it at the last screening, find it two years before and find it on the case for four years before. So this could have been over four years early detection. And we know that when breast cancer is detected early, it's over 90% chance that a patient survives. That could have been practically a life saved with A I. OK. So that's the impact what we can have on a single patient, right? So let's see what, what does it mean in reality. So what you see here is how a doctor looks at a particular case. This is a medium complexity case. It's not like a higher complexity case. You see a couple of nodules, but also it's relatively clean. Uh In this particular case, again, two doctors missed the cancer and the A I found the cancer. Ok. This is not a very, very hard case, this is a medium case, but the doctors have to go through hundreds and hundreds of cases every day. There's just not enough energy or attention for the doctors to do their jobs. Well, unless they get good tooling and good help. And I believe a I is one of the fundamental ways we can, we can, we can help. In fact, that's exactly how about what we have shown in live use. So what does it mean? It means we can have extremely strong impact on a patient on the life of a single patient. It also means because we're talking about software, this is automatically scalable. Like you bring a piece of software, a box to a hospital and you can have doctors start saving more lives. That's practically what we're talking about. And so roughly, the impact that we could have that, that Kyron is focusing on is scaling it across the globe. So we're estimating we can probably help doctors save about 2 to 3000 lives every single year in the UK, 40 to 60,000 lives in the US and hundreds of thousands across the globe. OK. This is a piece of software that you bring across completely scab right now today. And this is not a hypothesis. This is based on actual evidence, actual the results that we have right now in multiple clinical trials and multiple multiple assessments across the globe. Um So why is, why is this big, why is it, why is it different from the past? Actually, computer science and computer vision has been used a lot, or at least there were a lot of attempts to be used for breast cancer screening, but not with too much success. And there's a couple of reasons for that, but I'm going to talk, talk you through how we had a slightly different approach. We took two things extremely seriously. And I think the importance here is what can you learn from that? How can you translate it to like how you use A I or how you, how you develop A I and the two components of that is taking A I seriously that it's a very different type of technology than humanity had before and taking the domain seriously, right? Like we're talking about a domain where any kind of decision impacts the life, right? Like either way either you can over diagnose or you can potentially miss a cancer. So taking the A I seriously and taking the domain seriously means that you need to collect a lot of data, you need to have extreme level of evaluation. I just need to make sure that when the software goes to healthcare, to actually impacting every day, the patients and the doctors, it has a positive impact rather than a negative impact. OK? So if you want to know the details, you can look it up, I have a number of papers and nature published and also a number of other publications, but I'm just going to distill for you some of the secret sauce that I believe we all need to need to think about. So hopefully, you can use that in your everyday practice. And the secret here is that A I is not just a model, you have a good model that you think is good, it is not going to necessarily or most likely not going to succeed in practice A I is a type of technology that we have never had before as humanity, right? Like if you have a piece of hardware or software, you know what it's used for, you test it in the lab and too much. That's how you're going to use it in everyday practice your software. But A I specifically, it's designed to be able to be intelligent and good on input and in circumstances that it has never seen before, right? This is a completely different. That's the difference from, from between A I and other technologies. So what does it need? Of course, the models need to be good, it needs to be good in the lab, but also it needs to be good wherever else you want to use it. So the algorithms have to be good. OK, fine whenever, but we need to validate that again, both in the lab as well as in real word. However, we never know because it's a black box, whether it still generalizes, does it still work? If I bring it to a specific hospital, you need to validate that locally in deployment. So that's the deployment stage. And also because it's a black box, it might work one day, but it might not work next week or might not work a month or a year later. You need to keep monitoring that it is still providing the same performance. So you need four things for A I to work in practice, you need to get the algorithms right. You need to validate it. You need to make sure it works in practice both upon deployment as well as over time, right? So please please, if you are impacting lives of it or, or, or uh you're having some kind of very, very important impact with A I do make sure all of those work because uh or all of those are covered. Otherwise it's not gonna be very good. So after that, I'm actually going to leave you with the words of uh one doctor and one patient who, who can tell you about what it means for them uh on uh having used the A I um in the UK. I remember when I was a child, they think they would eventually find a cure for cancer, but it will take a long time. And this is all just progression down the right road. We read 5000 mammograms a year. There is a lot of pressure on the service. I have staff shortages, people are burnt out. We have to make our decision first. Then we are allowed to save the opinion of Mia. So we press the little button, it opens up and shows us yellow circles that says, can you have a second look here. I was quite surprised to hear back from the penny didn't drop immediately until they said they had found a very small amount of cancer and they said it wouldn't have been picked up by the human eye. It was too small, but the computer earmarked it and that's how they found it at the moment we are using in an evaluation process. So we haven't integrated it fully into the system. We have modeled that if we did have ma in our workflow, that would be a big difference because it would make our turnaround time go from probably 14 days at the moment down to three days. The technology helps move us forward. This is the beginning large scale. We can make a difference. It's a lifesaver, a life changer. So I will just leave you with that. That I think when we have enough evidence that A I works, it's actually our moral obligation to use it. Sometimes we need to be very careful. Sometimes we need to go through all the steps to validate it. But when we do and it can have a very strong impact, we need to actually use it And, uh, if you want to learn more, feel free to contact me.

Dr. Sunita Mishra, Chief Medical Officer, Amazon Health Services  Noor Shaker, Founder and CEO, SPATIALX  Moderator: Eileen Burbidge, Executive Director, Fertifa; Founding Partner, Passion Capital; Co-chair, Fortune Brainstorm AI London

Brainstorm AI London 2024: Building Trusted Data Foundations For Breakthrough Innovation

Dr. Sunita Mishra, Chief Medical Officer, Amazon Health Services  Noor Shaker, Founder and CEO, SPATIALX, Eileen Burbidge, Executive Director, Fertifa; Founding Partner, Passion Capital

Laura Connell, Partner, Atomico, Katelin Holloway, Founding Partner, 776, James Wise, Partner, Balderton Capital Moderator: ​​ Alex Wood Morton, Executive Editor, Europe, FORTUNE  

Brainstorm AI London 2024: Navigating Opportunities In The European Market

Photography by Joe Maher / Fortune Brainstorm AI

Poppy Gustafsson, Chief Executive Officer, Darktrace   Lama Nachman, Intel Fellow and Director, Intelligent Systems Research Lab, Intel Corp.    Baroness Joanna Shields, OBE, Founder and CEO, Precognition Moderator: Ellie Austin, FORTUNE 

Brainstorm AI London 2024: Championing Responsible AI Within The Enterprise

Photography by Joe Maher Fortune Brainstorm AI

Poppy Gustafsson,Chief Executive Officer,Darktrace ,Lama Nachman,Intel Fellow and Director,Intelligent Systems Research Lab, Intel Corp.,Baroness Joanna Shields,OBE, Founder and CEO, Precognition

Merav Bahat, Co-founder and CEO, Dazz    

Brainstorm AI London 2024: Turning The Threat Of AI Into An Opportunity

So just imagine that we're not here in London at the Rosewood, but rather we're in Paris in the movie cinema at and the year is 1895. So what happens now is that all of us are just jumping on our feet, running through the door, shouting and screaming and really panicking. What happened to the people on that uh film when the first film was uh was presented, they couldn't tell right from wrong. Is that a real train or not? So they're really, really afraid that it's gonna go at them and crush them. It might be funny today, but that's the only, not the only example, right? Whenever there's a new technology, typically modern technologies has always been a little bit scary and I can share a few more examples even when you think about um consumer were always terrified of elevators, right? So just think about it, it makes sense just being hanging up there in the sky with just a few cables holding you up. This is terrifying. And then the phone, people thought that it, it's a device that helps communicate with the deaf and, and even today, an average American is more afraid on robots than death itself. But we all know that today is the things that we are afraid of more is A I? And are there good reasons to be afraid of A I? Obviously? Yes, because even if you think about news and social media that we cannot even tell what is real, what is not real, what is fake and our identities, they could be theft and, and uh breached so easily so everybody can become us and they can, you know, tap into our bank accounts and, and medical records and what have you. Um And that's, that's, you know, only the personal level at the business level and I'm sure that you discussed it thoroughly. Uh Today is uh people are afraid that that A I is becoming so smart that it's actually going after our jobs, they can become better doctors, better teachers, better um uh software developers than us. Um But the real threat and that it is already coming is definitely cybersecurity. We already see the Attackers, the bad guys using A I and data to, you know, to have way more sophisticated attacks. And at the higher scale, I will just give you an example of malicious female phishing email attacks. So I think it's one of the one of the attacks that people are more, you know, common and aware of. So in the past, Attackers would have just sent emails, right? It takes them forever to find addresses and all that today, like email with chat GP T it was um 1000 times more attempts than, you know, before chat GP T and when you think about the attack itself. So in the past, it would be like a simple email targeting you. Some people would drop you in the trap and others, you know, just ignore it. But today with charge GP T the Attackers actually automate the response. So in a way you have a full communication with the attacker and you don't know, you're not aware and it looks like a human being and it's very targeted and personalized. So way more people fall into the trap. So yeah, cybersecurity is already the attacker is already using A I for the for the bad. So can we use the I for the good obviously? Yes. Right. Even if you think about the other examples of the train and the telephone and the elevator and all these and all these technologies help us communicate better, take us from one place to another and save us a lot of time. We can even hardly imagine our lives without it. I think that that's going to be the same with A I like 5, 10 years from now. We cannot imagine our lives without A I. And can you can it be used for good? Of course, it can just think about medical doctors just using all this data to better have analysis of patients even like specifically, like in rare cases, when you need to get data from the entire world to to to give a better uh prescription and medicine for the for the patient. And the same with developers, they can develop with A I way faster and more accurately. So yeah, A I definitely could be used for the good. What we also see in cybersecurity, we're already using a illm data in order to, to defend our organizations better. You look at CIA which is um a, a cybersecurity agency in the United States. Um They're using terabytes of data just in order to identify anomalies in this data, which these anomalies can, can show that there are upcoming threats so they can help in advance alert on upcoming threats. An organization can get uh appropriately prepared even in Britain, the MI-6. Um the spies are using A I to disrupt the flow of weapons to Russia and the war against Ukraine. And when you think again about developers and organizations use A I to fix flows, security flows faster. This is something that definitely we're starting to see more, more so and this is exactly what we do with that, my company and we're not the only ones that we are trying to leverage A I for, for protection. So in a way when you think about businesses, they have been breached every 11 seconds, which creates a huge damage, economic damage in the world. And what we can help is is with detection, identification if very quickly identify the exposure. But we also help with the fix using A I. So we can, we can help fix uh code, code and security issues or vulnerability security vulnerabilities really quickly. Just to sum up every company will be an A I company. Like every company moving to the cloud, every company is a software company, security organizations and security teams will have to be uh A I teams. They have to defend organizations using A I. And more importantly, when you think about it, cybersecurity was always a data play, but now it's even bigger play and whoever wins is the ones that have the data. Good news is that the defense side now has all the data. So we're very, very well positioned to fight A I with A I and we will win. Thank you, everyone.

Jeremy Barnes, Vice President, Product, Platform AI, ServiceNow Rebecca Gorman, Co-founder and CEO, Aligned AI Raza Habib, Co-founder and CEO, Humanloop Moderator: Jeremy Kahn, FORTUNE

Brainstorm AI London 2024: Boosting Confidence In AI With Employees And Consumers

The biggest impediments businesses are encountering and trying to implement generative A I is around the issue of trustworthiness. These systems just do not seem reliable enough to use in many business contexts. And I'm curious, you know, businesses are very concerned about reputational damage. They're concerned about giving out inaccurate information. But we're going to explore here maybe ways we can try to solve some of those problems and make A I more controllable. And I think control is also a big, a big issue, particularly as we move from A I that can just generate content to A I that's actually going to take actions, which is where everyone says this is probably going, Jeremy. I'm going to start with you at service. Now, how have you tried to address this problem of trustworthiness? You know, you're trying to implement systems for a lot of customers at scale. You know, one of the biggest enterprise, uh tech uh service companies out there. How have you tried to deal with trustworthiness? Yeah. And it, it's something that you need to think about very carefully from the beginning. It's not something that you can bolt on. So I've been thinking about it for quite a number of years. Uh A big part of our approach there is to uh make sure that we understand the actual, uh you know, the, what we're solving for, for the end user. Uh There's a little bit of a, um, you know, a, a false idea that if you say that each of the components of the system you build is trustworthy, therefore, the entire system is, uh that's not entirely true. So you know how you work that to analyze the, the end state of it. Um And that comes really into how we design our products, uh design them, test them and, and build them. Uh So we have a fundamental research lab who's working on issues of uh of trustworthiness, published about 70 papers uh on the, on the subject. Uh We have a set of uh what we call our human centric A I guidelines. So that's ensuring that there is uh your transparency, that it's human centric that we have uh is accountable and that is inclusive as well. So each of our products goes through that kind of design process to ensure that we uh have those built in. And there's just a giant amount of testing, testing of the end uh product. Uh We use it internally before we release it and we've uh trained hundreds of people to uh do this testing. Uh And it's important, it's not just testing the model, it's testing the system end to end so that we can get some kind of, uh, uh, guarantee of, you know, it's not gonna do, do, do certain things because we've tested very carefully around edges to make sure that it is. I was gonna ask about edge cases. I mean, how do you identify those and how do you make sure that, you know, because you might have a system that seems like it's working and then you put in deployment and suddenly a user asked something that you weren't expecting and you get, you get a failure mode, right? Yeah. So, I mean, part of that is addressed with the uh you know, our human centry A I guideline. So we make sure that we have a human in the loop for any consequential decision. So there is the ability for the human to understand what's going on and, and override it. But a lot of it is the, the mentality of testing is you need to figure out, you know, you can't just test that it works. You have to test, you have to know where it's gonna break and then you can put guide uh you know, guard rails in around that. And so it's really, you know how you equip that team to be able to, to do that. So you get the outcome right? Raza, I want to go to human loop. You guys are, you know, creating systems for figuring out reliable prompts for LL MS can you talk about how you do that and, and how you're able to, to implement that because I think a lot of people struggle with that. They, you know, one particular subtle prompt works really well. And then the next week it doesn't work or you change a few words in the prompt and suddenly you get a completely different, different answer. Yeah. So I think, I think Jeremy touched on some of this already, right, in terms of having good processes for testing internally. But maybe if we zoom out for a moment, like it's worth understanding, like what's changed, that means that this stuff is harder than before. And I think fundamentally, if you think about large language models or generative A I compared to traditional software, machine learning, we've gone from a paradigm with traditional software where you run the software and every time you get the same result. So it's very easy to write a test for that. And then with machine learning, OK, it was stochastic, it was random, but at least I could calculate a number. I knew where the accuracy was. I could calculate precision or recall. We had ways of doing this. We're now applying these technologies to things that are much more subjective. And so actually even just saying what good looks like can start to become difficult. And so giving the first thing we do is give people tools to be able to measure performance in a variety of different ways. So building like a suite of different evaluators that are measuring different aspects of how well the system is working both during development and production and then iterating against those with some kind of test set. And, you know, Jeremy, you asked Jeremy asked Jeremy, you know, how do you sort of test for edge cases? And I guess I take two things of that one, you know, in some senses because it's subjective, the correct answer becomes what your users say. The correct answer is for some of these applications, not for factual ones, but if you're writing someone an email or doing something that's creative in nature, you're trying to optimize for end user experience, not for some internal metric. And you find the edge cases to a certain extent like you can cover as much as you want pre production. But what you also need to be able to do is if something goes wrong in production, know about it and if you don't have good evaluations in place, you can't even catch those edge cases to fix them. So I think a lot of people, you know, a worry is deploying systems that have edge cases that they don't know about. And we try to give people the tools to avoid that interesting Rebecca at aligned A I, you're all about sort of trying to create A I that has more conceptual understanding. Can you talk about why that's important and how you're trying to achieve that. Yes. So, um, if you think about artificial intelligence it's, it's kind of made, it made up of three pieces. Uh, there's computer science, there's, uh, statistical science and, and then there's data and, um, sometimes, uh, an artificial intelligence can go wrong because the, the computer science bit of it, um, isn't, isn't completely, uh, reliable, like in the Post, uh, post Master scandal we had here with Fujitsu in the UK. Um And sometimes it can go wrong because of, of um the statistical science, not uh it has inherent limitations. So anyone here who works in artificial intelligence knows about things like concept drift, um and data drift and all models um degrade over time. Uh 95% of models A I models have to be retrained within a year because uh because the world changes, so you might not think this affects large language models, but actually, uh G BT two was out before um the pandemic and it or it came out during the pandemic and it made absolutely incorrect uh statements about what was going on in 2020 which were a bit odd when you were trying to use it in 2020. Um So e even language models uh need need to adjust the world as it changes as our language changes. And how are you trying to, how are you trying to solve that? Um Well, Jeremy, I was gonna mention the third part which is data. Um and uh data is, is kind of the also the another wild card when you come to artificial intelligence because all artificial intelligence is built on data. Um So we make artificial intelligence that um it's our, our goal is to make artificial intelligence that can understand human concepts. Even in situations where the data environment has changed. The environment is is different from, from what it was yesterday. It is different from what humans expected. Human blind spots are. There's always the uh blind spots when humans look at data because we're the ones who collect it. We're the ones who label it, it looks right to us. Obviously, we've CRE created a data set that looks right to us. So the blind spots that artificial intelligence learns from this data is our own blind spots. And so it's very, very hard for humans to identify and recognize. So that's why we need to create artificial intelligence systems that can become aware of its own blind spots and extend that to new environments. And you've had some initial success with this. I mean, if you, I don't know if you want to talk about what you've done with content moderation and some of the results you've had there. But I I think that might be interesting that, you know, this is not complete science fiction people, you know, it's possible to actually use some of these techniques to achieve uh some success. Yeah. So um a uh one of the, the early open source um chat bots, uh open source language models that um that you would know the name of uh they released a chat bot that they had to shut down. And it was because uh they were using open A is content filter and a lot of, of toxicity was getting through. So they asked us if we could do better. So we looked at, at open A is uh content filter. They have it have an open source. Um uh They, they present open source of data sets that they, they use, et cetera. And it turned out that um their content filter worked great 97% or something on, on the um on the data sets they were trained on, but they didn't work well on different data sets of toxic content that, that, that model hadn't been trained on. So, Kin A I created a content filter uh that be the open A I content filter on data sets of toxic content that the artificial intelligence hadn't previously seen. Um I wanna do a quick sort of rapid fire uh set of questions here. One of the big issues is around hallucination uh that people are concerned about. I wanna ask you all whether you think this is solvable and if so on time scale, if you have a prediction for how many years out we are from solving this problem, if it's solvable or maybe you think it's not. So I'm curious, we'll go down the line. Raza. What about you? Do you think hallucination is, is a solvable technical challenge for these elements? Yeah. So I think, I think it is solvable technically, but I also think that it doesn't need to be solved for this to be useful productively. So there are ways we're used to designing user experiences that are fault tolerant. In some way. You go to Google search, you get a ranked list of links, you don't get an answer. And like people who are in perplexity, get citations back now. So I don't think it has to be solved to make it useful. But the reason I'm optimistic that we can solve it is if you look at the, when you train a large language model, it kind of goes through three stages, there's kind of pre training and then there's sort of fine tuning and then reinforcement learning from human feedback at the last stage. And if you look at the models before they are fine tuned on human preferences, they're surprisingly well calibrated. So if you ask the model for its confidence to an answer, that confidence correlates really well with whether or not the model is telling the truth, we then train them on human preferences and undo this. And so the knowledge is kind of there already. And the thing we need to figure out is like how to preserve it once we make the models more steerable. And but the fact that the models are learning calibration in the process already makes me very optimistic that it should be much easier to solve than most people. And what time frame would you guess? Oh, like within a year? Ok. Rebecca. What do you think? Uh, yeah, I think, uh, or not. Oh, yeah, obviously solvable. Right. And, and what time scale do you think? Um, II, I agree with that in a year. Uh, Jeremy. well, from just a model perspective, uh you do have to buy some, in some ways, you don't want them to only produce things that are in the training data. So they kind of need to, you know, it's part of their, their capability. So from a model perspective, uh it's an unsolved question, uh it's not gonna be solvable from a systems perspective. And user application perspective, I agree with rather, it's, you know, it's already solved in, in many cases. And uh when you can build a system around it, uh you know, you have other tools there. So you know, a model which will hallucinate can still be incredibly useful in producing value. Can I can I add one thing to that? It's not always a bug like sometimes it's a feature. So if we want to have models that will one day be able to create new knowledge for us, then we need them to be able to act as conjecture machines. We want them to propose things that are weird and novel and then be able to filter that in some way. And so like in some senses, and also if you're doing creative tasks, actually, like having the model to be able to sort of fabricate things that are going off, the data domain is not necessarily a terrible thing. I'd like to add to my as well. Yeah. So um as someone set up here yesterday, uh foundation models as they're created today are hallucination machines. It's just sometimes the case that their hallucination corresponds with reality. Um So, well, I think it's solvable in a year. I also think that uh well, obviously making the models bigger is, is not going to solve that problem. Would you agree that humans are hallucination machines? That uh that's a philosophical question. II, I would say yes, that we are in a different way. Interesting. I want to go to the audience for questions. Um Please raise your hand if you have a question and we'll get a mic to you and um anybody questions. If not, I'll keep going. There's a question there. And if you could please identify yourself when you stand up, that would be great SCT of real chemistry. So when new technology comes as adaptation, we adapt technology to humans, we also adapt humans to technology and we've seen that happening a lot with technology over the years. So I'm curious about your take on what's the biggest adaptation of humans to generative A I that you expect in the next few years, biggest adaptation of humans to gene A I Jeremy. You take that one first. Well, we saw it happen. Uh You know, when Google first came onto the scene, people just changed the way that they searched in order to, you know, type in queries, which would be achievable. So, uh you know, I think that will lead to uh you know, some of the uh challenges with those systems, you know, gradually as people learn how to steer them, uh you know, I improving um on the flip side, uh we probably don't wanna expect people to adapt. I think that's one of the great things about, you know, that the promise of all this new generative A I technology is it can meet humans where they are instead of humans having to kind of come, you know, if you've used a chatbot from pre gene A A I, you had to, you could get it to work, but you had to really know exactly how to, to phrase things. And so um yes, I think it'll happen but I hope that we don't count on it too much. Interesting. Uh Other questions from the audience, please raise your hand if you have questions. Um I want to ask about the sort of the Air Canada chatbot incident because that's a case where, you know, it was using some sort of rag mechanism, it retrieves a document, it thinks says something but it has not summarized the document correctly and then you have a user, you know, a customer in that case, getting incorrect information and ultimately suing successfully. You know, I think that there's a lot of companies out there, that sort of incident. I, I'm curious, like how you deal with that situation, uh, since you're Canadian as well. Jeremy, I'll go to you first. Um, but yeah, what was, what's your reaction to the incident? Yeah, I'm on, I'm on an air Canada flight tomorrow, so it's gross to my heart. Listen, that's our, that's our kind of our, our core business. And so, you know, there's a um it's a question of how you design these systems. Uh you know, if you're just designing it to just with a positive metric in mind of, you know, is it able to get to the end of the conversation? Not, is it able to, to, uh you know, be accurate and you don't design the system, right? That's kind of what you get and the a lot of these systems here. Yeah, we talk about, you know, prompt engineering and I'd say, you know, the engineering is very much uh in, in inverted uh commas here. Uh We see that already there's this kind of prompt depth where, you know, there are these prompts which don't actually represent what we wanted to do but have been tested on, you know, a small uh set of examples. And so again, it's how you take something which seems to work in a proof of concept. You probably don't just wanna put it straight into production, you know, with real customers who, you know, have expectations and terms and conditions and things like that. And so it's, you know, how you bring it in the enterprise context so that it's able to uh you know, knock off some of those rough edges. That's where that the tough work is. And yeah, that needs to be done. Prompt. That's an interesting concept rather. I mean, is that something you, you've been uh considering or, or? Yeah. No, absolutely. I mean, in some senses, my answer, this feels like a softball question for me because like, how could I, I can, could have fixed this or they should have used human loop, right? Um But yeah, the management of prompts like versioning them, having history of them being able to test them rigorously, being able to have the domain experts be involved in that development. Like I do think that the Air Canada incident was like, completely avoidable. Like there's nothing fundamental about the technology that meant that had to happen. I just don't think that they had done enough around testing of the city. Yeah, I, I really disagree. What, what we've seen in enterprises over the past several months is that they're really cool demos and the really cool prototypes the CIO s brought to their CEO S, they just haven't been able to get them reliable and robust enough to, to bring them into production, at least. So, I, I agree there's a perception of that, but we've done it enough times now that I think it's just a question of like, so how would it, I'm curious on what, what would they should they have done to solve that? So, I think if it had sufficient guard rails in place, like they, they gave the Chatbot way much hard, like wide range than what it should have been able to say. So you can put these things in much narrower rails and like we used to do this with Chatbot systems where you do intent inference and then you kind of like follow up with smaller pieces of usage and they more or less gave people like almost raw access to Chat GPT with a little bit of rag attached and sure that's a dangerous thing to do, but like most people shouldn't do that. Right. Yes, I should have used service now on that. We've had, you know, hundreds of 13 years of work to kind of make that work. And, you know, a lot of it is just an intern builder system and then it's like we wanna push into production. So if you're understanding the, what I could do, so don't just give real access to, to chat GP T. Um, on that note, we're gonna have to end it because we're out of time. But thank you very much for, to my panelists and thank you all for listening. Thank you.

Emran Mian, Director General, Digital Technologies and Telecoms, Department for Science, Innovation and Technology (DSIT) Robyn Scott, Co-founder and CEO, Apolitical Guy Williams, Head, UK Defence, Palantir Technologies Moderator: Alan Murray, Chief Executive Officer, FORTUNE

Brainstorm AI London 2024: AI’s Impact On The Geopolitical Landscape

Emran Mian,, Innovation and Technology (DSIT), Robyn Scott, Apolitical, Guy Williams, Palantir Technologies, Alan Murray, FORTUNE

Thank you. We have a lot to talk about and a very short time to talk about it. Enron, I'm eager to talk to you about what can government actually do to mitigate the downside risks of A I, Rob? And I'm eager to talk to you about how can government use A I to just be better at what it does. But I, I think I have to start with you uh because you're in the defense business, uh you work for pun here but work closely with the UK government in the last 48 hours we've all seen on television, a pretty extraordinary uh event that Iran sent more than 300 drones, ballistic missiles, cruise missiles aimed straight at Israel. And we're told that 99% of them were shot out of the sky before they hit the ground, which is if true. Extraordinary. What is this event that we've all witnessed in the last 48 hours? Tell us about the changing face of national defense. I think the thing that's really highlighted for me is um if you go back in time, only, not even 10 years, but say four or four or maybe even three military operations are all about the speed of decision making decisions at the speed of relevance I think is a term often used. And historically, we've seen things like if you go all the way to before something is launched, it might be looking at satellite imagery to understand what things have moved, what changes have taken place. Now, with the introduction of computer vision models, you're able to take a two kilometer by five kilometer image. And within a second, identify all of the things that you might be interested in, whether it's changed or whether it is something that you, you recognize historically, that might have taken analyst hours to get to that point on one single image. And now it would take thousands of 300 of them with a pretty limited time. Yeah, exactly. And so the idea of identifying where these things might be coming from and deciding these are the ones of interest and then maybe tasking things, the collection assets that you might have, which could be sensitive or what have you and then passing that structured data through a model to then understand what are the interesting things do you care about all the things that might be good indicators? And more so it's a secret, it is a sea change. Totally, the compression of decision making times has been, you were directly involved. My understanding is you were directly involved in working with Ukraine to very much the same weaponry, by the way, both the offensive weaponry and the defensive weaponry, uh that Russia was using against Ukraine. What did you learn in that experience? Uh The Ukraine example is, is fascinating and very unique in the fact that you have seen, um, a complete tech sector nationalized into the defense sector and the speed of change that they've been able to deliver in terms of transformation and the use of A I and other technologies has been incredible. Well, they had some help, a little help. Yeah, but I mean, again, one of the, one of the earlier um commentators are talking about the, the getting your data in a healthy place and then run the models across it. And that's a very important part. The thing is we're fortunate in both of those cases that the defensive capabilities were so strong, but these are going to empower offensive capabilities as well, right? I mean, it's gonna be, it's gonna be a race. Yeah, I think the idea of strategic strategic competition is definitely, definitely with us and I think in terms of governments investing into future technologies and A I is, is gonna be pretty um pretty game changing, deterministic uh Emma and I, I want to back up a little bit and, and get to the broader question of what A I means for governments, uh which is the question you wrestle with every day in your position with the UK government, I guess. The thing is someone said this from the stage earlier today is this is moving so fast, so fast. How can government hope to keep up with the pace of change and mitigate the negative consequences in any way? Yeah, so a big part of our approach has been to do things differently. So we've been much more open to bringing in expertise from outside of government than perhaps sometimes we are as so we brought in lots of expertise from A I firms from academia, including Ian Hogarth who chairs our A I Safety Institute I think was speaking earlier in the conference and we've been much more open to the expertise and sometimes we are as government. We've also moved really, really quickly and in order to move as quickly as that, you really need political support all the way up to the top of government. And we've absolutely had that we've had a prime minister who's chosen to prioritize this set of issues. And in particular, he lent into saying that the UK would host the first A I Safety Summit which we ran in November last year. And I think that's another important lesson for us in terms of dealing with some of the risks or some of the potential downsides of artificial intelligence. We've got to do it with other governments. We've got to do this together. We've got to build international governance around this and that's why it's been so important to us to kind of begin to lead that conversation. So we hosted the first summit last year, we're co hosting the next summit with Korea next month, we are also trying to collaborate really extensively on A I safety testing with other countries. So our A I safety institute just signed a partnership agreement with the same institute in the US. And we're talking to these capabilities as they begin to be developed in other countries as well. So kind of that collaborative approach I think is really fundamental. And so what is sort of the top three things that you think you can accomplish with this A I safety effort? So I think the first thing is to bind this set of countries together, to build that connective tissue between the countries which have frontier A I firms and which you would expect to be at the frontier of A I being deployed across the wider economy. I think the second thing I'd say is there's still a lot to do in terms of understanding the science of A I safety. So there's a lot of discussion about what's the right approach to regulating frontier A I. And our view has been that the best thing to do first is to figure out what does good A I safety testing look like and set standards for that and to do that in collaboration with other people who are at the frontier of doing that work. And that's what the A I Safety institute been focused on. We want to understand what really good safety testing looks like, do that in collaboration with the firms and do that in collaboration with other countries. And is the end result, you give a seal of approval to certain models and not to others. What does this look like? Five or 10 years down the road? I think it's hard at this point to say what it looks like. Five or 10 years down the road. I think our approach at the moment is let's establish the basic science, let's figure out what good safety testing looks like. And alongside that, let's use our existing regulatory system for the harms that are manifesting potentially here. And now, and I think that's another important part of our approach. The digital markets that we're talking about here are markets that we already regulate in important ways. We have just passed legislation to allow us to better regulate for harms that occur online. Um Many of those harms will now also be created with A I, but we've, we've just regulated for that. So we should use that regulatory framework to deal with A I generated harms in the same way as we were planning to use it for other things. Yeah. Robin. Uh uh you're working uh from a different angle on these issues. You're trying to figure out how can A I make government better at what it does. Can you talk a little bit about what you're doing and where you're seeing results. Yeah. So the perspective I bring to this is running a political, which is a global network used by about a quarter of a million public servants. We also do online training and as part of that, we have a government A I campus and we've trained already about 10,000 public officials around the world. So that's our lens. The headlines are the potential is immense. The Turing Institute came out with a report recently which speaks to speak to this in the UK context, but it's generalisable. There are about a billion transactions a year between government and citizens. About 100 and 40 million of those are complex but repeatable tasks. And of those uh the the institute estimates that about 84% can be uh automated with A I. So incredible potential to cut into the work of government, improve citizen services, improve the return on taxpayer dollars. There's also a secondary reason for doing this and for adopting A I, because better users of A I are better regulators of A I and they're better buyers of A I. And I think, you know, the UK really has shown leadership on the general adoption front in government as well. And is there a willingness to do this gets back to the retraining issue? We were talking about a few minutes ago. Is there a willingness in government to really embrace the effort? Yeah, absolutely. So it's very interesting to look at government's current data sort of position on A I. First of all, somewhat surprisingly, your average public servant, we've done a lot of polling on this is more positive than negative on the potential of A I that doesn't hold necessarily for prior technologies. And given all the talk about concerns, we've been surprised how generally optimistic they are. There is much more optimism in emerging markets versus developed markets. The conversation in emerging markets is all about this is going to solve personalized education. This is going to bring great health care. The conversation in developed markets tends to have much more of a bent to work towards safety and harms and economic nationalism is also uh becoming a more and more important uh force, sorry A I nationalism rather. And we've seen big plays recently in the UAE and India and expect to see many more around the world. So all those are context for adoption, but there is general willingness to use it. The vast majority of public servants have already played with generative A I sometimes I think under approved guidance, probably sometimes somewhat off the books and that's only going to increase pretty dramatically. Yeah, I'm glad you mentioned economic nationalism because one of the things that makes this conversation about geopolitical effects uh interesting and challenging is the existence of China not represented on this, on this stage, China is certainly an A I superpower. There are some people out there who seem to think it will become the leading A I superpower. Uh and it has a very different background set of values that could be on the other side of any uh uh actual uh uh warfare. Um It, it takes a very different approach to many of the questions uh to use it a simple example. Uh The UK has said very clearly, we're not going to use facial recognition or other technologies to pick people out in a massive crowd crossing the street. I believe China has done the exact opposite so that we can make great gains in social cohesion if we have the ability to do that. So how, how does China's strength in A I affect these conversations, Emran, I'll start with you. Yeah. So our approach has been to include China in the international conversation about safety. Uh So China will represent it at the A I Safety Summit in Bletchley Park. And we think that's important because China produces a lot of fundamental research in A I. Uh and it has a very busy and a very active and a very interesting A I ecosystem. Um So having China as part of this conversation, we think is important. But at the same time, we've been really clear as a government that where China is engaged in activities that affect or harm UK interests, we will say so clearly and plainly. Uh so we've done that in the past few weeks we have called out where China has been engaged in uh offensive cyber operations in the UK. And we think that those two things are entirely consistent. Well, there are a lot of people in the US these days who think maybe they aren't consistent and that should we be sharing all our research with China if we're going to be in conflict over whether it's cyber or actual warfare? Yeah. And that's why our approach will not be to share all of our research with China. We're very clear on there are areas of research that you, that we will protect and that we do protect Robin. How do you think about this? We're looking very much at the practice, the applied side of A I in government. And one of the interesting general admissions, I think there's a general admission around thinking around capability in government. There's a lot of talk about, we must do this, there's a lot of talk about adoption, but there isn't the focus on building the kind of diffusion capability. So you can actually roll it out across government. So we look at that and then we look at how the exchange of best practice happens because the field is moving so fast, you are never going to win. If you just take top down guidance or uh you know, directives, you've got to talk to others who are in the field. And I think one of the interesting dynamics is actually the vast majority of countries are quite good at collaborating and sharing best practice. We see a lot of that on our platform, China kind of doesn't take part for the most part explicitly or at least in the same way in that sharing of best practice. So the question I'm interested in is is there going to be a siloing around how A I is rolled out in the Chinese government versus elsewhere? And should we be trying to get more permeability so we can, so we can be mutual learning around non controversial things? Like how do you get great services to citizens? How do you bring down the carbon footprint of these A I technologies? We heard about that earlier. Yep and Guy in defense, how, how do you think about China? Um Well, I mean, as you point out that facial recognition is reportedly like fantastic, but when it comes to things like large language models, apparently there's a slight limitation because they um restrict elements of the internet available to specific areas. So there is a, there is a theory that they might be slightly behind but to pick up Robin's Point there, I think what's really important is almost not ignoring it but just more internal focused of looking at an organization and in parallel to the regulatory and the safety side is like, how do you really push the use of these technologies to win? Like what are you doing day in day out to make the most of this tech to actually improve your, improve your work. And there's often this concern about losing jobs, but we like to view it as um in a marvel context of Team Iron Man versus Team Ultron. And it's how do you bring this technology to humans so that they can be 1000 times better at their job? And with the view to how you actually win in the long run, was that a national defense answer? Yeah. Well, I, I think so. I think if, when you're an analyst working in the defense in defense intelligence, if an operational commander is, how do you, how do you bring? But you, but you have to be thinking about, do our cap are our cap, do their capabilities match ours are our capabilities, period. That's the nature of defense. And I think that's the important thing of, of balancing the innovation and driving innovation forward in parallel to the safety side rather than letting the safety side restrict what you do in terms of innovation. Ok. Last question, I'd like to ask all of you to answer. If A I ran the government, what would we be doing differently today? What would we be doing that we aren't doing? And I'm going to make you go first. Um, we'd be doing meaningful work, not busy work. It would be my blunt answer. Still too much busy work in government. I think, I, I think it, it's very hard and, and A I can help us get rid of it. I hope so. Yeah, Robin. So there's been some data suggesting this is an aside that a majority of people, I think it's younger people in Europe actually want A I politicians. So they want A I to run the government as opposed to the politicians, as opposed to the flesh and blood runs of the coming, coming to government sooner than you may think. But in terms of what that would look like, assuming it's benevolent and aligned A I, I think the the potential the dream is that you get much better, much more personalized services, hyper personalized services to citizens and you can invest a lot more in those because you are saving money because of so many efficiencies that you're also finding Iran final word. Yeah, I'd add, I think government is a complex activity and it's too often con noted as being an elite activity. And I think A I gives us some real opportunities to, to deal with that. I think the way in which A I can help to simplify and explain things to citizens, I think has real power to it. I think the way in which it can help to simplify interactions with citizens, I think has real power to it. Um So I think breaking down that elitism that I think a lot of people experience in their interactions with government, I think, yeah, I can help us with that. Well, thank you all, you've given us a lot to think about. Thanks for being part of brainstorm A I.

 Amy Challen, General Manager, Artificial Intelligence Sachin Dev Duggal, Founder and Chief Wizard, Builder.ai Jenny Griffiths, Vice President, Data Science, Oracle Moderator: May Habib

Brainstorm AI London 2024: Navigating Roadblocks And Catalysts To Enterprise-Wide Adoption

Hey, hi again, everybody. Uh really excited to talk about getting those po CS out of purgatory and into scale uh with a fantastic panel. Uh First to kick us off how many folks have got scaled use cases in production for generative A I in the audience hands up. OK. Just what we thought. Um How many folks if you know, are spending more than a million dollars a year on generative A I? More than 3 million, more than 5 million, more than 10. OK. Well, you guys know what my answer is. Um but uh that's great. Uh uh insight and data folks are early and the, the stat, this is accenture BC G. Lots of folks who pulled executives fewer than 10% of the generative A I explorations are in production. Uh There is a graveyard of unused chat applications when those use cases do go to production. So I do think um we are in a um a bit of a trough from a adoption uh and scaling uh perspective and let us dive into um uh this conversation with, with that in mind. Um You all have scaled use cases in production and I would love for us to actually just maybe rapid fire, share those before getting into what you had to overcome to get to scales. And then we definitely want to take questions in, in this session. Uh Sachin, I'd love to start with you at Builder. What are you guys doing? How are you using A I internally? Um We are proud partners, mutual customers. So um tell us a little bit about how you guys are using A I internally. Sure. So, you know, at builder, we use the entire length and breadth. So, right, the way from, you know what you call classical A I machine learning, both supervised, unsupervised. Um We actually, the core foundations of our platform are built on a knowledge graph. So graph theory is really important because it creates the reasoning engine to give generative A I guard rails for all intents and purposes. Um but also um is a really strong method for reinforced learning. Um And um and then we use computer vision. Um So when we produce applications, you know, the old way of testing was you had a human that looked at an application compared to, to design. Uh It takes a really long time, we can test an application across thousands of devices in 30 seconds using computer vision. Um And then obviously, which everyone's looking forward to is we use a lot of generous technology. Um Now, I would say uh we have our own work especially on the small um sort of small language model side. Uh But the, the, the use of commoditized LLM SI use commoditized Lucy because I know it's a new technology but it's commoditized, it's actually quite important because we work with just about everyone um both from an image video and sort of the the tech side. And that allows us to do everything from user story generation based on ragging with the knowledge graph through to what we just launched last week, code generation. Um where the the our error rate is is single digit percentage um in in code produce because it plugs into the rest of our plumbing. So, really across the spectrum? Awesome. Thanks for sharing, Amy. What about you at Shell? What are we doing? Um We're doing a lot because we have the remit of the whole of Shell, which is a very large integrated value chain energy company. Um but some of our best established use cases and when I say best established, I mean, 10 years, it's things like predictive maintenance which you can use, you could use it in traditional oil and gas space, but also um wind farms, also the Holland Hydrogen one where we're, which we're building uh to be operational a year or two, which will create um green hydrogen from the Holland Coast, North Wind Farm that is being designed to have these integration as well. So that's a use case where we've got um So tens of thousands of pieces of equipment um under predicted maintenance models. Um Other examples are, are, are eb charging network. Um We've got optimized charging across that to ensure that um it will charge as efficiently as possible from where you're at. And then you've got other applications um such as um our marketing and loyalty pieces. Again, really obvious use case people have been doing it for years, but these have been a cut for years. So we've got a long tradition of making all the mistakes putting into production, scaling it. And that has put us in a much better place for the newer use cases and things like TV. I come out, everyone gets really excited. It's the same problem. It's, it's not the technology, which is the problem, it's the scaling, it's the implementation to processes, it's the change management. And so we have got that framework to do this. We have got these practices and we're doing the same again. So it's, it's been a really good couple of years in terms of the improvements in the models and improvements in the capabilities. But we are just putting it into our um, existing frameworks and existing ways of working in order to scale it and get the value. Awesome. We're gonna come back to that. That's great. And Jenny, what about you? You are both building A I into your products and using A I internally. So maybe tell us about both those sides of skilled use cases. Now, that's a really good point. We do have a kind of two pronged approach. Um So my group um I'm in Fusion A I and basically we develop embedded A I applications for Oracle Fusion Suite. Um and that covers pillars like uh human capital management, uh supply chain management, customer experience, uh finance. So loads of kind of different applications, but we try and use the same core underlying technology for loads of different customers and loads of different applications. So it makes us think about how you can design A I at scale. Um And yeah, when it comes to what we've been doing with generative A I, we have 25 use cases go live just last quarter alone, um A lot of others as well. So we're beginning to see this kind of customer adoption. And from my point of view, what's been really lovely to see is kind of people lean in at scale for kind of business knowledge and business application, but also on a more academic level of we now have customers really engaging with us about how A I models are put together what kind of data we're feeding systems with. So we're seeing this kind of business awareness and kind of academic awareness rise hand in hand, which has been really great to see awesome the end user adoption that you are seeing. Let's talk now, just your and your internal use cases for for generative A I, we have heard over a couple of panels around the reskilling challenge and the training challenge, training almost feels like an understatement or the kind of skill that you need uh in a workforce to be able to really use A I as leverage. Right? It's such a different type of technology. Can anybody speak to the types of things they did to really get scaled end user adoption inside of their companies? That would be good advice. So I think, um you know, I I'll sort of start off, we've looked at the implementation in two different lenses. One is where are you using this to remove human variants? And where are you using this to remove entire human workload and they're very different or, you know, tasks that someone was doing? Um And so where you're using it to remove variants, there is a, there's a co piloting or a companion experience, right? Because you're usually arming the human being. So for example, Natasha is ra I and we'll speak to customers, but at the same time, we'll speak to one of our customer success team and tell them when a customer this is, you actually need to answer with this. So there's a, there's a benefit collective insight, but that is very jarring for someone who's not used to having someone listening in their ear and then telling them what to ask. Um and, and so I think there is a, there is an upward training process in that and you need to get people comfortable with the fact that there's an on the call. Uh I think the removing work is a little bit easier because you're basically saying, hey, listen, uh as a good example, you used to manage your day. We have customer success team. They used to manage their day. You're not gonna manage your day anymore. Someone you're gonna have this assistant that's gonna move your calendar around and sort things out so that we can use the right time slots through the right things because they're all connected. Um There, it's more of an education to get people emotionally comfortable with the idea that, you know, they're sort of one step removed from how they were running a particular task. Very helpful. Yeah. And the way that we've been approaching it, at least to my team is that we've kind of segmented it into the different user base and depending on who it is, who will be consuming the technology. We apply different ways of educating the end customer. So for data scientists with my team, this time last year, we introduced, we called it LLM Friday, which was terribly branded from my point of view. Um But it was basically just my team, every Friday just took the time off tasks just to learn about LL MS because it's this huge new technology that's come in and we needed to be experts quickly. So we've gone to kind of that extreme around education, to the kind of day to day education around how our customers are actually engaging with this. So it could be hints and tips around prompt engineering. And we have a center of excellence to be formed to kind of lead people on that journey, help them engineer those prompts. But then, you know, we're leaving that knowledge with them as time goes on and then you've got the actual outreach to customers um doing events, videos and podcasts, basically, any data that we can feed people around how to get the most out of these systems. And that's what we're seeing is really important. Yeah, absolutely. Um I was in a conference recently about 50 A I CEO S and um there was a one of those like Slido, you know, respond to um the survey question and the number of um CEO S that had built uh A I features into their products versus the number of CEO S where there was meaningful PNL impact from using A I internally like 10 to 1, right? And so the, the onus for actual adoption is being pushed on to the customer's customer. Um And but by the A I vendors, um and I, I do think um the, the scale of change is really one of those blockers to, to wide enterprise uh adoption. So I love that you're doing that. Um I do think that um uh inside of even the A I companies, the um the the steepness of that learning curve, right is really being felt, it's only getting steeper back to you Amy at Shell. So you all have got a uh a deep hit history of integrating A I into your processes and getting a workforce comfortable um with relying on A I to make super critical like life and death types of decisions. Tell us a little bit about the framework that you're now adopting in the generative A I world to scale these use cases. What can you share with the audience? I mean, boring answer is not that different. It's like because again, it's like any change program, you just have to go through the steps and it will really depend on the individual use cases as the other panelists have mentioned about exactly, you know, what will be the blocker to change. So for an example, like with the particularly with safety use cases, you might call them parallel running is a really good one. So we have this technology called operator round by exception where you basically complement and operators around on an asset with um computer vision technology to sort of detecting what's going on. Now, you run it everything in parallel the old system, the new system for a long time and identify the variances before you take anything out. Um That can just once you've seen something working for how long you need days, weeks, months people start building trust they start realizing what's going on. They also, it's a big part of like how, you know, you've got the right model, right? This is the feedback loop. Um And I think that's, that's really the key for me. This is agile scaling where you always have a joint team. It's always a technical team, but also a business team. Never do data science without the business. You're always integrating the feed lap loop at every stage and you're always listening to what people are saying and understanding or if it's not being used, why it's your problem, it's not the user problem and then only scaling up and only taking to the next level when you're actually comfortable with that. And I think there's just no way out of that. It's never the case that you get something and it's perfect because it just doesn't work like that. So it thank you so much. OK. Questions. Um We've got time for a couple that is a fast hand. You in the middle. Yes, please uh tell everybody where you're from, who you are before you ask your question, Sam. Yes. Business school Barcelona. So we've talked a lot about employee training, user adoption and so on. What about the executives? How do we get the executives to a point that they can redesign organizational processes, structures and even business models? Oh, I love that question. There are so many execs who have to fill in airtime at board meetings. Right now and those 10 minutes last for a very long time. So, um, he would like to take executive training. So, you know, one of the things that we've said to our executive team and, and that we have a weekly war meeting, which is sort of weekly action review last week, next week. Uh, and I sort of put a man to everyone's like in every, every second meeting I wanna know a problem you wanna solve because I think less about education, less about process is actually first agreeing, what is the problem we're trying to solve? And is it really a problem that needs to be solved? And then, and then the question is how will you use the entire landscape of it? I not necessarily just generative. How will you use the landscape of how to solve that problem? Right. And where do you need it? That's just our experience. Yeah, I love that. Yeah, I'd be inclined to agree actually, if an executive comes to me with a problem, I always try and get to where they're trying to be and what KP it is that they want to impact rather than how they think they want to get there. And that's not because they might, they might know the answer, but it's kind of good for the data scientists behind the scenes to verify the approach that you're taking and that's really come into fruition kind of this year with generative A I because a lot of people think they want generative A I's case. But actually it turns out they want a deterministic output. They want the same thing again and again and then it's up to me to go, well, in that case, you should be looking at plastic A I, and it's not worse or it's not better. It's about that kind of education around which technologies are appropriate or which, which stages and the only way you can get there is actually talk about where you want to get to right at the very end. Yeah, I love that. I think it's really easy for people to come to you and say they want Blockchain and JV, I in an app and you're like, I think you need to business process automation and it's not because people are stupid. It's because they're curious, which is great and then you can have a conversation and we've had no problem at all having interested executives in all of this and the curiosity is great, but ultimately, there's no substitute for expertise. So it's always a joint conversation with people who really know their business about what can be done and what's sensible, helpful. Thank you. Other questions. Yes, back there. Hi there, Mark Riley. Mark Riley from Matheson A I. Um my clients have got clear vision and road maps and plans, but the stumbling block is always identifying ro I getting it past the CFO. How do we, how do we make roi a meaningful prediction. Yeah, I love this question. I can take that too. Um So it is so imperative to get the CFO on the same page and I actually think not getting them on the same page as executives uh really can result in some twisted incentives like execs, not sharing the actual ro I because they don't want the budget taken away from them. And so especially in the early days, um where you want to be able to reinvest savings into more explorations and not disincentivize execs from really taking costs out of the business. Um The Roy conversation, you know, should not be politicized. Um We track Roy with our customers weekly, not everybody is doing that, but it certainly requires being able to partner with procurement and finance and have this be kind of board level um priority uh to make that happen. But it's absolutely essential. Uh I think a big part of why Po CS that seem to be successful at a small scale aren't scaling is the CFO doesn't actually know how much this is going to cost at scale. Everything is priced by tokens and you have no idea what it's gonna cost the organization to run this at scale. So including them is just incredibly important. Um making sure that Ry is part of um the actual POC because that builds the scaffolding for what you're gonna take to production from people to process. Is it so important. Yeah, and, and look, and I, I just said whenever we've done an implementation and we, we do this entire thesis beforehand, which is we think it's gonna cut cost by this. Here is the cohort of things we need to test it across, here's the period we need it for. Um And depending on how close we get to the result that becomes the gatekeeper for whether we move to production or not, you know, if, if the cost to have that ro I obviously exceeds the RO I, then it's not worth doing. Um And I think that's, that's built in a really good discipline to test very early. Uh versus where we've seen in the past is it's tested too late. Uh And at this point, it's like, well, has it not shipped? Versus you might actually make the call and we've done this a few times. It's just not worth shipping. Or the approach we've taken is not the right approach because the more you generate, the more cost you have, but you could use classical A I or knowledge graphs, for example, to reuse something you previously generated and then actually makes it much more economical. Wonderful. Thank you folks so much. Thanks to the audience. Well, that's it.

Sanjeevan Bala, Media and Entertainment Leader, Non Executive Director, Bakkavor, SThree; Advisor, DataIQ, Gartner Clare Hickie, Chief Technology Officer, EMEA, Workday   Stuart Martin, Director, Group People Operations and People Platform Leader, Lloyds Banking Group

Brainstorm AI London 2024: Making AI Work At Work

Sanjeevan Bala, Gartner, Clare Hickie, Workday, Stuart Martin, Lloyds Banking Group

So we're going to take a few minutes now to hear from three people in the audience who have direct experience with using these technologies in business. Uh The tough part about this is they each have two minutes and only two minutes to introduce themselves and tell you what they're doing that may be relevant to you. And Sanji Van Bala, uh please you go first. Thank you. So, I'm I've been the uh chief data and A I officer for a number of media entertainment companies. And this is true of many of these companies. We're trying to do three things, effectively acquire consumers, get to watch more content and moize that content. We've deployed A I across all three of those use cases. And in some areas we've seen reductions uh that would take us process that three months down to three minutes. Other areas we've deployed uh increase in advertising yields. My top four learnings. Firstly, this is all about change and people change, the golden ratio is 70 2010 70% on people change 20% on technology and data, 10% on the algorithms. Secondly, uh proof of concept, I wouldn't bother with proof of concepts. Because proof of concept is proof of cost. I've moved to a model of proof of value because you map the value with business stakeholders and get the business stakeholders to buy into that value case. Third, learning is the last mile fix the last mile first. So that's the process in marketing the technology and what people skilling do you need to do to enable the last mile of value realization and finally develop FOMO measure the benefits and get more to talk about those benefits. The other stakeholders then want to have some of those benefits as well. Boy, that's, that's great advice. If you all didn't write it down. 70 2010 rule, which is uh 70% of it is about the people. Yeah. Um uh Number two was uh come on, help me last mile, get the last mile and create FOMO uh uh to, to, to get, get people in. Thank you for that. Next up is Claire Hickey from work day, Claire, please. Hi, thank you very much. Can I just say it's really hard for an Irish person to talk in two minutes. Um So my name is Claire Hickey. I'm the CTO for me at work day. Fascinating afternoon. Um One thing I'm gonna pick up on is their own skills in particular and it is around one is obviously the fear of the job lost and is the computer going to take over my jobs and everything that goes with it. Can I just say three things on this one is who knows what's going to happen, but it's always going to need humans. And the biggest right that we have and it's been demonstrated time and time today is the right of decision making. No matter what other skill may be taken by the computer, the power of the decision is not going to be taken away. So let's upskill on that one alone. The second one is upskilling without A I, it's really hard to upskill. And the third one is use A I to identify the talent gap in your organizations and use it to be able to upskill your employees in your organizations. It costs, it's six times more expensive to recruit externally than it is to absolutely upskill your workers and you are going to be able to retain that talent at the same time. So start using A I to equally develop your talent and identify your talent, but equally look at it in terms of what you need from a skills gap perspective internally to help your employees upskill. But where are you going with your organizations to remain with that competitive edge? What do you need from a skills perspective and identify it even externally to be able to fill that gap? So you're all successful. Did I do that in two minutes? And it was great. It was great. Uh uh Thank you for that. I think the emphasis on upskilling is so so important. And thank you for re emphasizing that. And finally, Stuart Mark Martin from Lloyds Banking Stuart, I, I'm going to face this way. Uh Certainly so two minutes to talk about some of the challenges solutions uh and lessons learned ain't going to happen. Uh So I'll focus really just on the challenges that I've, that I've seen in quite a few of them were quite familiar from, from today. So first thing to acknowledge that quite a number of people probably think this is a fad that's going to pass them by. That's a challenge. We have so few years ago, we probably sat here talking about we are or the metaverse or something to solve the world's crisis. So it's good to see it's going to stay. And so the challenge we have is to make sure people don't sit on the sidelines and think it can pass them by and not participate. So I think how we get, how we do that is going to be important. So this not seen as a bit of a fad. Uh that leads me to the second challenge. I think we all have to really face into and that is that we need a very different approach to the total world of work. So I have an hr hr background. So A I is impacting and will continue to impact every single facet of our, the way we work, what we do, how we do it when we do it, where we do it, which device we use to do it and which uh which even which environment we used to do it. So if you haven't picked up on that yet, um you know, that's going to be a substantial challenge. And I think as businesses and as senior leaders, our challenge is how do you uh grapple with that? How do you make it, uh you know, something real and how do we choose to work in that sort of environment? Very, very complex but very exciting third challenge, speed and complexity. Um you know, things are happening so fast. I think someone asked earlier about, you know, what's gonna happen the next decade? I think when you think about what's happening next week. So what's the, you know, what's the speed things are having such speed and complexity, everyone's trying to study the next unicorn, this is faster, this is better, this is more cost effective. And so I think the risk we have is that our workforce will just tune out and not participate in in all of this. And, and I've been given the nod to close up so that the final one I'll keep it quite short is, you know, how do we uh the risk we have is um engaging our evolving workforce, multi generational uh where we are today and working longer if we need to. So how do we um you know, increase uh the workforce and make work fundamentally more engaging and increase the digital literacy that we need to be successful. Great. Thank you very much. Thank you all three of you for sharing those. Stuart. I love the fact you talked about how this is going to change every single facet of what you do because I think that's one of the things that large organizations are struggling with. Everyone's doing experiments, everybody, everyone's doing trials. But the question that this next panel that we're going to have is going to address is how do you get from the trial stage to implementing these technologies at scale?

 Lan Guan, Chief AI Officer and Head, Center for Advanced AI, Accenture      Jaime Teevan, Chief Scientist and Technical Fellow, Microsoft Moderator: Alan Murray, FORTUNE

Brainstorm AI London 2024: The Future Of Work

Thank you. I thank you, Jamie II. I wanna start with the big question and we can get that out of the way and move on. Uh uh It is remarkable what these technologies can do and it conjures up the image of fully automated robots who can do pretty much anything you want them to do. Do either of you think there's any realistic possibility in our lifetimes that A I will reduce the number of aggregate aggregate jobs out there. Are we really talking about eliminating people's jobs? You want to go first one? Yeah, sure. Uh That's a great question there. A short answer to that is uh I think in the short term there was some job impact. Uh But it's my belief that in the long run, this is a positive sum game. Um So I really want to pivot the conversation to uh you know, around. OK. How do we understand the, you know how A I would substitute or compliment, right? What humans are doing every day so that we can get our people ready for that? Do you? So it's the wrong conversation. It's a mislead. Do you agree with that Jamie? Um, yeah, you know, there's, there's, I don't like the hesitation our jobs are gonna change for sure. Of course, that's not the big disruption. And there are pretty much two ways that companies deal with disruptions. One is to drive efficiencies, get really good at what they're doing and the other is innovation to figure out how to do new things. And I would say we essentially, we're seeing both of those and, and like, the efficiency thing is like, easier when I think about how I use a, I, I'm like, oh, I can write my email. So you start with efficiencies. And so, and that's sort of when we're not very creative, it's where mind goes. But like, innovation is a real opportunity. But we, and we've seen this with every wave of technology that we've had before. People thought, you know, 100 100 years ago, people thought, what are all these people who work on farms gonna do? That's not an issue anymore? So, well, I, I'm just curious with the group. Uh Can we get a quick show of hands if you think that 2030 40 years out, we're actually gonna have a problem creating jobs. Would you raise your hand? Ok. There are a few people back there. I'm gonna come back to you later on, but it's not many and, and I, I would, I would tend to agree, agree with that. So if we start with that as the premise, then the question is how are jobs gonna change? What, what, what activities are you no longer going to be doing and what activities do you need to learn to? Do you want to start with? Yeah. Sure. Uh So I, I think that goes back to my point earlier, how do we get our workforce ready? Right. What kind of skills do they need to grow more? And uh how, how do we get the people ready? Uh you know, for this kind of transformation? I it's my personal belief that, you know, the skills like uh what human is really good at, right? Things like communication, skill being, empathetic, uh you know, doing analytical thinking, structure thinking, these are the cognitive skills that A I, you know, it is not doing quite well yet, right? So uh I think we need to focus on uh having people uh humans focus on applying this kind of skills in our daily job. Um So let me give you one uh uh a frame of reference. Uh So I'm a big fan of uh uh you know, this uh Nobel prize winner and the Princeton scholar Daniel Kman, um who just passed away a couple of weeks ago, right? So his famous theory of system, one system thinking system, one system two thinking, I think it's a perfect uh you know, theoretical ground of what I was describing, right? System one thinking basically is fast in instinctive kind of thinking and oftentimes introduce human bias system. Two thinking is a slow analytical thinking and think about if you know in the future uh stage how A I and humans are working together A I is focusing on fast, you know, this kind of action taking and then the humans are focusing on things that we're really good at being empathetically. Uh you know, doing the structure thinking, being analytical, I think the you know uh you know, like for example, in context, in this case, we are you see that is happening, right? A I is helping our context and agents to you know, do this kind of system one thinking so that they can actually listen to the culture script, right? They can actually be summarizing all the, you know, agent notes for agents. So the agent can actually focus on having meaningful conversations with the customers. So there are many, many examples like that such a dramatic change from where we were just a few years ago, I mean, in in any kind of a conversation about advice to young people and what they should do in their career. In fact, it was a cover of Business Week magazine. Learn to code. You all need to go out and learn to code. Well, in fact, the the the machine is doing a pretty good job taking care of the coding. And what you're saying is we need to be better people, we need to learn to empathize we need better judgment. We need to understand how to motivate people. It's a completely different set of skills. It's a completely different set of skills, Jamie. I know you do a lot of research on this. This is, this is your job. What are you early days? But what are you learning about the skills people need and the skills that are being taken care of? Yeah, I mean, I actually one of the interesting emergent properties of this current generation of language models is actually that they're pushing on some of these meta cognitive skills as well. So in many ways, we're like, oh A I is gonna do all the drudge work so we can have the deep conversations. Um but we're starting to see like one of the things that's really cool is a I is actually quite good at planning. It's quite good at figuring out how to break paths down and, and allocate them. It's quite good at actually connecting in an empathetic way with people and understanding how your message is going to land. And so that really like it increases actually the meta cognitive burden on people. And I think it's um and important for us to think about how to take advantage of it to actually support our meta cognitive abilities. How do we come up with good conversations? How do we think about things critically? Yeah. Well, let's dive into that. I mean, so how, so as organizations, both of you work with companies. How do you, how do you prepare people for that? How do you, you know, you made the point at the beginning about many companies are taking the easy route early on and saying, let's just use technology to replace things we're already doing as opposed to use it to rethink what we do. How do, how do companies get themselves ready to do the right thing here? Yeah, I mean, and that's a huge question. If you think about it all of a sudden, we're giving people this empty prompt books and saying, like, say whatever you want and that's overwhelming. We've only been talking to our computers for a few months, you know, maybe a year if you're an early adopter, um we've been talking to people for a long time but we haven't actually been communicating uh with the machine. Yeah, you, you wrote a great piece for the Harvard Business Review. I'd love for you to share some of what you said there because all of us are experimenting with this stuff. We're writing prompts. Actually, there's an art to writing prompts. So how do we become better prompt writers? What makes a good prompt? I mean, so it's a good question and I'll have, you know, it is like one of the top academic questions as well. So when you look at the academic literature, a lot of the papers around prompt strategies and how you prompt uh a language model and that's because a good prompt strategy will actually give you better outputs than a bigger, more powerful uh model. Um So actually, one of the things that's important is to recognize that a language model isn't a person. And when you're prompting it, you need to prompt it a little differently. So uh when we were chatting backstage, we did a fair amount of grounding. This is that process of like getting to know each other. But because they're humans, I don't have to do the same level of grounding that I have to do with a language model with a language model. You have to be super explicit and say like you are playing this role and this, I want you to produce an output in this format for this audience. So that's very explicit and that's because it's not a human. Um But there's other things that actually you can use to your advantage with the language model not being human. Um You know, I can't be like, oh my gosh, I said something really dumb. Let's just start over and try again. When I'm chatting with these folks with a language model, you can actually just erase all your history and erase all the context and start the conversation again. And so you can actually start doing things like um I don't know if you know the concept of the wisdom of the crowds where you like ask a bunch of people how many jelly beans are in a jar. And if I ask this whole room, we'd on aggregate, get the right number of jelly beans. Uh You can start doing that with the language model. You can ask it to play different roles or produce answers in different contexts and then look at the sum total um of, of what kind of answers you might get it. Yeah. No, I think that's a great question. That's the question. I'm talking to C three clients all day long, right? I think from our perspective, I really think there are two things. One is this is time to get your talent strategy ready, right? So I have a bold prediction. I don't know if, you know, Jamie agree with that or not, right? I think, you know, in the future, basically, you look at all your roles, you know, they fall into two camps, one of the two camps, right? You're either builder of A I, right? Or you are the consumer of A I focusing on consumption side, right? So again, it's time to think, to rethink your talent strategy and bridge the gap. And I think the second thing which is kind of obvious, right? Time to uh you know, really drive the this kind of the concept of a continuous learning, right? So we learn, you know, new skills and you know, the skill definition could be changing. But what we can do is get ourselves ready and, and so how do you do that. I know accenture takes this very seriously. And someone told me once you spend something like a billion dollars a year on training, uh, uh, exercises, how do you, uh, how do you prepare the consumers of the technology? What is the, what's the nature of training that you give people? Yeah. Very good question. This could be our long conversation, but I'm gonna give you the right. That's what I think, Ellen num, I think the number one important thing is to have the top down support, you need to have the, you know, top CEO level support that, you know, basically articulating the importance of this talent transformation because we have 700,000 employees, right? Probably a lot of them are gonna be, you know, consumer for A I. So we actually have a program called TQ, right? TQ stands for technology quotient. So I'm one of the faculty members for TQ, right? And of course, a lot of our latest episodes is about generative A I. So this is kind of a program that we ask our every single employee to take. But that's not possible without the strong support from the leadership team. I I think that's one of the key building blocks. I want to call up uh Jamie uh a somewhat separate issue. Uh If you look at the last 23 decades of technology development, it seems pretty clear to me that it is increased inequality within societies that technology has enabled some people to move much faster and further and accumulate uh wealth and income uh than others. I've seen some early studies. I know it's very early days, but I've seen some early studies that suggest that generative A I may be different in that the biggest productivity gains often seem to be with the least skilled experienced workers. Do you think that's true? Um It's, it's a really important question that you're asking. Um And it, and it's a really interesting like generative A I feels different to me as a technology that's distributing and that it's distributing at a global scale so fast. So it, it is, it is different to be bringing this technology to every country and every location at the same time. It brings up all sorts of interesting challenges about how do you do that? How do you manage all the different regulations and reactions that are happening? Um uh It's actually very interesting because one of the emergent properties of the language models to spend their ability to understand multiple different languages. So we should have co pilot, for example, in all of our tier one languages out the gate um in terms of the research that you were referencing. So it's been a very consistent finding across these um sorts of tasks that are um the ones we were talking about that are the, the efficiency kinds of tasks that people are doing them a lot faster. And a a lot more efficiently and then particularly the people who are less skilled at those tasks are so like, I, if I'm not a great coder, I can actually generate code. That's quite good where I'm an amazing coder, it'll provide me some efficiency and benefit, but it won't be as, as significant. Um I do think this innovation question comes into play though because it starts becoming like, how are we figuring out the new ways to use the model and, and that's gonna be a place where I think it's actually our responsibility to make sure we distribute the technology and figure that out and distribute our learnings in a way that builds equity and like benefits the whole world because that's gonna be what benefits all of us the most is how we do it, do it, how we do it matters. Exactly. Let, let me open it up. See if there are any, there's a, a question Olivier and then a question right over here. Uh Right there. Yes. And then uh right, right here. I apologize to people in the back. The lights are very bright and it's hard to see beyond the first couple of tables. But Olivia, go ahead. Olivia, neuroscientist and entrepreneur in 96 Gary Kasparov wins against A I in 97 he loses, creates freestyle chess where humans can use technology in order to compete. And in 2005 amateurs beat grandmasters because they had a better process. Don't you think that the one thing we need to teach people if possible is process plus, I mean, I think that, that to me connects with this notion of meta cognition and like, how are you thinking about using things? And, and that feels like the real opportunity and actually the, the things I get really excited about with the model when we sort of talking about this innovation frontier, um you know, our uncreative selves are sort of thinking about like, how do we get A I to obey us? And I think our creative selves are like, how do we get A I to challenge us and get to think about things in new ways. Um So that's what I get excited about right there, please. Yes. Um Parliament from University of Birmingham here in the UK. Um And I wanted to go back to the point of uh OK. Jobs are going to change in ways that are somewhat unpredictable and my job as an educator as well as a researcher. Yeah, you're gone. That was kind of my question. I didn't want to project my existential crisis in public. But as you can tell, we, you know, we have been disrupted as a higher education institution. I'm actually professor of computer science, particularly impact us. So my, my, my question is really, what is the role, uh how is the role of higher education changing? It's not just the pedagogical approach which is changing because of A I. But what are your expectation as basically downstream from higher education in regards of how we best, how can we best prepare the next generation to basically be ready to move forward? And you understand that Microsoft is already making the tool that's going to replace you. Yes, we have a separate chat and I and I, I wanna so as a computer scientist, this is, there's a metaphor that I use actually. And, and partly my role, Sata uh created my role about five years ago to help the company deal with disruption and think about how, how things are going to change. And I would say that we're all right now responsible for building something new. We're figuring out an entire and that's not gonna slow down, that disruption is coming. And I actually think the whole world needs to start leading like scientists. So I think your science and what you teach, you know, and that means like experiment and develop hypotheses and try things out. But that's not the only part of the scientific process that means learning and building on the state of the art, you know, reading broadly and like writing that related work section and figuring out what you're doing, it means disseminating knowledge. So so that others can build on it. But even more importantly for validation because that's the way that people will challenge you look at things differently, you know, and it means thinking about the externalities and sort of the long term effects. So I actually think that scientific framework is amazing, you're going to give him superpowers. Yeah. Yeah, I mean, you're bringing superpowers to the technology. Land final word. Um So it's definitely exciting time, right? Uh Ellen and Jamie, I think to me A I is a team sport A I is not just, you know, as what we've been talking about, it's not just being done by technical experts or, you know, uh uh limited to ac three conversation boardroom conversation. I think this is a time that everybody has a role to play, right? The democratization of A I and everybody needs to play their own role, right? So, um this is a time that nobody wants to get left behind and uh happy to work with all of you on this. Uh Jamie. Thank you so much. I think that was encouraging. I feel a little better than I did. Thanks so much. Thank you.

Presenter: Ami Badani, Chief Marketing Officer, Arm

Brainstorm AI London 2024: The Importance Of AI Chip Development

Brainstorm AI London 2024: The Importance Of Ai Chip Development

Hi there. I think we're all here because we believe that A I will have a profound impact, probably the most profound impact that will be in our lifetime. It'll be bigger than the internet, bigger than the smartphone revolution, bigger than decades of innovation prior like the invention of the light bulb, but it's only A I is only on its tipping point. In fact, you've seen use cases today like sports, financial services, entertainment and we're only getting started. If you think about how A I will become mainstream over the next decade, you think about chat G BT, it's at 200 million users in just over a year. That's the fastest growing application that we have ever seen. Secondly, you think about self driving cars, one in 10 cars by 2030 will be self driving cars and that will all be because of artificial intelligence. 3rd, 75% of office applications, office productivity applications will incorporate some form of generative A I in some way, shape or form. So as you can see A I will become mainstream, but we're on a collision course with reality and it's not sustainable. There are two things that we need to really advance artificial intelligence and that's compute and power. Let's talk about compute A I models are more hungry, more data hungry than they than they have ever been before. They're increasing in complexity. They're increasing in size and they're increasing in scope. And what that drives is the need for more compute chips. In fact, if you recall open A I had their latest, so a model that's their latest text to video model. It takes 100,000 A I chips working at full compute capacity and full power consumption in order to train. So a that's a huge amount of compute chips and that's just one large language model of any of many. Then you think about smaller language models that are going to grow in scale and be quantized and apply to things like smartphones, your thermostat in your home and other home appliances, your PC laptops, your inference in the cloud. And then you think about large language models, small language models and that's really driving the demand for compute chips, but it's not about driving the demand for compute chips. It's really about power efficient compute chips. And that's really gonna help us carry the advancements of artificial intelligence. Power efficiency is in arms DNA. In fact, 99% of smartphones use an arm power efficient chip. And it's the same design philosophy that we're applying to the advancements of artificial intelligence. In fact, if you think about a I it comes with a cost and that cost is unfortunately power. In fact, large language models have an insatiable demand for power. So without the need to address power, we won't have advancements in artificial intelligence. And if you think about the future of artificial intelligence, it's all going to be about power efficient chip designs. Now, if you look at this graph, what it really shows is data center, power consumption, most A I training and large language models are all done in the data center today. And that's 2% of the global electricity consumption, which is an enormous amount. But if I talk, you know, I spoke about earlier that A I will become mainstream. And when A I becomes mainstream, the biggest tax that we'll see is our power grid and we won't be able to continue the advancement of A I without addressing power. In fact, by 2030 artificial intelligence will have triple the load than what it looks like today. To put this into perspective, 25% of the US power grid will be because of artificial intelligence. So we really need to address this power problem head on. And if you look at chat G BT, it requires 15 times more energy than a traditional web search. So that's a huge amount of energy. And that's what arm is really focused on is driving power efficient compute chips. In fact, if you look at customers like Google, Microsoft, Amazon Oracle they are all designing power efficient chips because they know they have to solve the power problem first in order to solve and drive the advancements of artificial intelligence. So in order for companies to have a competitive advantage, they really need to think about power efficiency in solving the power problem. In order to advance artificial intelligence.

Edward J. “EJ” Achtner, Head, Office of Applied Artificial Intelligence, London, HSBC   Alexandra Mousavizadeh, Co-founder and CEO, Evident Insights  Brian Mullins, Chief Executive Officer, Mind Foundry  Moderator: Massimo Marioni, Europe News Editor, FORTUNE

Brainstorm AI London 2024: Strategies To Stay Competitive

Edward J. “EJ” Achtner, Office of Applied Artificial Intelligence, London, HSBC, Alexandra Mousavizadeh, Evident Insights,  Brian Mullins, Mind Foundry, Massimo Marioni, FORTUNE

OK. Hello everyone. Thanks for staying with us. Um So financial services and Gener A I Banks have a huge opportunity to significantly reduce the time that it takes to perform banking operations, uh financial analyst tasks and that hopefully then empowers the employees by increasing their productivity. Now, Alexandra, I know your company has done a lot of research in this space. Tell us who's leveraging generative A I. Well, and what can we learn from them? You mean who's leading, who's lagging? I, no, no, I mean, we do um the, the index that we create and I'll get back to what that is, looks at not only generative A I but sort of the whole A I spectrum. So yes, we um we produced, it's now 14 months ago, we released the first um index measuring the biggest banks in North America and Europe on their A I capabilities or their A I maturities. And um it was uh interesting to see who, you know, who leads and why. And uh essentially we, what we do is we do a full 360 of the entirety of the bank's A I footprint and we look at elements like um the A I talent stack, we look at innovation, we look at leadership, we look at operating model and then we also look at the transparency of responsible A I and um put this index together. Um And the leaders were, it was interesting to see that in the, in the top 10 or if you look at the top 20 they were, they were dominated by North American banks. There's some um uh some reasons I think that uh that could be, we can get into that in a minute. Um The top bank was uh JP Morgan, um was followed by capital one and then R BC and they're very different banks. But what they did have in common or what they do have in common is having been out the gate and very forceful on, on, on, you know, being very clear about what that A I vision is. And then all of that fo that followed was um pushing very hard to hire uh A I talent um setting up research labs, uh really thinking about uh their innovation strategy and structure, uh research and patents and partnerships and vendors and so on. And then looking at the operating model all with a view to sort of how can we get the time from ideation to production down. Um So that's what we created, that's what we established in, in January last year. And then we did the second iteration of the index in November and it's on an annual cadence. So be very interesting to see if anything shuffles between now and the next update, which is in October. And you mentioned JP Morgan got ahead almost by being early adopters and, and, and getting out the gate quickly, obviously, you know, we can't, we can learn from that going forward, but you can't be an early doctor anymore. What can financial services learn from what those top banks have done apart from getting to it early? What, what are they doing really well that can be replicated? Yeah. Yeah, it's, it's, it's a question around sort of how can one catch up now or um, I mean, one thing I would say is that the, the banks that are leading are really doubling down. So there is a bit of AAA gap that is growing between sort of the, the leaders and the index and those further behind because there is such an advantage, there is an advantage in being first mover because you've established a reputation to draw in A I talent. But that said there's certainly um any bank that decides to really, to really focus in on this now and sort of looking at what are the leading banks doing and, and taking what is best uh approach for that specific bank? Absolutely. That can be um, a bank can catch up in terms of being very clear about the vision, being very arti you know, articulated both internally and externally to make sure that A I is the absolutely, you know, most important thing for any leader of the line, any line of business and putting in place. What uh what is um important for talent acquisition and retention. So what is it that makes the bank attractive? A I talent has a lot of other places to go than banks. So um one needs to make the bank a really attractive place to work. So that is, you know, can you, can you show your research, is there uh access to be active and sharing on open source communities? Are there really interesting problems to work on? Is it a priority of the bank? Uh and so on. So there are many ways to make that, you know, skills and training ongoing in the bank and so on. So those are sort of one of the, you know, some of the many ways that banks can, can focus on it now and catch up now. We've got EJ here from HS BC. They're very aggressively making moves in the A I space. I think I've heard you say that HS BC has nearly 1000 applications of A I within the bank. Could you talk us through them all, please? No, not sure. Here we go. What are your favorite EJ? What are your favorites? So that is that is accurate. Um We do have uh approximately 1000 applications across HSB CS operations that use artificial intelligence. Um the oldest of which go back nearly a decade with some of our original machine learning models. Um And as you might imagine, for our 62 market operational footprint, um all businesses, all functions are, you know, heavily invested. Um And really making sure that we're driving the leading edge of responsible and ethical A I with respect to generative. Um I think it's important to say a few things. So as you might imagine, we're testing and learning, you know, a range of generative A I, you know, use cases that are likely to scale. Um And I think it's also important to distinguish between proof of concepts, pilots and production. Um And what I see out there is a lot of great energy, a lot of great momentum. But I think what also needs to be stated particularly in financial services. And I know there's a many different sectors represented here today, but when it comes to banking, financial services, perhaps health care, ok. We do have a differentiated standard of care. We do have a differentiated standard of, you know, regulatory compliance and those are good things, ok? We should embrace that, ok. Um And so for us, the focus is on that fine balance, you know, between bridging from proof of concept into production that's going to take time. And as you might imagine, there's some lower risk types of use cases related to knowledge management all the way up to and including other types of use cases that candidly, even if it were in our risk appetite, it's our impression that in some respects the technology, the tooling is not yet mature enough for production grade applications. Um So we're really, I think proceeding cautiously, but at the same time, we are optimistic about this across, you know, all businesses and functions. Now, Brian with, with anything fun and, and potentially profitable. There comes risk. Unfortunately. Um What specific risks come with deploying generative A I in, in finance like algorithmic trading or fraud detection? What can you tell us about risk and how we can manage that? Yeah, I, I think that uh all the risks come with it. I think it's really important to understand what the models can and can't do. Um And, and whether or not that they're fit for purpose. Um I is the decision that is the highest risk decision to make. Um We see generative models uh which in and of itself is a little bit of a misnomer because all A I, all all machine learning models kind of generate an answer. But, but in, in essence, I think what we're talking about is, is foundation models that are trained on large amounts of data and, and are, are predicting tokens and they are statistically encoding knowledge that exists in the training data. What does that mean? Um You'll get some generalisability um but they're not truly Generali A I. Um they will, they will go off track, they will invent answers. Um This isn't a bug, it's a feature, it's how they exist, they invent these, these answers. Hallucination is not an accident. Sometimes we just like the hall hallucinations. But if you understand that, then you can see that there are some very powerful things that you can do with it. It's just not going to solve all your problems. Um And, and I think that once, once you know that and understand that you can choose the applications where they'll do the best or you can combine them with other machine learning methods to create a powerful solution that gets the flexibility from a user interaction standpoint of something like a large language model put together with a more deterministic model for forecasting to actually plug in the numbers that you'll use as, as part of the ultimate solution. Um So, so I think we really need to be thinking about it not as a silver bullet but as another arrow in the solution quiver. Um And, and then then really understanding um that uh the, the other silver bullet that doesn't exist is the idea of a guard rail. Um And, and, and we hear this term thrown around a lot. Uh But, but it's, it's used almost as a, we're gonna Sprinkle some magic guard rail on top and it's going to suddenly make the thing do what it can't do or stop it from doing what we don't want it to do. And, and I don't think we have enough experience, you know, with a I as, as an industry to say that that guard rails are improving things. In fact, uh often is the case when you add complexity to a system, you make it brittle, uh more likely to break. And, and so I think we should be skeptical when we see something that is not performing. Uh that the answer is we could add some guardrails to it and make it perform. Um We need to be cautious and, and, and really ask the questions about suitability uh on uh at the, at the beginning of a project before we decide whether we'll use something like a, a large language model. And do you think it's fair to say that one of the main factors involved with, with mitigating that risk is the people and the people that are operating these systems now with that? And ejr, I've, I've seen you talk about this before, you know, how can, how can financial institutions ensure that their, their workforce is adequately skilled in order to deal with A I? Because I've had a lot of people talk about a lot of very senior people in A I talk about, you know, it's not a I that's going to take your job. It's someone who can use A I who's going to take your job. So how can uh financial institutions as, as I say, ensure that their workforce is, is on top of their game when it comes to A I and that can mitigate risks on behalf of consumers. Sure. So I don't think we talk about this topic enough. I'm glad we're discussing it now. You know, you can come at this from the direction of the highest and best, you know, kind of productivity output. You can come at this from the direction of responsible and ethical use. You can come at this from the perspective of the law and you could also come at this from the perspective of what's the highest and best way for an individual, a teammate to achieve their personal or professional best, all roads lead to the same thing. And that's making sure you have an engaged, educated and formed workforce that's using these products and capabilities in a way that best meet all of those criteria. And the beauty of that is, you know, is that if you do it the right way, you build great products with product market fit, you have an engaged and informed workforce that differentiates you from a talent perspective, you're able to demonstrate to your key regulators that you're doing this and you have delighted customers. So no matter again what your starting point is, it's imperative that whatever sector you're in, you know, that you really have very thoughtful detailed plans on really employee reskilling. Now, I'll go to the audience to take some questions if there are any I just want to get some thoughts from the panel on what emerging gen a trends can business leaders monitor over the next decade, say and, and what steps are necessary to prepare them for these types of developments that we're going to see to start with. Brian, who looks the most worried. Yeah. No, II, I think that uh you know, I would, I would echo exactly what he said about um the, the idea of humans working with A I and, and really supporting them to learn how to use the tools. Um That's where you're going to get the most clarity. And I'm really excited whenever the user interface kind of gets broken down so that people without A I skills can start to use them and use them in their work. Because every time somebody in an organization uses A I, you get an answer quickly or it synthesize a whole lot of information quickly, but it kind of makes the information, you know, from an information theory standpoint, we'd say it increased the entropy. But really, it's just, it's making the picture a little bit blurry and the person looks at it figures out what's in it and then when they make a decision, they make it more sharp, right? Because they've combined a whole bunch of things from, from the rest of their life and the rest of the role that they do. And, and that helps correct for the weaknesses of these generative models and the more you can put those together, I think the better and, and I'm really excited when I see these U or user experience being developed in products that let people actively participate, not, not in a box ticking human in a loop exercise, but where I'm adding my creativity, then they're adding the creativity and more. And I think that's where we're going to see um you know, real valuable generative A I being deployed in the world. And Alexandra your, your top trend to monitor. So I just, yeah, I just wanted to agree with that. I mean, we, we, we monitor A I talent flows um in a, in a lot of detail in the and because the hypothesis when generative A I was, you know, when Chat G BT four was released was that there was going to be a huge um demand for prompt engineers, right? And so, but that actually didn't happen because banks were looking internally to do exactly what you're saying, you know, put, put it in the hands of everyone and, and figure out how we can, how one can use it for what and solve what problems. So there wasn't that there's been, you know, there's been other types of hires that we've seen instead and um looking at, you know, talent coming from academia, more looking at talent actually coming from big tech that have been part of, you know, behind the, the the growth of and the development of the tools so they know that but, but also not trying to build that in house, but trying to create the, the know how and the talent in inside the bank and upskilling and, and actually putting it in the hands of everyone to some have had that uh strategy, put it in the hands of everyone and then sort of see what surfaces from that and, and look at what, what um what problems, what ideas can come from that and ensuring that there is this, I know a lot of people talk about innovation mindset, but that's actually more important than ever because if you've got someone again and sort of inside the line of business or running it, that has a I to the front of mind all the time as a, as a solution, then then you're going to come up with many more ideas for use cases and use cases that are in the pipeline that then can be tested in terms of selection, which is looking at complexity and Roy that's attached to each of the use cases. So on uh on, on talent, right? So that's uh that's actually went in a slightly different direction than we anticipated, but upskilling and, and some that's, you know, what the banks are doing internally and across, you know, not just senior leadership, but across uh the whole talent stack is something we monitor quite closely interesting. Um I'll throw open to the floor. If there's any questions for the group, gentlemen in the middle. Oh, sorry. Hi. Um Ajay from uh sales force. Uh quick question and this is from a financial institution perspective. Banks and financial institutions have gotten really good with managing structured data and data is so critical to have the right A I output at the end of the day. Uh Can you give us a insight of what uh financial institutions or what you individually are trying to do when managing unstructured data? Because so critical to have a great outcome of any large language model? Sure, I'll take that very briefly. So, you know, again, another topic that's not discussed enough if you expect to get high quality output, even with a well trained, you know, workforce, you know, your house of data, that foundation must absolutely be as strong as it can be structured, unstructured where it is under what type of use, geographic, etcetera, full stop, especially in banking, financial services, health care, et cetera. So again, that strong foundation for your, your data is an absolute prerequisite, especially if you need to get demonstrable, repeatable, high quality outputs that you can stand in front of a customer or regulator to ultimately prove that you have product market fit. OK, guys, I think we'll leave it there. Uh EJ, Alexandra Brian, thank you so very much.

 Victor Riparbelli, Co-founder and CEO, Synthesia Mati Staniszewski, Co-founder and CEO, ElevenLabs Moderator: Jeremy Kahn, FORTUNE

Brainstorm AI London 2024: The Future Of Synthetic Media

Hi, welcome back. I hope you had a good break. Um I'm very excited to be joined with uh some of the, the CEO S of two of the hottest sort of start ups in the gene of A I space at the moment. Um And as you heard from Ellie, uh both working in synthetic media, which is maybe using tech to generate something, but maybe also using, you know, existing video, still images or voice to generate media and content to show you how this works. First, we've got two very exciting demos. First, we're going to go to Victor who's going to show you a little bit about how synthesia product works. Victor. Go ahead. Thank you so much. Yeah, I think we should give a visual in just a second here, but maybe before we jump into that. Uh So these are on a mission to make video easy for everyone and that's, that's something a lot of people have attempted before us. But we take kind of a new approach to this, right? We don't think of this as building smaller and more affordable cameras. We don't think of this as like slightly better editing apps that run in your phone. We're just building technology that eventually is going to be able to replace the entire physical production process of using studios, cameras, actors, microphones and all those stuff. We want to take that entire workflow and make it into something you can do entirely from behind your desk. Um Today, we're a fast platform and we help predominantly enterprise create more videos to communicate better with their stakeholders, which could be employees, could be customers. And I think we'll get a visual up here now of how it actually works and how easy it actually is to do. So this is the platform and in some ways it's sort of model a little bit on powerpoint. That's how easy we want to be able to, to do. And what we're showcasing here is our A I video assistant which uses LL MS to essentially help you get to a draft of your video in, in just a few moments. So what you're seeing here is someone putting in uh the URL to an article, of course, a fortune article because we're here today giving the system a little bit more context of what it is that we're trying to do with this video. And in just a second, the system will then go in, it will pass the content and it will give us a draft on editable copy that we can then kind of tune to what exactly we want it to be like now in the context of an enterprise, this may not be uh an article like this, this would probably be a knowledge based article. It could be a case study that you want to share with your customers or anything else really, you want to turn into a video as you can see here. Now, um we begin to actually write the script and it is of course using all the data that it's kind of taking from this URL. The idea here is not that this is a final video that you can just publish, but it is to take you 70 80% of the way. Instead of having what we internally call the blank screen of death, we give you something to actually begin kind of working from and we don't just write the script, we actually also give you the visuals. So as you'll see here, there's some text on the screen, there'll be some bullet points, there'll be a bunch of things that you can sort of edit yourself. Of course, everything can be edited. We have these avatars which is the sort of core of our product. Um And I think after this video is done, we of course see what the video looks like, but it's just to give you a quick visual of how easy it actually is to make videos with these systems. And it's one of the things that excites us the most about this new wave of generative tools and it's awesome if we can make it easier for Hollywood people and video production professionals to make video. But I think what's much, much more interesting is taking everyone in the world, including everyone here and making you into video creators and in some ways it sounds a bit science fiction in other ways. I think it's just natural evolution if you go back 4050 years in time and it wasn't part of most people's job to write texts, right? You had like secretaries, you had people in companies who would write things on typewriters and send them to the right people. Then we all got computers and keyboards and now I'm pretty sure everyone in this room you write as like part of your your daily tasks, right? Then powerpoint came along. We all became designers without really knowing probably all of you here know how to operate a powerpoint. And obviously, we think that the next iteration of this is going to be video, which is just a much, much better way of compressing information than text. At least for most people out there. I think with that, let's see the demo of what the final video looks like. This uses one of our latest technology, our latest model called Express One, which essentially teaches the avatars how they should perform and behave. But let's see the video. Imagine stepping into a room where the air crackled with the energy of innovation, a place where the future of A I and its impact on our world is not just discussed but shaped. Welcome to Fortune brainstorm A I in London. A gathering that promises to be the nexus of A I's brightest minds from leading technology companies at Fortune brainstorm A I in London. Expect to dive deep into conversations that matter. Hear from Google, deep minds, vice president and faculty A I CEO, as they unveil the future of A I and its transformative potential on society. Microsoft's chief scientist alongside accenture's chief A I officer will explore the profound changes. Generative A I is set to bring to the workplace. This event is not just about listening but engaging with roundtable sessions and ample opportunities for networking. I hope you enjoy the event. Great, very impressive given how quickly it all generates. Um Yeah, absolutely. Thank you. And now we'll hear from Maty and about 11 labs which is uh you may know about voice cloning. Let's see how this works. Thank you, Jeremy. I will attempt doing it live and maybe that's a very quick introduction. 11 labs is a Audi research and deployment company with the goal of making content university accessible across voices and languages. And what I will show you now is free demos, quick demos of how some of those technologies come to life. We should get a visual any second now to the screen. And the first of those building blocks is a one that most of you might be familiar with, with foundational text to speech. And that's one of the breakthroughs that we've, we've, we've figured out of how you can take existing text. And by understanding the nuance of the context, turn it into emotions and into the right intonation with a specific voice. So queuing up, I will use 11 apps to help me with the introduction to all of you and we'll see how that comes across. Ladies and gentlemen, welcome to the Fortune A I Brainstorm summit. We are thrilled to have you join us from around the globe for this dynamic gathering of minds and ideas today. Hm, we stand on the brink of new discoveries and innovations in the field of artificial intelligence that promise to redefine what's possible in business society and beyond, you could hear the pauses, you could hear the information, you could hear the excitement and also a little bit of ponder with the hm uh as, as as the A I was speaking and this is one of the voices from one of voice actors we work with. And actually there is plenty of those voice actors as part of our platform. We now have over thousands of voices which created clones of their voices and then shared it while every time it's being generated, they earn compensation and return as well. And whether you want this voice, a voice with an Australian accent with a different style or gender, this is all possible within the platform. But this is not again. And one of the things that's super exciting about the technology that it not only allows you to create the content in the language of the speaker, but actually take the content and turn it into other languages while preserving the same voice and the same characteristics. And I'll swap over to a demo that some of you might be familiar. It's a John F Kennedy speech from moon landing. And we'll see how initially he speaks in English and then I will flip it to Spanish and to Hindi life and you will see how some of those characteristics come across while we play the demo. But why some say the most, why choose this as our goal? And they may well ask why Kala Montana Masala Porque A Sua Atlantico hum for J for John. That's one small step for man. I didn't say I read it and you can see the emotions come across while the moon and the liftoff happened. And that's something we're excited about how you can take that content and enable those global audiences to watch it while still enjoying that original experience. But of course, in the audio world, there is so much more than speech and the voices. And as you think about the future about our work is how we can enable some of that work across some of the tangential domains, the sound effects, the music, how we can bring it all together with the video and truly make it an immersive experience and to close it off. I also show you something that you might be familiar from text to video work with Sora from Open Ayes as well where we'll take some of the stitches from the videos and we'll supplement it with A I generated sound effects. What we are trying to do is hit that unmute button with those videos that you've probably seen online out there. The real in a place beyond imagination where the horizon kisses the heavens. One man dares to journey where few have ventured armed with nothing but his wit and an unyielding spirit. He seeks the answers to mysteries that lie beyond the stars. And that's the goal to make all the all the content out there accessible across voices, across languages, across sounds. And uh and thank you. Well, I think you all agree. Those are pretty impressive demos. Um And you know, amazing technology also a little bit scary. Um And I think looking at this, a lot of people immediately think about uh some of the negative use cases around deep fakes, around fraud. Um I know there's already been some concern around 11 labs. Uh You know, whether people have used voice clones to uh perpetuate frauds. Uh how are you guys trying to, to prevent that from happening? Um And to make sure these technologies are not used uh for harmful uses. Uh Victor, maybe you go first actually. Yeah, sure. I think, I think any most new technologies, we immediately go to all things, things go wrong. Right. And I think in this case, that's definitely right. These technologies will be used by bad actors to do bad things. For sure. I think we should not be staring about that at all. And so for us, the the sort of safeguarding the technology has always been, has been been part of the company from day one. So we found the company in what we call our, our ethical framework, which is the free c consent, control and collaboration. Um It's a long topic, but I think that the kind of fundamental keystones for us is around consent. So never ever recreate anyone's voice or video avatar without the how do you, how do you guarantee consent? So we have a KYC style check when you go through, right? You submit your avatar footage. It is reviewed by a human being. You have to stay out of specific uh specific sentences to make sure you can make your clone. So essentially it is impossible today to go in and take some youtube videos or something and make a clone or someone that is just like not possible. Um And that we intend to keep it that way controls about content moderation. So we employ pretty heavy content moderation. There are quite strong stance on what you're allowed to create what you're not allowed to create. I think for most of us, the sort of what we call the red content is easy to agree on hate speech, violence, swearing, things like that. What gets harder is, well, someone's making a video about cryptocurrencies, for example. Right. Are you talking about what a great technological invention that is? Or you're trying to lead me into some schemes that promise I'm going to get rich in 10 days. Right. And we do our best to catch that stuff. It's a huge effort. It's of course a lot of computers and A I, but it's also a lot of humans sort of figuring out who are the bad faith actors on the platform. So that's a big, that is essentially internal product for us that said it is an incredibly small amount of users we catch trying to do this, but we do think it is our responsibility to, to take some stance on that. And the last one is Maddie. I want, I want to get you in there. How, how are you 11 labs are you trying to prevent this from misuse? Yeah, I think first of all, Jeremy, you're right, the defects are a serious concern, as we're saying about this year, air generated content and audio images videos coming coming across. Um And it's something that's a concern because open source other commercial models are becoming more active and, and there will be bad actors that will use and abuse them to the to the to the cases at 11 labs, it's education making everyone aware of the technology. Second is traceability. All the content that's generated by 11 apps can be traced back to the specific user specific account. And as they use the technology, they will need to verify themselves through different steps depending on the technology they use. And the last one is traceability, but also how you can embed the traceability as part of detection. So effectively all the content out there should be known as they are generated and there should be tools that allow you to quickly get that information as a viewer, then we release our tool publicly that allows everybody to use it and find whether it was 11 apps or not and reported back to us. So right now that that exists, if I run across some content and audio clip on the internet, on X or on linkedin or whatever I can check and see if it was 11 labs generated. Yes, you can, we can you can directly go on our website, upload that clip and and get probability of how likely it is. And now we are working with some partners to extend it across other technologies. So some of the open source model, other commercial models, that's part one and then part two because as you said, you can run into that in social media, but you might not be aware that it's A I generate in the first place. So how we can collaborate with social media, with telecommunication companies to check it on the fly while that content is being sent to you. I want to ask a question about the ethics of some of this. There's an interesting use case on the JFK speech. There have already been some cases of politicians in, in various countries wanting to appeal a certain audience in a language that they don't speak and they have translated their speeches in real time. I think maybe even using some of this technology to, to appear to speak those languages and they've said, oh, that's legitimate, I'm just trying to reach a different audience, you know, with my content, with my political content. But other people have said no, that's a complete red flag. You don't actually speak that language, you're presenting yourself with something you're not. What do you have a policy around this? And also at synthesia, do you have a policy around the creation of political content? Yeah, so we don't allow political content today. Um I think they'll change over time. Uh But I think we, we take a generally quite permissive stance on what you can use these technologies for. We also may be the enterprise. Um I do think it's a very interesting philosophical question. I think in five years time, something like that, no one will be talking about this anymore and everyone will be doing it. But I think as with any new technology, you will have this sort of years, right. Where people are trying to figure out what's right, what's wrong. Um, you could say, is it ok for a politician to have a ghostwriter to write a piece in fortune? Right. They didn't actually write it themselves. Is that worse? The same? Is it ok. Um, so I think, you know, all new technologies come with these sort of interesting questions and dilemmas and some of them will ring out to be very important and very, very true. And I think some of them will, will, will care less about in the future. And, and that idea of a policy definitely echo some of the vector pieces as well and where uh it's going to happen at one stage. And now as a society, we need to come together and figure out what's the right way of approaching that. Currently, we don't allow people around to impersonate any of that work or pretend that they are part of the candidates. Uh uh uh And the p and do the political speech or political content. Great. I want to take one question from the audience because I, I think I only have time for one. But if, if there's a question, please raise your hand and I'll try to come to you right here. I see as a gentleman here at this table, if you raise your hand again. Oh, great. If you could stand up and I yourself, uh it's Mark Salmon from the times, um uh when um Sadiq Khan had a deep fake of him, uh made the Met investigated and they concluded that they, they couldn't take any action. Um Do you think it's possible to write a law uh in this country to combat against that kind of deep fake? Um And do you think that's desirable, should there be laws against deep fakes? Obviously, I'm not a lawmaker. So I can't give you a definitive answer that I can give you my personal opinion. I think ultimately deep fakes is an advanced version of impersonation and impersonation is in most cases illegal if you do it for deceptive reasons, like fraud, for example, I definitely think we should include deep fakes of that, maybe even make the punishment harder if you're doing, using a deep fake for malicious impersonation. But where to draw the lines legally, I think it is very difficult to say that's not my, my my and to add uh one quick note to that as you, as you rightly ask, this content is out there, there's plethora of tools that can generate that content and what's the true solution? There's definitely the legislature that, that, that uh the countries can pass. But then also on the technical level, what we can do and beyond trace of and being able to detect it as you think here is out how this could work as part of the society. And one thing we are advocating for is beyond water marking. What's A I approved A I maybe there's a version of water marking what's real content and for Sadiq Khan or any other candidate to be able to transmit a message and that message is decoded. This is really Sadiq Khan that's speaking. So we don't only detect for what's fake but actually verify what's true. Excellent. Uh We're out of time. Thank you very much for the demos and thank you very much for the conversation. Uh Put your hands together, please for Maddie and.

Matthew Candy, Global Managing Partner, Generative AI, IBM Consulting Chris Clements, Digital Products Lead, Wimbledon Maggie Murphy, Chief Executive Officer, Lewes FC Moderator: Ellie Austin, FORTUNE 

Brainstorm AI London 2024: AI’s Impact On The Sports Industry

Matthew Candy, Global Managing Partner, Generative AI, IBM Consulting Chris Clements, Digital Products Lead, Wimbledon Maggie Murphy, Chief Executive Officer, Lewes FC, Ellie Austin, FORTUNE 

Matt, Chris and Maggie. Thank you so much for being here. Now, Matt and Chris, I think we should just explain how you two work together. So IBM is the digital partner of Wimbledon, which is why we have the two of you on stage together. Just so there's no surprises. Um Chris, I want to start with you first. So A I is being used in various ways at Wimbledon. Could you maybe highlight a couple of the cases that you think have been most transformative to date? Yeah, absolutely. So, yeah, like you said, there's a, there's a number of ways that we've been utilizing A I. Um I think the first one is around distilling data and being able to deliver engaging fan experiences at the back of that. Now, when I talk about that, it all starts with the fan, understanding their needs, understanding the pain points and understanding what we can serve them to get them more engaged with Wimbledon and get them more engaged with the tennis. So, what we've done over the years um is we've done a number of different initiatives starting with our Power Index. So what this was was a dynamic index, which enabled it enabled users to be able to see how players are progressing in terms of their form as we moved on through the tournament. So it's something a lot more dynamic than existing rankings. So users could understand exactly who they should be looking out for in terms of the big matches that are coming out that then fed through to our match insights, which are a percentage likelihood to win for each player on a match that's coming up. So, you know, for a fan, they can see where the big matches are, who's likely to play, especially for those fans that are less engaged, they don't follow the tour year round and where are they consuming this? So that's on our digital platform as well as on social media and then this year or in 2023 we took that a step further with our draw analysis. So what that was was an analysis of how difficult or a player's draw was from the very start. So that enabled a user to be able to select a player and see what their path through to the final looked like. So they could see where the big matchups were coming, where they were going to have a slightly more challenging match up. And then they could see that in order to be able to fully understand how the draw was likely to play out. And I think we've got some video footage which I'd love to run showing how this interface works. And you can also see it down here, maybe Chris and Matt, you could talk us through what we're seeing here. This is a representation of the tennis and all of the data points which we're utilizing in order to come up with some of those stats. So tennis is an extremely data rich sport, but it's not always the easiest for your average fan to understand. So what we're doing is utilizing A I to have to take all of those data points and distill that into something which is easy for anyone to understand so that they can fully engage with the event. And Matt who is the demographic that you're targeting here because it doesn't seem like if I think about, you know, my parents who love Wimbledon, I can't imagine they're going to be tuning into the app to, to engage with all this data. So I think it's aimed at all demographics. So, you know, only a small number of people get to experience physically, the Wimbledon, the amazing Wimbledon environment each year. So this is all about extending the reach the richness of the brand, um you know, and just the beauty of the game and sport. And so that second screen experience that people can access through the ipad app, IOS Android through the web platforms. AX is an amazing companion effectively to the live broadcast. And so, you know, we end up with all types of demographics accessing that in order to drive a deeper level of engagement with the game and with the sport, Maggie, I'm going to come to you in one second, bearing all of this in mind as we integrate a I further into the viewing experience does what it mean to be a sports fan change. So, um does what it mean. I don't think it does. I think it allows us to actually make the most of people's time. And I think if we all think about us in the room, everybody has chosen to spend time here today and time is probably the most valuable commodity that we've got. And so if we're able to create experiences that allow people to personalize the use of that time and be able to access the information that they want, you know, and certainly being able to create, you know, personalized content, et cetera. You know, I think the technology provides an amazing way to make the most of people's time. Maggie. Let's bring you in. So in addition to being CEO of le football club last year, you were part of a really interesting initiative. You were part of a team that created an A I campaign for a hypothetical female FIFA presidential candidate to illustrate the fact that there are very few women at the top levels of football. And before we ask you about it, I want to show a clip of the campaign and our candidate is called Hope Sogni. Ok. So Maggie, first of all, hope looks incredibly realistic. And secondly, why did you decide to get involved with this project? Yeah. Well, I mean, first of all, I don't know anything about A I, so I'm probably in the wrong room. Um I do know a fair amount of uh about people and unfortunately, I wish that hope didn't have to exist. Um but she exists because of human made challenges, which is the fact that it's very, very rare for females, for female candidates to get elected in football governance. You might find there are plenty of very very good women that are working in, in football. But when it comes to elections, which is generally how the women's football, where how football is governed, they don't always get elected. It's almost like their, their whole woman. This is a little bit too radical. And so it was we came together and decided to create a candidate and fed this person with the views, the insight, the research, the data from a whole bunch of women around the world who are working in football to create this person. Um Hopefully who someone like Gianni Infantino could actually ask questions to, to maybe get a woman's perception on a particular challenge and the challenges are real. It's a case of the fact that take the balance or ceremony last year. You know, one of the biggest awards ceremonies in the world, you hardly have any women that are in the decision making roles. And so it was put during a women's international calendar. So all the top stars were actually playing football that night. Um And, and then you also get really specific issues around a number of rape allegations where female players are actually going through a system. But the people that are perpetuating the rape, for example, their own presidents of the football associations are the people that they are required to report the allegations to. And so hope was kind of trying to create a system where we can feed in all of these, these ideas and have a candidate that can actually speak to and represent women that if they spoke up in some circumstances, they could actually face retribution. Did hope have any impact on changing the game. Yeah. Well, there was certainly a lot of buzz and I think that's what we wanted to do. We wanted to start a conversation. I don't really want hope to be elected. I'd rather there be a woman out there that is elected and that is the point. It was just to show that there are these situations around the world where women are not getting there is that kind of glass ceiling that's still in place. So we have plenty of reactions, plenty of discussions, plenty of conversations and even some conference conferences thought about actually profiling her and platform her and giving her the stage. So that's exciting. Um We missed a trek not having hope today, but we're thrilled to have you. Um, Matt. So in addition to working with Wimbledon IBM works at Seville football club with the golf masters at the US Open. Um I have a question about how you decide what use cases to pursue. So I saw a clip on Twitter or X of what was A I Enabled commentary at the golf and some people were loving it and some people were saying, if this is what it sounds like, I'm pressing mute, how do you make sure that viewers want the new technologies that you're introducing and that it's not just innovation for innovations sake. Yeah. So I mean, we've just obviously masters concluded yesterday, been partner with them, I think for over 25 years now. And so this year we had spoken voice commentary for every shot for every hole for every player um in both English and Spanish. And so the key objective there. And also similarly, if you look at what we're doing with Wimbledon and the US open is to basically be more inclusive. And so if we can provide effectively the ability for, you know, people in their natural language to be able to access deep and rich commentary around the game that they're watching, um You know, we've done similarly with Wimbledon around the highlight clips and the highlight reels that we generated last year and layering a natural language, spoken voice commentary ever found full of emotion and excited about the game. Yeah. No. So, so I think there's a lot of work that we've done taking open source large language models and the small model and training it specifically and not just on language because this isn't a language translation issue. This is around picking up the nuances of the game and expressing effectively what people are seeing in a way in which it's going to build emotion. And certainly we started with English and Spanish as two of the big constituents of viewing groups. But we absolutely see this is something that will open up to other languages beyond. So I think, I think it's important to say that actually what, what you're not aiming to do here is replace the human commentators. Actually, what we're looking to do is find ways to supplement them. So, so, so for instance, the highlights clips, the highlights clips of Wimbledon last year, we had generative A I commentary that was overlaid onto them. What that was offering was context as to the clip that you were watching. So it's not about trying to bring out that human emotion. It's not John mcenroe screaming at how exciting the point is what it is is just providing that extra little bit of context which in you to understand what you're seeing and why it's important. So, so it's, it's about finding the right use case for it. It's not necessarily about replacing or trying to or trying to come up and shoehorn these to places where they're not necessarily needed. It's about finding where the gaps are in the fan experience. And then we can look at the technology and whether or not the technology can work to alleviate some of those, some of those challenges that fans are having. Sorry, I was going to say this point of augmentation is really important because, you know, and I think if I look at the video generated highlight clips that we produce, that's not to replace content producers, but it's effectively to allow the A I to take a decision, looking at crowd noise and player, facial gestures and fist bumps to auto curate. What we believe is the most exciting point of the game. And then allowing a content editor to push that highlight reel and reduce the cycle time for getting stuff out there basically and allow more content and a quicker cycle time. And Chris Wimbledon is a very unique example because it's a tournament that's so steeped in heritage. And so the balance between preserving that culture and innovation is a particularly unique one. I know that the A TP has announced that it's going to be using electronic line judges across its tour as of next year. I think Wimbledon obviously has in person line judges and I wonder has using A I to make judgment calls on court been discussed for the future of Wimbledon. So this year we are, we are planning on keeping human line judges, obviously with any new and emerging technology, we'll be looking at how that progresses. We work very closely with the other tennis bodies and we'll be learning from their experience, but our decision will be down to what we think provides the best experience both for players and for fans so it could be in the future. No call back. Ok, Maggie. Um, aside from hope, how else can A I be used to bring more diversity to the game? Oh, it's funny, I was thinking about um how A I could be used in, in general and I run a fan owned club. You know, we have 2000, 402,500 owners in 41 different countries. And I think, you know, we're all very aware as a non A I specialist about targeting and segmenting your audiences. But it does make me think how about if we were able to segment to the point that rather than getting a monthly or, or a weekly newsletter, you're actually getting a video from me. So I was telling that person exactly what they want to hear because we're able to understand exactly what they want so that they're actually getting a video in their inbox where I'm saying, hey, hey, Dave, how are you doing? You saw the results last week and it is, it actually could personalize the experience, the real challenge in all of this is football is about people. It's about that human connection and I would worry that you would lose some of the value of me actually writing an email to somebody if they are receiving a monthly video from me that gives them what they know they want to hear. But it's not actually me saying it. So I think that's the thing that we have to, to to toe the line on a little bit. Um Matt, I'm gonna ask you the next question, then we're gonna open it up to the floor. So get ready if you have anything to ask. Um I'm interested in IB M's partnership with CV FC. Um And specifically, I know that similarly to what we had doctor be talking about the use of A I to enhance scouting a challenge there is privacy and the sharing of personal biometric data that belongs to players. What kind of guard rails or regulations do you have in mind when you're thinking about that particular use? Yeah. So I guess for all of the work that we do in this space, so we have a platform that we've built in IBM called What's Next, which is our generative A I technology. As part of that, there's a governance platform and toolkit. And so where we put these models into use that governance toolkit allows you the ability to monitor, manage for biased drift, explainability, observable and live use. Do you think about all of the regulation that's coming in around the world and making sure that you're compliant with all of that. Um And so certainly, if I look at the work we're doing with Sevilla football club, um So this is a product that we built for them called Scout Advisor. It's powered by our A I platform. Um And basically taking a lot of the quantum quo data that they've got around kind of potential players, about 200,000 player reports that they have and basically producing a very simple digital product to allow their 20 plus scouts that they've got to be able to access that information, to get simple summaries of that information and the data that's there to help them take better decisions. So again, it's not to replace them, but it's basically to help them sift through the huge volumes of data that they've got. I mean, these models are amazing at summarizing, being able to kind of chat with information and being able to classify information. And so when you think about generative A I, there's many uses of it in a non generative way. And so, you know, using using this this type of technology to basically help get these insights and gems from the data there to allow them to find talent. Does anyone have any questions for the panel? Yes, we've got one right here at the front. Please tell us your name and your company. Yes. Hello, Jamie TV, from Microsoft. Uh few years ago I went to the League of Legends World Championship and the sport my Children watch most is Fortnite. Um, it seems like there's a lot of opportunity for a, I, particularly in the e-sports domain. I was wondering if you could say anything about that. Well, yeah. Uh, so, uh, actually funnily enough, uh, Wimbledon entered Fortnite, um, last year, um, with, uh, race to Wimbledon, which was a mini game, uh, which we launched on, on Fortnite last year. Um I think, um, there's definitely uh a lot of potential in that es sports domain. It's very interesting because what we're trying to uh to, to match up is, uh as Edie mentioned before our heritage and, you know, the fact that Wimbledon is a brand that people know and they associate with certain things with the fact that we're trying to, to move forward and we're trying to deliver more engaging experiences through a variety of different ways and means. And at the center of all of that is the human element of it is the fact that, you know, the, the remarkable feats of sports people are remarkable because they are human and you can't quite believe it at times, but they are in the e sports Domain. Potentially there is a little bit more leeway to be able to be a bit more expansive in some of that. Um And I think it's maybe something that sits a little bit more, more, more naturally there. Um I just want to finish with a quick fire question for each of you. Um So we're obviously in a room of people who largely work in the business world. What lessons from the application of A I in sport in your experience? Would you like everyone here today to take to their respective sectors? Maggie? Let's start with you. I mean, number one for us when we were designing, hope you had to have proper human inputs that even though the brilliant creative agency, dark courses were the ones that put it together. They needed me and they needed my data, my database of incredible women to be able to feed it in. And if we are not being diverse in the way that we create A I, we're going to lose a lot of opportunities but also sideline a lot of people as well. Yeah, I think it's about being led by your users and your customers not being led by the technology and also knowing the limitations of the technology. So in our instance, for instance, we work specifically with what the next, which is the IBM platform because it can be specifically trained in our domain. So we can guarantee the quality of the outputs. It's going to talk like you would expect Wimbledon to talk for instance, so definitely be led by our users and know the limitations of the technology. And then I think um look II I guess a lot of the conversation around A I and generative A I at the moment is on productivity, right? And people looking at how this technology can be used to drive productivity and processes within the enterprise. So hr and procurement supply chain customer service and it absolutely can and it's absolutely gonna have a profound impact in, in those areas of business. But I think what the sports world demonstrates, um you know, in, in in spades is the use of this technology to drive new types of products and services and a deeper engagement with your customers and fans. And so I think there's a lot that business, every industry can learn about the way in which the sports industry drives really deep personalized engagement through using A I data and analytics. Matt Chris Maggie. Thank you so much for joining us today.

Presenter: Dr. Ryan Beal, Co-founder and CEO, Sentient Sports 

Brainstorm AI London 2024: How AI Is Enhancing The Scouting  Process And The Fan Experience

I'm Ryan Bill. I'm CEO and co founder of sent in sports where we help professional sports teams and organizations um utilize cutting edge A I technologies. So initially, we focused on scouting in football and helping scouts in 2020 who couldn't go to games find a new way to assess players. So at the time, a lot of the analytics in the space was focused on backwards looking data. So it's how players are performing at their current clubs and with their current teammates. However, in football, when a team, when a player moves all of this changes, so the player moves into a new team um into his new teammates. So you can see an example here for Harry Kane, um for his Bayern Munich move in the summer. Um He moved into the new team, had new teammates and then we could simulate a number of different things. So the first of these was team chemistry and this effectively looked at how well he would work with his new teammates based on things like his languages. He speaks how good he is as a player. And if he's played with any of these players before we then had tactical fit, which is a prediction of how well he'll fit in with the manager's tactics and whether or not he will suit the formation and style of the manager and then finally return of investment, which was a prediction of the suppliers like to sell for a profit in the future. So what this gave to decision makers was forward looking risk assurance effectively to help them make multimillion pound decisions. And the key thing we learned while working with some of the top clubs around Europe on this was around explainability and for a decision maker to be able to trust these models, they really had to understand what underpinned them and what drove those decisions. So you can see we were able to unpick some of these and provide different ways of doing it. So what we found with generative A I over the last few years as it's come to the mainstream is that it's able to solve different problems of sports. And one of these in particular is how they can start to generate new revenue from their fan bases. So at the moment, sports revenues are generated very much from physical sources, things like their ticketing, their merchandizing as well as TV, rights and broadcasting. Whereas in the future, we expect this to shift and it's be very much focused on digital experiences and the new ways that they can interact with their global fan bases. So we have built a system to be able to test this, the system is called scout G BT. Um And we'll show an example of this now and it's able to interact with all of our back end analytics. It is able to access all of the things we just showed and all of the data that we've got on football. So in the example we see here, um we've got an Arsenal fan who's asking questions of whether or not they think the arsenal will win against Bayern Munich. This week, the system is then able to go out and access all of the data we've got and simulate whether or not arsenal will win in that game. The fan can then ask what position should ask and look to strengthen in the summer and the system is then able to access all of the data we've got on their squad depth. So it will provide this back to the fan as well as providing a assessment and a chart of their squad. So it's able to do all of this in real time with real time data as well. Then finally, we can ask, you are the top five left back to ask Shahine to solve these, these weaknesses that we identified. And then through this, we can provide that back to the fan um as you can see here. So this is what's known as a Rag LM, which is a retrieval augmented generation system. And this allows us to provide an interface to all of that back end data that we've got, which allows the fan to access a domain specific and real time LLM, which provides a difference to things like chat GPT and anthropic exist at the moment, which ultimately are all trained on the same things like the open web. Whereas this can give it access to much more domain specific areas. So the more that a fan uses this system, the more that we can start to learn about what the fan likes, what their preferences are and ultimately, that can lead to ultra personalized content generation. So in the example here, we've learned that the fan supports Arsenal and we've learned that they are interested in transfers. So using that information, we can serve them with this article here, which is a simulation of Alfonso Davis potentially joining Arsenal in the summer. And then this is displayed onto the screen to the user. Now, if the user likes this article, they're able to ask questions to a virtual journalist so that we can then provide articles on the fly to their own preferences. So you can see here, they've asked for the same article, but for mbappe move to Real Madrid and then that's been produced and served back to the user. This is also supplemented with things like charts and images which are all displayed on the flight and eventually be multi modal with things like podcasts where if we know a user is at a train station and has half an hour to burn, we can serve them with an automated personalized podcast. This also leads into multi languages. So if we know that a user is in Thailand and they speak different languages to what we might expect where the clothes was based, then we're able to serve them with content that is unique to their interests, like the club and their players as well as unique to their consumption preferences in terms of length. And then also in their local language, which can allow sports teams to provide a much deeper connection to their global digital fan bases and ultimately generate more revenue. Thank you for listening and I'm happy to answer any questions later today.

Paula Goldman, Chief Ethical and Humane Use Officer, Salesforce Moderator: Nick Lichtenberg, Executive News Editor, FORTUNE

Brainstorm AI London 2024: Building The AI We Want

Hey, everyone. Thanks for joining us today. We are wanting questions from the crowd. So if anything we say resonates with you, if you've got a question, please raise your hand. Uh say your name and your title. We would love to have your questions for Paula. I'm super excited to talk to Paula today because she's sales forces first ever. Here's the title Chief Ethical and Humane Use Officer. And you've been doing that for five years, but you've been in ethical Technology for a decade. So you've been thinking about these things that the the wider world is now very aware of for about 12 months now. Um So describe to us your journey with Ethical Tech. Um Well, I should say I'm a deep believer in the ability of technology to open up incredible things for people in society. And I spent a lot of time working on tech for good. Um And about a decade ago at the time, I was working for Pierre, a Miar, the founder of ebay, we kind of looked up and said, oh, wow, tech is no longer the underdog tech needs guardrails. Um And, and it's been an amazing set of uh set of uh developments. But as you were just saying, it's also really gratifying to be in a moment where the field of tech ethics is in the news all the time. And that it's, it's actually, you know, one of the top questions we get from our customers when we're talking about A I is around, can we trust it? So it becomes a set of innovations and features on the product, right? And sales force has this concept of the human at the helm. And that to me that conjures up uh someone on a ship, the human at the helm of the ship, I think our art is very uh apt today. We've got this human, but they're driving all these gears. Um Does that define your philosophy at sales force? Yeah. So I mean, if you think about, so think about where we are in generative A I, right? So you just mentioned people have been captivated for about a year, a year and a half. Um And at phase one, a lot of people were talking about human in the loop. Actually, I feel that that term comes from the military, right? It was the notion of like having a person check before a really consequential decision was made by a machine. Um I think that's no longer good enough, right? And I think what is really important right now is we're moving from A I just generating content, I say just because it was blowing our minds, you know, to taking action on our behalf, the notion of co pilots, right? Um We need next level controls. Um We need people to be able to understand what's going on across the A I system. And most importantly, we need to be designing A I products that are taken into account what A I is good at and bad at, but also what people are good at and bad at in their own decision making and judgment. Yes. So you said co pilot there, I think that's an important word because uh we hear a lot about autopilot um and various discussions around A I but you insist it's a co pilot situation where we've got the human who's got the A I co pilot next to them. Right. That's right. Right. II I flew here. Uh I didn't go into a plane that was flying on its own. Right? And I think the idea of A I is that it's gonna augment human ability, it's gonna augment human judgment, it's gonna augment our ability to do our job better and more productively and to do that, we need to make sure that we've got the two strands working side by side. Right. So what have you found about what people are good at and what A I is good at which, which role should, which copilot have? Well, let me give you a few examples that I think are really incredible as we think about human at the home. Right. I think we do a lot of work. Uh, A I for service workers. Right. A I for service. Um, we, uh, one of our earliest generative A I products, um, we used with a major luxury consumer brand and their service workers. Right. And, uh, what they found was that, um, because the, the service agent didn't have to spend as much time looking up the answer to a routine question. They actually were able to do what really only people could do, which is connect with their customers. Right? And they found that all of the sudden service agents were 20% more doing product conversions. They were sales people now, right? Or think about the doctor that uses A I for note taking and is able to actually pay attention to what their patient is saying and what they're not saying and ask better questions and get to a better judgment or diagnosis at the end of that call. Really A I is already transforming jobs, people with their A I co pilots, uh people are becoming sales people overnight or doctors are becoming a new type of doctor. Yeah, it's changing. Well, it's definitely changing the nature of jobs and it's freeing up people to do things that are going to become higher value for the companies they're working in. Do you have any questions from the room so far? Uh One over here in the back there? And Tom Whitaker from law firm Burgess Salmon. Really interested to hear about A I ethics. How do you ensure that those who talk about, um A I ethics not only talk the talk but walk the walk. How do you ensure compliance? That's a very interesting question. So, I, I, what I would say is, you know, it's really important actually, I think a ethics is becoming a skill in all of our jobs, But the way that we approach it as sales force is actually like building it into the product itself, right? So you and I were talking about this a little earlier. The question of um how do you build friction into the product that people actually know how to use things? Mindful, mindful friction is a term that we use, right? So I'll give you an example, we have a marketing segmentation product. So it helps you, you want to send an email campaign, you want to generate an audience for that email a segment. So sometimes using demographics for that purpose is fine. Sometimes actually, it's not fine. It introduces bias. You might be overlooking audiences that you might, you know, might be better to target people who buy dresses recently than just women of a certain age group, right? And so, um the question then is how do you design a product? So that those types of questions which are matters of judgment are front and center. Well, we did a small thing in that product interface, where if the A I generates a segment that has demographics, those demographics are unchecked and you simply have to check them to bring them in. We also have another example, a model builder product um that enables people to build prediction of models or bring in their own generative models into our Einstein one platform. And there again, if you're building a model and you say, let's say you bring in zip code or postal code into your model that can be correlated with race. Again, sometimes that is OK. Sometimes that introduces unwanted bias, we have a little warning toggle that pops up, right? So there are many layers to the question compliance, you know, starting with regulation and a lot of other topics. But the piece that I'm actually most excited about is how you design products that it's actually, you know, what to trust and where you should take a second look and apply human judgments the A I outcome, right? Have you been surprised while designing mindful friction there in terms of things you think you can trust versus things, you know, you can trust. I, I think that the part that's been really interesting is like how do you design um controls that allow the human to look at the totality of what's going on in A I to the point about human and loop versus human at the home. Um One of the things that we built into our N 21 products was an audit trail. Um, so that you can say let's, you, you had a whole email campaign, you generated a million emails. Um, but there were the use 50 over here that got edited consistently before they were sent on to the customer. Maybe you have a problem with the knowledge article that you use. So that's like a signal to you. Exactly. 50 emails out of a million. Yeah, I think increasingly increasingly, you know, that's so we're at stage one, here's an audit trail. Increasingly, I think we're heading towards systems that can detect anomalies like that and encourage and prompt the human to take a second look at it. Fascinating. Do we have another question from the room over here? Thank you. Uh Bartik Organski from lever at lever, we coach human skills. Absolutely love what you're doing. He used the term human intelligence, which is, which is amazing and of course, much more important than in many aspects artificial intelligence for humans. How do you define or measure human intelligence? How long do you have? Obviously this is uh yeah, this has been debated the ages. I mean, I think there's a, there's a fierce ongoing debate about what is the nature of artificial intelligence, what is the nature of human intelligence. But I think the way that we measure it that measured, at least it kind of in the context of what sales force is building is their business outcomes that our products are um intended to achieve. And again, to be able to free up that sort of the service agent working to develop a deeper human connection or for example, we were working with an insurance company, a health insurance company in the UK. And again by freeing up service workers to deal with more routine, to not have to deal with the more routine insurance cases, but then pay more attention to their vulnerable populations and engage with them by phone. It's that type of interaction, those types of human connections that I think are deeply, deeply important. And on a part of the part of the landscape of human A I interaction and human intelligence that I think I would argue it cannot be automated, interesting. So human intelligence is about person to person interaction. Would you say, I mean, I would say that that is one factor of one factor, but the human intelligence is much broader than that. Of course, yes, I want to go back to your academic background. You have a phd from Harvard and this was just uncanny to me when I met you and how unorthodox ideas go mainstream. And you've been working in tech ethics for a decade now and now it's the most mainstream topic that there is. Uh so isn't that uncanny to you? Can you believe that that's what you studied? And here you are at Fortune brainstorm A I in London talking about this and it, it's the top issue in the world right now. I love hearing you say that. Um No, I think, you know, and when I started getting interested in tech ethics is because I saw that there was an, an unfilled need and that this needed to become, you know, super important. Um And so it is very gratifying that it has arisen in importance. And I hope that um I hope that the attention that's being paid to these issues of trust continues and is not kind of a momentary thing because you know, we talk about we, I'm sure all of you have watched these different waves of A IA I summers and A I winters. Um I think there's no doubt right now about the capabilities of A I, right, that you know, that it is the the the capabilities that are proceeding very, very quickly developing very quickly. My concern would be that it's possible that the next A I winter is caused by trust issues with A I or people adoption issues with A I, right. And so it is, it is very important to me that these types of trust innovations in products and the types of uh the types of training and on the job work and the sort of people adoption within companies. I think that's what's gonna continue to unlock A I productivity and gains for companies. Yes, we have just a moment for one final thought from you. One prediction. Of what you see next? You were just saying trust, do you think will be at the center of it? I also think since meeting you that you're kind of a psychic that you, your doctorate, you predicted your future career, you predicted the tech ethics would be the big question. So what is your clairvoyant view of what's, what's coming next? I think. So, two things. It is the sort of the company is paying attention to the people side of A I, I think we're gonna see a lot more attention not only to the interface and the sort of the, how, how you design A I for trust. Um But also how do you bring people along on the journey? How do you allow that? How employees to experiment? So that's one big thing. The second big thing I think is data. We just have not talked enough about data. And I think this year we will start to data governance, data governance as part of the regulatory conversation, which has been largely missing. Um And I think that's also like, I think that that's the second topic. Data. OK. All right. Well, we, we're gonna leave it there, Paula. Thank you so much for joining us. It was a pleasure. Thank you, everyone.

Josh Berger CBE, Founder and Chairman, Battersea Entertainment Lynda Rooke, Actor; President, Equity Alexandra Shannon, Head of Strategic Development, Creative Artists Agency Moderator: Ellie Austin, Deputy Editorial Director, Fortune Live Media, FORTUNE; Co-chair, Fortune Brainstorm AI London  

Brainstorm AI London 2024:  Responding To AI’s Disruption Of Entertainment And The Arts

Josh Berger CBE, Founder and Chairman, Battersea Entertainment Lynda Rooke, Actor; President, Equity Alexandra Shannon, Head of Strategic Development, Creative Artists Agency Moderator: Ellie Austin, Deputy Editorial Director, Fortune Live Media, FORTUNE

Josh Berger CBE, Founder and Chairman, Battersea Entertainment Lynda Rooke, Actor; President, Equity Alexandra Shannon, Head of Strategic Development, Creative Artists Agency, Ellie Austin, FORTUNE

Hello again, everyone. And hello, Josh Linda and Alex. Thank you so much for being here. Alex actually has just flown in this morning from New York. So it's hot marks to her for being here, Linda. I want to start with you now. So equity represents a whole range of artists. We were talking backstage. You said in addition to actors, musicians, wrestlers, models, radio stars, um which A I use case do you think will be most disruptive to the members of equity? Probably screen uh and to some extent life as well live theater because of streaming of shows, et cetera that suddenly come in since the pandemic or during the pandemic. Uh but particularly a film and television as well as voice as well. Yeah, voice as well, the replication of, of voice. And I know that you're campaigning quite hard at the moment to raise, aware of these issues because there are no uh formal regulations in place. What would you, what guard rails would you like to see around A I in the arts industry? Well, there, there's a number of issues. It's, it's to do with transparency for a start of knowing where where your image is, what the rights are, where it's being used, et cetera, consent as well and control over that image, you know that you have the right to withdraw that consent, et cetera. And the biggie remuneration. Basically, if you're going to use our image and our voice, et cetera, we want to be paid and we want to know about it and we want to be paid properly. We're going to come back on to what properly means in a second. Josh. Now you've worked in the media TV, film theater industry for decades and you were at Warner Brothers where one of the things you did was you ran the global Harry Potter franchise, which is something very close to my heart. And I'm sure many millennials in the audience as well. Um I want to focus on the films for a second. If that first Harry Potter film was coming out in the A I age, how would it production be different and how would the end product look different? It's a good question. II, I mean, the first film, even as compared to the last films, uh there was a kind of revolution in the visual effects industry in this country as well. And Potter kind of brought on what is now the world leading the FX uh industry based based here. And uh and so in some ways, it was a step in that direction because they were computer generated images in the in the latter films and in the earlier films, not nearly as much, um you know, the the castle was, for example, a real, it was up at an E for example. Uh and then you'll see that there was a much more digital uh imagery in, in the later films as, as technology advanced. Well, in the A I era, it would be possible then to do more of that and it might look even more real or uh and it would be created, you know, with a AAA verbal prompt in 60 seconds instead of many, many people working to create it. Now again, that presupposes that we've worked out a lot of those uh a lot of these issues about remuneration for people who would normally be doing that work. Humans are doing that work as opposed to a machine. And then of course, the machine that would create the visuals would be building them based on scraping all of the visuals that are already that have already been created by somebody. So again, the remuneration for the work and the attribution of that work because the transparency is really important. And then again, I don't, I don't suspect it's, I don't suspect anybody knows where every single image in every piece of text was ever created because that's what's out there and that's what's being scraped and that's what's creating, you know, the foundation of the A I which then creates visuals and text and scripts and everything else. Now, Alex at CAA, you've come up with a pretty interesting and innovative way to manage the thorny issue of digital doubles. And you've created something called the CAA Vault, which allows your clients, your actors to create their own digital doubles and then have control over how, when they used and be paid to them. Can you talk us a bit through how that came about? Sure. So it's no surprise to anyone in this room that a few years ago, we noticed that on the one hand, there was this rampant misuse of our client's name, image likeness, voice, body of work without their consent. And on the other hand, we were noticing interesting opportunities in companies who wanted to work with our clients and their digital rights, but didn't have a way to do that in an ethical talent based mechanism. And so we decided to launch what we call the CIA Vault. It is on the one hand, we are proactively creating our talents, digital likeness, we are scanning their image, we're scanning their voice, we're scanning their likeness. Um And we are then storing that on their behalf. Um The purpose of this is on the one hand, we know that the law is going to take time to catch up. And so this is a mechanism for our clients to actually create and own and have permissions around their digital identities. And we think over time that will actually help the law catch up faster because now there is a legitimate way to work with one of our clients. There is a legitimate way to compensate them and to give them credit for their work. Um And on the opportunity side, um for those who are interested and not all are, but for those who are interested, um this provides a way for us to help set a precedent for anyone who wants to work with one of our clients and their digital identity. There is a mechanism to have them be compensated. There is a mechanism to work with them in the right way and unlock opportunities. Have you had to convince your clients to take advantage of this is some people skeptical what's been the take up? Absolutely. Uh It runs the gamut. On the one hand, you have clients who are very skeptical of this space and on the other side, you have those who are quite interested and thinking about how to take advantage of this space. And the clients that we're scanning are actors and actresses. But it is also musicians and athletes and digital influencers and politicians. It, it runs the gamut because we think that by starting and starting to set the precedent that we think this will ultimately benefit everybody. Um We all know sort of the conversations we were talking as we were walking in about some of the conversations over the last year or so. Um We can't just stop at the top in any field. It has to be a place where we can set the precedent that everyone benefits from one more question to you. And then Linda and Josh, I'm gonna bring you in on this subject. You talked about remuneration. How is it calculated? If one of these digital doubles is used by a studio, what are the metrics for deciding what the client should be paid for that use? It's a great question and the market is, is forming to be honest, uh it is happening in real time. But the idea is that if you're to work with somebody's digital double, you aren't working with that because you think you can work for that person in a more cheap way that it's creating some big cost efficiency for you at the end of the day, if you are working with somebody, the the value of that person to a brand, for example, um if it's a brand endorsement deal, the value is in that person representing your brand, whether it is them fully in person and performing, whether it is using some of these A I tools powered by a digital double to do post production fixes or to help enhance the value um to think about localization of content, for example. OK. So it's more a scheduling or convenience tool rather than a cost cutting measure. Linda Josh, what do you make of this, Linda? Would you be encouraging your members to get involved I think if you, you have status or profile in the industry, there's probably a very good reason to sort all that side of things out, you know, and I'm sure it's going on, but I represent 50,000 members, not of all whom you will know who they are, et cetera. Some just starting out in the industry and they won't have the power, et cetera to be able to, to work through those problems as well. You know, uh all the financial wherewithal to do that. Um So those are the issues around making sure that everybody's protected added to which dry, say the other people who are not in the room to be able to defend themselves are those who are no longer with us where somebody's image and voice, et cetera, who is, who is dead, who is then being brought back to life without the consent, without their consent so that it's states are trying to manage that. Well, yes. Uh But there are also the other thing, of course, when a lot of programs were made in the seventies, eighties, nineties, what was going on? There was no A I so within what we have is called collective agreements, there was nothing to control the use of those images and rights because back then, as you know, we were dealing with C DS and cassettes and everybody's cell phone or mobile phone was the size of a brick. So have changed now. Uh and unfortunately relying on the agreements that were in place and the contracts that were signed, something has to be updated. So, are you speaking to the government about this? Yes, we are. Yes, there's a, there's a number of fronts on this one. There's the intellectual property side of things and there's also GDPR as well. Uh There's a number of routes plus, of course, if they could adopt one B, option one B of the Beijing treaty, that would be useful because so much of our work in English speaking countries, we work a lot with the USA. What would that entail? Well, basically, we, we work with our collective agreements uh because we obviously all the time, um the USA and Canada film over here, et cetera and our actors work over there and there's a kind of multiplicity around that and it would just ease the problems around uh property rights, et cetera. OK. Um Josh, you were saying backstage that you were in L A around the time that there was the S A afra strikes last year um which those strikes were about the streaming economy. They were also about then lack of regulation around A I and on the whole, it seems that humans won as it were, there's now guardrails around what can happen with A I in a writer's room and also when it comes to talents doubles, did, how do you feel about the outcome of the strikes? And maybe what could we learn from them here in the UK. First of all, I think it's too early to know what the outcome is. I, I think the headline sounded good, um, for the actors and for the writers and it was good that the studios finally capitulated and, and did something because for too long they were immovable on the subject and it cost a lot of people, you know, a lot of, a lot of work and a lot of money that they desperately needed a lot more than the studios who were putting them in that position. And, you know, honestly, they could have, they could have gotten to that deal months faster in, in my view. But um but aside from that, I still think it's, it's too early because if you calculate also the cost of the strike uh in lost business for everybody. I mean, you know, uh Warner Brothers Discovery uh restated their numbers based on the strike impact and it, the, the, the number were, the numbers were horrendous. And again, you know, if they had worked that into their negotiating position, again, David Zaslav would have pushed a lot faster. I think I, I would have thought so. And um but uh so I think it's too early to do the cost benefit and it's, it's sort of a step in the direction of not resolution but just some motive vivendi while we see what happens with A I and how A I is is going to be used. Uh So po more positive than negative because there was a deal. Um I mean, the, the real negative case was just that kind of stand off for all that time and there was an acknowledgment among, among the studios that for example, if you write something with A I, you can't, you know, a I can't be the credit who write if anything, it's, it's gotta be a human being. Um I mean, there's just interesting data most recently just with CEO si mean half of CEO S now are using A I and, and passing off that work is their own, which you probably saw just in the last couple of days. So, I mean, the, the fact is this tool is going to be used by many, many, many people um including in our, in our industry and we need to figure out how to, again how to, how to pay people. There's numerous court cases at the moment between big tech companies and the New York Times or the Authors Guild um over the fact that original source material that's used to train the A I is never credited. So I guess it's also an IP law question as well. Um I'm gonna open it up to the audience now to see if anyone has a question for our panel. If you do. Um please raise your hand. Um We will come to you. If not, I will continue with my questions for now. Ok. I, oh no, we do have one question and please state your name and the company that you're from. Thank you. Hello, thank you. My name is Jack Dane. I'm the podcast producer for global government forum. I just want to sort of pitch three scenarios and ask the panel how plausible they think they will be looking forward into the future with the ability to take the likenesses of celebrities deceased. Though it comes with ethical problems or still alive and create kind of 3d models of them. How plausible is it that we'll see, say the Beatles headlining in Glastonbury in 2035 2nd scenario, um, could actresses and actresses based on the sort of work that you're doing with them at the moment potentially work forever and if that's the case, well, I guess like Children really would never have to work again. But the point being, could that actually be, is that plausible if the value that they put on their work isn't that they do it in person, but that they simply bring their likeness and their, their data to it? And I guess the third would be, do you think it's plausible that public figures may want to take advantage of that too? Might we live one day in a world of immortal presidents and prime ministers? Ok. Why don't we? That's a scary thought. Sounds horrific. Um, let's have each of you take one of them. So Alex, maybe if you start with the Beatles headlining Glastonbury question. Yeah. So they're, they're fascinating questions and I think these technologies are, are only going to continue leading to this explosion of creativity. I think we're only scratching the surface for what's possible in the long run. Um Certainly thinking through how the technology is applied and thinking through the human centric element is, is paramount. Um But I think that world where, you know, you think about the legacy of somebody, you think about their ability to continue making impact years and years and years to come. I do think it is an interesting one. I equate it very much to when, you know, digital cameras were coming out. It's my favorite analogy with this space because that didn't get rid of the traditional art form of painting and the traditional kind of in person artwork. It just created this whole new incremental set of work and it sort of expanded the pie in many ways and grew the pie. And so I very much think that, you know, is there a scenario such as the, the Beatles example? I'm not sure we're seeing versions of that with even here in the UK, with Abba Voyage. Um which I think is a fascinating one where if you think about the dynamics of a group of people that otherwise may not be comfortable traveling, may not all be able to be in the same place, they're still able to reach fans and engage with fans in the right way, but they were in control of that. And it's, it's a model that puts them front and center. I think those sorts of experiences and ways to continue reaching fans for generations to come, I do think is, is a powerful opportunity. And then let Linda briefly on the question of active actresses working forever. Yeah, I mean, I, I understand where you're coming from with that kind of thing. But the truth is it's, this is an industry that demands change and innovation on a constant basis, there's always a fresh new face, whatever being looked out for. So whilst there is a potential, I guess for somebody to be playing, you know, the Harry Potter Kid forever and a day, the point being is that in the end that you the audiences will demand different faces changes to those things, we we'll move on, you know, Laurence Olivier is no longer with us. We've enjoyed looking at his films, et cetera, but we want fresh faces, we want different, we want other younger people coming through. Uh there could be the chance of, you know, you you never age, you know, the Peter Pan effect. That would be an interesting one. But, but overall, II I think it's in our collective agreements, write it in there, pay as well and send it on to the families et cetera. But in the end, you the audience will tire of that repetition. Trust me. I want to end on a more upbeat note. Has anyone here got an example in mind of a piece of art that's been created using a I that they think is spectacular and wouldn't otherwise have come to be. Yes, Josh, I thought you were gonna ask me that question about the politicians. I mean, can I just, can I just answer it very quickly if the questioner asked that question of Donald Trump, he'd say yes, I wanna live and be the president forever. That was the first thought. And that's like the biggest nightmare in history. Um So said as an American, um uh what was the other question? The way A A I is the need for good? And Damien Hirst did a, an NFT something called the currency, which was really cool and his machine created, but he did another series. I forget what it's called. The spun paintings or the spun colors. I mean, they're really, really beautiful and they're, I, and you can, you can kind of build your own actually and sort of tell the machine what you want and you can see and, you know, he puts his name on it as, you know, as he does and he's good at that. But um they, they really are stunning actually. And that, those, that's a ir and there's a marketplace now for that, a rair in the NFT space. Uh Josh Alex LT, that's all we've got time for. It's been a joy speaking to you. Thank you so much for joining us. Thank you. Go over here, I've got the.

Balbir Bakhshi, Chief Risk Officer, London Stock Exchange Group Ian Hogarth, Chair, Phasecraft, AI Safety Institute; Partner, Plural Connor Leahy, Founder and CEO, Conjecture; Co-founder, EleutherAI Moderator: May Habib, Co-founder and  CEO, Writer; Co-chair, Fortune Brainstorm AI London  

Brainstorm AI London 2024: Striking The Right Balance In AI Regulation

Balbir Bakhshi, London Stock Exchange Group, Ian Hogarth, Phasecraft, AI Safety Institute; Partner, Plural, Connor Leahy, Conjecture; EleutherAI, May Habib, Fortune Brainstorm AI London  

Really excited to get into the conversation. Uh National approaches may differ is an understatement. Thank you, Jeremy. I'm excited to get into it with everybody. Ian, I'd like to start with you. You have been writing about not just the opportunity for A I to transform society but the risks as well for two decades. And you now chair the A I Safety Institute in the UK, $100 million investment pound investment by the government to make sure that the ecosystem for A I in the UK is the right balance of grabbing the opportunity as well as mitigating the risk. So walk us through what you think is kind of state of the art. You guys have uh published the A I progress report, safety progress report. Uh not too long ago. What is the U K's pro innovation stance on managing A I safety? And, and how do you compare it to what the EU is doing and, and maybe what's happening in the US as well? I think um I think in the UK, I mean, the Prime Minister is, I think he would sort of describe himself as a, as a techno optimist. You're very excited about the potential of technology and start ups to transform the economy. And so I think the UK sort of tried to charter a course which really is sort of fundamentally built around empiricism. So um he created this uh A I safety institute um with the intention of basically creating uh research capacity inside government. So we've recruited some great researchers from uh from industry, from academia to come into government. So people like um Yarn Gal who's a professor at Oxford, um Jeffrey Irving, who was previously in charge of um alignment and safety at deep mind, Google, deep mind, um Jade Leung, who was previously at open A I and, and these people have come into government and we've built a research team and the research team is essentially looking at a variety of different risks at the frontier and trying to assess kind of how meaningful they are and then feeding that up to people making decisions about regulation, people making decisions about policy. So the idea is to sort of put, put risk on an empirical basis so that the government can make appropriate decisions about how and when it regulates. Um And so the, the the approach really is building state capacity, building, the capacity to understand what's going on. Um I think in, in Europe and in the US, there's been maybe a bit more of a kind of earlier move towards regulation with the EU A I Act and the Biden executive order, but the UK hasn't yet regulated. Um, we've just built this kind of empirical capacity. Um And in terms of the, you know, the, the innovation risk trade off. I mean, I think, I think that, um, maybe a good place to frame it would be around uh the ability of A I systems to write software. So I think it's something that potentially is very, very exciting because there's so many things that we could build if we, we we had more capable kind of software tooling. So, you know, I remember when I first kind of encountered more powerful programming languages, it was like, it was kind of like having a superpower, right, the ability to do, you know, right, code faster and, and, and and ship features faster. And so I think that for example, A I that makes coding more productive is, is potentially really, really positive in that we can generate loads of new applications and value for society. Um Obviously, there's always a risk with software which is, you know, um cyber crime, cyber attacks. And so, you know, one of the work streams that we have at the Safety Institute is just understanding how much these kind of frontier models would give a sort of malicious actor uplift in trying to perpetrate some kind of cyber attack. And we work with uh you know, we work in tandem with organizations like GCHQ and N CS C National Cybersecurity Center on that. So that's a good example of an area where there's lots and lots of opportunity and lots of opportunities for value creation for society. But then there's also this risk to mitigate. Yeah, thank you, Ian Connor. You have really, I think um kicked off the initial open source work with GP TJ and we build models at writer and I remember looking at that project um with, with a keen interest to see how it develops. Um You've also expressed grave concerns. I believe the exact wording was God like a I could be super. Um So there's definitely um a AAA large plurality of um A I researchers who, who agree with you, you've since started lecture um which is a, a effort to build models with bounded. Can you walk everybody through what you think that means? Um Do you still believe you guys are two years in? And it's, this is a, a for profit company. Um Tell us about what you have learned um on the frontier side to Ian's point around the potential to build in benevolence to humanity inside of these, these uh non deterministic model. So fundamentally, the way I got into a I, originally when I was a teenager was I was thinking about how can I do the most good for the world? How can I solve the most problems eventually? I figured out, you know, I'm just one guy with one brain. But if I figure out intelligence. Well, then I can make a lot of brains and they could work on all the problems. I could solve all the diseases and all the problems. Great. So I'll go do that. How hard can it be. Um, and so turns out relatively hard, but maybe not as hard as you might think. Um, so when I eventually though I figured out that I was thinking about that, the hard problem of A I of A G I is not how to build it. It's how to control it. It's much easier to build something that is intelligent, that does something than something that does what you want. Because think of yourself like what is what we want? That's a really hard question. Even someone we love very dearly, we could sometimes hurt them or we can misunderstand what they want even if we're trying our very, very best. Now, imagine a system built by for-profit companies to maximize profits. But presumably, uh what do you think that thing is going to do? Who knows if we build systems that are smarter than humans, which is the intent stated goal of all the major players that's better at business, science and technology, politics and everything else. And we don't know how to control it, which we don't currently know how to do. Please think for a moment, what do you think happens? Seriously? What happens? And so this is the question I, I've always been working on. Well, then what so well, even having the luxury of it, then what is a luxury? So conjecture, we work on this problem on the tech, what I call the technical problem or even hypothetically how can you make an artificial intelligence system do what you want? What does that even mean? So we focus on a sub problem of this which we call bounded where the question of how can you know what a system won't do? So this isn't even asking what it can do or guaranteeing that it will do the right thing. It's just can you guarantee that it will never do specific things? This is currently extremely difficult as I'm sure anyone here who has tried to use a GP T model in production can tell you. So we work on um this is a paradigm that we personally call cognitive emulation where we try to work on very, very precisely emulating the exact cognitive processes that humans use using LMS trained very, very, precisely on very, very precise tasks. So we know what these systems can and cannot do. This is not a full solution to the problem. It is a an important step forward to how can we get systems that in your business and your applications to bring you real value, do what you want them to do, do it in ways that is legible and comprehensible to humans and they don't go off the rails not enough for a good future in totality. Of course, since you know, as we are on a panel here about regulation, the problem of A I in general of full job automation, which is something that we will see in the near future is a question that is not just technical, it is also a social and a political problem. It is a humanist problem. And you know, techno optimism is nice. I love technology. I have worked with technology my entire life, I've worked with open source. Um I've been deeply involved in movements. I love technology, but uh techno optimism is not a replacement for humanism. If we just build better and better technology, it does not mean we may build a better future that would make a better place for people. We will make a better place for capital. Let me tell you those stock prices are going to go up. I guarantee you that. Now that's the prediction I'm happy to make. But will it be a better world for us to live in? I think that is a very difficult question. Yeah, I appreciate that. Um that perspective, the the gap between the the haves and have nots in a I enablement. Um When we think about like people at work who can be as productive as the next 20 or 30 colleagues and, and folks who, you know, haven't learned those skis. It's definitely not a recipe for, for social cohesion. And you think about um the government's role in regulating the social ills and harms of social media. And it is not a pretty picture of, you know, their role to be able to really bring the best of this next um platform shift in that um uh Harry World Bill Beer, uh as chair of the Responsible A I Institute of the group at um London Stock Exchange. Um you are really in the middle of trying to figure out how to incorporate A I internally in uh in, in, in your business while keeping, you know, one eye out for what the government is doing and is likely to um to to regulate right in the UK, nothing is law yet around um where and where not to invest and then also where the models and the technology is going. So um take us through, you know, a day in the life of beer trying to really help hopefully like push the envelope internally on the innovation side while balancing um all of the the risks and potential harms. Yeah. Thank you. I mean, se is an exchange, a clearing house, but most of our business is data and analytics. So, um when we started looking at the control side of things at the beginning of last year, um we established a set of principles aligned to our values responsible A I principles. Um and then we set up a steering committee uh to really govern what, what was going on because there was a huge, huge amount of hype in Q one last year and we wanted to get a handle on. OK, what's going on in a 25,000 person organization? How can we control it? So we started, so I I established that as an oversight group, not be because, because I'm the risk officer because this is a truly multidisciplinary activity and we see it fundamental to business transformation. So we had a, an overall um steering committee that really collected use cases, evaluated them, looked at feasibility commercially as well, but also looked at the risk. Um and then a product level working group, an efficiency, working group and a responsible A I working group. And that's really the construct that we've used ever since. And we keep the principles updated as new regulations come out. So they're in line with the EU A I Act as N brought out standards, we've aligned them to that. So all of these are very helpful, but we largely have taken a self regulation route. And in terms of managing the risk, I think our approach, there has very much been upgrading, constant upgrading of our capabilities, so particularly data management, security, but also model risk management and thinking about how, how A I systems can regulate other A I systems. And we've recently tested a couple of our LM models. So we're working very closely with Microsoft in developing A I tools that really expose our data and analytics and make clients lives easier. And then we're also working on tools to make internal efficiency improvements like a Q A tool to improve customer inquiries. And as we looked at our model validation approach and designed that in conjunction with some professors, we used LM as a judge as part of that as well as human oversight. And that showed interesting results particularly for out of domain questions that we might throw at a model. So that actually didn't surpass our expectations, but actually was very helpful in feeding back into the model development life cycle. So very much evolving uh the risk management approach at the same time. But, but the self regulation approach I think is really helpful, keeping it updated as new regulations sort of and principles arise. I love that 11 framework I've really found helpful in, in talking to execs about um A I safety is the concept of guardrails. And I think what I'm hearing here is there is a level of guard rails that you know, the the the tech vendors right are going to be competing to build into their systems. There is a level of guard rails that responsible executives like beer are going to be building into their own responsible A I and governance frameworks at a company level. And then there are A I guard rails that the government should be uh insisting on. And so with that framing in mind, um could we do rapid fire maybe starting with with Connor, you know, what are the, the, the top one or two guard rails, right? Uh If you do the, the vendor level, Connor, uh well, go then the bal bear government level and then, um our company level and then, uh Ian with, uh with government level, top one or two guard rails that folks should, should be building into products. Oh, God. Um uh I mean, for your own business, this highly recommend that you do not trust any outputs of LMS in general. I don't think there's any clear technical solutions there other than, you know, filters and the like, but even though it don't really cut it, if you're asking about the general societal level questions, these are the kinds of questions that I don't think companies should even have to answer. It's not, this might be controversial but it's not, shouldn't be the responsibility of oil companies to solve climate change. It should be the responsibility of government to stop oil companies from calling in climate change or at least make them pay to clean it up after they cause the mess. So really guardrails should be something that is coming down from the governmental level in terms of societal level risks. As for risks that apply to like a given business application or given thing. I can just say like, uh you know, careful what your consultants tell you A I is super unreliable at the moment. It is not a human level of reliability at the moment for basically any application that isn't fun fiction writing. So, just be very careful there. All right. Well, Connor is throwing the responsibility over to you guys and what are the top one or two guard rails that you would you think the UK? Let's start with the UK. Um, or keep it to the UK. Should, should insist on, I think that, um, I would say in most of our efforts right now are really trying to understand whether large models um enhance any existing national security threats. And so that would, that would seem to be a really clear area where, you know, we've already, we already are doing work to mitigate against, you know, for example, cyber attacks. And so understanding how a model can and can't contribute to some sort of malicious, um you know, code based attack would be the kind of thing. But again, like, I think we're still quite early in building the empirical base for talking about this stuff. Um But, you know, the great thing about what we're doing at the Safety Institute is we are working very closely with the, the experts in the UK national security community. People who are experts in biosecurity, experts in cybersecurity and nuclear security, who can help us kind of understand the level of uplift these systems truly provide because they think about these risk day in day out. Um I think like that, that's the sort of I guess the easy answer, I think the sort of thing I think we would want to be really on the lookout for, with these models is something like deceptiveness. You know, you wouldn't want a model that is deceptive. Um, and that's the, that's the sort of thing that's a bit broader than national security that I think it would be very important to, to, to, to sort of put guardrails in around. Yeah. Thank you. Yeah, I think um so we would worry a lot at a corporate level around being responsible for any misinformation, disinformation. Um So in, in terms of ensuring that you, you're putting things out there for people to make decisions on that are robust, I think having watermarked content and auditability. So, so that's what we're trying to build into our A I tools so that clients can see what was used to produce the output. And I think um just in terms of copyright type issues, I think having really strong data management and tagging your data as to what's the licensing agreement you have and where, where can you use it and who can have what, what are the permissions associated with that data? Give you just a bit more robust assurance that what you're producing at the end doesn't infringe anyone else's copyrights. And, but both of these areas you're already seeing some litigation that is concerning for organizations. So I think data management is, is probably quite fundamental. It's the fuel for, for A I. So getting that right. I think puts you in a much better place uh in your usage inside of London Stock Exchange. Have you created a guardrail around using models that are not trained on copyrighted data? We're seeing that now in some artists that are not trained. Yeah. So, so I think there is um have you gone that far? We are the main thing that we are focused on is actually um training models on our own data and a lot of our data, we actually contract some, some we create ourselves and some we actually have license agreements with third parties. So in doing so, we've been really careful. It's very easy for our own data. But where we actually contract data from somebody else, we're very careful about how and when we would use that within the training of, of an LLM that we would use with a client so trained and not fine tuned. Yes. Yeah. Very interesting. Well, I can't believe it, but that is all we have time for. Uh would love, I would have loved to do a Q and A session with everybody, but we'll have to save that for the break. So, thanks everybody.

Zoubin Ghahramani, Vice President, DeepMind, Google; Professor, Information Engineering, University of Cambridge Dr. Anne Phelan, Chief Scientific Officer, BenevolentAI Dr. Marc Warner, Co-Founder and CEO, Faculty Moderator: Jeremy Kahn, AI Editor, FORTUNE; Co-chair, Fortune Brainstorm AI London 

Brainstorm AI London 2024: How The Latest Breakthroughs In Generative AI Will Transform Business And Society

Brainstorm AI: How The Latest Breakthroughs In Generative Ai Will Transform Business And Society

Zoubin Ghahramani, DeepMind, Google; Information Engineering, University of Cambridge, Dr. Anne Phelan, BenevolentAI, Dr. Marc Warner,  Faculty, Jeremy Kahn, FORTUNE

Thank you all for being with me and thank you all for joining us. So general A I is creating a lot of buzz. A lot of excitement about what is coming in the future from these models. I want to start by talking a little bit with been who's Amelia to write about Gemini. This is a very powerful model that Google has, has developed and has just come out with a couple of new versions of what is so special about Gemini Zubin. And how is this potentially transformative? Let's start there. Yeah, thanks Jeremy. It's a pleasure to be here. So, um first of all, Gemini is, is a whole family of models going from on device models to our very largest models. And uh what's really exciting about Gemini is that they're built multi modally from the ground up so they can process uh images, audio, video, et cetera alongside text. And our latest version of Gemini um actually has uh what's called a, a 1 million token context window, which means that it can process and keep in memory a tremendous amount of information much larger than any of the other models out there. It's like a whole book's worth of material. It's many books, a whole podcast, a video, an entire code base. And you know, if you think of that, not just if you think of that in terms of like the short term memory that the model has, that you can query and interact with the capabilities of that. We've only really scratched the surface of that. Interesting. Now, I know when Gemini came out at least one version of the model, there was an issue around the image generation capability of the model that in an effort to try to make sure that the images were you represent a diverse set of people and overcame some of the historical biases in the data that people complained about with these models. You guys created a model that it was actually very hard to get it to generate images of white people even in historically appropriate context for that. And it was quite embarrassing for you. How do you, how do you overcome issues like that? Because I think, I think that's one of, I think that's one of the problems now is that people want a model that you can just say, OK, well, you know, be diverse when that's appropriate and when it's not appropriate, you know, be correct to the historical context, but you can't just tell the model that apparently. So yeah, how do you overcome the first of all? Thanks Jeremy for asking, you know, it's, it's actually you know, it really highlights the fact that these challenges are difficult, right? First of all, you know, we want the tools, the generative A I tools to be relevant to people all around the world. So if you ask a model to create images of, let's say a nurse in a hospital or a school teacher or you know, a board member of a company, it has to be relevant to people all around the world. Google is a global company. We have, you know, people using our tools on every continent. Um but the guard rails that we put in uh in that case, we were clearly wrong. We came out, we actually said that we then um pulled the image generation for Gemini off so that we can improve it and put it back out in a way that's more acceptable to more people. Um You know, this is really part I think of the iteration process where you create tools, these are cutting edge tools, you put them out there, people find them really useful, sometimes, sometimes they find them offensive, right? And you know, I think this is a a healthy thing that the whole A I community is really grappling with right now. Mark I want to get you in here. You faculty help a lot of different businesses, try to figure out how to implement A I technology, help the government do this to some extent. I think, you know, this all comes down to reliability. You have these very powerful systems were very excited about the sort of headline capabilities, but then they start playing around with it and they're like, oh, but it's not reliable. How do I? And I think a lot of business leaders are struggling with the idea of how do I implement something that's not reliable? What advice are you giving people around that? Yeah. So we think that people need to think about these in a human centric way. So they are tools to be used in combination with people. And so if you're careful and you're thoughtful about how you design that in from the start, you give yourself this ability to sort of tune the workload between the person and the machine and that ultimately, well, one puts you in a much better place right now because the models aren't perfect and will have small failure modes that can mess up a particular workflow. But two, it does also future proof you to an extent because over time, you can tune that balance between the person and the machine so that as these developing capability and the truth is, nobody knows how fast they're going to develop and how far they're going to get. But as they do develop in capability, you will be able to tune that workflow as you go. And so you'll have this nice possibility of being relatively future proofed and benevolent. You're using these to help with drug discovery a lot of people are very excited about the potential transformational effect on science. Maybe you can talk a little bit about where you see this going and how big an impact you think these models will have on science. Yes, actually your comments resonated really clearly for me because we do think of it as human in the loop we have now the capacity to surface huge amounts of data by medical data. The field is expanding at a colossal rate, all mixed data, patient level data and to be able to carefully curate and amalgamate all that to be able to pose biological questions to our data foundations is something that we've given an awful lot of thought and some elements are moving really quickly in terms of automation and some are very heavily reliant on the human in the loop kind of process. But fundamentally drug discovery is very, very difficult, it's expensive, it's time consuming something like 95% of drugs don't make it through clinical development to launch and it costs something in the region of $2.5 billion per drug. So anything that we can do even incrementally to increase our probability of success in each incremental process step of the process and inevitably bring down costs is going to have a massive societal impact with the use of A I is carefully applied to drug discovery. And when you talk about carefully applying, I mean, can you talk a little bit about what's involved there and, and how do you prevent some of the potential failure modes? As Mark was saying of these models? Yeah. So I mean, I think incrementally, the first step of drug discovery is identifying the target that you want to modulate. So we spent a lot of time thinking about trying to integrate all biomedical data so that we can understand the signaling pathways and mechanisms of health and disease. So we can choose the right target because fundamentally, if you have the wrong target, doesn't matter how good you are, the rest of it, the whole thing's doomed really. So starting really carefully with the with the right target is a is a big first step. And actually, as we use some of these more generative models, really thinking carefully about the prompt and asking very careful questions of the technology so that you don't get these kind of hallucinatory answers that you can very carefully. It's not just one question, one answer. It's a series of very carefully crafted questions that get you a more and more refined answer back that then we can use, you know, actual data that we can use for the next step of the process. Interesting speaking of questions, we're going to go to questions from the audience in a minute. So please think of your questions. Um When you talk about using these tools for science, I know deep mind has done a lot of work around this alpha fold, you know, a huge success. Um recently, though there was a paper you guys had on material science that had been published in nature came up with, you know, thousands of new materials with unknown properties of science. It sounded very exciting. There's recently been some criticism though that a lot of those materials are not really synthes, sizeable, not easy to manufacture. Maybe we're getting ahead of ourselves. I mean, as someone who's been in this area for a long time, looking at the research potential of these models, how are you thinking about it? Are we, are we over hyping? What these can do for science? So I don't think we're over hyping because I think actually one of the most exciting things that A I can do for humanity is uh advancing our ability to create new technologies, new other technologies. So let's talk about material science. This is a very exciting area. Um uh So the paper in question um actually uh had uh 2.2 million new stable crystals that were automatically generated from the A I model. And if you think about what new materials can help you do, they can help you um design uh better batteries, they can help you um design better uh photovoltaic solar panels and all sorts of things that will help us address, you know, medical devices, all sorts of things that helps us address um you know, societal challenges. Now, um the uh the process that we stand by the results of the paper, it was peer reviewed in nature, which is really the top journal academically. Um, and, you know, it's part of a healthy scientific uh uh process where you put out some results and people come out and question them, you go back and forth and, you know, hopefully that the truth comes out. But the paper is really exciting along along with a lot of the other work that has been done in A I for the sciences in, you know, many areas like biology with Alpha fold or, you know, fusion energy and mark, do you think the hype sort of getting ahead of ourselves here in terms of what, what these models can accomplish? Uh Well, I mean, I, I like, I think that A I for science that deep mind does is unbelievable and totally brilliant. But I do think in other contexts it's easy to over claim for these models. So, like, you know, you look at when we were promised or fully autonomous cars and it's taken a bit longer than we expected. And I think if people are thinking in the context of like, these are just going to automate away jobs, one for one that will be like much further away than most people anticipate, hopefully much further away as well. I think it's both true and good that it would be further away. But none of that is to say anything other than like the science these guys are doing is totally amazing, right? Uh I wanna go to questions from the audience. So please, if you have, if you have a question, please raise your hand and please wait till mic gets to you and I will call you. There's a question here if we can get a mic to this uh lady here. Uh And if you could please state your name and, and your affiliation. Um, when you stand up. Hi, I'm Natalia Yashu. I lead product in a net tech called Learn Light. Um, and actually I just wanted to respond to the last comment you made around. Well, it takes us longer than we sometimes expect to progressive technologies partly is due to our acceptance or no acceptance of risk. Maybe we could have progressed far faster with autonomous cars if we're ready for more accidents, right? So it feels to me like it's the same with um A I, so I wonder how do you think we can actually find that balance between accepting some risk? Is there less a risk that we're ready to accept because it's technology and not a human error or actually do we need to? That's, that's a great question mark. I want you to answer that first. Like when you're advising people on that question, like how much risk should I accept? How do they, how do you reach that balance was obviously context dependent. So if somebody is saying we want an algorithm that will put a red jumper or a blue jumper top of our website. Then, you know, it's basically use whatever you want. It doesn't, it doesn't matter too much. Then in something like a medical context where the ultimate decision can in principle come down to life or death care much, much, much, much more greatly about um about that risk tolerance. Um At the moment, you know, I don't think there is anything better or more thorough than working with the organization themselves because it is important to remember that even though A I is a new technology, the thing you're using it for is still the thing you are always doing in almost all cases. So if you're a doctor, you are always making life and death decisions and now A I is coming into that. So whether that was like a bureaucratic decision or a technological decision or some simpler algorithm, in any case, you are used to wrestling with exactly the ethical dimensions of that problem. And so in the context of any given problem, there's almost always like a large amount of pre existing thought that you can tap into carefully and, and when you're dealing with a process that, you know, costs 2.5 billion and takes 10 years, how do you view that sort of risk assessment? And, and can you, you know, is it, oh, if we can save any time and money off? Of that. It's worth it or, you know, not always. Yeah, I think, I mean, there is a risk associated with it, but I think the risk can be quite a positive force for disruption. So when you think about drug discovery, as in designing the actual drug, we have very experienced and very seasoned medicinal chemists who can design, they're augmented now with all sorts of different predictive tools to enable them to make different molecules. And sometimes the tools will suggest something the chemist would never ever have thought of. And they inherently think well, it could be, that's not right, you know, that's too dangerous. But I should then, you know, you kind of think, well, actually maybe components of that could be useful. So I think it's the risk management if you, you've got to be open to the notion of risk and what it can do in terms of disruption, but there has to be like guard rails and controls around how you view that, that kind of prediction. So I think, yeah, the scope for this as long as it's carefully monitored, right? Other questions from folks in the room, please raise your hand. There's a question right here, please stand up and say, yeah, he l Southern Methodist University and the board of Directors of Hitachi. Um at lunch, we spoke about the importance of the triangle between on one hand, it backbone. On the other hand, data analytics, data science and domain knowledge you talked about the human in the loop to make sure we get the right quality. But often the person with the domain knowledge, it's the hardest to get into the loop. Can you, can you give some practical advice? How do we make sure that we actually get this human in the loop in the right way? That's a good question mark. Do you want to take that? Yeah. Well, we think of it as human in the loop and human over the loop. So initially, when you start building a system, you probably do want a human actually in the loop as in nothing goes out without human seeing it. But over time, as you build confidence, you can have a human in over the loop putting policies to guide it. Now, the problem is that you do actually have to build your systems in a slightly different way. Like if you want to make them properly human centric, it requires a different set of slightly different set of technologies, a slightly different set of design principles. So we've thought quite a lot about how you do that and have not not published it in an easily accessible form. So I, I promise I will write a blog and I will send it around. Uh but they're like in essence, the way you do it is you start by designing for human and machine to work together and then you structure the algorithms in a more modular fashion so that you basically inherently build in more explainability and more causality into the system. Fascinating other questions here. There's a gentleman here hand up uh marksman from the Times and how, how do you address the issue of concentration of power in A I given the resources needed to build models, etcetera and compute. How do we, how do you answer that one from Google Jeremy? Um So, yeah. No. Uh I, I think it's a, it's a very good question. Um When we look at the, uh what's happened in A I over the last few years, um it's actually really exciting to see uh how many start ups have come out with state of the art models. It seems like, you know, uh every week we have a new entrant into the field, the barriers to entry have been continuing to decrease with open models. We've also contributed to that. We, you know, we've published a lot of our work, but we also have produced open models and then if you look at the um cloud providers, um the whole model is to make these uh tools available very widely. Um So I think it's actually a pretty healthy ecosystem of competition right now. Um It's, you know, although it's expensive to train the very largest models, once you train it, once you can make it available and then people can build on it and use it. And we've seen an incredible thriving of the ecosystem So, um, you know, we'll have to see what happens over the next few years. But, um, I'm, I'm, you know, not concerned about that myself, but, you know, and that brings up an interesting question though, which we were talking about earlier about sort of, do you, do you build these models yourself? Do you buy them? Do you use open source models? How is benevolent sort of looking at that? And then also in the industry of pharmaceuticals, like there's this issue about power, all the big pharmaceutical companies have traditionally had a lot of data. How have you guys as a start up sort of thought, compete with that? Yes, in terms of the models, we have a hybrid model. Actually, we have some proprietary capabilities, we have open access capabilities, we have open access capabilities that we sort of customized and made them more bespoke to our needs. So, you know, in case of access to data, we've systematically brought in a whole range of different data types from the literature or makes compound structures, all this kind of totality of biomedical data so that we can reason over it. But we will always be have smaller data foundations and the really big pharma who have got decades of drug discovery data in house and that's for them to build their models and reason over. But for us, you know, there's a surprising amount of literature publicly available, well, literature and other data modalities that if you systematically build, you can get really decent picture of human biology and mark when you're talking about customers and clients, how do you advise them on this issue of bill versus buy? And do they use an open source model or do you use the proprietary model that seems to do better on a benchmark? What we, I mean, we think about A I is just better software, right? So it's the perennial build vs by question and the only two answers we know are wrong is all build or all buy. Anything in the middle is almost, is at least justifiable. And then our view is that it depends on how it compares to your capabilities and your competitive advantage. So if you have like if it's close to your competitive advantage, you want to be leaning to more towards, more towards build. If it's further from your competitive advantage, you want to be leaning more towards buy. Fantastic. That's all the time we have for today with this group of panelists. But thank you very much for being here and thank you all for listening and we'll get on to the next panel. Thank you very much.

Brainstorm AI London 2024

Brainstorm AI London 2024: How Deutsche Telekom Used AI to Finish Beethoven’s Unfinished Symphony

Latest video

Dr. Matthias Röder, Chief Executive Officer, Karajan Institute; Co-founder and Managing Partner, The Mindshift; Director, Beethoven X – The AI Project

Videos (23 videos)

Brainstorm AI London 2024: How Deutsche Telekom Used AI to Finish Beethoven’s Unfinished Symphony

Brainstorm AI London 2024: From Sports To The Workplace

Brainstorm AI London 2024: Combating Deep Fakes And Protecting Public Trust

Brainstorm AI London 2024: Three Ways AI Is Already Impacting Political Campaigns

Brainstorm AI London 2024: Boosting Breast Cancer Detection

Brainstorm AI London 2024: Building Trusted Data Foundations For Breakthrough Innovation

Brainstorm AI London 2024: Navigating Opportunities In The European Market

Brainstorm AI London 2024: Championing Responsible AI Within The Enterprise

Brainstorm AI London 2024: Turning The Threat Of AI Into An Opportunity

Brainstorm AI London 2024: Boosting Confidence In AI With Employees And Consumers

Brainstorm AI London 2024: AI’s Impact On The Geopolitical Landscape

Brainstorm AI London 2024: Navigating Roadblocks And Catalysts To Enterprise-Wide Adoption

Brainstorm AI London 2024: Making AI Work At Work

Brainstorm AI London 2024: The Future Of Work

Brainstorm AI London 2024: The Importance Of AI Chip Development

Brainstorm AI London 2024: Strategies To Stay Competitive

Brainstorm AI London 2024: The Future Of Synthetic Media

Brainstorm AI London 2024: AI’s Impact On The Sports Industry

Brainstorm AI London 2024: How AI Is Enhancing The Scouting Process And The Fan Experience

Brainstorm AI London 2024: Building The AI We Want

Brainstorm AI London 2024: Responding To AI’s Disruption Of Entertainment And The Arts

Brainstorm AI London 2024: Striking The Right Balance In AI Regulation

Brainstorm AI London 2024: How The Latest Breakthroughs In Generative AI Will Transform Business And Society