Brainstorm A.I. 2022 - Salvador Dali Meets Wall-E

Conferences

Brainstorm A.I. 2022 - Salvador Dali Meets Wall-E


across our conversations, we've been discussing how AI is evolving and how it's being integrated into the fabric of all types of organizations. Now we'd like to share a demo on AI is remarkable creative abilities brought to you by one of the world's leading AI research labs. Open AI imagine creating original digital images simply by describing with words what you would like to see in a bespoke demo. Let's meet Valley. A nod to both Wally, the 2008 animated movie about an autonomous robot and Salvador Dali, the surrealist painter. Please welcome to the brainstorm Ai stage. Mark Chen, research scientist at Open I AI together with dolly. Hi everyone. Thank you for the kind introduction. And I'm really excited to be here at fortune brainstorm AI And today I'm here to talk to you about why what and how of dolly too. And um I want to sincerely apologize if you're here to talk about chat GPT but we'll leave that for another conference. So before I begin, I want to talk a little bit about open AI and our core mission. So our core mission is twofold. First we want to build safe and beneficial artificial general intelligence and by A G. I. We mean systems that can match or outperform humans at most economically valuable tasks. It's equally important though to ensure that A. G. I benefits all of humanity and not just a small slice. So earlier this year we announced dolly to and we released it in a platform called labs and as Fortune pointed out this represented a big step forward in text image generation And the big breakthrough here was approaching photo realism. By which I mean that at a quick glance, it's difficult to tell whether some of these images are computer generated or human generated. So at its core, dolly provides an interface where a user can provide a prompt, which is a sentence, a couple sentences of text and dolly can render a high resolution image that matches this prompt. So if I said something like fortune brainstorm ai conference in the style of Picasso, you might see an image like shown on the screens over here. So let's peer under the hood a little bit and talk about how dolly too was trained. So you can imagine dolly to at the beginning of training like a small child who has very little concept of what text or images are all about. And you start showing this child many flash cards, one at a time. Each flashcard has an image and a caption on it. And you can imagine that after seeing a million hundreds of millions of these flash cards that the child starts to form some associations like when it sees the word panda, it might say, oh hey in the image there's a fuzzy animal that's black and white. And so it actually learns through seeing so many images and text pairs, this own internal language which were representing by this strip of colors uh, for representing both vision and language. Now in order to generate a new image, What the model does is it takes the prompt, let's say fall colors next to a lake and it turns it into its internal language or representation. And then it uses what's called a diffusion model to render it into a highly realistic image. Now, if you've been following our work since Dolly one, this was using a technology more similar to gpt three, where we're generating the image row by row and then column by column within the each individual row. In contrast what powers dolly too is a diffusion model and you can see a visualization of how these are working in the slide. So you can see that the image starts out as very grainy, noisy textures. And over the course of generation it resolves into something that's crisp and sharp. So as I said at its core value to has texted image capabilities, which means that given a prompt we can create an image that's true to its concept, attributes and styles. And just to illustrate this point, let's say that I have a prompt like the following say it's a sea otter with sunglasses and a suit, classic Hong kong film actor, action movie close up with dramatic lighting. So even though sea otters wearing sunglasses and suits don't appear in our data set at all. Our models able to generalize and basically compose all these elements together to form a realistic image. And it's also not just able to combine these objects together. It also respects the mood lighting that you want and the overall ambiance that you want. Another example here is if I ask for vibrant and smiling, bowling balls rolling down a bowling alley in the style of digital art, it's creative enough to imagine that the finger holes could be eyes and the thumb hole could be a mouth and it really synthesize it in the aesthetic of like this kind of neon bowling alley vibe. So text image isn't the only capability of dolly. There's also in painting in which you can make realistic edits to existing photographs. And to give you an example, say I start with this image of a yellow couch in a modern living room and I don't really like this couch. I want to change it to something like chic grade chair so I can just mask out the middle part of the image and ask for a C grade chair and I can get one in its place and notice that this respects all the lighting and shadows that are implied by the original image, The lights coming in from the right side and you can see that the shadows are generated appropriately. One other interesting thing is that you can change these edits in succession so I don't have to stop here. I can ask for a modern fireplace on the wall and so you can imagine this kind of very iterative editing procedure with images. Next we have out painting which is powered by the same technology as our in painting. So what we can do here is we can take an original image and draw around the borders of it to synthesize a larger, arbitrary aspect ratio image. So let's take a famous painting, let's say the girl with a pearl earring and artist august camp use our tool which you can kind of see visualized here. And drew around this painting to visualize what could have been in the background and you can see that plausibly this girl was in a very messy kitchen and living room. And finally one thing we can do is variations, which means that we can take an input image and synthesize other images that look like this image but with slightly different features to give you an example, I can start with a dog dressed in formal attire and if I run it through our Variations engine, I can get these other images and you can notice that you know, they're all dogs dressed in formal attire but they have different ties are facing slightly different ways. And we've seen artists respond really well to this, they can visualize their work from different perspectives or take some art that they feel like isn't quite right and run it through the variations engine to find out and and come up with new inspirations. So finally, I want to talk about a couple of practical applications of dali. So when we launched ali, one interesting class of users who came to us were urban planners and in particular this one organization Better Streets ai they were occupied with. How do we show people what our streets could look like if we designed them around pedestrian experiences rather than vehicles and you can see that here, they've taken some city streets and rendered pedestrian and walkable spaces. And this illustrates the power of showing people, not just telling people next when you were creating dolly One tagline that we always thought about was we wanted to create an image generator for your imagination or image, search for your imagination and Microsoft this fairly literally. Um if you're not able to find the images that you want in being searched, you're able to go and synthesize your own images. Of course the potential for video games is also very clear, and in this case we had a video game designer create a character sprite sheet and also all the background and playable elements using dali and actually create a playable video game out of it. One kind of niche but fairly heartwarming application is in generating graphics for Children's storybooks. So Children's story books, of course they have this like very crayon drawn aesthetic. Um and it's very vibrant colors and we've noticed that many people have been bringing their story books to life. Uh here is one particular example where M I. T. Professor, he had a Children's book that he wrote a while ago and made images for them. And finally I want to showcase Kela, which is an end to end uh fashion ideation and design platform. And this is an application where users can go and use dolly to synthesize different clothing that they feel like could suit them and get it made and shipped to them in the real world. So this illustrates one really strong point of using Dali. It allows you to personalize images so that they suit individual tastes. And this is also a big reason that I was so invested in creating dolly for myself. I'm not a very artistic person and I feel like it's so empowering to give myself the ability to create images that are personalized for me. And to conclude, we went through what dolly is all about, how it works, uh the different modes in which you can use that you can use it for text image for in painting, for out painting for variations. We went through some applications for how you can deploy in the real world and I still think the most compelling applications for this technology are still to come. Thank you very much