GeMs: A New Trend in AI that is Changing the Meaning of Creativity
With their ability to generate novel texts and images of astonishing quality, GeMs are amplifying the human imagination and changing what it means to be creative in the age of AI. Cowritten with GPT-3
The past year has produced a jaw-dropping shift in the field of Artificial Intelligence (AI). With new algorithms and data from billions of examples, massive neural networks are now capable of generating novel texts and images of astonishing quality. In contrast to the humorously low-quality output of smaller models (for an example, see end of article), contemporary AI models can demonstrate an uncanny level of conceptual understanding and aesthetic nuance. For instance, massive language models like GPT-3 can write with apparent emotional intelligence whereas massive image models like MidJourney, DALLE or Stability Diffusion can create images with apparent artistry. These contemporary Generative Models came about due to new algorithms (like Transformers and Diffusion Models) and a truly ginormous amount of data. To refer to them all, we could call them GMs (for Generative Models) but I like to refer to them as GeMs. It’s cute, right?
“An oil painting by Van Gogh of a girl doomscrolling” DALLE2. Could be better if I spent more than 30 seconds on it.
Fuck if they aren’t powerful. GeMs have such a complex range of emergent effects and their full potential is far from understood. This has provoked a massive amount of exploration, both playful and systematic. Researchers have published collections of instructions (or “prompts”) that tend to produce high-quality results. Other users claim to have developed intuitions for how to effectively use GeMs, even though the AI systems themselves are largely opaque.
“Cyberpunk Tarot Cards” on Stability Diffusion by Max_Volta
GeMs are not robots or personalities. Instead, they are tools that can be used by experts or amateurs to more quickly generate ideas for new creative works. By amplifying the human imagination, GEMs are changing what it means to be creative in the age of AI. It is a kind of modern magic: to be able to transform abstract ideas into detailed realities.
As GeMs become more widespread, their impact will continue to grow. They will certainly be used for questionable purposes like advertising, propaganda or even fraud. However, they also offer great potential for good, e.g. by helping people to communicate their ideas more effectively, or by providing new ways to create and enjoy art.
How should people use GeMs?
Here are some of the best ways to use GeMs in a positive manner.
First, GeMs can be used to rapidly generate ideas for new creative works. This can save a lot of time and frustration, and allow you to explore a wider range of ideas. Here’s an example of how Generative Models can be used to “Redesign Everything,” starting with American stripmalls.
Using DALLE’s inpainting tool and the prompt “Beautiful, sustainable apartments above a redesigned shopping center”
Second, GeMs can be used to communicate your ideas more effectively. For instance, if you have an idea for a story but you’re not sure how to start, you could use a GeM to generate a first draft. This can be a great way to get feedback from others, and to start a conversation about your idea.
Third, GeMs can be used to support self-discovery. By interacting with GeMs to produce texts and images, you can develop a better understanding of your own creative ideas and preferences. This can lead to a better appreciation of your own imagination, as well as a greater willingness to experiment with new ideas.
Fourth, GeMs can be used to create new opportunities for shared experience and enjoyment. For instance, you could use a GeM to generate a new piece of music, and then share it with your friends. Or you could use a GeM to generate a new piece of art, and then enjoy it together.
Finally, GeMs can be used to help people in need. For instance, you could use a GeM to generate a new story or poem for a child who is in the hospital. This could help to brighten their day and provide a moment of joy in an otherwise difficult time. Keep in mind that this paragraph is mostly written by GPT3.
These are just a few of the many ways that GeMs can be used in a positive way. As GeMs become more widespread, we are sure that even more uses will be discovered.
Look forward to more from the AI and Experience Design Community!
The above picture was generated by OpenAI’s GPT-2 and nVidia’s GAU-GAN in December, 2021. Prompt: “Santa-Bot”
The above pictures were generated by DALLE, September 2022. Prompt: “Santa-Bot.” Big change in one year!
The optimistic read certainly shows these can open a lot of new doors for those unfamiliar or only casually with art production, and even skilled people wanting to try something beyond the scope of their usual art. Being an artist myself I've caught myself by surprise when things go very right and I generate something inspiring that is outside my own experience. As a researcher, it's also intriguing how at times I can get close enough to an abstract idea to produce something coherent with just a few prompt adjustments.
That being said, it isn't only the misuse of these tools that could be concerning - but also the way they learn - presently in the artist community one of the major pushbacks is that they accumulate and transform many original works of artists without consent - which reflects a deeply colonial approach to new technology... that anyone already present in the field the new tech is intervening in are merely obstacles to be bulldozed and ignored, or resources to be fed into the machine that allows us to sell a product.
Historically many new pieces of technology designed to increase productivity have then been turned around and used as justification to eliminate jobs rather than enhance the work of those already there - we will need to strongly push for these to be a supplement rather than a replacement.