I’ve been learning a lot lately about how the currently popular AIs that generate images from text prompts work. Now that I can find virtually everything I’ve ever published in those databases, I think it’s time to explain this to others:
I regularly revise and add to this article when I learn something new. The last revision was Jan 13th, 2023!
Where does the material used to train AIs come from?
Text-to-image models take user input in the form of text and produce images that match the prompt. To learn this capability, the model is trained with a huge collection of images and corresponding descriptions, which have been extracted from the internet via data mining or scraping, and collected as a dataset. The generated images simulate as closely as possible the statistical relationship between the texts in the dataset and the images in the dataset.
Data used include copyrighted works of artists and private data from the public. As text-to-image models create images based on probability and statistics, they’re susceptible to reproducing biases, stereotypes, and copyrighted works.
“Stability AI funded the creation of the biggest and most utilized database (…). LAION 5B, originally created on the pretext of “research” contains 5.8 billion text and image data, including copyrighted data and private data, gathered without artists, individuals, and businesses’ knowledge or permission. MidJourney, Stability AI, Prisma AI (Lensa AI) & other AI/ML companies are utilizing these research datasets that contain private and copyrighted data, for profit. They did so without any individual’s knowledge or consent, and certainly without compensation.” – Concept Art Association
From what I understand, Stability AI plans to start training Stable Diffusion V3 as early as January 2023 – and any data contained in the LAION 5B database at that point will be used for training purposes.
In the music industry, where in parallel to visual Stability Diffusion, Stability AI is training Dance Diffusion, the situation is entirely different: “Dance Diffusion is also built on datasets composed entirely of copyright-free and voluntarily provided music and audio samples. Because diffusion models are prone to memorization and overfitting, releasing a model trained on copyrighted data could potentially result in legal issues. In honoring the intellectual property of artists while also complying to the best of their ability with the often strict copyright standards of the music industry, keeping any kind of copyrighted material out of training data was a must.” – Weights and Biases, 26.9.22
Are my Images included in these Databases?
Spawning provides a platform at haveibeentrained.com where you can search and opt-out the usage of specific images, one by one. Stability AI has apparently agreed to remove the corresponding images. This is far from good (better would be Opt-in, i.e. no use without the owner’s consent, plus compensation), but at least it is a start!
The catch is that many images are not so easy to find there. If you can’t find anything under your own name, projects or alt tags, you can still get quite a surprise when uploading images of your works. I assume it’s pretty much impossible to track down all works and all their copies and have them removed, but at least you can reduce.
Apparently, Stability AI has made use of the databases of some online portals – not only Artstation or Deviant Art, where protests are already being voiced, but also Pinterest, where it looks like the alt text is read differently, making it more difficult to find the images. The perfidious thing about it is that on Pinterest you don’t share your works yourself, as on portfolio portals, but that users there share the works of others, so there are hundreds of copies…
Does this concern me if I am not an Artist?
LAION’s datasets seem to consist largely of art, but also contain an infinite number of photos and confidential documents of private individuals. In addition, AI operators offer users the option of uploading and modifying images, which enables those users to effortlessly create plagiarism and also to damage the reputation of private individuals.
Moreover, the problem affects everyone who values art in any way or who thinks it is important that (professional) art continues to exist and that future generations have any reason at all to learn how to draw and paint, to express themselves creatively, and to maybe love it so much that they might even consider art as a profession.
But isn’t it beautiful when Robots dream the World?
Despite all my reservations, I can absolutely relate to the fascination of AI images. I actually find them quite exciting myself – especially where AIs make mistakes that humans would never make, and develop an awkward charm all their own in the process. I also really adore this idea of robots dreaming of the world – but that’s just not it!
It’s tech companies making a lot of money from the works of others, used without consent or compensation.
It’s particularly alarming that with some AIs, you can enter not only your desired motif, but also the desired style – with the name of the artist! For images generated this way, not the artist is paid, but the operators of the AI model. There are already people using the resulting works commercially, and even mentioning name of the artist there…
In addition, AI operators offer users the option of uploading and modifying images, which enables those users to effortlessly create plagiarism and also to damage the reputation of private individuals.
Both shows clearly that image-generating AIs were never meant to be, as often claimed, just a tool (if they were, they’d be adapted just as quickly and enthusiastically as any other tool!), but on the contrary, they’re meant to be a replacement for artists. In the long run, humans will hardly be able to keep up with price, speed and perfection of AIs.
AI learning is often described as “training”, which can be thought of like human artists copying others, in order to eventually find their own style. AIs, on the other hand, have no style to find, because they assemble their works purely from probabilities in their “teaching material.” By the way, those tiny snippets of text in backgrounds of AI-generated images are fragments of signatures of those whose visuals were used without consent!
“It’s funny how recognizing AI art nowadays is just the same old rules as recognizing the fae in old tales. “Count the fingers, count the knuckles, count the teeth, check the shadows…” … and under no circumstances should you make deals with their kind.” – Erkhyan (@firstname.lastname@example.org on Mastodon), 23.12.2022
What’s the difference to Artists getting inspired?
Inspiration is not the same as imitation or collage. Humans are inspired by all kinds of things, such as nature, literature, feelings, smells, cloud formations – or simply their everyday life and everyday objects. For some, a small part of the inspiration for the type of realisation may also be the art of others; many start with copying in order to develop their own style from it. Unlike AIs, however, even a person who has never seen a picture in their life is quite capable of depicting surroundings and thoughts, in their own personal way. AIs (still) lack the personality for this.
Generally speaking, if you want to create a picture as a professional artist, you usually don’t even look at the works of others; for many, this is actually rather a hindrance. After a client briefing and / or an idea for a picture, many first do research, but not for styles (you have your own!), but for details of the desired motif that you can’t draw from your own imagination. Templates are needed, for example, for technical details such as the construction of a bicycle, the anatomy of a plant or an animal. These templates are not drawn directly, but usually serve as an aid to thinking.
Isn’t AI just another Tool for Artists?
I assume that in the long run, there will be applications of AIs that actually facilitate the work of artists (some already exist, e.g. AI-supported addition of backgrounds in Photoshop), but (a) training with copyrighted image material, (b) prompts and image output in the style of a certain artist, both without bespoke artist’s consent or compensation, as well as (c) the option of uploading anyone’s images and having them manipulated, are just fundamentally wrong.
What are the future perspectives in this?
The point that worries me most about AI text-to-image generators is how completely they devalue and randomize art, and with it, the most human act of actually creating things with your mind, hands and – most importantly! – heart, and how all the personality, the knowledge, the curiosity and the learning, the love and the pain, the lovingly placed details, and the stories those details tell at second or third sight, become completely irrelevant.
If we continue on this path, some artists of our generation may still find some nice old-fashioned work, but for future generations, there will be no point in learning how to draw, and to actually create. Can you imagine telling a child that the one thing they’re really good at and that makes them truly happy is not a possible career, but a bot job?
I think we – and not only the artists, but all of us! – have the responsibility to not let that happen.
Does generating images make you an artist?
Despite all that, I can’t blame those who generate AI images (and pay for them with money as well as data) – even if I find it strange when they call themselves “artists”. If you commission a human illustrator to visualize your ideas, you wouldn’t think of calling yourself an artist, let alone the creator or copyright holder of the resulting work!
The problem is the companies, not the users. The users just use what is offered to them.
From cave to oil paintings, book illustrations or advertising graphics to everday doodles, art is the very expression of a human need for visualization. On one hand, this helps understand the appeal of generating AI images, but on the other hand, that’s exactly why it’s so wrong to leave the most beautiful part of it, the actual creation, to the bots.
Many think that this creation process is not so much different with humans than with AIs: You type in a text query and *zing*, out comes an image (or series of images). It may be surprising to some that it’s not like that.
Creating images is quite a lot of work (for humans). You come up with an idea (possibly after a briefing from the client), dive into it deeply, research all the details, consider image compositions, make sketches and drafts, and finally create the image – with lots of revisions to make sure that every detail, every line, every pixel (and of course every finger! :)) is exactly where it should be, and that you have found the one solution that is just right! Personally, I love to see the warmth, enthusiasm and lifeblood of creators in their images, and how this transfers to the viewer.
I hope that many more generations will be able to experience this happiness of (real) creation.
Spawning AI, who advocate fair use of images in AI and provide tools like haveibeentrained.com (s.a.), differentiate between “artists” and “promptists”, which I consider linguistically correct. The work of a promptist is probably most comparable to that of an art director or picture editor, who thinks up motifs, commissions illustrators and supervises their work. That is certainly a type of art in itself, but not the same as actually creating, like an artist does.
Aren’t several problems mixed up here?
The problem clearly has more than one dimension, but is very often mixed up in the usual discussions (which I reproduce here in the form of frequent questions), and is then very difficult to separate. Because I consider this separation to be very important, I am trying to break down the different problems more clearly here:
On the one hand, it’s about (a) using copyrighted works for training without the artists’ consent or payment (which is explicitly ruled out for music, on the grounds that AIs tend to reproduce copyrighted works!), (b) entering names to copy their style, and (c) uploading and modifying other people’s images. AI operators train models from databases created for research, but sell commercial rights to works. In my opinion, all this should be stopped urgently!
On the other hand, it is about what AI can and is allowed to do. The more the visible difference between AI-generated images and the images of human artists blurs, the more absurd it seems to equate both types of images, and even those who create them. For me, those are two completely different things, f.e. because AI images would not even exist without templates! Moreover, I find competitions between artists and promptists or a juxtaposition of portfolios just as sensible as having running robots compete against athletes. A sensible solution would be a clearly recognisable, non-erasable watermarks, and separate categories instead of AI images competing with the styles of their templates.
Lines blur though when human art is mixed with AI-generated images, i.e. the use of AIs as a tool, f.e. to generate ideas, work out details, vary one’s own works or prepare material for collages. I suspect that sooner or later many professional human artists will be working that way, and the artistic output will compare to works created without AI.
How can these ethical issues be solved?
It’s probably not possible to put the genie back in the bottle, but we urgently need an ethical solution that doesn’t involve intellectual property theft! The justifiable minimum would be to train AIs exclusively with public domain, maybe Creative Commons (I’m not sure about the legal situation concerning those) image material.
Copyright-protected image material should definitely have to be approved in advance by the respective author (“Opt-in”), those authors should be fairly compensated and participate in any income generated with the help of their work and/or their names. Furthermore, I think that a two-stage authorisation, i.e. for (a) a general database and prompts that do not request an author’s name or style, and (b) prompts that actually do, would make a lot of sense.
I hope that there will be legislation very soon that effectively prevents the unethical practices I described, but such things do not usually happen overnight. In the meantime, we can do much more than just watch and wait! The more people understand what is happening and how unethical it is, the less people will (hopefully) want to work with it.
I would like to ask anyone who appreciates art made by real people (like me :)) to keep learning about this, and to educate others! In the meantime, there are initiatives of artists who stand up against AI-generated images (more precisely, their training with copyrighted images) and collect donations! In EU, there are Mestieri del Fumetto, in the US, the Concept Art Association. On their linked Gofundmes, problems and solutions are explained very clearly!
I will discuss further possibilities of resistance in detail in the sequel to this post
Ultimately, I would absolutely love to see not only artists and their direct environment, but also our entire range of clients, as well as everyone who enjoys art and wants to make art possible for future generations, up to and including those who enjoy to generate images via AIs, commit to ensuring that AI models work ethically.
As soon as I hear anything else that I consider relevant in this context, I’ll include it here and in my Post on Facebook, and I’ll post again separately if necessary. This post is very welcome to be shared! If you like, you can use my respective posts on Facebook, Twitter, Instagram or LinkedIn and follow me there as well for further information.
I am writing this post to educate, help and share my own views. I’m happy to include fact corrections or helpful resources, but I have neither the time nor the nerve to discuss with anyone whether theft is okay. Talking about what AI is allowed to do or how useful it might be for creators is sth we can do once companies stop using copyrighted material for training. Under the current conditions, from my side, it’s a very clear No to AI-generated images.
The image accompanying my post is a variation of my colleague Alexander Nanitchkov’s logo you might have seen on ArtStation and social media profile images, which is how I first became truly aware of the problem.
Collection of Related Links
Here I collect not only sources mentioned in the post, but also other links that I find informative and important. The list is regularly updated (last update on January 13th, 2023), the latest additions are marked in colour. If you think of anything else that should definitely be listed here, I would be happy to receive a comment or a mail!
★ Great video about the issues with AI art, “Why Artists are Fed Up with AI Art” by Sam Does Arts, 24.12.22 ➔
★ Detailed Video “The End of Art: An Argument Against Image AIs” by Stephen Zapata Art, 18.10.22 ➔
★ Detailed explanations “AI Art Explained Easily: Development, Problems & Tips” by Sandra Süsser, 27.12.22 ➔
★ “I asked Chat GPT to write a song in the style of Nick Cave and this is what it produced. What do you think?” ➔
★ Article “Stability AI plans to let artists opt out of Stable Diffusion 3 image training”, Ars Technica, 15.12.22 ➔
★ Article “Why I’m Done Using And Boosting AI Art” by Chuck Wendig, Terrible Minds, 8.12.22 ➔
★ Article “AI selfies — and their critics — are taking the internet by storm” by Washington Post, 8.12.22 ➔
★ Essay “The Alt-Right Manipulated My Comic. Then A.I. Claimed It.” by Sarah Anderson, 31.12.22 ➔
★ Fundraiser “Help protect our art and data from AI companies” by MeFu / Mestieri del Fumetto (EU) ➔
★ Fundraiser “Protecting Artists from AI Technologies” by Karla Ortiz, Concept Art Association (USA) ➔
★ “Getty Images is suing the creators of AI art tool Stable Diffusion for scraping its content”, J. Vincent, The Verge ➔
★ Stable Diffusion Litigation by Matthew Butterick and Joseph Saveri Law Firm ➔, Case Website ➔
★ “AI Image Models Illegally Using Art without Permission in their Datasets”, by Stop Consumer Harm Reports ➔