I’ve been learning a lot lately about how the currently popular AIs that generate images from text prompts work. Now that I can find virtually everything I’ve ever published in those databases, I think it’s time to explain this to others:
Opening Remarks
I regularly revise and add to this article when I learn something new. The last revision was May 23rd, 2023!
Where does the material used to train AIs come from?
Text-to-image models take user input in the form of text and produce images that match the prompt. To learn this capability, the model is trained with a huge collection of images and corresponding descriptions, which have been extracted from the internet via data mining or scraping, and collected as a dataset. The generated images simulate as closely as possible the statistical relationship between the texts in the dataset and the images in the dataset.
Data used include copyrighted works of artists and private data from the public. As text-to-image models create images based on probability and statistics, they’re susceptible to reproducing biases, stereotypes, and copyrighted works.
“Stability AI funded the creation of the biggest and most utilized database (…). LAION 5B, originally created on the pretext of “research” contains 5.8 billion text and image data, including copyrighted data and private data, gathered without artists, individuals, and businesses’ knowledge or permission. MidJourney, Stability AI, Prisma AI (Lensa AI) & other AI/ML companies are utilizing these research datasets that contain private and copyrighted data, for profit. They did so without any individual’s knowledge or consent, and certainly without compensation.” – Concept Art Association
As far as I understand, Stability AI intends to start training Stable Diffusion V3 as early as January, Stability AI started training Stable Diffusion V3 in April 2023 – using all the data that was in the LAION 5B database at that time.
In the music industry, where in parallel to visual Stability Diffusion, Stability AI is training Dance Diffusion, the situation is entirely different: “Dance Diffusion is also built on datasets composed entirely of copyright-free and voluntarily provided music and audio samples. Because diffusion models are prone to memorization and overfitting, releasing a model trained on copyrighted data could potentially result in legal issues. In honoring the intellectual property of artists while also complying to the best of their ability with the often strict copyright standards of the music industry, keeping any kind of copyrighted material out of training data was a must.” – Weights and Biases, 26.9.22
Are my Images included in these Databases?
Spawning provides a platform at haveibeentrained.com where you can search and opt-out the usage of specific images, one by one. Stability AI has apparently agreed to remove the corresponding images. This is far from good (better would be Opt-in, i.e. no use without the owner’s consent, plus compensation), but at least it is a start!
Even if it is too late for the current training now – the next one will come for sure, and hopefully other operators will agree to remove the works which have been opted out by their creators / rights holders per haveibeentrained.com.
The catch is that many images are not so easy to find there. If you can’t find anything under your name, projects or alt tags, you can still get quite a surprise when uploading images of your works. I assume it’s pretty much impossible to track down all works and all their copies and have them removed, but at least you can reduce. Meanwhile, there is also the option for website owners to unsubscribe their entire URL, and I recommend everyone to do just that.
Apparently, Stability AI has made use of the databases of some online portals – not only Artstation or Deviant Art, where protests are being voiced loudly, but also Pinterest, where it looks like the alt text is read differently, making it more difficult to find the images. The perfidious thing about it is that on Pinterest you don’t share your works yourself, as on portfolio portals, but that users there share the works of others, so there are hundreds of copies…
Does this concern me if I am not an Artist?
LAION’s datasets seem to consist largely of art, but also contain an infinite number of photos and confidential documents of private individuals. In addition, AI operators offer users the option of uploading and modifying images, which enables those users to effortlessly create plagiarism and also to damage the reputation of private individuals.
Moreover, the problem affects everyone who values art in any way or who thinks it is important that (professional) art continues to exist and that future generations have any reason at all to learn how to draw and paint, to express themselves creatively, and to maybe love it so much that they might even consider art as a profession.
But isn’t it beautiful when Robots dream the World?
Despite all my reservations, I can absolutely relate to the fascination of AI images. I actually find them quite exciting myself – especially where AIs make mistakes that humans would never make, and develop an awkward charm all their own in the process. I also really adore this idea of robots dreaming of the world – but that’s just not it!
It’s tech companies making a lot of money from the works of others, used without consent or compensation.
It’s particularly alarming that with some AIs, you can enter not only your desired motif, but also the desired style – with the name of the artist! For images generated this way, not the artist is paid, but the operators of the AI model. There are already people using the resulting works commercially, and even mentioning name of the artist there…
In addition, AI operators offer users the option of uploading and modifying images, which enables those users to effortlessly create plagiarism and also to damage the reputation of private individuals.
Both shows clearly that image-generating AIs were never meant to be, as often claimed, just a tool (if they were, they’d be adapted just as quickly and enthusiastically as any other tool!), but on the contrary, they’re meant to be a replacement for artists. In the long run, humans will hardly be able to keep up with price, speed and perfection of AIs.
AI learning is often described as “training”, which can be thought of like human artists copying others, in order to eventually find their own style. AIs, on the other hand, have no style to find, because they assemble their works purely from probabilities in their “teaching material.” By the way, those tiny snippets of text in backgrounds of AI-generated images are fragments of signatures of those whose visuals were used without consent!
“It’s funny how recognizing AI art nowadays is just the same old rules as recognizing the fae in old tales. “Count the fingers, count the knuckles, count the teeth, check the shadows…” … and under no circumstances should you make deals with their kind.” – Erkhyan (@erkhyan@yiff.life on Mastodon), 23.12.2022
What’s the difference to Artists getting inspired?
Inspiration is not the same as imitation or collage. Humans are inspired by all kinds of things, such as nature, literature, feelings, smells, cloud formations – or simply their everyday life and everyday objects. For some, a small part of the inspiration for the type of realisation may also be the art of others; many start with copying in order to develop their own style from it. Unlike AIs, however, even a person who has never seen a picture in their life is quite capable of depicting surroundings and thoughts, in their own personal way. AIs (still) lack the personality for this.
„The actual scary part is that AI may not understand the vendettas or racist pedophile fantasies that come out of its mouth, but it only learned to speak those words by imitating us. It’s just a mirror. All it is doing is imitating our behavior.“ – Joshua Diederich
Generally speaking, if you want to create a picture as a professional artist, you usually don’t even look at the works of others; for many, this is actually rather a hindrance. After a client briefing and / or an idea for a picture, many first do research, but not for styles (you have your own!), but for details of the desired motif that you can’t draw from your own imagination. Templates are needed, for example, for technical details such as the construction of a bicycle, the anatomy of a plant or an animal. These templates are not drawn directly, but usually serve as an aid to thinking.
Isn’t AI just another Tool for Artists?
I assume that in the long run, there will be applications of AIs that actually facilitate the work of artists (some already exist, e.g. AI-supported addition of backgrounds in Photoshop), but (a) training with copyrighted image material, (b) prompts and image output in the style of a certain artist, both without bespoke artist’s consent or compensation, as well as (c) the option of uploading anyone’s images and having them manipulated, are just fundamentally wrong.
Ultimately, training AI models on copyrighted images without the consent, payment and naming of the creators, as well as prompting with styles and names, is an appropriation not only of the work and style, but also of the skill, experience, perspective, ultimately of the entire personality of the respective artist – and if works generated in this way then even enter into commercial competition with the works of that very artist, I would call that expropriation.
“There is no ethical way to use the major AI image generators. All of them are trained on stolen images, and all of them are built for the purpose of deskilling, disempowering and replacing real, human artists.” – Molly Crabapple
What are the future perspectives in this?
The point that worries me most about AI text-to-image generators is how completely they devalue and randomize art, and with it, the most human act of actually creating things with your mind, hands and – most importantly! – heart, and how all the personality, the knowledge, the curiosity and the learning, the love and the pain, the lovingly placed details, and the stories those details tell at second or third sight, become completely irrelevant.
“Humans doing hard jobs on minimum wage while robots write poetry and paint is not the future I wanted” – Karl Sharro
If we continue on this path, some artists of our generation may still find some nice old-fashioned work, but for future generations, there will be no point in learning how to draw, and to actually create. Can you imagine telling a child that the one thing they’re really good at and that makes them truly happy is not a possible career, but a bot job?
I think we – and not only the artists, but all of us! – have the responsibility to not let that happen.
Does generating images make you an artist?
Despite all that, I can’t blame those who generate AI images (and pay for them with money as well as data) – even if I find it strange when they call themselves “artists”. If you commission a human illustrator to visualize your ideas, you wouldn’t think of calling yourself an artist, let alone the creator or copyright holder of the resulting work!
The problem is the companies, not the users. The users just use what is offered to them.
From cave to oil paintings, book illustrations or advertising graphics to everday doodles, art is the very expression of a human need for visualization. On one hand, this helps understand the appeal of generating AI images, but on the other hand, that’s exactly why it’s so wrong to leave the most beautiful part of it, the actual creation, to the bots.
Many think that this creation process is not so much different with humans than with AIs: You type in a text query and *zing*, out comes an image (or series of images). It may be surprising to some that it’s not like that.
Creating images is quite a lot of work (for humans). You come up with an idea (possibly after a briefing from the client), dive into it deeply, research all the details, consider image compositions, make sketches and drafts, and finally create the image – with lots of revisions to make sure that every detail, every line, every pixel (and of course every finger! :)) is exactly where it should be, and that you have found the one solution that is just right! Personally, I love to see the warmth, enthusiasm and lifeblood of creators in their images, and how this transfers to the viewer.
I hope that many more generations will be able to experience this happiness of (real) creation.
“We compiled a set of 62 professional digital artists, each with a large amount of copyrighted work online. We found that when Stable Diffusion is prompted to imitate the artists, the artists could be classified from the image successfully an average of 52/62 times (82.74%) and at best 58/62 times (93.54%).” – Study by Guo et al, MIT/Harvard
Spawning AI, who advocate fair use of images in AI and provide tools like haveibeentrained.com (s.a.), differentiate between “artists” and “promptists”, which I consider linguistically correct. The work of a promptist is probably most comparable to Google image search, or an art director who thinks up motifs, commissions illustrators and supervises their work. That is certainly a type of art in itself, but not the same as actually creating, like an artist does.
Aren’t several problems mixed up here?
The problem clearly has more than one dimension, but is very often mixed up in the usual discussions (which I reproduce here in the form of frequent questions), and is then very difficult to separate. Because I consider this separation to be very important, I am trying to break down the different problems more clearly here:
On the one hand, it’s about (a) using copyrighted works for training without the artists’ consent or payment (which is explicitly ruled out for music, on the grounds that AIs tend to reproduce copyrighted works!), (b) entering names to copy their style, and (c) uploading and modifying other people’s images. AI operators train models from databases created for research, but sell commercial rights to works. In my opinion, all this should be stopped urgently!
On the other hand, it is about what AI can and is allowed to do. The more the visible difference between AI-generated images and the images of human artists blurs, the more absurd it seems to equate both types of images, and even those who create them. For me, those are two completely different things, f.e. because AI images would not even exist without templates! Moreover, I find competitions between artists and promptists or a juxtaposition of portfolios just as sensible as having running robots compete against athletes. A sensible solution would be a clearly recognisable, non-erasable watermarks, and separate categories instead of AI images competing with the styles of their templates.
Lines blur though when human art is mixed with AI-generated images, i.e. the use of AIs as a tool, f.e. to generate ideas, work out details, vary one’s own works or prepare material for collages. I suspect that sooner or later many professional human artists will be working that way, and the artistic output will compare to works created without AI.
How can these ethical issues be solved?
It’s probably not possible to put the genie back in the bottle, but we urgently need an ethical solution that doesn’t involve intellectual property theft! The justifiable minimum would be to train AIs exclusively with public domain, maybe Creative Commons (I’m not sure about the legal situation concerning those) image material.
“The AI Argument boiled down is pretty simple. Either you believe everyones personal data and skills should be available indiscriminately for any purpose, or you oppose this and believe in data privacy, the ownership and value of human effort and asking consent to use them.” – Eric Bourdages
Copyright-protected image material should definitely have to be approved in advance by the respective author (“Opt-in”), those authors should be fairly compensated and participate in any income generated with the help of their work and/or their names. Furthermore, I think that a two-stage authorisation, i.e. for (a) a general database and prompts that do not request an author’s name or style, and (b) prompts that actually do, would make a lot of sense.
I hope that there will be legislation very soon that effectively prevents the unethical practices I described, but such things do not usually happen overnight. In the meantime, we can do much more than just watch and wait! The more people understand what is happening and how unethical it is, the less people will (hopefully) want to work with it.
I would like to ask anyone who appreciates art made by real people (like me :)) to keep learning about this, and to educate others! In the meantime, there are initiatives of artists who stand up against AI-generated images (more precisely, their training with copyrighted images) and collect donations! In EU, there are Mestieri del Fumetto, in the US, the Concept Art Association. On their linked Gofundmes, problems and solutions are explained very clearly!
I will discuss further possibilities of resistance in detail in the sequel to this post
»What can we do against AI-generated images?« (Part II) ➔
Ultimately, I would absolutely love to see not only artists and their direct environment, but also our entire range of clients, as well as everyone who enjoys art and wants to make art possible for future generations, up to and including those who enjoy to generate images via AIs, commit to ensuring that AI models work ethically.
Closing Remarks
As soon as I hear anything else that I consider relevant in this context, I’ll include it here and in my Post on Facebook, and I’ll post again separately if necessary. This post is very welcome to be shared! If you like, you can use my respective posts on Facebook, Twitter, Instagram or LinkedIn and follow me there as well for further information.
I am writing this post to educate, help and share my own views. I’m happy to include fact corrections or helpful resources, but I have neither the time nor the nerve to discuss with anyone whether theft is okay. Talking about what AI is allowed to do or how useful it might be for creators is sth we can do once companies stop using copyrighted material for training. Under the current conditions, from my side, it’s a very clear No to AI-generated images.
The image accompanying my post is a variation of my colleague Alexander Nanitchkov’s logo you might have seen on ArtStation and social media profile images, which is how I first became truly aware of the problem.
Collection of Related Links
Here I collect not only sources mentioned in the post, but also other links that I find informative and important. The list is regularly updated (last update on May 23th, 2023), the latest additions are marked in colour. If you think of anything else that should definitely be listed here, I would be happy to receive a comment or a mail!
★ Sequel (Part II) to this post, “What can we do against AI-generated images?” by Iris Luckhaus ➔
★ German Translation of this Post, “Statement | Nein zu AI-generierten Bildern” by Iris Luckhaus ➔
Basics
★ “Stable Diffusion Objectively Succeeds at Copycatting Specific Artists’ Styles”, Study by Carl Gu et al, 10.5.23 ➔
★ Detailed explanations “AI Art Explained Easily: Development, Problems & Tips” by Sandra Süsser, 27.12.22 ➔
★ Great video about the issues with AI art, “Why Artists are Fed Up with AI Art” by Sam Does Arts, 24.12.22 ➔
★ Detailed Video “The End of Art: An Argument Against Image AIs” by Stephen Zapata Art, 18.10.22 ➔
★ “I asked Chat GPT to write a song in the style of Nick Cave and this is what it produced. What do you think?” ➔
Press
★ “Der A.I.-Horror: Wie KI die Tyrannei des Mittelmaß fördert” von Markus Haage / Neon Zombie, 23.5.23 ➔
★ “AI machines aren’t ‘hallucinating’. But their makers are” by Naomi Klein / The Guardian, 8.5.23 ➔
★ “Die künstliche Intelligenz löst Probleme, die wir nicht haben” von Alexander Brentler / The Jacobin, 27.4.23 ➔
★ “A.I. Is Sucking the Entire Internet In. What If You Could Yank Some Back Out?”, H.T. Murphy / SLATE, 27.3.23 ➔
★ “Ethisch verwerflich und eine massive Gefahr für den Schutz des Urheberrechtes”, Nino Kirst / Page, 25.1.23 ➔
★ “The Alt-Right Manipulated My Comic. Then A.I. Claimed It.” von Sarah Anderson / New York Times, 31.12.22 ➔
★ “Stability AI plans to let artists opt out of Stable Diffusion 3 image training”, B. Edwards / Ars Technica, 15.12.22 ➔
★ “Why I’m Done Using And Boosting AI Art” Ramble by Chuck Wendig / Terrible Minds, 8.12.22 ➔
★ “AI selfies and their critics are taking the internet by storm”, Tatum Hunter / Washington Post, 8.12.22 ➔
Tools
★ Toolset “Have I been Trained” for finding and opting-out of Images from the LAION Dataset (mentioned above) ➔
★ “Stable Attribution” findet die Bilder, die von KIs (wahrscheinlich) zur Erstellung von Bildern benutzt wurden ➔
★ “Lexica” – nicht nur ein Bildgenerator, sondern auch eine Art Suchmaschine für KI-generierte Bilder ➔
★ App that protects from futre scraping, “NO AI (AI Watermark Generator)” on GitHub (PC, Mac will follow) ➔
★ App “Glaze” – a filter that keeps AIs from training the specific style of artists; University of Chicago ➔
Initiatives
★ European Initiative, Petition and Fundraiser “EGAIR – European Guild for Artificial Intelligence Regulation” (EU) ➔
★ German Initiative “KI aber fair” von Kunst- und Kulturverbänden wie u.a. der Illustratoren Organisation (DE) ➔
★ Fundraiser “Help protect our art and data from AI companies” by MeFu / Mestieri del Fumetto (EU) ➔
★ Fundraiser “Protecting Artists from AI Technologies” by Karla Ortiz, Concept Art Association (USA) ➔
★ “Stop AI stealing the show: Campaign to strengthen performers’ rightsin response to the rise of AI” (UK) ➔
Legal
★ “Getty Images is suing the creators of AI art tool Stable Diffusion for scraping its content”, J. Vincent, The Verge ➔
★ Stable Diffusion Litigation by Matthew Butterick and Joseph Saveri Law Firm ➔, Case Website ➔
★ “AI Image Models Illegally Using Art without Permission in their Datasets”, by Stop Consumer Harm Reports ➔
Thanks for taking the time to compile this Iris, really appreciate it. Having worked in digital technology within amd for government & large corporates for many years, I am also equally fascinated from a tech perspective, about how AI works and how it is and will continue to transform and evolve how people live and work. But there is ALWAYS a flipside to the potential positive of tech (or anything, really) – and that is the fundamental questions around the ETHICS of use – ie. who are the decision makers, and implementers of the tech, and what is there agenda? Or in other words, What’s in it for Them? Money and power / influence will always be the core driving force, as that is how current systems everywhere are structured. And where there is tech hype, there will always be large corporates (& government agencies) involved. It’s interesting to note that these business models and systems however whilst making a LOT of $$$ for for profit companies (from trading our data, art etc) they are failing to address the needs of our most vulnerable people in communities – and increasingly society as a whole. Case in point: the United States of America (Gov in dire debt, its big tech and media corporates that own the world’s data, $$ and set the agenda – both directly and indirectly). I’m in Australia and hope that we don’t attempt to replicate the US model or else we’re doomed too. European and Canadian governments and businesses, and communities have always seemed to me to be relatively more focused on decision making, governing and advocating for people whilst still retaining sound, profitable, sustainable and ETHICAL business models. It is possible to do both.