GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models

Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, Ben Y. Zhao
Computer Science, University of Chicago
{shawnshan, jennacryan, ewillson, htzheng, ranahanocka, ravenben}@cs.uchicago.edu

Abstract
arXiv:2302.04222v1 [cs.CR] 8 Feb 2023

Recent text-to-image diffusion models such as MidJourney

and Stable Diffusion threaten to displace many in the pro-
fessional artist community. In particular, models can learn to
mimic the artistic style of specific artists after “fine-tuning”
on samples of their art. In this paper, we describe the design,
implementation and evaluation of Glaze, a tool that enables
artists to apply “style cloaks” to their art before sharing on-
line. These cloaks apply barely perceptible perturbations to
images, and when used as training data, mislead generative
models that try to mimic a specific artist. In coordination with Figure 1. Sample AI-generated art pieces from the Midjourney community
the professional artist community, we deploy user studies to showcase [55, 71].
more than 1000 artists, assessing their views of AI art, as
well as the efficacy of our tool, its usability and tolerability of
perturbations, and robustness across different scenarios and many have taken the open sourced StableDiffusion model,
against adaptive countermeasures. Both surveyed artists and and “fine-tuned” it on additional samples from specific artists,
empirical CLIP-based scores show that even at low perturba- allowing them to generate AI art that mimics the specific
tion levels (p=0.05), Glaze is highly successful at disrupting artistic styles of that artist [32]. In fact, entire platforms have
mimicry under normal conditions (>92%) and against adap- sprung up where home users are posting and sharing their own
tive countermeasures (>85%). customized diffusion models that specialize on mimicking
specific artists, likeness of celebrities, and NSFW themes [14].
1 Introduction Beyond open legal questions of copyrights, intellectual
property, and consent, it is clear that these AI models have
It is not an exaggeration to say that the arrival of text-to- had significant negative impacts on independent artists. For
image generator models has transformed, perhaps upended, the estimated hundreds of thousands of independent artists
the art industry. By sending simple text prompts like “A pic- across the globe, most work on commissions, and attract cus-
ture of a corgi on the moon” to diffusion models such as tomers by advertising and promoting samples of their artwork
StableDiffusion or MidJourney, anyone can generate incredi- online. First, professional artists undergo years of training to
bly detailed, high resolution artwork that previously required develop their individual artistic styles. A model that mimics
many hours of work by professional artists. AI-art such as this style profits from that training without compensating the
those in Figure 1 have won awards at established art conven- artist, effectively ending their ability to earn a living. Second,
tions [71], served as cover images for magazines [49, 90], and as synthetic art mimicry continues to grow for popular artists,
used to illustrate children’s books [61] and video games [79]. they displace original art in search results, further disrupting
More powerful models continue to arrive [34, 58, 83, 85], cat- the artist’s ability to advertise and promote work to potential
alyzed by VC funding [60, 97, 98], technical research break- customers [77]. Finally, these mimicry attacks are demoraliz-
throughs [4, 12, 39, 48, 54], and powered at their core by con- ing art students training to be future artists. Art students see
tinuous training on a large volume of man-made art scraped their future careers replaced by AI models even if they can
from online art repositories such as ArtStation, Pinterest and successfully find and develop their own artistic styles.
DeviantArt. Today, all of these consequences have indeed occurred in
Only months after their arrival, these models are rapidly the span of a few months. Art students are quitting the field;
growing in users and platforms. In September 2022, Mid- AI models that mimic specific artists are uploaded and shared
Journey reported over 2.7 million users and 275K AI art for free; and professional artists are losing their livelihoods
images generated each day [31]. Beyond simple prompts, to models mimicking their unique styles. Artists are fighting

1
back via lawsuits [23, 37], online boycotts and petitions [24], 2 Background: AI Art and Style Mimicry
but legal and regulatory action can take years, and are difficult
to enforce internationally. Thus most artists are faced a choice In this section, we provide critical context in the form of basic
to 1) do nothing, or 2) stop sharing samples of their art online background on current AI art models and style mimicry.
to avoid training models, and in doing so cripple their main
way to advertise and promote their work to customers. 2.1 Text-to-Image Generation
In this paper, we present the design, implementation and Since Text-to-image generation was first proposed in
evaluation of a technical alternative to protect artists against 2015 [52], a stream of research has proposed newer model
style mimicry by text-to-image diffusion models. We present architectures and training methods enabling generation of
Glaze, a system that allows an artist to apply carefully com- higher-quality images [19, 22, 36, 47, 63, 89, 100, 103, 106].
puted perturbations to their art, such that diffusion models The high level design of recent models used for AI art gen-
will learn significantly altered versions of their style, and be eration [18, 66, 69] is shown in Figure 3. During training,
ineffective in future attempts at style mimicry. We worked the model takes in an image x and uses a feature extractor
closely with members of the professional artist community to Φ to extract its features, producing Φ(x). Simultaneously, a
develop Glaze, and conduct multiple user studies with 1,156 conditional image generator G takes in a corresponding text
participants from the artist community to evaluate its efficacy, caption (s) and outputs a predicted feature vector G(s). Then
usability, and robustness against a variety of active counter- the parameters of G are optimized so the text feature vector
measures. G(s) matches the image feature vector Φ(x). At generation
Intuitively, Glaze works by taking a piece of artwork, and time, a user gives G a text prompt s0 , and G outputs an im-
computing a minimal perturbation (a “style cloak”) which, age feature vector G(s0 ). A decoder D then decodes G(s0 ) to
when applied, shifts the artwork’s representation in the gener- produce the final generated image.
ator model’s feature space towards a chosen target art style. Compared to earlier models based on generative adversarial
Training on multiple cloaked images teaches the generator networks (GANs) or variational autoencoders (VAE) [19, 22,
model to shift the artistic style it associates with the artist, 36, 63, 89, 106], more recent models [65, 69, 74] leveraging
leading to mimicry art that fails to match the artist’s true style. diffusion models produce significantly higher quality images.
Our work makes several key contributions: Feature extractor (Φ) is used to reduce the dimensionality
of the input image to facilitate the generation process. The
• We engage with top professional artists and the broader extractor Φ and decoder D are often a pair of variational
community, and conduct user studies to understand their autoencoder (VAE) [66, 69], i.e., extractor (encoder) extracts
views and concerns towards AI art and the impact on their image features and decoder map features back to images.
careers and community. Training Data Sources. The training datasets of these
• We propose Glaze, a system that protects artists from style models typically contain image/ALT text pairs scraped from
mimicry by adding minimal perturbations to their artwork to the Internet. They are extremely large, e.g. LAION [80] con-
mislead AI models to generate art different from the targeted tains 5 billion images collected from 3 billion webpages.
artist. 92% of surveyed artists find the perturbations small These datasets are subject to minimal curation and gov-
enough not to disrupt the value of their art. ernance. Data collectors typically only filter out data with
• Surveyed artists find that Glaze successfully disrupts style extremely short or incorrect text captions (based on an auto-
mimicry by AI models on protected artwork. 93% of artists mated text/image alignment metric [80]). Since copyrighted
rate the protection is successful under a variety of settings, images are not filtered [80], these datasets are rife with private,
including tests against real-world mimicry platforms. sensitive content, including copyrighted artworks.
• In challenging scenarios where an artist has already posted
significant artworks online, we show Glaze protection re- 2.2 Style Mimicry
mains high. 87.2% of surveyed artists rate the protection as
In a style mimicry attack, a bad actor uses an AI art model to
successful when an artist is only able to cloak 1/4 of their
create art in a particular artist’s style without their consent.
online art (75% of art is uncloaked).
More than 67% of art pieces showcased on a popular AI-art-
• We evaluate Glaze and show that it is robust (protection sharing website leverage style mimicry [55].
success > 85%) to a variety of adaptive countermeasures.
Style mimicry techniques. Today, a “mimic” can easily
copy the style of a victim artist with only an open-source text-
Ethics. Our user study was reviewed and approved by our to-image model and a few samples of artwork from the artist.
institutional review board (IRB). All art samples used in ex- A naive mimicry attack directly queries a generic text-to-
periments were used with explicit consent by their respective image model using the name of the victim artist. For example,
artists. All user study participants were compensated for their the prompt “a painting in the style of Greg Rutkowski” would
time, although many refused payment. cause the model to generate images in the style of Polish

2
 Model training

Feature extractor ground truth

(Φ) image features
training image similarity
loss
“a dog Generator generated
running” (G) image features
training prompt

Image generation
Original artwork Mimicked artwork “a dog Generator Image
by Hollie Mengert in Hollie’s style in space” (G) decoder
generation prompt
Figure 2. Real-world incident of AI plagiarizing the style of artist Hollie generated generated image
Mengert [3]. Left: original artwork by Hollie Mengert. Right: plagiarized image features
artwork generated by a model trained to mimic Hollie’s style. Figure 3. High level model architecture of text-to-image models.

artist Greg Rutkowski. This is because many of Rutkowski’s Artists have spoken out against style mimicry in nu-
artworks appear in training datasets of these generic models merous venues, focusing particularly on how it violates
labeled with his name. their intellectual property rights and threatens their liveli-
Naive mimicry can succeed when the artist is well-known hoods [15, 35, 41, 87, 92, 94, 95, 99]. Others have taken direct
and has a significant amount of art online, but fail on other action. The Concept Art Association raised over $200K to
artists. In more recent mimicry attacks, a mimic fine-tunes fight AI art [16], and filed a class action lawsuit in the US
a generic text-to-image model on samples of a target artist’s against AI art companies [37]. In November 2022, artists orga-
work (as few as 20 unique pieces) downloaded from online nized a large protest against ArtStation [94], the large digital
sources. This calibrates the model to the victim artist’s style, art sharing platform that allowed users to post AI artwork
identifying important features related to the victim style and without identification. Anti-AI images flooded the site for sev-
associating these regions in the feature space with the victim eral weeks, until ArtStation banned the protest images [26].
artist’s name [29, 72]. This enables style mimicry with im- Members of the professional art community reached out to
pressive accuracy. The entire fine-tuning process takes less us in Sept 2022. We joined online town halls and meetings
than 20 minutes on a low-end consumer GPU1 . alongside hundreds of professionals, including Emmy winners
Real-work mimicry incidents. The first well-known inci- and artists at major film studios. After learning more, we be-
dent of mimicry was when a Reddit user stole American artist gan an active collaboration with multiple professional artists,
Hollie Mengert’s style and open-sourced the style-specific including award-winning artist Karla Ortiz, who leads efforts
model on Reddit [3]. Figure 2 has a side-by-side comparison defending artists and is lead plaintiff in the class action suit.
of Hollie’s original artwork and plagiarized artwork generated The artists helped this project in multiple ways, by 1) sharing
via style mimicry. Later, famous cartoonist Sarah Andersen re- experiences about specific ways AI-art has impacted them and
ported that AI art models can mimic her cartoon drawings [2], their colleagues; 2) sharing domain knowledge about what is
and other similar incidents abound [56, 101]. acceptable to artists in terms of perturbations on their art; and
Several companies [67, 79] have even hosted style mimicry 3) helping to widely disseminate our user study to members
as a service, allowing users to upload a few art pieces painted of their professional organizations, including the Concept Art
by victim artists and producing new art in the victim styles. Association and the Animation Guild (TAG839).
CivitAI [14] built a large online marketplace where people Evaluation via Direct Feedback from Artists. Our goal
share their customized stable diffusion models, fine-tuned on is help artists disrupt AI models trying to mimic their artistic
certain artwork. style, without adversely impacting their own artwork. Because
“success” in this context is highly subjective (“Did this AI-art
3 Collaborating with Artists successfully mimic Karla’s painting style?”), we believe the
only reliable evaluation metric is direct feedback by profes-
Next, we explain our collaborative relationship with profes- sional artists themselves. Therefore, wherever possible, the
sional artists, and its significant impact on our key evaluation evaluation of Glaze is done via detailed user studies engaging
metrics in this paper. We also summarize key results from our members of the professional artist community, augmented
first user study on views of AI art and mimicry by members by an empirical score we develop based on genre prediction
of the artist community. using CLIP models.
We deployed two user studies during the course of this
1 It takes an average of 18.3 minutes on a GTX 1080 GPU project (see Table 1). Both are IRB-approved by our insti-

3
 Survey # of artists Content
1) Broad views of AI art and style mimicry(§3.1) Many artists (> 89% artists) have already or plan to take
Survey 1 1156 2) Glaze’s usability, i.e. acceptable levels of cloaking (§6.3)
3) Glaze performance in disrupting style mimicry (§6.3)
actions because of AI mimicry. Over 95% of artists post their
1) Additional performance tests (§6.3) artwork online. Out of these artists, 53% of them anticipate
Survey 2 2) Robustness to advanced scenarios (§6.4)
(Extension to Survey 1)
151
and countermeasures (§7) reducing or removing their online artwork, if they haven’t
3) Additional system evaluation (Appendix A) already. Out of these artists, 55% of them believe reducing
Table 1. Information on our user studies: the number of artist participants and their online presence will significantly impact their careers.
where we report the results of the studies. We sent Survey 2 to some specific
One participant stated “AI art has unmotivated myself from
participants from survey 1 who volunteered to participate in a followup study.
uploading more art and made me think about all the years
I spent learning art.” 78% of artists anticipate AI mimicry
tution. Both draw participants from professional artists in- would impact their job security, and this percentage increases
formed via their social circles and professional networks. to 94% for the job security of newer artists. Further, 24% of
The first (Survey 1, §3.1, §6.3), asked participants about their artists believe AI art has already impacted their job security,
broad views of AI style mimicry, and then presented them and an additional 53% expect to be affected within the next
with a number of inputs and outputs of our tool, and asked 3 years. Over 51% of artists expressed interest in proactive
them to give ratings corresponding to key metrics we wanted measures, such as personally joining class action lawsuits
to evaluate. We select a subset of participants from the first against AI companies.
study to participate in a longer and more in-depth study (Sur- Professional artists thought AI mimicry was very successful
vey 2) where they were asked to evaluate the performance of at mimicking the style of specific artists. We showed the
Glaze in additional settings (§6.3, §6.4, §7, and Appendix A). artists examples of original artwork from 23 artists, and the
artwork generated by a model attempting to mimic their styles
3.1 Artists’ Opinions on Style Mimicry (detailed mimicry setup in §6). 77% of artists found the AI
model successfully or very successfully mimic the styles of
While we expected artists to view style mimicry negatively, victim artists, with one stating “it’s shocking how well AI can
we wanted to better understand how much individual artists mimic the original artwork.” Additionally, 19% of participants
understood this topic and how many perceived it as a threat. thought the AI mimicry is somewhat successful, leaving only
Here we describe results from Survey 1 to gather perceptions < 5% of artists rating the mimicry as unsuccessful. Several
of the potential impact of AI art on existing artists. artists also pointed out that, as artists, upon close inspection
Survey Design. Our survey consisted of both multiple they could spot differences between the AI art and originals,
choice and free response questions to understand how well but were skeptical the general public would notice them.
people understand the concept of AI art, and how well the A significant concern of most participants, surprisingly, is
models successfully imitate the style of artists. Additionally, not just the existence of AI art, but rather scraping of existing
we asked artists about the extent to which they anticipate the artworks without permission or compensation. As one partic-
emergence of AI art to impact their artistic activities, such ipant stated: “If artists are paid to have their pieces be used
as posting their art online and their job security. A handful and asked permission, and if people had to pay to use that AI
of professional artists helped disseminate our survey to their software with those pieces in it, I would have no problem.”
respective artist community groups. Overall, we collected re- However, without consent to use their artwork to train the
sponses from 1,207 participants, consisting primarily of pro- models, “it’s incredibly disrespectful to the artist to have their
fessional artists (both full-time (46%) and part-time/freelancer work ‘eaten’ by a machine [after] many years to grow our
(50%)) and some non-artist members of the art community skills and develop our styles.”
who felt invested in the impact of AI art (4%). Of the par-
ticipants who consider themselves artists, their experience 4 Preliminaries
varied: <1 year (13%), 1-5 years (49%), 5-10 years (19%),
10+ years (19%). Participants’ primary art style varied widely, We propose Glaze, a tool that protects artists against AI style
including: animation, concept art, abstract, anime, game art, mimicry. An artist uses Glaze to add small digital perturba-
digital 2D/3D, illustration, character artwork, storyboarding, tions (“cloak”) to images of their own art before sharing on-
traditional painting/drawing, graphic design, and others. line (Figure 5). A text-to-image model that trains on cloaked
Key Results. Our study found that 91% of the artists have images of artwork will learn an incorrect representation of
read about AI art extensively, and either know of or worry the artist’s style in feature space i.e., the model’s internal
about their art being used to train the models. Artists expect understanding of artistic styles. When asked to generate art
AI mimicry to have a significant impact on artist community: pieces in victim’s style, the model will fail to mimic the style
97% artists state it will decrease some artists’ job security; of the victim, and instead output art pieces in a recognizably
88% artists state it will discourage new students from studying different style.
art; and 70% artists state it will diminish creativity. “Junior Here, we first introduce the threat model, then discuss exist-
positions will become extinct,” as stated by one participant. ing alternatives to the AI style mimicry problem. We present

4
 post scrape
online artwork
Style-specific
model
Victim artist generate
fine-tune
AI company model
train Artwork
...
mimicking
download victim’s style
Large dataset of text Text-to-image model Mimic’s actions
and image pairs model (generic)

Figure 4. High level overview of the mimicry attack scenario. The mimic scrapes copyrighted artwork from the victim artist and uses these to fine-tune a
pre-trained, generic text-to-image model. The generic model is trained and open-sourced by an AI company. The mimic then uses the fine-tuned model to
generate artwork in the style of the victim artist.

the intuition behind Glaze and detailed design in §5. 4.2 Potential Alternatives and Challenges
4.1 Threat Model A number of related prior works target protection against
invasive and unauthorized facial recognition models. They
Here we state assumptions for both the artists protecting their proposed “image cloaking” as a tool to prevent a user’s im-
own art and the users training models to replicate their artistic ages from being used to train a facial recognition model of
style. We refer to these AI art model trainers as “mimics.” them [10,13,27,81]. They share a similar high level approach,
Artists. Artists want to share and promote their artwork by using optimized perturbations that cause cloaked images to
online without allowing mimics to train models that repli- have drastically different feature representations from original
cate their art styles. Sharing art online enables artists to sell user images. It is possible to adapt existing cloaking-based
their work and attract commissioned work, fueling their liveli- systems to protect artists against AI style mimicry. Protection
hoods (§3). Artists protect themselves by adding impercep- system would compute a cloak on each artwork in order to
tible perturbations to their artwork before sharing them as perturb its feature space representation to be different from its
shown in Figure 5. The goal of the Glaze cloak is to disrupt unperturbed representation. This can succeed if the cloak sig-
the style mimicry process, while only introducing minimal nificantly shifts the artwork’s feature representation, making
perturbation on images of the artwork. resulting models generate dramatically different artwork.
We assume the artists have access to moderate comput- We found that in practice, however, existing solutions are
ing resources (e.g., a laptop) and add perturbation to images unable to introduce large-enough feature space shifts to
of their artwork locally before posting online. We also as- achieve the desired protection. This is due to the properties of
sume artists have access to some public feature extractor feature spaces in text-to-image models. Face recognition mod-
(e.g., open-source models such as Stable Diffusion). We begin els classify identities, so their feature spaces mainly represent
with assumption that artists use the same feature extractor as identity-related information. On the other hand, text-to-image
mimics (large majority of mimics use the open-source Stable models reconstruct original images from extracted features,
Diffusion model). We later relax this assumption. so their feature spaces retain more information about the
Mimics. The mimic’s goal is to train a text-to-image model original image (objects, locations, color, style, etc.). Thus,
that generates high-quality art pieces of any subject in the producing the same shift in feature representation in a text-
victim’s style. A mimic could be a well-funded AI company, to-image model is much harder (requires more perturbation
e.g., Stability AI or OpenAI, or an individual interested in the budget) than in a classification model. This observation is
style of victim artist. We assume the mimic has: validated by prior work showing that adversarial perturba-
tions are much less effective at attacking generative mod-
• access to the weights of generic text-to-image models well- els [30,42,68,88]. Specifically, [42,88] found that adversarial
trained on large datasets; attack methods that are effective at attacking classifiers are sig-
• access to art pieces from the target artist; nificantly less effective at attacking autoencoders. We empiri-
• significant computational power. cally confirm that existing cloaking methods cannot prevent
AI mimicry (§A.1 in Appendix). We show that Fawkes [81]
We assume the attack scenario where the mimic fine-tunes
and LowKey [13] perform poorly in this setting, even when
its model on images of the artist’s artwork (as shown in Fig-
artists add highly visible cloaks to their artwork.
ure 4). This is stronger than the naive mimic attack without
For generative models, concurrent work [78] proposes Pho-
fine tuning. Finally, we assume the mimic is aware of our pro-
toGuard, a method to cloak images to prevent unauthorized
tection tool and can deploy adaptive countermeasures (§7).

5
 Artist (V) Original artwork (originals) Cloaked artwork is similar to
the originals in input space
Original artwork Cloaked artwork Cloaked artwork

GLAZE
Feature extractor (Φ) GLAZE
Style transfer to
Target style (T) “oil painting by Van Gogh”
Optimizes cloaks for
original artwork
Style-transferred artwork (targets)
Mimic scrape artwork

fine-tune ge
generate
Cloaked artwork is similar
to targets in Φ’s feature space

a) Style transfer b) Cloak optimization

Style-specific Fails to mimic
Cloaked artwork
model victim artist Figure 6. High level overview of how Glaze perturbs the style-specific fea-
tures of the artwork. a) Glaze style transfers the original artwork to a different
Figure 5. Overview of Glaze, a system that protects victim artists from AI style, which changes its style but leaves other features unaltered. b) Glaze
style mimicry by cloaking their online artwork. (Top) An artist V applies optimizes a cloak that makes the artwork’s features representation match that
the cloaking algorithm (uses a feature extractor Φ and a target style T ) to of the style-transferred art, while constraining the amount of visible changes
generate cloaked versions of V ’s art pieces. Each cloak is a small perturbation to the artwork.
unnoticeable to human eye. (Bottom) A mimic scrapes the cloaked art pieces Style transferred artwork with target:
from online and uses them to fine-tune a model to mimic V ’s style. When Original artwork
“Oil painting” “Abstract painting” “Cubism painting”
prompted to generate artwork in the style of V , mimic’s model will generate
artwork in the target style T , rather than V ’s true style.

image edits (inpainting) on cloaked images. Similar to exist-

ing cloaking systems, PhotoGuard tries to indiscriminately
minimize all information contained in an image (i.e., the norm
of the feature vector) to prevent models from editing the
image. Thus, it is also not effective at mimicry prevention
(validated by empirical results in Appendix A.1). Figure 7. Example style-transferred artwork with different target styles.

Design Challenges. The main reason that existing cloak-

ing methods fail to prevent AI mimicry is because they indis- 5 Disrupting Style Mimicry with Glaze
criminately shift all features in an image, wasting the cloak
perturbation budget on shifting unnecessary features (e.g., In this section, we introduce Glaze, its design intuition fol-
object shape, location, etc.). Protecting artist’s style requires lowed by the detailed algorithm.
only shifting features related to the artistic style of victim.
This can be achieved if a text-to-image model learns to draw 5.1 Design Intuition
objects similar to those drawn by the victim artist as long
as the model cannot mimic the artist’s unique style. Thus, Our key intuition is to identify and isolate style-specific fea-
optimal protection from mimicry requires concentrating the tures of an artist’s original artwork, i.e., the set of image fea-
cloak on style-specific features. tures that correspond to artistic style. Then Glaze computes
cloaks while focusing the perturbation budget on these style-
specific features to maximize impact on stylistic features.
Unfortunately, identifying and separating out these style-
As discussed, identifying and calculating style-specific fea-
specific features is difficult. Even assuming the existence
tures in model’s feature space is difficult due to the poor
of interpretability methods that perfectly explain the feature
interpretability of model features and how art style manifests
space of a text-to-image model, there is no clear way to math-
differently across artworks. We overcome these two chal-
ematically define and calculate “artistic styles.” In all like-
lenges by designing a style-dependent and artwork-dependent
lihood, any definition would change across different styles.
method that operates at image space. Given an artwork, we
For example, “impressionist” likely correlates more strongly
leverage “style transfer,” an end-to-end computer vision tech-
with color features, whereas “cubism” correlates with shape
nique, to modify and isolate its style components. “Style
features. Even across multiple art pieces in the same style, the
transfer” transforms an image into a new image with a differ-
style may manifest differently.

6
ent style (e.g., from impressionist style to cubist style) while example, Fauvism and Impressionism are distinct art styles
keeping other aspects of the image similar (e.g., subject matter that often look visually similar to the untrained eye. Image
and location). of an impressionist painting style cloaked to Fauvism might
We leverage style transfer in our protection technique as not produce a visually discernible effect on model-generated
follows. Given an original artwork from the victim artist, we paintings. Note that an artist can maximize their ability to
apply style-transfer to produce a similar piece of art with a avoid mimicry if they consistently style cloak all their artwork
different style, e.g., in style of “an oil painting by Van Gogh” towards the same target T .
(Figure 6 a). The new version has similar content to the orig- For a new user, Glaze uses the following algorithm to ran-
inal, but its style mirrors that of Van Gogh. We show more domly select T from a set of candidate styles reasonably
style-transfer examples with different target styles in Figure 7. different from V ’s style. The algorithm first inspects a pub-
Now, we can use the style-transferred artwork as projection lic dataset of artists, each with a specific style (e.g., Monet,
target to guide the perturbation computation. This perturbs Van Gogh, Picasso). For each candidate target artist/style, it
the original artwork’s style-specific features towards that of selects a few images in that style and calculates their feature
the style-transferred version. We do this by optimizing a cloak space centroid using Φ. It also computes V ’s centroid in Φ
that, when added to the original artwork, makes its feature using V ’s artwork. Then, it locates the set of candidate styles
representation similar to the style-transferred image. Since whose centroid distance to V ’s centroid is between the 50 to
the content is identical between the pair of images, cloak op- 75 percentile of all candidates. Finally, it randomly selects T
timization will focus its perturbation budget on style features. from the candidate set.
Step 2: Style transfer. Glaze then leverages a pre-trained
5.2 Computing Style Cloaks style-transfer model Ω [69] to generate the style-transferred
Using this approach, we compute style cloaks to disrupt style artwork for optimization. Given each art piece x ∈ XV and
mimicry as follows. Given an artwork (x), we use an existing target style T , it style transfers x to target style T to produce
feature extractor to compute the style-transferred version of x style-transferred image Ω(x, T ).
into target style T : Ω(x, T ). We then compute a style cloak δx , Step 3: Compute cloak perturbation. Then, Glaze com-
such that δx moves x’s style-specific feature representation to putes the cloak perturbation, δx for x, following the optimiza-
match that of Ω(x, T ) while minimizing visual impact. The tion defined by eq. (1), subject to |δx | < p. Our implemen-
cloak generation optimization is: tation uses LPIPS (Learned Perceptual Image Patch Simi-
larity) [105] to bound the perturbation. Different from the
min Dist (Φ(x + δx ), Φ(Ω(x, T ))) , (1) L p distance used in previous work [9, 43, 73], LPIPS has
δx gained popularity as a measure of user-perceived image distor-
subject to |δx | < p, tion [13, 45, 70]. Bounding cloak generation with this metric
ensures that cloaked versions of images are visually similar
where Φ is a generic image feature extractor commonly used
to the originals. We apply the penalty method [57] to solve
in text-to-image generation tasks, Dist(.) computes the dis-
the optimization in eq.(1) as follows:
tance of two feature representations, |δx | measures the percep-
tual perturbation caused by cloaking, and p is the perceptual
perturbation budget. min ||Φ(Ω(x, T )), Φ(x + δx )||22 + α · max(LPIPS(δx ) − p, 0)
δx
As discussed in §5.1, the use of the style-transferred image (2)
Ω(x, T ) guides the cloak optimization in Eq (1) to focus on where α controls the impact of the input perturbation. L2
changing style-specific image features. To maximize cloak distance is used to calculate feature space distance.
efficacy, the target style T should be dissimilar from artist’s Upload artwork online. Finally, the artist posts the cloaked
original style in the feature space. We discuss our heuristic artwork online. For artists already with a large online presence,
for selecting target styles in §5. they can cloak and re-upload artwork on their online portfolio.
While updating online images is not always possible, Glaze
5.3 Detailed System Design can be effective even when the mimic’s model has significant
Now we present the detailed design of Glaze. Given a vic- amount of uncloaked art (§6.4).
tim artist V , Glaze takes as input the set of V ’s artwork to
be shared online XV , an image feature extractor Φ, an style- 5.4 On the Efficacy of Style Cloaks
transfer model Ω, and perturbation budget p. Note that in Glaze’s style cloaks work by shifting feature representation
many cases, a single model (e.g. Stable Diffusion) provides of artwork in the generator model. But how much shift do we
both Φ and Ω. need in order to have a noticeable impact on mimicked art?
Step 1: Choose Target Style. The selected target style T Two reasons suggest that even small shifts in style will
should be sufficiently different from V ’s style in model feature have a meaningful impact in disrupting style mimicry. First,
space to maximize chances of disrupting style mimicry. For generative models used for style mimicry have continuous

7
output spaces, i.e., any shift in image feature representation its genre (e.g., impressionism, cubism). We randomly sam-
results in changes in the generated image. Because genera- pled 30 art pieces from each artist to use in style mimicry
tive models are trained to interpolate their continuous feature attacks. Generic text-to-image models found online have
spaces [93, 96], any shift in the model’s representation of been trained on some artwork from these artists. Using this
art style results in a new style, a “blend” between the artist art simulates a more challenging scenario in which a famous
and the chosen target style. Second, mimicked artwork must artist attempts to disrupt a model that already understands
achieve reasonable quality and similarity in style to the artist their style.
to be useful. Small shifts in the style space often produce
Mimicry attack setup. We recreate the strongest-possible
incoherent blends of conflicting styles that are enough to dis-
mimicry attack scenario, based on techniques used in real-
rupt style mimicry, e.g., thick oil brushstrokes of Van Gogh’s
world mimicry incidents [3, 5, 25, 53, 72, 101], that works as
style mixed into a realism portrait.
follows. First, we take art pieces from the victim artist V and
These two factors contribute to Glaze’s success in more generate a text caption for each piece using an image caption-
challenging scenarios (§6.4), and its robustness against coun- ing model [51]. Then, we append the artist’s name to each
termeasures (e.g. adversarial training) that succeed against caption, e.g., “mountain range by Vincent van Gogh”. Some
cloaking tools for facial recognition (§7). example images and their captions are shown in Figure 16 in
Appendix. Finally, we fine-tune a pre-trained generic text-to-
6 Evaluation image model (details below) on the caption/image pairs.
We use 80% of the art pieces from the victim artists to fine-
In this section, we evaluate Glaze’s efficacy in protecting tune models that mimic each artist’s style, reserving the rest
artists from style mimicry. We first describe the datasets, for testing. We fine-tune for 3000 optimization steps, which
models, and experimental configurations used in our tests. we find achieves the best mimicry performance (Figure 17 in
Then we present the results of Glaze’s protection in a variety Appendix). We then use the fine-tuned, style-specific model
of settings. Due to Glaze’s highly visual nature, we evaluate its to generate mimicked artwork in style of each victim artist.
performance using both direct visual assessment by human We query the model using the generated captions (which
artists in a user study, and automated metrics (see §6.2 for include V ’s name) from the held-out test artwork set. We
details). generate 5 pieces of mimicked art for each text caption using
Summary of results. Over 93% of artists surveyed believe different random seeds and compare these to the real victim
Glaze effectively protects artists’ styles from AI style mimicry art pieces with this caption. Additional details on training and
attacks. Protection efficacy remains high in challenging set- generation parameters, as well as its sensitivity to random
tings, like when the mimic has access to unprotected artwork. seed selection and the number of training art pieces are in
Glaze also achieves high protection performance against a Appendix A.2.
real-world mimicry-as-a-service platform. Of our 1156 artist Text-to-image models. We use two state-of-the-art, public,
participants, over 92% found the perturbations introduced by generic text-to-image models in our experiments:
cloaking small enough not to disrupt the value of their art, • Stable Diffusion (SD): Stable Diffusion is a popular and high-
and over 88% would like to use Glaze to protect their own performing open-source text-to-image model [84],trained
artwork from mimicry attacks. on 11.5 million images from the LAION dataset [80]. SD
training takes over 277 GPU months (on A100 GPU) and
6.1 Experiment Setup costs around $600K [85]. SD uses diffusion methods to
generate images and achieves state-of-the-art performance
Mimicry dataset. We evaluate Glaze’s performance in
on several benchmarks [69]. Viewed as one of the best open-
protecting the styles of the following two groups of artists:
source models, SD has powered many recent developments
• Current artists: 4 professional artists let us use their art- in text-to-image generation [1, 7, 46, 58, 67, 79]. We use SD
work in our experiments. These artists have different version 2.1 in the paper [85], the most up-to-date version as
styles and backgrounds (e.g., full-time/freelancers, water- of December 2022.
color painters/digital artists, well-known/independent). Each • DALL·E-mega (DALL·E-m): DALL·E-m-mega, an updated
provided us with between 26 to 34 private original art pieces version of the more well-known DALL·E-m-mini, is an open-
for our experiments. We use perceptual hashing [40] to ver- source model based on OpenAI’s DALL·E-m 1 [66]. The
ify that none of these are included in existing public datasets model leverages a VAE for image generation and is trained
used to train generic text-to-image models (e.g. [11, 80]). on 17 million images from three different datasets [11, 82,
• Historical artists: We also evaluate Glaze’s protection on 91]. Training takes 2 months on 256 TPUs [17]. While
195 historical artists (e.g., van Gogh, Monet) from the DALL·E-m performs worse than diffusion-based models
WikiArt dataset [75]. The WikiArt dataset contains 42,129 like SD, we use it to evaluate how Glaze generalizes to
art pieces from 195 artists. Each art piece is labeled with different model architectures.

8
Glaze configuration. We generate cloaks for each of vic- w/o Glaze w/ Glaze (p=0.05)
Generic Artist
tim V ’s art pieces following the methodology of §5.3. First, model dataset
Artist-rated CLIP-based Artist-rated CLIP-based
PSR genre shift PSR genre shift
we use the target selection algorithm to select a target style
Current 4.6 ± 0.3% 2.4 ± 0.2% 94.3 ± 0.8% 96.4 + 0.5%
T . We choose from a set of 1119 candidate target styles, col- SD
Historical 4.2 ± 0.2% 1.3 ± 0.2% 93.3 + 0.6% 96.0 + 0.3%
lected by querying the WikiArt dataset with artist and genre Current 31.9 ± 3.5% 6.4 ± 0.8% 97.4 ± 0.2% 97.4 + 0.3%
DALL·E-m
names, e.g., “Impressionism painting by Monet” 2 . We then Historical 29.8 ± 2.4% 5.8 ± 0.6% 96.8 ± 0.3% 97.1 + 0.2%
style transfer each victim art piece into the target style lever- Table 2. Glaze has a high protection success rate, as measured by artists and
aging the style transfer functionality of stable diffusion model CLIP, against style mimicry attacks. We compare protection success when
(stable diffusion model has both text-to-image and style trans- artists do not use Glaze vs. when they do (with perturbation budget 0.05).
fer functionality). We test the sensitivity of our protection
results to the choice of style-transfer model in Appendix A.2. we define CLIP-based genre shift rate as the percentage of
Finally, we optimize a cloak for each art piece using Eq. 2 by mimicked art whose top 3 predicted genres do not contain V ’s
running the Adam optimizer for 500 steps. It takes an average original genre. A higher genre shift rate means more mim-
of 1.2 mins on Titan RTX GPU and 7.3 mins on a single Intel icked art belongs to a different genre from the victim artist,
i7 CPU to generate a cloak for a single piece of art. and thus means more successful protection.
In our initial experiments, we assume Glaze generates To calculate the genre shift we use a set of 27 historical
cloaks using the same image feature extractor as the mimic genres from WikiArt dataset and 13 digital art genres [33] as
(e.g. SD’s or DALL·E-m’s feature extractor). We relax this the candidate output labels. In Appendix A.3, we show that a
assumption and evaluate Glaze’s performance when artists pre-trained CLIP model is able to achieve high genre classifi-
and mimics use different feature extractors in §6.4. cation performance. We report the average CLIP-based genre
shift for all 199 victim artists across all mimicked artworks.
6.2 Evaluation Metrics We use CLIP-based genre shift as a supplemental metric to
evaluate Glaze because it is only able to detect style changes
We evaluate our protection performance using both visual
at the granularity of art genres. However, mimicry attacks also
assessment and feedback from human artists, and a scalable
fail when Glaze causes the mimicked artwork quality to be
metric. Here, we describe the setup of our evaluation study
very low, something that CLIP cannot measure. Measuring
and define the exact metrics used for evaluation.
the quality of generated image has been a challenging and
Artist-rated protection success rate (Artist-rated PSR): ongoing research problem in computer vision [6, 38, 44].
The user studies ask artists to rate the performance of Glaze.
We generate a dataset of mimicry attacks on 13 victim artists
6.3 Glaze’s Protection Performance
(the 4 current artists and 9 randomly chosen historical artists)
across 23 protection scenarios (including ones in §7). For Style mimicry success when Glaze is not used. Mimicry
each participant, we randomly select a set of mimicry attacks attacks are very successful when the mimic has access to a
out of these 13 × 23 settings and ask them to evaluate protec- victim’s original (unmodified) artwork. Examples of mim-
tion success. For each mimicry attempt, we show participants icked artwork can be found in Figure 8 (more in Figure 23
4 mimicked art pieces and 4 original art pieces from the victim in Appendix). The leftmost two columns of Figure 8 show a
artist. We ask participants to rate the success of Glaze’s pro- victim artist’s original artwork, while the third column depicts
tection on a 5-level Likert scale (ranging from “not successful mimicked artwork generated by a style-specific model trained
at all” to “very successful”). Each mimicry attempt is evalu- on victim’s original artwork when Glaze is not used. In our
ated by at least 10 participants. We define artist-rated PSR user study, over > 95% of respondents rated the attack as suc-
as the percent of participants who rated Glaze’s protection as cessful. Table 2, row 1, gives the artist-rated and CLIP-based
“successful” or “very successful.” Our user studies primarily genre shift for mimicry attacks on unprotected art.
focus on artists, as they would be most affected by this tech- SD models produce stronger mimicry attacks than DALL·E-
nology. We found though, that not all current artists despise m models, according to our user study (see Table 2). This is
AI art, and some view it as a new avenue for a different form unsurprising, as DALL·E-m models generally produce lower-
of artistry. quality generated images. CLIP-based genre shift does not
CLIP-based genre shift: We define a new metric based reflect this phenomenon, as this metric does not assess image
on CLIP [62], using the intuition that Glaze succeeds if the quality.
mimicked art has been impacted enough by Glaze to be classi- Glaze’s success at preventing style mimicry. Glaze
fied into a different art genre from the artist’s original artwork. makes mimicry attacks markedly less successful, as shown
We leverage CLIP model’s ability to classify art images into in Figure 8. Columns 5 and 6 (from left) show mimicked
art genres. Given a set of mimicked art targeting an artist V , artwork when the style-specific models are trained on artwork
2 One artist may paint in multiple styles, resulting in multiple candidate protected by Glaze. For reference, column 4 shows an exam-
target styles from a single artist. ple style-transferred artwork Ω(x, T ) used to compute Glaze

9
 Mimicked art GLAZE target Mimicked art
Original artwork when GLAZE not used style when GLAZE is used

Artist A
(Karla Ortiz)
Oil painting
by Van Gogh

Artist B
(Nathan Fowkes)
Abstract expressionism
by Norman Bluhm

Artist C
(Claude Monet)

Cubism by Picasso

p = 0.05 p = 0.1
Glaze perturbation size
Figure 8. Example Glaze protection results for three artists. Columns 1-2: artist’s original artwork; column 3: mimicked artwork when artist does not use
protection; column 4: style-transferred artwork (original artwork in column 1 is the source) used for cloak optimization and the name of target style; column
5-6: mimicked artwork when artist uses cloaking protection with perturbation budget p = 0.05 or p = 0.1 respectively. All mimicry examples here use SD-based
models.

Success of cloaking protection

Willingness to post cloaked artwork
No protection
p=0.03
Perturbation Artist-rated CLIP-based
Cloaked, p=0.05
budget PSR genre shift p=0.05
Cloaked, p=0.1 0 (no cloak) 4.6 ± 1.4% 2.4 ± 0.8% p=0.1
0.05 93.3 ± 0.6% 96.0 ± 0.3%
Cloaked, p=0.2
0.1 95.9 ± 0.4% 98.2 ± 0.1% p=0.2

0% 20% 40% 60% 80% 100% 0.2 96.1 ± 0.3% 98.5 ± 0.1%
0% 20% 40% 60% 80% 100%
not successful at all not very successful very unwilling somewhat unwilling
somewhat successful successful neutral somewhat willing
very successful
Table 3. Performance of our system (artist-rated very willing
Figure 9. Glaze’s cloaking protection success in- protection success rate and CLIP-based genre shift
Figure 10. Artists’ willingness to post cloaked art-
creases as cloak perturbation budget increases. The rate) increases as the perturbation budget increases.
work in place of the original decreases as perturba-
top row of the figure shows baseline performance (SD model, averaged over all victim artists).
tion budget of the cloaks increases.
with the mimic trains on uncloaked images (p=0).

cloaks for the protected art pieces. Overall, Glaze achieves to use Glaze.
> 93.3% artist-rated PSR and > 96.0% CLIP-based genre We find that artists are willing to add fairly large Glaze per-
shift (see Table 2). Glaze’s protection performance is slightly turbations to their artwork in exchange for protection against
higher for current artists than for historical artists. This is mimicry. To measure this, we show 3 randomly chosen pairs
likely because the historical artists’ images are present in of original/cloaked artwork to each of the 1,156 artists in our
the training datasets of our generic models (SD, DALL·E- first study. For each art pair, we ask the artist whether they
m), highlighting the additional challenge of protecting well- would be willing to post the cloaked artwork (instead of the
known artists whose style was already learned by the generic original, unmodified version) on their personal website. More
models. than 92% of artists select “willing” or “very willing” when
How large of perturbations will artists tolerate? Increas- p = 0.05. This number only slightly increases to 94.3% when
ing the Glaze perturbation budget enhances protection per- p = 0.03. Figure 10 details artists’ preferences as perturba-
formance. We observe that both artist-rated and CLIP-based tion budget increases. (see Figure 11 for examples of cloaked
genre shift increase with perturbation budget (see Figure 9, Ta- artwork with increasing p). Based on these results, we use
ble 3, and Figure 18). Given this tradeoff between protection perturbation budget p = 0.05 for all our experiments, since
success and Glaze protection visibility on original artwork, most artists are willing to tolerate this perturbation size.
we evaluate how perturbation size impacts artists’ willingness Surprisingly, over 32.8% artists are willing to use cloaks

10
 Original p = 0.05 p = 0.1 p = 0.2 Artist w/o Glaze w/ Glaze (p=0.05)
dataset Artist-rated CLIP-based Artist-rated CLIP-based
PSR genre shift PSR genre shift
Current 6.2 ± 0.5% 3.8 ± 0.3% 92.5 ± 0.5% 94.2 + 0.3%
Historical 7.2 ± 0.6% 3.3 ± 0.4% 92.1 + 0.3% 93.9 + 0.4%
Table 4. Performance of Glaze against real-world mimicry service (sce-
nario.gg). Mimicry service achieves high mimicry success when no protec-
tion is used. When Glaze is used, the mimicry service has low performance.

Glaze’s protection remains highly successful (left half of Fig-

ure 12)—the style of mimicked artwork remains distinct from
Figure 11. Original artwork and cloaked artwork computed using three dif-
ferent cloak perturbation budgets. artist’s true style. Artist-rated and CLIP-based genre shift
measurements confirm this observation. Artist-rated PSR is
> 90.2%, while CLIP-based genre shift is > 94.0%. The PSR
with p = 0.2, which is clearly visible to human eye (see Fig- is slightly higher when the two feature extractors only differ
ure 11, high resolution version in appendix Figure 20). While in architectures (Φ-B to Φ-A) than when they differ in both
we are surprised by this high perturbation tolerance, in our architecture and training data (Φ-C to Φ-A).
follow-up free response artists noted that they would be will- Mimic has access to uncloaked artwork. Another chal-
ing to tolerate large perturbations because of the devastating lenging scenario is when the mimic gains access to some
consequence if their styles are stolen. One participant stated uncloaked artwork from victim artists. This is a realistic sce-
that “I am willing to sacrifice a bit image quality for protec- nario for many prominent artists with a large online presence.
tion.” Many artists (> 80%) also noted that they have already As expected, Glaze’s protection performance decreases when
used traditional, more visually disruptive techniques to protect the mimic has access to more uncloaked artwork (right side
their artwork online when posting online, i.e., adding water- of Figure 12). As the ratio of uncloaked/cloaked art in the
mark or reducing image resolution. One participant stated mimic’s dataset increases, the mimicked artwork becomes
that “I already use low to medium resolution images only for more similar to artist’s original style. Yet, Glaze is still rea-
online posting, thus this would not impact my quality control sonably effective (87.2% artist-rated PSR) even when artists
too much.” can only cloak 25% of their artwork. This validates our hy-
pothesis in §5.4 that cloaking will have a noticeable effect as
6.4 Glaze’s Protection Robustness long as the mimic has some cloaked training data.
Next, we test Glaze’s efficacy in more challenging scenarios. A mimic with access to a large amount of uncloaked art-
First, we measure performance when the mimic uses a dif- work is still an issue for Glaze. Fortunately, in our user study,
ferent feature extractor for mimicry than the one used by the we found that 1) many artists constantly create and share new
artist to generate the cloak. Second, we measure what hap- artwork online, which can be cloaked to offset the percent-
pens when the mimic has uncloaked artwork samples from the age of uncloaked artwork, and 2) many artists change their
victim. Due to the poor mimicry performance of DALL·E-m, artistic style over time. In our user study, we asked artists
we focus our evaluation using SD as the generic model. to estimate the number of unique art pieces they currently
have online (M) and the estimated number of art pieces they
Artist/mimic use different feature extractors. In the real
anticipate uploading each subsequent year (Y ). Among artists
world, it is possible that the mimic will use a different model
with an existing online presence, over 40% have Y /M > 25%,
(and thus a different image feature extractor) for style mimicry
meaning that one year from now, > 20% of their total online
than the one used by the victim artist to cloak their artwork.
artwork would be cloaked (if they start using Glaze immedi-
While the feature extractors may still be similar because of
ately). More than 81% of artists also stated that their art style
the well-known transferability property between large mod-
has changed over their career, and half of these said that theft
els [20, 59, 81, 86, 102], their differences could reduce the
of their old, outdated styles is less concerning.
efficacy of cloaking. We test this scenario using three feature
extractors—Φ-A, Φ-B, and Φ-C. Φ-A and Φ-B have different
6.5 Real-World Performance
model architectures (autoencoder-KL [69] vs. VQ-VAE [66])
but are both trained on the ImageNet dataset [21]. Φ-A and Next, we test Glaze against a real-world style mimicry-as-
Φ-C have different model architectures (autoencoder-KL vs a-service system, scenario.gg [79]. Scenario.gg is a web
VQ-VAE) and training datasets (ImageNet vs. CelebA [50]). service that allows users to upload a set of images in a specific
In our experiments, the victim artist uses one feature ex- style. The service then trains a model to mimic the style
tractor (either Φ-B or Φ-C) to optimize cloaked artwork, and and returns an API endpoint that allows the user to generate
the mimic trains their style-specific models with SD models mimicked images in the trained style. The type of model or
using Φ-A. Despite the difference in victim/mimic extractors, mimicry method used by the service is unknown.

11
 Feature extractors used by artist and mimic Percentage of artwork cloaked
Artist: no cloaking Artist: Φ-A Artist: Φ-B Artist: Φ-C
0% cloaked 25% cloaked 50% cloaked 75% cloaked
Mimic: Φ-A Mimic: Φ-A Mimic: Φ-A Mimic: Φ-A

Attempts to
mimic artist A

Attempts to
mimic artist B

Artist-rated PSR 4.3 ± 0.2% 93.5 + 0.6% 91.3 + 0.5% 90.2 + 0.8% 4.3 ± 0.2% 87.2 ± 1.1% 90.1 ± 0.8% 91.5 ± 0.9%

CLIP-based genre shift 1.4 ± 0.2% 96.0 ± 0.3% 94.8 ± 0.4% 94.0 ± 0.4% 1.4 ± 0.2% 90.3 ± 0.8% 93.8 ± 0.4% 94.7 ± 0.3%

Figure 12. Glaze remains successful under two challenging scenarios. Left: when artist and mimic use different feature extractors. Right: when artists can only
cloak a portion of their artwork in mimic’s dataset. Bottom of the figure shows artist-rated PSR and CLIP-based genre shift for the corresponding setting.

Gaussian noise level JPEG compression level

Denoised Upscaled
σ = 0.05 σ = 0.1 σ = 0.15 20 15 10

Attempts to Attempts to
mimic artist A mimic artist A

Attempts to Attempts to
mimic artist B mimic artist B

Artist-rated PSR 92.9 ± 0.5% 91.2 ± 0.7% 91.6 ± 0.5% 89.3 ± 1.2% Artist-rated PSR 93.4 ± 0.8% 92.3 ± 0.6% 87.4 ± 0.9% 85.3 ± 1.3%

Figure 13. Glaze’s protection performance remains high as mimic adds an Figure 14. Glaze’s protection performance remains high as mimic adds JPEG
increasing amount of Gaussian noise to the cloaked artwork. Even when the compression to the cloaked artwork. Even when the mimic also upscales the
mimic adds denoising (last column), Glaze’s protection persists. mimicked images (last column), Glaze’s protection persists.

Glaze remains effective against scenario.gg. We ask efficacy of each countermeasure on the 13 victim artists from
scenario.gg to mimic the style from a set of cloaked or un- §6.2. Here, we focus on artist-rated PSR metric, because many
cloaked artwork from 4 current artists and 19 historical artists. countermeasures trade off image quality for mimicry efficacy,
Table 4 shows that when no protection is used, scenario.gg and CLIP-based metric does not consider image quality.
can successfully mimic the victim style (< 7.2% protection Image transformation. A popular approach to mitigate
success). The mimicry success of scenario.gg is lower the impact of small image perturbations, like those intro-
than our mimicry technique, likely because scenario.gg duced by Glaze, is to transform training images before using
trains the model for fewer iterations due to computational them for model training [8, 28]. In our setting, the mimic
constraints. When we use Glaze to cloak the artwork and could augment the cloaked artwork before fine-tuning their
upload the cloaked artwork, scenario.gg fails to mimic the model on them to potentially reduce cloak efficacy. We first
victim style (> 92.1% artist-rated PSR and > 93.9% CLIP- test Glaze’s resistance to two popular image transformations,
based genre shift rate) as shown in Table 4. adding Gaussian noise and image compression. We also con-
sider a stronger version of this countermeasure that then tries
7 Countermeasures to correct the image quality degradation introduced by the
transformations.
We consider potential countermeasures a mimic could employ Transforming cloaked artwork does not defeat Glaze’s pro-
to reduce the effectiveness of Glaze. We consider the strongest tection. Figure 13 shows that as the magnitude of Gaussian
adversarial setting, in which the mimic has white-box access noise (σ) increases, the quality of mimicked artwork decreases
to our protection system, i.e., access to the feature extractor as fast as or faster than cloak effectiveness. This is because
used and protection algorithm. In our experiments, we assume models trained on noisy images learn to generate noisy im-
the mimic uses the SD model as the generic model and test the ages. We observe a similar outcome when mimic uses JPEG

12
 Number of robust training steps ages from the robust feature space. Finally, the mimic uses the
1K steps 3K steps 5K steps 10K steps robust generic model for style mimicry as in §6. We discuss
the detailed robust training setup in Appendix A.4.
Attempts to
Glaze performance remains high, even if the mimic ro-
mimic artist A bustly trains the generic model for many iterations before
using it for style mimicry (see Figure 15). As the model be-
comes more robust, the mimicked artwork is less impacted by
Attempts to cloaking (less influenced of the target style). However, robust
mimic artist B
training greatly degrades mimicked image quality, prevent-
ing successful mimicry. Overall, the artist-rated PSR remains
Artist-rated PSR 92.2 ± 0.8% 89.3 ± 1.3% 91.3 ± 0.9% 95.3 ± 0.3% > 88.7%. To mitigate robust training’s impact on image qual-
ity, we explore an alternative robust training method, where
Figure 15. Glaze’s protection performance remains high against robust train- we robustly train a new feature extractor designed to remove
ing countermeasure proposed by Radiya et al. . The protection performance cloak’s impact while operating in the original feature space
first decreases then increases as mimic robustly trains the model with an
increasing number of steps. (thus no need to change the image generator). We found this
robust training approach is also ineffective (details in §A.4).
As discussed in §5.4, Glaze remains reasonably effective
compression (Figure 14), where image resolution and quality against Radiya et al. because 1) the continuous output space
degrade due to heavy compression. Artists-rated PSR de- of the generative model, and 2) high quality requirement of art
creases slightly but remains above > 87.4% across both types generation. Robust training reduces cloaking’s effectiveness
of data transformations. Artists consider Glaze’s protection but cannot completely remove its impact. In the classification
to be successful when mimicked artwork is of poor quality. case (facial recognition), this reduced effectiveness only man-
The mimic can take this countermeasure one step further ifests in small changes in classification confidence (compared
by reversing the quality degradation introduced by the nois- to no cloaking) and often does not change the discrete classifi-
ing/compression process. Specifically, a mimic can run image cation outcome. However, in the context of generator models,
denoising or image upscaling tools on the mimicked artwork the continuous output space means that even less-effective
(e.g., ones shown in Figure 13 and 14) to increase their quality. cloaks still directly affect the mimicked artwork. Combined
We found this approach improves generated image quality with the high quality requirement, the reduced protection ef-
but still does not allow for successful mimicry. For denoising, fect is enough to disrupt style mimicry, as shown in Figure 15.
we ran a state-of-the-art CNN-based image denoiser [104] Additional robust training simply degrades generation quality,
that is specifically trained to remove “additive Gaussian noise” rather than reducing cloaking efficacy.
(the same type of noise added to cloaked artwork). The last
column of Figure 13 shows the denoised image (using the
noisy mimicked image when σ = 0.2 as the input). While the 8 Discussion
process removes significant amounts of noise, the denoised
artwork still has many artifacts, especially around complex Here we conclude with a discussion of the limitations of the
areas of the artwork (e.g., human face). We observe similar current system, and outline plans for ongoing work.
results for image upscaling, where we use a diffusion-based Limitations. The first limitation of our approach is that
image upscaler [83] to improve the quality of compressed im- protection relies on artists cloaking a portion of their art in the
ages (Figure 14). Overall, our artist-rated protection success mimic model’s training dataset. This is particularly challeng-
rate remains > 85.3% against this improved countermeasure. ing for established artists because 1) their styles have matured
Radiya et al. [64] robust training. Radiya et al. [64] over the years and are more stable, and 2) many of their art
design a robust training method to defeat cloaking tools like pieces have already been downloaded from art repositories
Fawkes [81] and Lowkey [13] in the face recognition setting. like ArtStation and DeviantArt. Someone can simply mimic
At a high level, this method augments the attacker’s training these artists’ style using only old artwork collected before
dataset with some cloaked images generated by the cloaking the release of Glaze. While it is a win for the artists in terms
tool and the correct output labels. Training on such data makes of preventing mimic from training on newer artwork, these
the model more robust against cloak perturbations on unseen artists must rely on data collectors providing an opt-out and
cloaked images at inference time, and thus, can potentially removal option in order to stop style mimicry.
circumvent the protection. Second, a system like Glaze that protects artists faces an
We test if this robust training approach can defeat Glaze. inherent challenge of being future-proof. Any technique we
We assume the mimic first robustly trains the feature extrac- use to cloak artworks today might be overcome by a future
tors in their generic models using cloaked artwork generated countermeasure, possibly rendering previously protected art
by Glaze, and then trains the generator model to generate im- vulnerable. While we are under no illusion that Glaze will

13
remain future-proof in the long run, we believe it is an impor- [15] C LARKE , L. When AI can make art – what does it mean for creativ-
tant and necessary first step towards artist-centric protection ity?, 2022. https://www.theguardian.com/technology/2022/
nov/12/when-ai-can-make-art-what-does-it-mean-for-c
tools to resist invasive AI mimicry. We hope that Glaze and reativity-dall-e-midjourney.
followup projects will provide some protection to artists while
[16] C ONCEPT A RT A SSOCIATION. Protecting Artists from AI Technolo-
longer term (legal, regulatory) efforts take hold. gies, 2022. https://www.gofundme.com/f/protecting-artis
Ongoing Work. We are actively developing Windows and ts-from-ai-technologies.
Mac versions of Glaze for use for members of the artist com- [17] DAYMA , B. DALLE Mega - Training Journal, 2022. https://wand
b.ai/dalle-mini/dalle-mini/reports/DALL-E-Mega-Train
munity. We also recognize that a significant portion of the
ing-Journal--VmlldzoxODMxMDI2.
artist community may lack access to these tools because of
[18] DAYMA , B., PATIL , S., C UENCA , P., S AIFULLAH , K., A BRAHAM ,
lack of awareness or computational resources. We are ac- T., L E K HAC , P., M ELAS , L., AND G HOSH , R. Dalle mini, 2021.
tively exploring approaches for more proactive art mimicry [19] DAYMA , B., PATIL , S., C UENCA , P., S AIFULLAH , K., A BRAHAM ,
protection for groups of artists that do not require individual T., M ELAS , L., AND G HOSH , R. Dall·e mini, 7 2021.
effort. [20] D EMONTIS , A., M ELIS , M., P INTOR , M., JAGIELSKI , M., B IGGIO ,
B., O PREA , A., N ITA -ROTARU , C., AND ROLI , F. Why do adversar-
ial attacks transfer? explaining transferability of evasion and poisoning
References attacks. In Proc. of USENIX Security (2019), pp. 321–338.
[21] D ENG , J., D ONG , W., S OCHER , R., L I , L.-J., L I , K., AND F EI -F EI ,
[1] AI R ENDER. AI Render - Stable Diffusion in Blender. , 2022. https:
L. Imagenet: A large-scale hierarchical image database. In Proc. of
//blendermarket.com/products/ai-render.
CVPR (2009), IEEE, pp. 248–255.
[2] A NDERSEN , S. The Alt-Right Manipulated My Comic. Then A.I. [22] D ING , M., YANG , Z., H ONG , W., Z HENG , W., Z HOU , C., Y IN , D.,
Claimed It., 2022. https://weirdwonderfulai.art/resources L IN , J., Z OU , X., S HAO , Z., YANG , H., ET AL . Cogview: Mastering
/midjourney-style-chart-by-robomar-ai-art/. text-to-image generation via transformers. Proc. of NeurIPS 34 (2021),
[3] BAIO, A. Invasive Diffusion: How one unwilling illustrator found 19822–19835.
herself turned into an AI model, 2022. https://waxy.org/2022/ [23] D IXIT, P. Meet the three artists behind a landmark lawsuit against AI
11/invasive-diffusion-how-one-unwilling-illustrator art generators. BuzzFeedNews, January 2023.
-found-herself-turned-into-an-ai-model/. [24] E DWARDS , B. Artists stage mass protest against AI-generated artwork
[4] BALAJI , Y., NAH , S., H UANG , X., VAHDAT, A., S ONG , J., K REIS , on artstation. Ars Technica, December 2022.
K., A ITTALA , M., A ILA , T., L AINE , S., C ATANZARO , B., ET AL . [25] E D XD. How to Use DreamBooth to Fine-Tune Stable Diffusion
ediffi: Text-to-image diffusion models with an ensemble of expert (Colab), 2022. https://bytexd.com/how-to-use-dreamboot
denoisers. arXiv preprint arXiv:2211.01324 (2022). h-to-fine-tune-stable-diffusion-colab/.
[5] B ERNUY, F. The guide to fine-tuning Stable Diffusion with your own [26] E LIAÇIK , E. Does ArtStation become PromptStation?, 2022. https:
images, 2022. https://tryolabs.com/blog/2022/10/25/the-g //dataconomy.com/2022/12/no-to-ai-generated-images-a
uide-to-fine-tuning-stable-diffusion-with-your-own-i rtstation/.
mages. [27] E VTIMOV, I., S TURMFELS , P., AND KOHNO , T. Foggysight: A
[6] B LAU , Y., AND M ICHAELI , T. The perception-distortion tradeoff. In scheme for facial lookup privacy. arXiv preprint arXiv:2012.08588
Proc. of CVPR (2018), pp. 6228–6237. (2020).
[28] F EINMAN , R., C URTIN , R. R., S HINTRE , S., AND G ARDNER , A. B.
[7] C ANTRELL , C. Stability Photoshop plugin (beta). , 2022. https://
Detecting adversarial samples from artifacts. arXiv:1703.00410
exchange.adobe.com/apps/cc/114117da/stable-diffusion.
(2017).
[8] C ARLINI , N., AND WAGNER , D. Adversarial examples are not easily [29] G AL , R., A LALUF, Y., ATZMON , Y., PATASHNIK , O., B ERMANO ,
detected: Bypassing ten detection methods. In Proc. of AISec (2017), A. H., C HECHIK , G., AND C OHEN -O R , D. An image is worth one
pp. 3–14. word: Personalizing text-to-image generation using textual inversion.
[9] C ARLINI , N., AND WAGNER , D. Towards evaluating the robustness arXiv preprint arXiv:2208.01618 (2022).
of neural networks. In Proc. of IEEE S&P (2017), IEEE, pp. 39–57. [30] G ONDIM -R IBEIRO , G., TABACOF, P., AND VALLE , E. Adversarial
attacks on variational autoencoders. arXiv preprint arXiv:1806.04646
[10] C HANDRASEKARAN , V., G AO , C., TANG , B., FAWAZ , K., J HA , S.,
(2018).
AND BANERJEE , S. Face-off: Adversarial face obfuscation. arXiv
preprint arXiv:2003.08861 (2020). [31] H EIDORN , C. Mind-Boggling Midjourney Statistics in 2022, 2022.
https://tokenizedhq.com/midjourney-statistics/.
[11] C HANGPINYO , S., S HARMA , P., D ING , N., AND S ORICUT, R. Con-
ceptual 12m: Pushing web-scale image-text pre-training to recognize [32] H EIKKILA , M. This artist is dominating ai-generated art. and he’s
long-tail visual concepts. In Proc. of CVPR (2021), pp. 3558–3568. not happy about it. MIT Technology Review, Sept 2022.
[33] H OARE , A. Digital Illustration Styles, 2021. https://www.theill
[12] C HEN , W., H U , H., S AHARIA , C., AND C OHEN , W. W. Re-
ustrators.com.au/digital-illustration-styles.
imagen: Retrieval-augmented text-to-image generator. arXiv preprint
arXiv:2209.14491 (2022). [34] I VANENKO , N. Midjourney v4: an incredible new version of the AI
image generator, 2022. https://mezha.media/en/2022/11/11/
[13] C HEREPANOVA , V., G OLDBLUM , M., F OLEY, H., D UAN , S., D ICK - midjourney-v4-is-an-incredible-new-version-of-the-a
ERSON , J., TAYLOR , G., AND G OLDSTEIN , T. Lowkey: Leveraging i-image-generator/.
adversarial attacks to protect social media users from facial recogni-
[35] JAVAID , M. The Magic Avatar you paid $3.99 for is probably stolen,
tion. arXiv preprint arXiv:2101.07922 (2021).
artists say, 2022. https://www.washingtonpost.com/technolog
[14] C IVITAI. What the heck is Civitai?, 2022. https://civitai.com/ y/2022/12/09/lensa-apps-magic-avatars-ai-stolen-dat
content/guides/what-is-civitai. a-compromised-ethics/.

14
[36] J IA , C., YANG , Y., X IA , Y., C HEN , Y.-T., PAREKH , Z., P HAM , H., [55] M IDJOURNEY. Community Showcase, 2022. https://www.midj
L E , Q., S UNG , Y.-H., L I , Z., AND D UERIG , T. Scaling up visual and ourney.com/showcase/top/.
vision-language representation learning with noisy text supervision. [56] MURPHY, B. P. Is Lensa AI Stealing From Human Art? An Expert
In Proc. of ICML (2021), PMLR, pp. 4904–4916. Explains The Controversy, 2022. https://www.sciencealert.c
[37] J OSEPH S AVERI L AW F IRM LLP. Class Action Filed Against Sta- om/is-lensa-ai-stealing-from-human-art-an-expert-exp
bility AI, Midjourney, and DeviantArt for DMCA Violations, Right lains-the-controversy.
of Publicity Violations, Unlawful Competition, Breach of TOS, 2023.
[57] N OCEDAL , J., AND W RIGHT, S. Numerical optimization, series in
https://cybernews.com/news/artists-unite-in-legal-bat
operations research and financial engineering. Springer, New York,
tle-against-ai/.
USA, 2006 (2006).
[38] K ARRAS , T., A ITTALA , M., H ELLSTEN , J., L AINE , S., L EHTINEN ,
[58] N OVEL AI. NovelAI changelog, 2022. https://novelai.net/up
J., AND A ILA , T. Training generative adversarial networks with
dates.
limited data. Proc. of NeurIPS 33 (2020), 12104–12114.
[59] PAPERNOT, N., M C DANIEL , P., AND G OODFELLOW, I. Transfer-
[39] K AWAR , B., Z ADA , S., L ANG , O., T OV, O., C HANG , H., D EKEL , T.,
ability in machine learning: From phenomena to black-box attacks
M OSSERI , I., AND I RANI , M. Imagic: Text-based real image editing
using adversarial samples. arXiv:1605.07277 (2016).
with diffusion models. arXiv preprint arXiv:2210.09276 (2022).
[60] P EREZ , M. AI Art Generator Cupixel Rakes in 5M From Craft Store
[40] K E , Y., S UKTHANKAR , R., H USTON , L., K E , Y., AND S UK -
JOANN, 2022. https://www.builtinboston.com/2022/08/10
THANKAR , R. Efficient near-duplicate detection and sub-image re-
/cupixel-raises-5m.
trieval. In Proc. of MM (2004), vol. 4, Citeseer, p. 5.
[61] POPLI, N. He Used AI to Publish a Children’s Book in a Weekend.
[41] K ELLY, C. Australian artists accuse popular AI imaging app of
Artists Are Not Happy About It., 2022. https://time.com/62405
stealing content, call for stricter copyright laws, 2022. https://www.
69/ai-childrens-book-alice-and-sparkle-artists-unhap
theguardian.com/australia-news/2022/dec/12/australian
py/.
-artists-accuse-popular-ai-imaging-app-of-stealing-c
ontent-call-for-stricter-copyright-laws. [62] R ADFORD , A., K IM , J. W., H ALLACY, C., R AMESH , A., G OH , G.,
[42] KOS , J., F ISCHER , I., AND S ONG , D. Adversarial examples for AGARWAL , S., S ASTRY, G., A SKELL , A., M ISHKIN , P., C LARK , J.,
ET AL . Learning transferable visual models from natural language
generative models. In Proc. of SPW (2018), IEEE, pp. 36–42.
supervision. In Proc. of ICML (2021), PMLR, pp. 8748–8763.
[43] K URAKIN , A., G OODFELLOW, I., AND B ENGIO , S. Adversarial
examples in the physical world. arXiv preprint arXiv:1607.02533 [63] R ADFORD , A., M ETZ , L., AND C HINTALA , S. Unsupervised rep-
(2016). resentation learning with deep convolutional generative adversarial
networks. arXiv preprint arXiv:1511.06434 (2015).
[44] K YNKÄÄNNIEMI , T., K ARRAS , T., A ITTALA , M., A ILA , T., AND
L EHTINEN , J. The role of imagenet classes in fr\’echet inception [64] R ADIYA -D IXIT, E., H ONG , S., C ARLINI , N., AND T RAMÈR , F. Data
distance. arXiv preprint arXiv:2203.06026 (2022). poisoning won’t save you from facial recognition. arXiv preprint
arXiv:2106.14851 (2021).
[45] L AIDLAW, C., S INGLA , S., AND F EIZI , S. Perceptual adversarial
robustness: Defense against unseen threat models. arXiv preprint [65] R AMESH , A., D HARIWAL , P., N ICHOL , A., C HU , C., AND C HEN ,
arXiv:2006.12655 (2020). M. Hierarchical text-conditional image generation with clip latents.
arXiv preprint arXiv:2204.06125 (2022).
[46] L EVINE , G. A New Stable Diffusion Plug-In For GIMP & Krita,
2022. https://80.lv/articles/a-new-stable-diffusion-p [66] R AMESH , A., PAVLOV, M., G OH , G., G RAY, S., VOSS , C., R AD -
lug-in-for-gimp-krita/. FORD , A., C HEN , M., AND S UTSKEVER , I. Zero-shot text-to-image
generation. In Proc. of ICML (2021), PMLR, pp. 8821–8831.
[47] L I , W., Z HANG , P., Z HANG , L., H UANG , Q., H E , X., LYU , S., AND
G AO , J. Object-driven text-to-image synthesis via adversarial training. [67] REPLICATE. replicate/dreambooth, 2022. https://replicate.co
In Proc. of CVPR (2019), pp. 12174–12182. m/replicate/dreambooth.
[48] L I , Y., FAN , H., H U , R., F EICHTENHOFER , C., AND H E , K. Scal- [68] R IACHI , E., AND RUDZICZ , F. Understanding adversarial attacks on
ing language-image pre-training via masking. arXiv preprint autoencoders.
arXiv:2212.00794 (2022). [69] ROMBACH , R., B LATTMANN , A., L ORENZ , D., E SSER , P., AND
[49] LIU, G. The World’s Smartest Artificial Intelligence Just Made Its O MMER , B. High-resolution image synthesis with latent diffusion
First Magazine Cover., 2022. https://www.cosmopolitan.com models. In Proc. of CVPR (2022), pp. 10684–10695.
/lifestyle/a40314356/dall-e-2-artificial-intelligenc [70] RONY, J., G RANGER , E., P EDERSOLI , M., AND B EN AYED , I. Aug-
e-cover/. mented lagrangian adversarial attacks. In Proc. of ICCV (2021),
[50] L IU , Z., L UO , P., WANG , X., AND TANG , X. Large-scale celebfaces pp. 7738–7747.
attributes (celeba) dataset. Retrieved August 15, 2018 (2018), 11. [71] ROOSE , K. An A.I.-Generated Picture Won an Art Prize. Artists
[51] L UO , Z., X I , Y., Z HANG , R., AND M A , J. Vc-gpt: Visual conditioned Aren’t Happy., 2022. https://www.nytimes.com/2022/09/02/t
gpt for end-to-end generative vision-and-language pre-training. arXiv echnology/ai-artificial-intelligence-artists.html.
preprint arXiv:2201.12723 (2022). [72] RUIZ , N., L I , Y., JAMPANI , V., P RITCH , Y., RUBINSTEIN , M., AND
[52] M ANSIMOV, E., PARISOTTO , E., BA , J. L., AND S ALAKHUTDINOV, A BERMAN , K. Dreambooth: Fine tuning text-to-image diffusion
R. Generating images from captions with attention. arXiv preprint models for subject-driven generation.
arXiv:1511.02793 (2015). [73] S ABOUR , S., C AO , Y., FAGHRI , F., AND F LEET, D. J. Ad-
[53] MCDONALD, A. THIS AI CAN DRAW ME!!! – DREAMBOOTH versarial manipulation of deep representations. arXiv preprint
& STABLE DIFFUSION, 2022. https://techteamgb.co.uk/202 arXiv:1511.05122 (2015).
2/12/05/this-ai-can-draw-me-dreambooth-stable-diffu [74] S AHARIA , C., C HAN , W., S AXENA , S., L I , L., W HANG , J., D EN -
sion/. TON , E., G HASEMIPOUR , S. K. S., AYAN , B. K., M AHDAVI , S. S.,
[54] M ENG , C., G AO , R., K INGMA , D. P., E RMON , S., H O , J., AND L OPES , R. G., ET AL . Photorealistic text-to-image diffusion models
S ALIMANS , T. On distillation of guided diffusion models. arXiv with deep language understanding. arXiv preprint arXiv:2205.11487
preprint arXiv:2210.03142 (2022). (2022).

15
[75] S ALEH , B., AND E LGAMMAL , A. Large-scale classification of fine- [94] WEATHERBED, J. ArtStation is hiding images protesting AI art
art paintings: Learning the right metric on the right feature. arXiv on the platform, 2022. https://www.theverge.com/2022/12/23/
preprint arXiv:1505.00855 (2015). 23523864/artstation-removing-anti-ai-protest-artwork
[76] S ALEHI , M., A RYA , A., PAJOUM , B., OTOOFI , M., S HAEIRI , A., -censorship.
ROHBAN , M. H., AND R ABIEE , H. R. Arae: Adversarially robust [95] W EEKMAN , K. People Have Raised Serious Concerns About The
training of autoencoders improves novelty detection. Neural Networks AI Art App That’s All Over Your Instagram Feed, 2022. https:
144 (2021), 726–736. //www.buzzfeednews.com/article/kelseyweekman/ai-art-a
pp-lensa-instagram-photo-trend-problems.
[77] S ALKOWITZ , R. AI is coming for commercial art jobs. can it be
stopped? Forbes, Sept 2022. [96] W HITE , T. Sampling generative networks. arXiv preprint
arXiv:1609.04468 (2016).
[78] S ALMAN , H., K HADDAJ , A., L ECLERC , G., I LYAS , A., AND
M ADRY, A. PhotoGuard: Defending Against Diffusion-based Image [97] W IGGERS , K. Scenario lands $6M for its AI platform that generates
Manipulation, 2022. https://gradientscience.org/photogua game art assets, 2022. https://techcrunch.com/2023/01/19/sc
rd/. enario-lands-6m-for-its-ai-platform-that-generates-g
ame-art-assets.
[79] S CENARIO . GG. AI-generated game assets, 2022. https://www.sc
enario.gg/. [98] W IGGERS , K. Stability AI, the startup behind Stable Diffusion, raises
101M, 2022. https://techcrunch.com/2022/10/17/stabilit
[80] S CHUHMANN , C., B EAUMONT, R., V ENCU , R., G ORDON , C., y-ai-the-startup-behind-stable-diffusion-raises-101
W IGHTMAN , R., C HERTI , M., C OOMBES , T., K ATTA , A., M ULLIS , m/.
C., W ORTSMAN , M., ET AL . Laion-5b: An open large-scale dataset
for training next generation image-text models. arXiv preprint [99] W ILLIAMS , T. Artists angry after discovering artworks used to train
arXiv:2210.08402 (2022). AI image generators without their consent, 2022. https://www.ab
c.net.au/news/2023-01-10/artists-protesting-artificia
[81] S HAN , S., W ENGER , E., Z HANG , J., L I , H., Z HENG , H., AND Z HAO , l-intelligence-image-generators/101786174.
B. Y. Fawkes: Protecting privacy against unauthorized deep learning
models. In Proc. of USENIX Security (2020), pp. 1589–1604. [100] X U , T., Z HANG , P., H UANG , Q., Z HANG , H., G AN , Z., H UANG ,
X., AND H E , X. Attngan: Fine-grained text to image generation with
[82] S HARMA , P., D ING , N., G OODMAN , S., AND S ORICUT, R. Con- attentional generative adversarial networks. In Proc. of CVPR (2018),
ceptual captions: A cleaned, hypernymed, image alt-text dataset for pp. 1316–1324.
automatic image captioning. In Proc. of ACL (2018), ACL, pp. 2556–
2565. [101] YANG , S. Why Artists are Fed Up with AI Art., 2022. https:
//www.youtube.com/watch?v=5Viy3Cu3DLk&ab_channel=Sam
[83] S TABILITY AI. Stable Diffusion 2.0 Release, 2022. https://stab DoesArts.
ility.ai/blog/stable-diffusion-v2-release.
[102] YOSINSKI , J., C LUNE , J., B ENGIO , Y., AND L IPSON , H. How trans-
[84] S TABILITY AI. Stable Diffusion Public Release. , 2022. https: ferable are features in deep neural networks? In Proc. of NeurIPS
//stability.ai/blog/stable-diffusion-public-release. (2014).
[85] S TABILITY AI. Stable Diffusion v2.1 and DreamStudio Updates [103] Z HANG , H., X U , T., L I , H., Z HANG , S., WANG , X., H UANG , X.,
7-Dec 22, 2022. https://stability.ai/blog/stablediffusio AND M ETAXAS , D. N. Stackgan: Text to photo-realistic image syn-
n2-1-release7-dec-2022. thesis with stacked generative adversarial networks. In Proc. of ICCV
[86] S UCIU , O., M ĂRGINEAN , R., K AYA , Y., DAUMÉ III, H., AND D U - (2017), pp. 5907–5915.
MITRA Ş , T. When does machine learning fail? generalized transfer- [104] Z HANG , K., Z UO , W., C HEN , Y., M ENG , D., AND Z HANG , L. Be-
ability for evasion and poisoning attacks. In Proc. of USENIX Security yond a gaussian denoiser: Residual learning of deep cnn for image
(2018). denoising. IEEE transactions on image processing (2017), 3142–
[87] S UNG , M. Lensa, the AI portrait app, has soared in popularity. But 3155.
many artists question the ethics of AI art, 2022. https://www.nbcn [105] Z HANG , R., I SOLA , P., E FROS , A. A., S HECHTMAN , E., AND
ews.com/tech/internet/lensa-ai-artist-controversy-eth WANG , O. The unreasonable effectiveness of deep features as a
ics-privacy-rcna60242. perceptual metric. In Proc. of CVPR (2018), pp. 586–595.
[88] TABACOF, P., TAVARES , J., AND VALLE , E. Adversarial images for [106] Z HU , M., PAN , P., C HEN , W., AND YANG , Y. Dm-gan: Dynamic
variational autoencoders. arXiv preprint arXiv:1612.00155 (2016). memory generative adversarial networks for text-to-image synthesis.
In Proc. of CVPR (2019), pp. 5802–5810.
[89] TAO , M., TANG , H., W U , F., J ING , X.-Y., BAO , B.-K., AND X U , C.
Df-gan: A simple and effective baseline for text-to-image synthesis.
In Proc. of CVPR (2022), pp. 16515–16525.
A Appendix
[90] T HE E CONOMIST. How a computer designed this week’s cover., 2022.
https://www.economist.com/news/2022/06/11/how-a-compu
ter-designed-this-weeks-cover. A.1 Comparison with Existing Cloaking Sys-
[91] T HOMEE , B., S HAMMA , D. A., F RIEDLAND , G., E LIZALDE , B., tems
N I , K., P OLAND , D., B ORTH , D., AND L I , L.-J. Yfcc100m: The
new data in multimedia research. Communications of the ACM 59, 2 Here, we discuss how we adapt existing cloaking tools for
(2016), 64–73. anti-mimicry protection. We show existing cloaking systems
[92] T RAN , T. H. Image Apps Like Lensa AI Are Sweeping the Internet, have limited effectiveness for anti-mimicry protection.
and Stealing From Artists, 2022. https://www.thedailybeast.co Adapting existing cloaking systems. Fawkes [81] gen-
m/how-lensa-ai-and-image-generators-steal-from-artis
ts. erates a cloak on user face images by optimizing the fea-
[93] U PCHURCH , P., G ARDNER , J., P LEISS , G., P LESS , R., S NAVELY,
ture space difference between the cloaked image and a target
N., BALA , K., AND W EINBERGER , K. Deep feature interpolation image. The target image is simply a face image of a differ-
for image content changes. In Proc. of CVPR (2017), pp. 7064–7073. ent person. We adapt Fawkes to anti-mimicry protection by

16
 switching the feature extractor from facial recognition to the
same one we use for Glaze. For the target image used, we
assume Fawkes randomly picks an artwork from a different
artist. Fawkes uses DSSIM to bound the input perturbation.
For a fair comparison, we change Fawkes perturbation from
DSSIM to LPIPS, ones used by Glaze.
The general design of Lowkey [13] is similar to Fawkes,
A crowd of people A man wearing armor except Lowkey does not optimize cloak images towards a
A woman in a white
in front of a building and holding a sword
dress by Karla Ortiz
by Nathan Fowkes by Kim van Deun target in feature space but simply optimizes cloaked images to
be different from the original one. We directly apply LowKey
for anti-mimicry protection: Lowkey maximizes the cloaked
artwork to have a different feature representation from the
original artwork.
Photoguard [78] works by minimizing the norm of the
image feature vector. It is equivalent to Fawkes when Fawkes
A room with a bed and A boat on a lake A vase on a white table selects the zero feature vector as the target for optimization.
wooden floor by Van Gogh by Arkhip Kuindzhi by Yiannis Moralis
For anti-mimicry, we have Photoguard to minimize the norm
Figure 16. Example data used for finetuning. Example artwork from different of feature representation of the cloaked artwork.
victim artists and their text captions. Performance comparison. Figure 22 show Fawkes,
Lowkey, and Photoguard have limited effectiveness at anti-
mimicry protection. Out of the three existing systems, Fawkes
achieves the high performance with 41.0% artist-rated protec-
tion success rate. While we can see small artifacts introduced
1K steps 2K steps 3K steps 4K steps by Fawkes and Lowkey, they are not sufficient to prevent
mimicry. In our tests, we use the same LPIPS perturbation
Attempts to level and the same feature extractor for optimization for all
mimic artist A cloaking systems.
Defeating Photoguard with random noise. We found
Photoguard is very sensitive to small random noises. When
Attempts to
mimic artist B we add Gaussian noise with σ = 0.03, its cloak effectiveness
is reduced by > 79% (the average L2 norm of the cloaked
image grows back from 29.3 to 119.4). After adding noise,
Artist-rated PSR 89.4 ± 1.5% 32.9 ± 4.3% 4.3 ± 0.2% 6.4 ± 0.8% we found their cloaks are no longer effective at their original
task of preventing image editing.
Figure 17. The success of style mimicry when the mimic finetunes the model
for an increasing number of iterations.
A.2 Additional information on style mimicry
Impact of fine-tuning on mimicry success. Figure 19
compares the mimicry performance when mimicking fine-
Mimicked artwork when perturbation budget equals to tunes on the victim artist’s artwork or directly using a generic
p=0 p = 0.03 p = 0.05 p = 0.1 p = 0.2 model. For less famous artists (Karla Ortiz and Nathan
Fowkes), fine-tuning significantly improves mimicry perfor-
mance. This improvement is limited to famous artists like Van
Gogh. We generate the images using text captions containing
the artist’s name, e.g., “a river by Nathan Fowkes”.
Details on training parameters. For stable diffusion, we
follow the same training parameters as the original paper [69].
We use 5 · 10−6 learning rate and batch size of 32. For a gener-
ation, we follow the default setting using the PNDM sampler
Figure 18. Mimicked artwork when artist uses an increasingly high pertur- and 50 sampling steps. For DALL·E-m, we also follow the
bation budget to protect his/her original art. same training setup [18] with a learning rate of 2 · 10−5 and
batch size 32. To generate images, we use the default setting
with a condition scale equal to 10.

17
 Mimicked art when mimic Mimicked art when mimic
Original artwork
finetunes on victim’s artwork directly uses a generic model

Artist A
(Karla Ortiz)

Artist B
(Nathan Fowkes)

Artist D
(Van Gogh)

Figure 19. Comparison of the mimicry performance between when mimic additionally fine-tunes the model on victim’s art pieces and when mimic directly
using a generic model. Column 1-2: victim artists’ original artwork; column 3-4: plagiarized artwork generated from a style-specific model fine-tuned on victim
artist’s art; column 5-6: plagiarized artwork generated from the generic SD model using the victim artist’s name.

1
Mimimcry success rate

0.8

0.6
Cloaked artwork
Original artwork
(p = 0.2) 0.4

0.2

0
5 10 15 20 25 30 35 40
Number of unique artwork

Figure 21. Mimicry success rate (1 - artist-rated PSR) increases as mimic

fine-tunes on more unique art pieces from the victim artist.

Using a different style transfer model. We evaluate our

system’s performance when using a different style transfer
model. We use the style transfer model offered by Midjour-
ney [34] while keeping the rest of the system identical. We
test the system on 10 victim artists with a cloak perturba-
tion budget of 0.05 and against a mimic using SD model for
mimicry. The average artist-rated protection success is 92.6%,
similar to the results when using stable diffusion as the style
transfer model.
Number of fine-tuning images’ impact on mimicry per-
formance. How many unique art pieces are needed for
Figure 20. High resolution view of original artwork and cloaked artwork
computed with perturbation budget 0.2. successful style mimicry? We answer this question but run-
ning style mimicry assuming mimic has an increasing number
of original artwork from the artist. We test this on 4 victim
artists (historical artists) and Figure 21 shows the mimicry
performance increases as mimic trains on an increasing num-

18
 Protection System
No protection Fawkes LowKey PhotoGuard Ours

Attempts to
mimic artist A

Attempts to
mimic artist B

Artist-rated PSR 4.3 ± 0.2% 41.0 ± 6.4% 37.6 ± 5.5% 11.5 ± 3.2% 93.5 + 0.6%

CLIP-based genre shift 1.4 ± 0.2% 29.6 ± 4.9% 23.6 ± 3.3% 7.3 ± 3.2% 96.0 ± 0.3%

Figure 22. Comparison of the protection performance across different cloaking systems. We adapt Fawkes, Lowkey, and Photoguard for style protection. Glaze
significantly outperforms existing systems.

ber of art pieces. The mimicry success (1 - artist-rated PSR) dataset, each containing the ground truth label from the
reaches 92% with 20 unique art pieces but continues to in- Wikiart dataset. Then we collect 100 artwork for each of
crease when the mimic trains on more artwork. the 13 digital art genres by searching the name of the genre
Impact of selecting random seed. For diffusion-based on ArtStation, one of the largest digital art-sharing platforms.
models (e.g., SD), artwork generation is controlled by a ran- We evaluate CLIP performance using top-3 accuracy as many
dom seed (random noise input at the beginning). Different art genres are similar to each other (e.g., impressionism vs
random seed leads to very different images, and thus, it’s a fauvism). CLIP achieves 96.4% top-3 accuracy on artwork
common practice for mimics to generate a set of artwork using from WikiArt and 94.2% for artwork from ArtStation.
different seed and select the best artwork. We investigate the
impact of random seed selection on Glaze protection. Given a A.4 Additional Information on Countermea-
style-specific model and a given text caption, the mimic gen- sures
erates 100 plagiarized artwork using different random seeds.
Similar to how we calculate CLIP-based genre shift, we then Here, we give details on the robust training method we used
use the CLIP model to filter any artwork that belongs to the in
same genre as the victim style. The result shows that for 4.3% Details on robust training. We follow prior work [76] on
of the time, the mimic is able to find at least 1 out of the robust training of autoencoder models. Mimic first uses Glaze
100 plagiarized artwork that passed CLIP filtering. While the to generate a large number of cloaked artwork using artwork
filtered artwork does belong to the same genre as the artist, from WikiArt dataset. Given the feature extractor Φ used by
we found they tend to have lower image quality. We verify mimic’s text-to-image model, mimic trains Φ to minimize the
this observation in our user study, and > 94.1% human artists following loss function:
rated the protection remains successful on these art pieces. We
believe the reason that some plagiarized artwork still shares
minΦ ||Φ(xcloaked ) − Φ(xorg )||22 (3)
the same genre as victim style after protection, is that text-
to-image models today are still imperfect and often output where xcloaked and xorg is a pair of cloaked and original art-
poor-quality images in rare cases with some random seed. work. This optimization effectively forces Φ to extract the
same feature representation for cloaked and original artwork.
A.3 CLIP-based metric To prevent the extractor from collapsing (e.g., output zero
vectors for all inputs), we regularized the training with the
We test CLIP’s performance in classifying artwork into the standard VAE reconstruction loss and trains the decoder D at
correct art genre. We take 27 historical genres from WikiArt the same time. Given the high discrepancy between features
and 13 digital art genres [33] as the candidate labels. We col- of cloaked and original artwork, this training process signifi-
lect a test dataset consisting of 1000 artwork from WikiArt cantly modifies the internals of Φ as well as the feature space.

19
Thus, the mimic needs to fine-tune the decoder D and gen- In this way, we do not need to retrain the generator as Φ2 ’s
erator G on the new robust feature space. We assume mimic feature space is trained to be similar to Φ’s and robust to
trains Φ for K steps on K different pairs of cloaked/original cloak perturbation. Similar to the first robust training design,
artwork, and then fine-tune D and G until converging. we train Φ2 on pairs of cloaked/original artwork until Φ2
Alterative robust training approach. The main reason converges. At generation time, we swap in Φ2 as the feature
that the above robust training method fails to defeat Glaze is extractor replacing Φ.
due to the degradation of image quality introduced by robust
training. The degradation is likely because the robust feature
space is less precise where small details in input images are
ignored by the robust feature extractor. We look at an alterna-
tive robust training approach where we do not need to retrain
the existing feature extractor and generative model.
The alternative approach trains a new feature extractor, We found that this alternative robust training approach is
Φ2 , that is robust to cloak perturbation and generates a similar not effective at defeating Glaze. After extensive training, Φ2
feature vector on cloaked images as ones generated by Φ on still fails to output sufficient similar feature vectors as Φ caus-
original images. Specifically, given a pair of cloaked/original ing the generator to output extremely low-quality images. The
images (xcloaked , xorg , we train the parameters of Φ2 to mini- artist-rate PSR on these mimicked artwork is over 95.1%. We
mize the difference between Φ2 (xcloaked ) and Φ(xorg ) while believe the reason for the poor countermeasure performance
keeping the parameters of Φ fixed: is that we enforce Φ2 to operate in the same feature space as
non-robust model Φ, which significantly limits Φ2 ’s ability
to be robust against cloak perturbations. As a result, we use
minΦ2 ||Φ2 (xcloaked ) − Φ(xorg )||22 (4) the first robust training design in §7.

20
 Mimicked art Mimicked art
Original artwork when GLAZE not used when GLAZE is used

Artist A
(Karla Ortiz)

Artist B
(Nathan Fowkes)

Artist C
(Claude Monet)

Artist D
(Van Gogh)

Figure 23. Additional example Glaze protection results for four artists. Columns 1-2: artist’s original artwork; column 3-4: plagiarized artwork when artist does
not use protection; column 5-6: plagiarized artwork when artist uses cloaking protection with perturbation budget p = 0.05. All mimicry attempts use SD-based
models.

21