a distill-style blog post

an example of a distill-style blog post and main elements

2410.18775 Shilin Lu et el.

↗ arXiv

↗ Hugging Face

TL;DR

Current watermarking faces threats from AI editing. W-Bench is introduced to tackle this, serving as the first benchmark to evaluate how well watermarks hold up against image editing methods like regeneration and video creation. Evaluations on existing methods reveal failures against these edits. This highlights the pressing need for methods to protect copyright in the AI era.

Addressing the issues, VINE enhances watermarking. It analyzes frequency of image editing, using blurring as a surrogate to boost watermark robustness during training. VINE uses diffusion model SDXL-Turbo for robust embedding. Experiments show VINE achieves outstanding watermarking under image editing, outperforming prior work in image quality and robustness.

Key Takeaways

W-Bench, a new benchmark, evaluates watermarking robustness against modern image editing techniques.

VINE, a novel watermarking method, enhances robustness against image editing while preserving image quality.

Blurring distortions can serve as surrogate attacks during training to bolster watermark robustness.

Why does it matter?

This paper introduces W-Bench, a benchmark that aids researchers in assessing watermark robustness against generative editing. The VINE model offers new ways to improve watermarking. This work opens new avenues for research, emphasizing the need to adapt to evolving editing.

---

Visual Insights

{(https://arxiv.org/html/2410.18775/x1.png)}

🔼 Figure 1 is a two-part figure summarizing the W-Bench benchmark. (a) shows the evaluation process flow, from watermark encoding to image editing and watermark decoding. (b) presents a visual comparison of 11 watermarking methods. Each method is represented by a diamond (size indicating encoding capacity, y-coordinate showing normalized image quality (average of PSNR, SSIM, LPIPS, and FID), and x-coordinate representing robustness (TPR@0.1%FPR across four editing types)). Four bars extend from the diamond, representing performance against four image editing types: image regeneration (left), global editing (top), local editing (right), and image-to-video generation (bottom). Bar length indicates normalized TPR@0.1%FPR; longer bars represent better performance.

read the caption

Figure 1: (a) Flowchart of the W-Bench evaluation process. (b) Watermarking performance. Each method is illustrated with a diamond and four bars. The area of the diamond represents the method’s encoding capacity. The y-coordinate of the diamond’s center indicates normalized image quality, calculated by averaging the normalized PSNR, SSIM, LPIPS, and FID between watermarked and input images. The x-coordinate represents robustness, measured by the True Positive Rate at a 0.1% False Positive Rate (TPR@0.1%FPR) averaged across four types of image editing methods, encompassing a total of seven distinct models and algorithms. The four bars are oriented to signify different editing tasks: image regeneration (left), global editing (top), local editing (right), and image-to-video generation (bottom). The length of each bar reflects the method’s normalized TPR@0.1%FPR after each type of image editing—the longer the bar, the better the performance.

Method	Cap $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	TPR@0.1%FPR $\uparrow$ (%) (averaged over all difficulty levels)
						Regeneration		Global Editing			Local Editing		I2V
						Sto	Det	Pix2Pix	Ultra	Magic	Ultra	CtrlN	SVD
MBRS (Jia et al., 2021)	30	27.37	0.8940	0.1877	6.85	99.53	99.35	83.50	7.50	88.54	99.60	89.16	13.55
CIN (Ma et al., 2022)	30	43.19	0.9847	0.0270	1.13	44.85	51.65	51.40	17.00	68.38	51.28	66.04	2.93
PIMoG (Fang et al., 2022)	30	37.72	0.9863	0.0289	3.43	82.85	71.18	72.78	40.14	81.88	74.30	64.22	14.33
RivaGAN (Zhang et al., 2019)	32	40.43	0.9702	0.0488	1.86	10.12	12.50	6.22	4.14	33.96	34.28	56.92	3.15
SepMark (Wu et al., 2023)	30	35.48	0.9814	0.0150	1.72	61.21	73.85	87.74	51.84	82.58	92.94	97.14	8.81
DWTDCT (Al-Haj, 2007)	30	40.46	0.9705	0.0136	0.24	0.09	0.00	0.04	0.06	0.04	0.32	0.56	0.01
DWTDCTSVD (Navas et al., 2008)	30	40.40	0.9799	0.0265	0.86	3.12	1.43	3.82	4.02	30.84	24.56	50.04	0.76
SSL (Fernandez et al., 2022)	30	41.77	0.9796	0.0350	3.54	1.76	9.70	25.06	10.58	50.10	25.28	31.46	3.65
StegaStamp (Tancik et al., 2020)	100	29.65	0.9107	0.0645	7.61	91.09	92.13	93.72	51.24	91.18	98.84	99.06	30.85
TrustMark (Bui et al., 2023)	100	41.27	0.9910	0.0026	0.86	9.22	34.20	77.72	43.48	85.90	76.62	59.78	39.60
EditGuard (Zhang et al., 2024d)	64	37.58	0.9406	0.0171	0.51	0.09	6.00	0.06	1.16	0.24	0.18	2.66	0.18
VINE-Base	100	40.51	0.9954	0.0029	0.08	91.03	99.25	96.30	80.90	89.29	99.60	89.68	25.44
VINE-Robust	100	37.34	0.9934	0.0063	0.15	99.66	99.98	97.46	86.86	94.58	99.96	93.04	36.33

🔼 This table presents a comprehensive comparison of eleven watermarking techniques’ performance across various image editing methods. It evaluates both the quality of the watermarked images (using PSNR, SSIM, LPIPS, and FID) and the robustness of the watermarks against four types of image editing: stochastic and deterministic image regeneration, global editing, local editing, and image-to-video generation. For each editing method, the True Positive Rate at 0.1% False Positive Rate (TPR@0.1%FPR) is calculated, indicating the robustness of the watermarking method against that specific type of editing. The table also shows the encoding capacity of each method. Higher values in PSNR, SSIM, and TPR@0.1%FPR represent better image quality and watermark robustness, while lower values in LPIPS and FID, indicate better perceptual similarity between the original and watermarked images. The best performing method in each category is shown in bold, and the second best is underlined.

read the caption

Table 1: Comparison of watermarking performance in terms of watermarked image quality and detection accuracy across various image editing methods. Quality metrics are averaged over 10,000 images, and the TPR@0.1%FPR for each specific editing method is averaged over 5,000 images. The best value in each column is highlighted in bold, and the second best value is underlined. Abbreviations: Cap = Encoding Capacity; Sto = Stochastic Regeneration; Det = Deterministic Regeneration; Pix2Pix = Instruct-Pix2Pix; Ultra = UltraEdit; Magic = MagicBrush; CtrlN = ControlNet-Inpainting; SVD = Stable Video Diffusion.

In-depth insights

W-Bench: T2I Edit

While the provided document doesn’t explicitly define a section titled “W-Bench: T2I Edit,” we can infer its potential scope. Considering the paper’s focus, such a section would likely detail the evaluation of watermarking schemes against image editing techniques powered by Text-to-Image (T2I) models. This benchmark, being part of W-Bench, would involve subjecting watermarked images to various T2I editing operations, and then assessing the ability to recover the embedded watermark. Key aspects likely covered include the specific T2I editing models employed, the range of prompts used to guide the edits, and the metrics used to quantify both the robustness of the watermark and the quality of the edited image. Furthermore, the results likely highlight the vulnerabilities of existing watermarking techniques when confronted with these advanced T2I editing capabilities, thus motivating the need for more robust solutions like VINE.

Freq. Analysis

The document includes a frequency analysis to understand how image editing techniques affect the spectrum of an image. This analysis aims to identify surrogate attacks, especially blurring distortions, that can enhance the robustness of watermarking against image editing. The key insight is that image editing tends to remove patterns embedded in high-frequency bands while those in low-frequency bands are less affected. This property is similar to blurring distortions. Understanding the frequency characteristics of image editing allows the model to learn to embed watermarks in the less-affected frequency bands, improving robustness. Blurring distortions are used as surrogate attacks during training to achieve this.

VINE: Prior boost

VINE (Visual Information NEtwork) leverages the power of pre-trained generative models as a prior boost for watermarking. It recognizes that watermarking can be framed as a conditional generation task. By adapting a powerful generator, the model is capable of embedding watermarks in a more robust and imperceptible manner, as it learns a more intricate data distribution for watermarked images. This prior knowledge is crucial for upholding both image quality and resilience. The method’s strength comes from this smart use of prior knowledge. It enables high watermark strength with minimal impact on image quality.

Quant. Editing

While the provided document doesn’t contain a heading explicitly titled “Quant. Editing,” the intersection of quantitative analysis and image editing suggests several key aspects. Such a field would likely involve quantifying the impact of various editing operations on images. This could entail measuring changes in image quality metrics (PSNR, SSIM), perceptual similarity (LPIPS, FID), or even the detectability of specific features or objects. Furthermore, “Quant. Editing” could focus on developing algorithms that automatically optimize editing parameters based on quantitative criteria. For instance, an algorithm might aim to minimize perceptual distortion while achieving a desired artistic style transfer. It might also be important for comparing effects brought by human edits and those that could be replicated in large language model driven image edits, so more quantifiable metrics would bring improvements there.

I2V still Limited

The phrase ‘I2V still Limited’ suggests that while progress has been made in various aspects of the research, the Image-to-Video (I2V) domain faces persistent limitations. This could mean that existing techniques struggle to maintain watermark robustness when images are transformed into video, possibly due to temporal inconsistencies or alterations during video generation. The algorithms may not handle dynamic scenes well, or the method that are used may not be robust enough when there is a change. The need for a more robust and adaptable method is therefore important. The complexity of watermarking videos is high, since it should consider frame consistency and potential distortions during the process.

More visual insights

More on figures

 > 🔼 This figure demonstrates how image editing affects an image's frequency components. The Instruct-Pix2Pix model is used as an example of an image editing process. The image is first Fourier transformed (ℱ(⋅)), and then the editing model is applied to both the original image and the Fourier transformed image. A comparison of the Fourier transforms of both the original and edited images reveals which frequency components are most affected by the editing process. The magnitude of the Fourier transform is shown on a logarithmic scale for better visualization of changes across different frequency bands. >

>

read the caption

> Figure 2: Process for analyzing the impact of image editing on an image’s frequency spectrum. In this example, the editing model Instruct-Pix2Pix, denoted as ϵ⁢(⋅)italic-ϵ⋅\epsilon(\cdot)italic_ϵ ( ⋅ ), is employed. The function ℱ⁢(⋅)ℱ⋅\mathcal{F}(\cdot)caligraphic_F ( ⋅ ) represents the Fourier transform, and we visualize its magnitude on a logarithmic scale. >

 > 🔼 Figure 3 visualizes the effects of image editing and various image distortions on the frequency components of images. The analysis, performed on 1000 images, reveals a consistent trend: image editing and blurring techniques (like pixelation and defocus blur) primarily affect the mid and high-frequency patterns, removing them largely while leaving the low-frequency components relatively untouched. Conversely, common distortions such as JPEG compression and saturation do not show this frequency-specific behavior. Notably, Stable Video Diffusion (SVD), a method that completely removes all frequency patterns, was excluded from the analysis due to the complete removal of any frequency information. >

>

read the caption

> Figure 3: Impact of various image editing techniques and distortions on the frequency spectra of images. Results are averaged over 1,000 images. Image editing methods tend to remove frequency patterns in the mid- and high-frequency bands, while low-frequency patterns remain largely unaffected. This trend is also observed with blurring distortions such as pixelation and defocus blur. In contrast, commonly used distortions like JPEG compression and saturation do not exhibit similar behavior in the frequency domain. The analysis of SVD is not included, as it removes all patterns, rendering them invisible to the human eye. A discussion on SVD can be found in Section 4.3. >

 > 🔼 This figure illustrates the architecture of VINE, a novel watermarking method. VINE leverages the pre-trained SDXL-Turbo model, a one-step text-to-image diffusion model, as its watermark encoder. A key innovation is the 'condition adaptor' module, which effectively combines the watermark information with the input image before feeding it into the VAE encoder. This fusion step ensures seamless integration of the watermark into the image. To enhance the perceptual similarity between the watermarked and original images, zero-convolution layers and skip connections are incorporated into the encoder-decoder structure. Finally, the watermark decoder uses the ConvNeXt-B model, augmented with a fully connected layer, to recover the 100-bit watermark. Importantly, during training, the SDXL-Turbo model's text prompt is set to 'null' to focus training on the watermarking task. Details of the condition adaptor are shown separately in Figure 9. >

>

read the caption

> Figure 4: The overall framework of our method, VINE. We utilize the pretrained one-step text-to-image model SDXL-Turbo as the watermark encoder. A condition adaptor is incorporated to fuse the watermark with the image before passing the information to the VAE encoder. Zero-convolution layers (Zhang et al., 2023) and skip connections are added for better perceptual similarity. For decoding the watermark, we employ ConvNeXt-B (Liu et al., 2022b) as the decoder, with an additional fully connected layer to output a 100-bit watermark. Throughout the entire training process, the SDXL-Turbo text prompt is set to null prompt. Figure 9 shows the condition adaptor architecture. >

 > 🔼 Figure 5 presents a comprehensive comparison of eleven watermarking techniques across three distinct image editing scenarios: stochastic image regeneration, global image editing, and local image editing. Each scenario presents various difficulty levels to assess robustness. The figure uses TPR@0.1%FPR (True Positive Rate at 0.1% False Positive Rate) to measure the watermark's resilience against each editing method, showing the percentage of correctly identified watermarks. It also displays image quality metrics (PSNR, SSIM, LPIPS, and FID) to evaluate the perceptual impact of each watermarking method on image quality, illustrating the trade-off between robustness and quality. Additional results involving different editing methods are provided in Figure 16. >

>

read the caption

> Figure 5: The performance of watermarking methods under (a) Stochastic regeneration, (b) Global editing, and (c) Local editing. Additional results are available in Figure 16. >

 > 🔼 This figure visualizes the frequency patterns of eleven watermarking methods by using a 2D Fourier transform. The magnitude of the frequency spectrum is shown on a logarithmic scale. The patterns are analyzed to determine how the watermarking methods distribute information across different frequency bands. The DWTDCT method is omitted because its pattern is very similar to DWTDCTSVD and both are too faint to clearly distinguish at this scale. The image should be zoomed to see the details. >

>

read the caption

> Figure 6: Frequency pattern visualizations for each watermarking method. The DWTDCT method is excluded because it closely resembles DWTDCTSVD and their pattern intensity is too weak to be discerned on the uniform scale. Please zoom in for a closer look. >

 > 🔼 Figure 7 displays the performance of various watermarking techniques when subjected to different image distortions at a resolution of 512 x 512 pixels. Each subplot represents a different type of distortion: (a) Gaussian blurring (varying kernel size), (b) brightness adjustments (changing brightness factor), (c) contrast modifications (changing contrast factor), (d) Gaussian noise (varying standard deviation), and (e) JPEG compression (varying compression factor). For each distortion type, the figure shows how the True Positive Rate at a 0.1% False Positive Rate (TPR@0.1%FPR), True Positive Rate at a 1% False Positive Rate (TPR@1%FPR), bit accuracy, and Area Under the Receiver Operating Characteristic Curve (AUROC) of each watermarking method change in response to the increasing level of distortion. This allows for a comparative analysis of the robustness of different watermarking methods against various common image processing manipulations. >

>

read the caption

> Figure 7: Performance of watermarking methods at a resolution of 512×\times×512 pixels under (a) Gaussian blurring, (b) brightness adjustments, (c) contrast modifications, (d) Gaussian noise, and (e) JPEG compression. >

 > 🔼 Figure 8 displays the performance of different watermarking methods when subjected to various image distortions at their original training resolutions. The distortions tested include Gaussian blurring, brightness adjustments, contrast modifications, Gaussian noise, and JPEG compression. The figure highlights how each method's robustness varies depending on the type and severity of distortion. Importantly, different watermarking methods were trained on different image resolutions (MBRS, CIN, PIMoG, and SepMark at 128x128; TrustMark, VINE-B, and VINE-R at 256x256; and StegaStamp at 400x400), demonstrating the impact of training resolution on robustness against common image processing artifacts. >

>

read the caption

> Figure 8: Assessment of watermarking methods at their respective training resolutions under the following conditions: (a) Gaussian blurring, (b) brightness adjustments, (c) contrast modifications, (d) Gaussian noise, and (e) JPEG compression. Training resolutions: MBRS, CIN, PIMoG, and SepMark were trained at 128×\times×128 pixels; TrustMark, VINE-B, and VINE-R at 256×\times×256 pixels; and StegaStamp at 400×\times×400 pixels. >

 > 🔼 This figure details the architecture of the 'condition adaptor' module used within the VINE watermarking model (Figure 4 of the paper). The condition adaptor's role is to effectively combine the input image and watermark data before this combined information is passed to the VAE encoder. The architecture consists of several fully connected and convolutional layers, each followed by a ReLU activation function. This design ensures that the relevant information from both the watermark and image is properly fused for optimal watermark embedding. >

>

read the caption

> Figure 9: Architecture of the condition adaptor in Figure 4. Each fully connected and convolutional layer is followed by an activation layer. >

 > 🔼 This figure displays the reconstruction quality for both stochastic and deterministic image regeneration methods. Stochastic regeneration introduces noise to the image, then uses a diffusion model to reconstruct a clean image. Deterministic regeneration uses the diffusion model to deterministically create a noisy image, and then reconstructs it. The x-axis in (a) represents the noise level (timestep), and the y-axis is the peak signal-to-noise ratio (PSNR) to measure the difference between the regenerated and original image. Similarly, the x-axis in (b) shows the number of sampling steps used to create the noisy version, and the y-axis is the PSNR. Higher PSNR values indicate better reconstruction quality, with lower values suggesting degradation due to higher noise levels. The plots show that as the difficulty of the regeneration task increases (higher noise levels or fewer sampling steps), the PSNR decreases. >

>

read the caption

> Figure 10: The reconstruction quality of (a) stochastic regeneration and (b) deterministic regeneration. The PSNR is calculated by comparing the regenerated image to the original image. >

 > 🔼 Figure 11 shows a comparison of image reconstruction quality for stochastic and deterministic image regeneration methods. The top row displays the original images used for testing. Each subsequent row illustrates the results of regenerating the image using stochastic methods with increasing levels of noise (60, 100, 140, 180, 220) and deterministic methods with increasing levels of sampling steps (15, 25, 35, 45). The figure shows that higher noise levels in stochastic methods and higher sampling steps in deterministic methods result in lower quality reconstruction. This degradation is apparent in the detail loss and/or introduction of artifacts. It is recommended to zoom in for a closer examination. >

>

read the caption

> Figure 11: The reconstruction quality of stochastic regeneration and deterministic regeneration. Please zoom in for a closer look. >

 > 🔼 This figure demonstrates the results of global image editing applied to images watermarked using different methods. Two example images are shown, each edited using three different methods (UltraEdit, InstructPix2Pix, and MagicBrush). For each image and each editing method, the original image, the watermarked version, and the edited watermarked version are displayed. The results show that despite being edited, the watermarks have a minimal effect on the final edited image, only causing minor visual changes. >

>

read the caption

> Figure 12: Different watermarks have minimal impact on the image global editing outcomes, resulting in only slight changes. >

 > 🔼 This figure displays the results of local image editing experiments using different watermarking methods. Two example images (a stop sign and a pizza) are shown with various local edits applied. Each row represents a different watermarking technique. The 'Unedited' column shows the original image, followed by columns showing the edited images using three different methods (UltraEdit, ControlNet-Inpainting, and Mask). The results demonstrate that the watermarks have minimal effect on the final appearance of the images after local editing, with only minor changes visible. >

>

read the caption

> Figure 13: Different watermarks have minimal impact on the image local editing outcomes, resulting in only slight changes. >

 > 🔼 This figure displays the results of applying image-to-video generation to images watermarked using various methods. The process involves taking a single watermarked image and converting it into a short video clip. Each row represents a different watermarking technique applied to two example images. The columns show the original unedited image, followed by the watermarked image and then a series of frames from the generated video clip. The caption indicates that the presence of the watermarks has a minimal impact on the image-to-video generation process, leading to only minor visual differences in the resulting videos. >

>

read the caption

> Figure 14: Different watermarks have little effect on image-to-video generation, leading to only minor changes. >

 > 🔼 Figure 15 presents a qualitative comparison of different watermarking methods by visually inspecting the residual images obtained after watermark removal. Two example images are used for comparison: a canal scene and a dog eating food. For each watermarking method, it showcases the original image, watermarked image, and the residual image resulting from subtracting the original from the watermarked image. This visual representation helps assess the perceptual quality and invisibility of the watermarks. The residuals should ideally be imperceptible (noise-like) to the human eye. Note that the quality of the residuals can be further analyzed by zooming into the image. >

>

read the caption

> Figure 15: Qualitative comparison of the evaluated watermarking methods. Please zoom in for a closer look. >

 > 🔼 Figure 16 presents a comprehensive evaluation of eleven watermarking techniques across five distinct image manipulation scenarios. Each subfigure (a-e) illustrates the performance metrics (TPR@0.1%FPR, TPR@1%FPR, bit accuracy, and AUROC) for each watermarking method under a specific image editing technique. (a) shows results for deterministic image regeneration using DPM-Solver, showcasing robustness at varying sampling steps. (b) and (c) evaluate global editing capabilities using UltraEdit and Instruct-Pix2Pix, respectively, across different text guidance scales, revealing methods that maintain watermark integrity under various editing prompts. (d) assesses robustness against local editing using ControlNet-Inpainting at different mask sizes, providing insights into the methods’ resilience to targeted image modifications. Finally, (e) shows the performance of each method when generating videos from images with Stable Video Diffusion, determining which watermarks can be successfully extracted in the generated video frames. >

>

read the caption

> Figure 16: The performance of watermarking methods under (a) Deterministic regeneration with DPM-Solver, (b) Global editing with UltraEdit, (c) Global editing with Instruct-Pix2Pix, (d) Local editing with ControlNet-Inpainting, and (e) Image-to-video generation with Stable Video Diffusion. >

More on tables

Config	Blurring	Watermark Encoder					PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	TPR@0.1%FPR $\uparrow$ (%)
	Distortions	Backbone	Condition	Skip	Pretrained	Finetune					Sto	Det	Pix2Pix	Ultra
Config A	\faTimes	Simple UNet	N.A.	N.A.	N.A.	\faTimes	38.21	0.9828	0.0148	1.69	54.61	66.86	64.24	32.62
Config B	\faCheck					\faTimes	35.85	0.9766	0.0257	2.12	86.85	92.28	80.98	62.14
Config C	\faCheck					\faCheck	31.24	0.9501	0.0458	4.67	98.59	99.29	96.01	84.60
Config D	\faCheck	SDXL-Turbo	ControlNet	\faTimes	\faCheck	\faTimes	32.68	0.9640	0.0298	2.87	90.82	94.89	91.86	70.69
Config E	\faCheck		Cond. Adaptor	\faTimes	\faCheck	\faTimes	36.76	0.9856	0.0102	0.53	90.86	94.78	92.88	70.68
Config F (VINE-B)	\faCheck		Cond. Adaptor	\faCheck	\faCheck	\faTimes	40.51	0.9954	0.0029	0.08	91.03	99.25	96.30	80.90
Config G (VINE-R)	\faCheck		Cond. Adaptor	\faCheck	\faCheck	\faCheck	37.34	0.9934	0.0063	0.15	99.66	99.98	97.46	86.86
Config H	\faCheck		Cond. Adaptor	\faCheck	\faTimes	\faCheck	35.18	0.9812	0.0137	1.03	99.67	99.92	96.13	84.66

> 🔼 This ablation study analyzes the effects of different components on the performance of the VINE watermarking method for image regeneration and global editing tasks. It shows how adding blurring distortions, changing the watermark encoder, incorporating a condition adaptor, using skip connections, and fine-tuning with Instruct-Pix2Pix affect the True Positive Rate (TPR) at 0.1% False Positive Rate (FPR), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), learned perceptual image patch similarity (LPIPS), Fréchet inception distance (FID), and other image quality metrics. Each configuration builds upon the previous one, highlighting the incremental contribution of each component. >

>

read the caption

> Table 2: Ablation study examining the impact of key components on image regeneration and global editing. Each configuration builds upon the previous one, with changes highlighted in red. >

Method	Resolution	Capacity $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	TPR@0.1%FPR $\uparrow$ (%)
MBRS (Jia et al., 2021)	128 $\times$ 128	30	25.14	0.8348	0.0821	13.51	100.0
CIN (Ma et al., 2022)	128 $\times$ 128	30	41.70	0.9812	0.0011	2.20	100.0
PIMoG (Fang et al., 2022)	128 $\times$ 128	30	37.54	0.9814	0.0140	2.97	100.0
SepMark (Wu et al., 2023)	128 $\times$ 128	30	35.50	0.9648	0.0116	2.95	100.0
StegaStamp (Tancik et al., 2020)	400 $\times$ 400	100	29.33	0.8992	0.1018	8.29	100.0
TrustMark (Bui et al., 2023)	256 $\times$ 256	100	40.94	0.9819	0.0015	1.04	100.0
VINE-Base	256 $\times$ 256	100	40.22	0.9961	0.0022	0.10	100.0
VINE-Robust	256 $\times$ 256	100	37.07	0.9942	0.0048	0.19	100.0

> 🔼 Table 3 presents a detailed comparison of different watermarking methods' performance. It assesses both the quality of the watermarked images and the accuracy of watermark detection under ideal conditions (no image distortions or edits). Image quality is measured using PSNR, SSIM, LPIPS, and FID. Detection accuracy is represented by the True Positive Rate at 0.1% False Positive Rate (TPR@0.1%FPR). The table shows each method's encoding capacity (number of bits embedded), and the best and second-best performance values for each metric are highlighted. This allows for a comprehensive comparison of the methods' effectiveness in terms of both image quality and robustness before any attacks are applied. >

>

read the caption

> Table 3: Comparison of watermarking performance, evaluating both image quality of the watermarked images and detection accuracy under normal conditions (no distortion or editing applied) at the original training resolution. The best value in each column is highlighted in bold, and the second best value is underlined. >

Method	Instruct-Pix2Pix			UltraEdit			MagicBrush
	$\text{CLIP}_{\text{dir}}\uparrow$	$\text{CLIP}_{\text{img}}\uparrow$	$\text{CLIP}_{\text{out}}\uparrow$	$\text{CLIP}_{\text{dir}}\uparrow$	$\text{CLIP}_{\text{img}}\uparrow$	$\text{CLIP}_{\text{out}}\uparrow$	$\text{CLIP}_{\text{dir}}\uparrow$	$\text{CLIP}_{\text{img}}\uparrow$	$\text{CLIP}_{\text{out}}\uparrow$
Unwatermarked Image	0.2693	0.7283	0.2732	0.3230	0.7268	0.3008	0.3025	0.7913	0.2930
MBRS (Jia et al., 2021)	0.2494	0.7385	0.2733	0.2919	0.6654	0.2891	0.2857	0.7816	0.2929
CIN (Ma et al., 2022)	0.2625	0.7232	0.2729	0.3152	0.7111	0.3010	0.2949	0.7841	0.2928
PIMoG (Fang et al., 2022)	0.2518	0.7021	0.2746	0.3010	0.6940	0.3024	0.2815	0.7662	0.2962
RivaGAN (Zhang et al., 2019)	0.2647	0.7317	0.2721	0.3168	0.7133	0.3003	0.3020	0.7948	0.2930
SepMark (Wu et al., 2023)	0.2659	0.7292	0.2743	0.3145	0.7181	0.3002	0.2975	0.7891	0.2936
DWTDCT (Al-Haj, 2007)	0.2644	0.7317	0.2734	0.3189	0.7250	0.3009	0.2959	0.7942	0.2934
DWTDCTSVD (Navas et al., 2008)	0.2581	0.7220	0.2751	0.3115	0.7118	0.3004	0.2869	0.7793	0.2939
SSL (Fernandez et al., 2022)	0.2583	0.7218	0.2752	0.3093	0.7065	0.3019	0.2896	0.7780	0.2944
StegaStamp (Tancik et al., 2020)	0.2436	0.6826	0.2697	0.2904	0.6886	0.3007	0.2663	0.7512	0.2944
TrustMark (Bui et al., 2023)	0.2634	0.7181	0.2729	0.3172	0.7146	0.2994	0.2943	0.7853	0.2936
EditGuard (Zhang et al., 2024d)	0.2722	0.7045	0.2722	0.3155	0.7170	0.3021	0.2882	0.7708	0.2940
VINE-Base	0.2743	0.7260	0.2743	0.3186	0.7189	0.2996	0.2977	0.7889	0.2931
VINE-Robust	0.2624	0.7248	0.2715	0.3176	0.7183	0.3001	0.2981	0.7953	0.2940

> 🔼 This table presents a comparative analysis of the editing quality achieved by various global image editing methods (Instruct-Pix2Pix, UltraEdit, and MagicBrush). The impact of different watermarking techniques on the effectiveness of these editing methods is also evaluated. For each editing method, the table displays the quality metrics CLIPdir (alignment of edits with prompts), CLIPimg (content preservation), and CLIPout (alignment of edited image to target caption). A control condition using an unwatermarked image is included for comparison. The analysis is conducted using a consistent image guidance scale of 1.5 and a text guidance scale of 7 for all watermarking and editing methods. >

>

read the caption

> Table 4: Comparison of editing quality for different global editing methods and the effect of different watermarks on image editing outcomes. All models use an image guidance scale of 1.5 and a text guidance scale of 7. >

Method	ControlNet-Inpainting			UltraEdit
	$\text{CLIP}_{\text{dir}}\uparrow$	$\text{CLIP}_{\text{img}}\uparrow$	$\text{CLIP}_{\text{out}}\uparrow$	$\text{CLIP}_{\text{dir}}\uparrow$	$\text{CLIP}_{\text{img}}\uparrow$	$\text{CLIP}_{\text{out}}\uparrow$
Unwatermarked Image	0.1983	0.7076	0.2589	0.2778	0.7519	0.2917
MBRS (Jia et al., 2021)	0.1846	0.7058	0.2588	0.2657	0.7175	0.2913
CIN (Ma et al., 2022)	0.1966	0.7042	0.2613	0.2745	0.7389	0.2922
PIMoG (Fang et al., 2022)	0.1828	0.6909	0.2600	0.2578	0.7371	0.2920
RivaGAN (Zhang et al., 2019)	0.1975	0.7117	0.2612	0.2748	0.7469	0.2937
SepMark (Wu et al., 2023)	0.1932	0.7126	0.2582	0.2716	0.7588	0.2921
DWTDCT (Al-Haj, 2007)	0.1982	0.7197	0.2602	0.2776	0.7558	0.2924
DWTDCTSVD (Navas et al., 2008)	0.1922	0.6995	0.2608	0.2705	0.7469	0.2940
SSL (Fernandez et al., 2022)	0.1911	0.6995	0.2604	0.2677	0.7380	0.2940
StegaStamp (Tancik et al., 2020)	0.1752	0.6684	0.2606	0.2439	0.7246	0.2919
TrustMark (Bui et al., 2023)	0.1959	0.7001	0.2594	0.2728	0.7451	0.2919
EditGuard (Zhang et al., 2024d)	0.1921	0.6944	0.2606	0.2696	0.7392	0.2923
VINE-Base	0.1953	0.7023	0.2591	0.2726	0.7494	0.2906
VINE-Robust	0.1951	0.7030	0.2591	0.2710	0.7475	0.2909

> 🔼 This table presents a quantitative analysis of the image editing quality using different watermarking methods. Specifically, it evaluates local editing performance using ControlNet-Inpainting and UltraEdit, considering three metrics: CLIPdir (correspondence between edits and prompts), CLIPimg (content preservation), and CLIPout (alignment of edited image with target caption). The results are shown for unwatermarked images and images watermarked with eleven different watermarking methods. All editing models used a consistent image guidance scale of 1.5 and text guidance scale of 7. >

>

read the caption

> Table 5: Comparison of editing quality for different local editing methods and the effect of different watermarks on image editing outcomes. All models use an image guidance scale of 1.5 and a text guidance scale of 7. >

Method	Running Time per Image (s)	GPU Memory Usage (MB)
MBRS (Jia et al., 2021)	0.0053	938
CIN (Ma et al., 2022)	0.0741	2944
PIMoG (Fang et al., 2022)	0.0212	878
RivaGAN (Zhang et al., 2019)	-	-
SepMark (Wu et al., 2023)	0.0109	928
DWTDCT (Al-Haj, 2007)	-	-
DWTDCTSVD (Navas et al., 2008)	-	-
SSL (Fernandez et al., 2022)	2.1938	1072
StegaStamp (Tancik et al., 2020)	0.0672	1984
TrustMark (Bui et al., 2023)	0.0705	648
EditGuard (Zhang et al., 2024d)	0.2423	1638
VINE	0.0795	4982

> 🔼 This table presents a performance comparison of various watermarking methods in terms of processing time per image and GPU memory consumption. The metrics were obtained by averaging the results from 1000 images. Note that DWTDCT, DWTDCTSVD, and RivaGAN are excluded from this comparison because their implementations are designed for CPU-only use. >

>

read the caption

> Table 6: Comparison of watermarking methods based on running time per single image and GPU memory usage. The results are averaged over 1,000 images. Since the implementations we employed for DWTDCT, DWTDCTSVD, and RivaGAN support CPUs exclusively, they have been omitted from the comparison. >

Full paper