Digital Camera World, in a report titled "Google’s chatbot refuses to make deepfakes. But Gemini’s new photo‑editing capabilities open the AI to far more potential for misuse", tested and assessed the image‑editing capabilities and risks of Google's updated chatbot Gemini. The report notes that Google has an explicit refusal policy to prevent "text‑only generation of deepfakes of specific real people": when users request generating or fabricating a specific real person using only text prompts, the model will refuse to output.
The problem lies in the image‑to‑image workflow. Gemini's new editing tools allow users to upload a real photo and then use masks, inpainting and image expansion to repeatedly modify a subject's expression, pose, clothing, lighting and background. The reporter demonstrated a process: upload a standard portrait, then use text prompts to change age, adjust facial expression, replace the background, save, and then continue editing the new image. After multiple iterations, the final image retained the subject's original facial features while displaying high‑fidelity realistic details — this "start from a real photo and progressively morph" pathway circumvents Google's refusal logic for text‑only requests about real people.
The report amplifies the risks: anyone who obtains a target's photo (from social media or public records) can use Gemini's editing chain to produce multiple versions of convincing "replacement" images. More worryingly, the model is already sufficiently subtle in retouching, makeup and expression tweaks, making it hard to immediately detect the composite as fake even under close forensic inspection. The report also notes that Gemini has protections for direct generation requests involving public figures, but current automated detection does not adequately cover editing requests where users supply an image and ask to "change the scene or behavior".
Google responded externally by emphasizing that it applies multiple safety layers to image generation, including restrictions on generating some public figures, built‑in abuse detectors, and published usage guidelines; however, Digital Camera World warns these measures are fragile in real‑world scenarios where source images are available. The author gives examples: swapping a benign photo's background for a controversial scene or altering a person's expression to a provocative one could create misleading visual evidence used for online harassment, extortion or political smear campaigns.
The report concludes with recommendations: first, model developers should strengthen traceability and watermarking across chained image edits, automatically embedding invisible but verifiable provenance markers; second, platforms should improve review and source‑tracing of uploaded photos, explicitly prohibiting multi‑stage composites using third‑party portraits without consent; finally, society should push for stricter legal and technical measures in parallel, such as digital content provenance standards and mandatory metadata declarations.
Overall, Digital Camera World's report recognizes Gemini's creative convenience in image editing, but issues a clear warning about the practical gap when protections target text‑only generation while leaving image iteration paths uncovered — as these tools proliferate, ease of creation and risk of misuse are teetering on a dangerous threshold.