Choose Between Generating, Editing, and Controlling an Image

Most frustration with image models comes from using the wrong workflow rather than the wrong prompt. Someone tries to change one object in a finished image by rewriting the prompt, gets a completely different picture, and concludes the model is uncontrollable. The model behaved correctly; the request was aimed at the wrong mechanism.

There are four mechanisms. Choosing among them takes about ten seconds once you can name what each one preserves.

Four workflows, four things preserved

Text-to-image starts from noise. It preserves nothing, which is exactly right when nothing exists yet and wrong the moment you have something worth keeping. Use it for exploration and for the first image in a series.

Image-to-image starts from your picture, adds noise to it, then denoises. It preserves large-scale structure — composition, pose, colour blocking — in proportion to how little noise you added. Use it for restyling, for grade changes, and for pushing a rough sketch toward a finish.

Inpainting restricts changes to a mask. Everything outside the mask is untouched by construction, not by persuasion. This is the only mechanism that reliably changes one object while leaving the rest identical. If your task contains the words "just change the," you want a mask.

Control signals condition generation on a structural input — an edge map, a depth map, a pose skeleton, a segmentation map. They preserve geometry while leaving appearance free, which is the opposite trade from image-to-image at low strength.

The common mistake is reaching for text-to-image and a longer prompt when the task is actually editing. Prompts are not a targeting mechanism. They describe the whole image, so they change the whole image.

Denoise strength is the only dial that matters in image-to-image

Image-to-image has one parameter that dominates everything else. It goes by different names — denoise, strength, variation — but it always means how much of your image is destroyed before reconstruction begins.

At very low values the model can only adjust texture and grain; your image survives almost intact and almost unchanged. Around a third, palette and lighting shift while composition holds. Around half, objects begin to move and small elements are reinvented — this is the range where restyling actually happens, and also where faces start to drift. Above two-thirds you are functionally doing text-to-image with a vague compositional hint, and above about 0.85 the input is decorative.

The practical consequence: when a restyle "loses the picture," the answer is almost never a better prompt. It is a lower strength plus more steps. And when a restyle "does nothing," raising strength in increments of 0.05 while keeping the seed fixed will find the threshold in three runs.

Fixing the seed while sweeping strength is what makes this a measurement rather than a guess. Two variables moving at once tells you nothing.

Control signals constrain geometry, not style

Control signals are the answer to "I need this exact layout with completely different content." Each type constrains a different property, and picking the wrong one produces either a copy or an unconstrained mess.

Edge maps are the tightest. They preserve outlines faithfully, which is ideal for product shots, packaging, and architecture, and too rigid for anything organic — an edge-conditioned portrait tends to inherit the original face.

Depth maps preserve spatial arrangement and volume while discarding surface detail. This is usually the best default for scene work: the room stays the same shape and everything in it can be re-imagined.

Pose skeletons preserve the figure's articulation and nothing else. Same pose, different person, different clothing, different setting.

Segmentation maps preserve regions and their labels — sky here, building there — which is the loosest and most forgiving of the four.

You can combine signals, and combining an edge map with a depth map at full weight will produce something that looks traced. Start with one signal, at a weight below full, applied to only part of the denoising schedule.

Practice: route five real tasks

Take five things you actually need to do this month and route each one. The routing questions, in order:

1. Does an image already exist that must survive? If no, text-to-image.
2. Must most of it survive pixel-identically? If yes, inpaint with a mask.
3. Must the layout survive while appearance changes? If yes, a control signal.
4. Otherwise, image-to-image, and your first decision is the strength.

Then run the one that is hardest and record what strength or control weight worked. That number is model-specific and it is the single most transferable thing you will learn this week.

Common mistakes

Prompting for a local change. "Same image but the mug is blue" will not give you the same image. Mask the mug.

Treating strength as a quality dial. It is a preservation dial. Higher is not better; it is less of your input.

Masking too tightly. A mask drawn exactly at an object's outline leaves the model no room to blend, and the seam shows. Feather the edge and include a little surrounding context.

Stacking control signals at full weight. Two hard constraints usually conflict, and the result looks like neither. One signal, partial weight, is the working default.