SDEdit refinement with LTX-2: two experiments, opposite outcomes
Ditto's denoising_enhancing improves a video by re-noising it partway up the diffusion schedule and denoising again. LTX-2 is a joint audio–video DiT, so the same trick can refine both modalities at once. Run in two settings, it produced opposite results — and the contrast is the actual finding.
The rule both results imply: SDEdit needs the refiner to hold a stronger prior than whatever produced the input. A cannot satisfy it — the student is a distillation of the refiner. B satisfies it in principle, but the measured gain is small enough that the honest summary is "marginal at best," not "works."
Experiment B — external clip marginal at 0.10
256×256 · 81 frames @ 16 fps · mono 16 kHz audio · empty prompt
This set ships a ground truth, so quality can be measured rather than inferred: source.mp4 is the clean conditioning clip, target.mp4 the clean continuation, and generated.mp4 a model's degraded attempt at it — blocky, flickering, with garbled text. The question is whether refinement moves the generation closer to target.
Source · generated · refined · target
Measured against ground truth
| Clip | PSNR vs target ↑ | SSIM vs target ↑ | Δ PSNR | Δ SSIM | Flicker |
|---|
Δ is relative to the unrefined generation. Flicker is mean absolute frame-to-frame difference — note the target's own flicker is 2.47, so lower is not better: the generation's 4.11 is excess noise, but 0.42's 1.66 is under-shooting real motion.
Zoomed crop — frame 40, 3× nearest-neighbour
Clips
Distance to ground truth vs. strength
PSNR against target.mp4, the only measure here that answers "did refinement get us closer to the truth". It peaks at 0.10, is already below the unrefined generation by 0.42, and collapses past that. The dashed lines are the unrefined generation and the static source — note that the static source outscores every generated variant.
Audio is inconclusive here, not improved. The source is bandwidth-limited 16 kHz mono with essentially zero energy above 4 kHz, so there is nothing to restore and the >4 kHz metric reads 0.0000 for every variant. Testing the audio claim needs a full-band stereo source.
Experiment A — self-distilled student fails
768×512 · 121 frames @ 24 fps · 48 kHz stereo · LTX-2 DMD student, 4 steps
Refining a 4-step DMD student with the same 22B checkpoint it was distilled from. Video sharpness improves at exactly one of five settings; audio degrades at all of them.
Clips unmute — audio degrades more audibly than the video
Quality vs. strength
Both indexed to baseline so they share one axis — no dual scales. Points at 0.10 and 0.27 use a 30-step fine schedule; 0.42 and above are the distilled model's own grid, whose lowest reachable level is 0.4219.
Measurements
| Variant | PSNR vs base | Sharpness | Audio RMS | Audio >4 kHz | MB |
|---|
Method
Both experiments use the same code path: VAE-encode (or take latents directly), re-noise both modalities to noise_scale = sigmas[k], and run the schedule's tail from k down. This is not new diffusion math — it is what stage 2 of LTX-2's two-stage pipeline already does; the addition is a scalar strength knob and a direct-latent input.
Strength is quantized to the schedule. The distilled grid is [1.0, …, 0.9094, 0.725, 0.4219, 0], so every strength from 0 to 0.42 lands on 0.4219 — there is no light-touch setting and no true no-op. The 0.10 and 0.27 points therefore use a 30-step LTX2Scheduler grid instead, which is off the distilled model's trained grid; taking smaller steps than trained is the safe direction, but it is a caveat, not a free lunch.
# Experiment B — an existing clip
python denoising_enhancer/enhance_video.py --video samples/generated.mp4 \
--denoise-strength 0.4219 0.725 0.9094
python denoising_enhancer/enhance_video.py --video samples/generated.mp4 \
--fine-steps 30 --denoise-strength 0.10 0.27
# Experiment A — DMD student latents (two phases: the 46 GB base won't load twice)
python -m dmd.infer_dmd --prompt-index 0 --save-latents p0_latents.pt --out p0_base.mp4
python -m dmd.enhance_dmd --latents p0_latents.pt --denoise-strength 0.4219 0.725 0.9094
Open questions
The CFG path (distilled=False with GuidedDenoiser) was not tested and may preserve detail far better in both settings. A descriptive prompt instead of the empty default would likely extend Experiment B's usable range past 0.42, since content would be anchored by text rather than by the residual latent alone. And LTX-2 works natively well above 256×256, so Experiment B's source is somewhat off-distribution regardless of strength.
One clip and one seed per experiment. Both trends are large and monotonic, but the magnitudes should not be treated as precise without a broader sweep.