Repository navigation
Fix flow sigma timestep precision in UniPC and DPM-Solver - #15226
linusfackler wants to merge 2 commits into
Conversation
mangeshraut712
left a comment
There was a problem hiding this comment.
Matching the conditioning timestep to the sigma the solver actually integrates makes sense for flow models.
| if self.config.use_flow_sigmas: | ||
| self.timesteps = ( | ||
| self.sigmas[:-1] * self.config.num_train_timesteps | ||
| ).to(device=device) |
There was a problem hiding this comment.
The same use_flow_sigmas -> torch.int64 truncation is in the other schedulers that support flow sigmas: scheduling_deis_multistep.py (L342), scheduling_dpmsolver_singlestep.py (L418), scheduling_dpmsolver_multistep_inverse.py (L361) and scheduling_sasolver.py (L340) on main. With only UniPC and DPM-Multistep changed, switching a flow pipeline between e.g. DPM-Multistep and DPM-Singlestep would change the timesteps the transformer sees, from fractional to truncated. Could the same branch go into those too (and their classes into the parametrized regression test), or is there a reason to leave them out?
Edge case: in DPM-Multistep, use_flow_sigmas is checked after the use_karras_sigmas / use_lu_lambdas / use_exponential_sigmas / use_beta_sigmas branches, and __init__ only rejects combinations among beta/exponential/karras, not with flow. With use_flow_sigmas=True plus e.g. use_karras_sigmas=True, self.sigmas holds the DDPM-derived Karras sigmas (up to ~14 for SD-style betas), so self.sigmas[:-1] * num_train_timesteps gives timesteps well beyond 1000 where _sigma_to_t used to keep them in range. Gating this on the flow branch actually having produced the sigmas (or rejecting flow + karras/lu/exp/beta in __init__) would avoid that.
What does this PR do?
Fixes #15167.
This PR fixes a timestep precision mismatch in
UniPCMultistepSchedulerandDPMSolverMultistepSchedulerwhenuse_flow_sigmas=True.Previously, both schedulers converted their timesteps to
torch.int64, truncating fractional values even though the numerical solvers integrate using floating-point sigma values. As a result, the model receives conditioning timesteps that do not exactly correspond to the sigma values used by the solver.For example, a sigma corresponding to timestep
954.1981would result in the model receiving timestep954.Changes
use_flow_sigmas=Trueby computing them directly fromsigmas[:-1] * num_train_timesteps.Testing
Reproduced the original issue locally using the reproduction script from #15167.
Regression tests:
Full scheduler test suite:
Ran
python -m pytest tests/schedulers/ -qboth before and after the implementation.DPMSolverSDESchedulertests fail identically on the unmodified codebase and with this fix applied. These failures are unrelated to the modified schedulers.No new dependencies are required.
AI-assisted development / self-review
ChatGPT was used to assist with drafting the implementation and writing regression tests. The changes were applied and tested locally.
Review notes:
The repository's dedicated AI self-review workflow has not yet been run.
Before submitting
Who can review?
@dg845 @yiyixuxu