Skip to content

Fix flow sigma timestep precision in UniPC and DPM-Solver - #15226

Open
linusfackler wants to merge 2 commits into
huggingface:mainfrom
linusfackler:fix/flow-sigma-timestep-precision
Open

linusfackler wants to merge 2 commits into
huggingface:mainfrom
linusfackler:fix/flow-sigma-timestep-precision

Conversation

@linusfackler

Copy link
Copy Markdown

What does this PR do?

Fixes #15167.

This PR fixes a timestep precision mismatch in UniPCMultistepScheduler and DPMSolverMultistepScheduler when use_flow_sigmas=True.

Previously, both schedulers converted their timesteps to torch.int64, truncating fractional values even though the numerical solvers integrate using floating-point sigma values. As a result, the model receives conditioning timesteps that do not exactly correspond to the sigma values used by the solver.

For example, a sigma corresponding to timestep 954.1981 would result in the model receiving timestep 954.

Changes

  • Preserve floating-point timesteps when use_flow_sigmas=True by computing them directly from sigmas[:-1] * num_train_timesteps.
  • Maintain the existing integer timestep behavior for non-flow configurations.
  • Apply the fix to both UniPC and DPM-Solver Multistep schedulers.
  • Add a parametrized regression test covering both schedulers, verifying that their timesteps retain floating-point precision and match the corresponding sigma values.
  • Update the existing UniPC test for combined flow and Karras sigmas to expect floating-point timesteps.

Testing

Reproduced the original issue locally using the reproduction script from #15167.

Regression tests:

  • Before the fix: 2 failed (integer timesteps).
  • After the fix: 2 passed.

Full scheduler test suite:

Ran python -m pytest tests/schedulers/ -q both before and after the implementation.

  • After the fix and test updates, the targeted regression tests and existing UniPC timestep test pass.
  • Four DPMSolverSDEScheduler tests fail identically on the unmodified codebase and with this fix applied. These failures are unrelated to the modified schedulers.
  • No additional failures were identified in the reported test runs.

No new dependencies are required.

AI-assisted development / self-review

ChatGPT was used to assist with drafting the implementation and writing regression tests. The changes were applied and tested locally.

Review notes:

  • Verified that non-flow timestep conversion remains unchanged.
  • Verified that the new regression tests fail before the fix and pass afterward.
  • Updated the existing UniPC assertion that depended on integer timestep behavior.
  • Confirmed that the four DPM-SDE failures also occur without the changes.
  • Full model-level numerical quality validation was not performed.

The repository's dedicated AI self-review workflow has not yet been run.

Before submitting

Who can review?

@dg845 @yiyixuxu

@mangeshraut712 mangeshraut712 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matching the conditioning timestep to the sigma the solver actually integrates makes sense for flow models.

Comment on lines +486 to +489
if self.config.use_flow_sigmas:
self.timesteps = (
self.sigmas[:-1] * self.config.num_train_timesteps
).to(device=device)

@mangeshraut712 mangeshraut712 Nov 10, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same use_flow_sigmas -> torch.int64 truncation is in the other schedulers that support flow sigmas: scheduling_deis_multistep.py (L342), scheduling_dpmsolver_singlestep.py (L418), scheduling_dpmsolver_multistep_inverse.py (L361) and scheduling_sasolver.py (L340) on main. With only UniPC and DPM-Multistep changed, switching a flow pipeline between e.g. DPM-Multistep and DPM-Singlestep would change the timesteps the transformer sees, from fractional to truncated. Could the same branch go into those too (and their classes into the parametrized regression test), or is there a reason to leave them out?

Edge case: in DPM-Multistep, use_flow_sigmas is checked after the use_karras_sigmas / use_lu_lambdas / use_exponential_sigmas / use_beta_sigmas branches, and __init__ only rejects combinations among beta/exponential/karras, not with flow. With use_flow_sigmas=True plus e.g. use_karras_sigmas=True, self.sigmas holds the DDPM-derived Karras sigmas (up to ~14 for SD-style betas), so self.sigmas[:-1] * num_train_timesteps gives timesteps well beyond 1000 where _sigma_to_t used to keep them in range. Gating this on the flow branch actually having produced the sigmas (or rejecting flow + karras/lu/exp/beta in __init__) would avoid that.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UniPC and DPM-Solver with use_flow_sigmas condition the model on integer timesteps while integrating with float sigmas

2 participants