Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux example Auto DeviceMap panic #1165

Closed
kharish89 opened this issue Feb 28, 2025 · 1 comment · Fixed by #1187
Closed

Flux example Auto DeviceMap panic #1165

kharish89 opened this issue Feb 28, 2025 · 1 comment · Fixed by #1187
Labels
bug Something isn't working

Comments

@kharish89
Copy link

I'm trying the flux model in example and have an issue with the device mapper set as auto
❯ cargo run --features metal --example flux --release

cargo run --features metal --example flux --release
Compiling mistralrs v0.4.0 (/Users/xxx/git/mistral.rs/mistralrs)
   Finished `release` profile [optimized] target(s) in 2.03s
    Running `target/release/examples/flux`
2025-02-27T08:25:19.294188Z  INFO mistralrs_core::pipeline::loaders::diffusion_loaders: Loading `flux1-schnell.safetensors` locally at `black-forest-labs/FLUX.1-schnell/flux1-schnell.safetensors`
2025-02-27T08:25:19.294276Z  INFO mistralrs_core::pipeline::loaders::diffusion_loaders: Loading `ae.safetensors` locally at `black-forest-labs/FLUX.1-schnell/ae.safetensors`
2025-02-27T08:25:19.294331Z  INFO mistralrs_core::pipeline::loaders::diffusion_loaders: Loading `transformer/config.json` locally at `black-forest-labs/FLUX.1-schnell/transformer/config.json`
2025-02-27T08:25:19.294338Z  INFO mistralrs_core::pipeline::loaders::diffusion_loaders: Loading `vae/config.json` locally at `black-forest-labs/FLUX.1-schnell/vae/config.json`
Error: .into_mapper does not work on Auto device map, convert it to a Map with DeviceMappedModelLoader::get_device_layers```

I checked the error is coming from the diffusion_model.rs which sets the loader to be Auto https://github.com/EricLBuehler/mistral.rs/blob/e2f9648cbb6e4b8a6ac1d9e44df5900210fa5862/mistralrs/src/diffusion_model.rs#L96 and the device_map.rs is setup to panic here https://github.com/EricLBuehler/mistral.rs/blob/e2f9648cbb6e4b8a6ac1d9e44df5900210fa5862/mistralrs-core/src/device_map.rs#L240
I'm not sure if I'm missing something, any help is appriciated.
@kharish89 kharish89 added the bug Something isn't working label Feb 28, 2025
@EricLBuehler
Copy link
Owner

@jorgeantonio21 this should be fixed in #1187.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants