Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ddp_find_unused_parameters_true strategy alias in Fabric #20037

Open
tesslerc opened this issue Jul 2, 2024 · 1 comment
Open

Add ddp_find_unused_parameters_true strategy alias in Fabric #20037

tesslerc opened this issue Jul 2, 2024 · 1 comment
Labels
bug Something isn't working help wanted Open to be worked on strategy: ddp DistributedDataParallel ver: 2.2.x
Milestone

Comments

@tesslerc
Copy link

tesslerc commented Jul 2, 2024

Bug description

The documentation (https://lightning.ai/docs/fabric/stable/api/fabric_args.html#strategy) states that to run DDP with "find_unused_parameters=True" we can use the strategy string "ddp_find_unused_parameters_true".
This fails with Fabric.

What version are you seeing the problem on?

master

How to reproduce the bug

Fails: `Fabric(strategy="ddp_find_unused_parameters_true")`

Succeeds: `Fabric(strategy=DDPStrategy(find_unused_parameters=True))`

Error messages and logs

ValueError: You selected an invalid strategy name: `strategy='ddp_find_unused_parameters_true'`. It must be either a string or an instance of `lightning.fabric.strategies.Strategy`. Example choices: auto, ddp, ddp_spawn, deepspeed, dp, ... Find a complete list of options in our documentation at https://lightning.ai

Environment

Current environment
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0): 2.3.1
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source): pip
#- Running environment of LightningApp (e.g. local, cloud): cloud

More info

No response

cc @justusschock @awaelchli

@tesslerc tesslerc added bug Something isn't working needs triage Waiting to be triaged by maintainers labels Jul 2, 2024
@awaelchli awaelchli changed the title [Fabric] Add strategy="ddp_find_unused_parameters_true" alias in Fabric Jul 5, 2024
@awaelchli awaelchli changed the title Add strategy="ddp_find_unused_parameters_true" alias in Fabric Add ddp_find_unused_parameters_true strategy alias in Fabric Jul 5, 2024
@awaelchli
Copy link
Member

awaelchli commented Jul 5, 2024

I think this was just forgotten. Let's add the alias for this strategy configuration. Contribution from anyone welcome of course!

@awaelchli awaelchli added strategy: ddp DistributedDataParallel help wanted Open to be worked on and removed needs triage Waiting to be triaged by maintainers labels Jul 5, 2024
@awaelchli awaelchli added this to the 2.4 milestone Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on strategy: ddp DistributedDataParallel ver: 2.2.x
Projects
None yet
Development

No branches or pull requests

2 participants