Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modin is unable to handle ray cluster scaling from 0 #4450

Open
RehanSD opened this issue May 10, 2022 · 1 comment · May be fixed by #4451
Open

Modin is unable to handle ray cluster scaling from 0 #4450

RehanSD opened this issue May 10, 2022 · 1 comment · May be fixed by #4451
Labels
P2 Minor bugs or low-priority feature requests

Comments

@RehanSD
Copy link
Collaborator

RehanSD commented May 10, 2022

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  • Modin version (modin.__version__):
  • Python version:
  • Code we can use to reproduce:

Describe the problem

When we have a ray cluster that autoscales from 0, the ray.cluster_resources() dictionary does not initially include any CPU resources, and so Modin errors out. We should probably catch this case, use a default, and warn the user to update NPartitions appropriately.

Source code / logs
@modin-bot
Copy link

This issue has been mentioned on Modin Discuss. There might be relevant details there:

https://discuss.modin.org/t/placement-group-in-ray/303/2

RehanSD added a commit to RehanSD/modin that referenced this issue May 10, 2022
@RehanSD
FIX-modin-project#4450: Ensure Modin successfully initializes when Ra…
2683d06 
…y cluster has no resources.

Signed-off-by: Rehan Durrani <[email protected]>
@RehanSD RehanSD linked a pull request May 10, 2022 that will close this issue
Open
8 tasks
@vnlitvinov vnlitvinov added the P2 Minor bugs or low-priority feature requests label Sep 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P2 Minor bugs or low-priority feature requests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

FIX-#4450: Ensure Modin successfully initializes when Ray cluster has no resources RehanSD/modin
3 participants
@RehanSD @vnlitvinov @modin-bot