Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider deferred tasks as running in heartbeat #40453

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

0x26res
Copy link
Contributor

@0x26res 0x26res commented Jun 27, 2024

close #40435

@boring-cyborg boring-cyborg bot added the area:Scheduler Scheduler or dag parsing Issues label Jun 27, 2024
@0x26res
Copy link
Contributor Author

0x26res commented Jul 5, 2024

@sunank200 I've noticed you are working on similar stuff (#36090). I'd be curious to have your opinion about this change.

Also, I have a question related to the underlying issue. I've noticed this happens if a lot of deferred tasks are submitted at once. The server I'm running this on is very small (2 CPU, little memory). I think this is means the timing of the heart beat is messed up by a straved CPU. Is there a way to configure my scheduler so that these deferred task are not all submitted at once, but throttled (but still eventually running in parallel)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Scheduler Scheduler or dag parsing Issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

deferred tasks get kill during heartbeat callback in some rare cases
1 participant