Skip to content

Fix race condition on GPU - #4542

Merged
JackCaoG merged 1 commit into
pytorch:masterfrom
ymwangg:fix_race_condition
Feb 1, 2023
Merged

JackCaoG merged 1 commit into
pytorch:masterfrom
ymwangg:fix_race_condition

Conversation

@ymwangg

@ymwangg ymwangg commented Jan 31, 2023 •

Copy link
Copy Markdown
Contributor

This is a temporary fix for #4541 until it can be fixed on tensorflow side.

Btw: this PR should also fix the flaky ddp test on GPU.

@ymwangg
ymwangg force-pushed the fix_race_condition branch from 168d1eb to 55652cd Compare January 31, 2023 21:31
Comment thread tf_patches/gpu_race_condition.diff Resolved
@JackCaoG

JackCaoG commented Feb 1, 2023

Copy link
Copy Markdown
Collaborator

Thanks @ymwangg !

@JackCaoG
JackCaoG merged commit 33e87a2 into pytorch:master Feb 1, 2023
@JackCaoG

JackCaoG commented Feb 1, 2023

Copy link
Copy Markdown
Collaborator

FYI @alanwaketan since we can enable the DDP test for GPU now.

mateuszlewko pushed a commit that referenced this pull request Mar 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants