Skip to content

[PT/XLA] Reorganize DDP tests - #793

Merged
will-cromar merged 2 commits into
masterfrom
wcromar/pjrt-ddp
Jan 28, 2023
Merged

will-cromar merged 2 commits into
masterfrom
wcromar/pjrt-ddp

Conversation

@will-cromar

Copy link
Copy Markdown
Collaborator
  • Set torch.distributed env vars for XRT DDP tests
  • Remove tests for custom PJRT DDP
  • Add a convergence test with DDP
  • Re-enable some PJRT tests and disable some XRT tests

Depends on pytorch/xla#4520

@will-cromar
will-cromar requested a review from JackCaoG January 27, 2023 17:53
mnist + convergence + v2_8 + timeouts.Hours(1) + mixins.Experimental,
mnist + convergence + v2_8 + timeouts.Hours(1) + tpuVm + mixins.Experimental,
mnist + convergence + v2_8 + timeouts.Hours(1) + pjrt + mixins.Experimental,
mnist + convergence + v2_8 + timeouts.Hours(1) + pjrt,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to add mnist conv test back? Would resnet be enough?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this one back since it's our only e2e test that works on TPU v2 that I know of. I can take it off of the schedule if there's another test that covers PJRT + TPU v2.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can keep it.

@will-cromar
will-cromar merged commit dabe096 into master Jan 28, 2023
@will-cromar
will-cromar deleted the wcromar/pjrt-ddp branch January 28, 2023 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants