Use random default port instead of a fix one

[magic282]

Before submitting

• [y] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
• [y] Did you read the contributor guideline?
• [y] Did you make sure to update the docs?
• [n] Did you write any new necessary tests?

What does this PR do?

Fixes #485.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[Borda]

pls, drop the as e

[Borda]

user numpy.random.randint()

[williamFalcon]

actually, this won’t work.
All processes need the same port. making the port random will break ddp

[Borda]

so what about random port which will be the save over the package?

[williamFalcon]

how? ddp starts a process independently for each GPU, and you can’t guarantee the same number.

[Borda]

You would have to set it in init, but I need to have look at it in detail...

[williamFalcon]

can’t set in init because the processes all run at different times without guarantee of the same start time

[magic282]

How about hashing the python cmd args to a get a fix random number for all processes as a workaround?

[Borda]

well if you set numpy.random.seed(42)then all random numbers are the same, it is commonly used for tests with random matrixes to have every time/run the same random numbers...

[williamFalcon]

random seed is standard for anything ML. that doesn’t solve the problem though. but hashing the command might be a good option