-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documention for send/recv OP #7904
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Please fix a few minor typos before merging. Thank you for the PR!
doc/design/send_recv.md
Outdated
|
||
## Recv OP | ||
|
||
The recv OP receives its input tensors, run the optimization block, and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
run -> runs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Done.
paddle/operators/recv_op.cc
Outdated
AddComment(R"DOC( | ||
Recv operator | ||
|
||
This operator will recieve tensor from send_op | ||
This operator receives its input tensors, run the optimize block, and serves the tensors requested by the send OP. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
run-> runs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Done.
17d72ef
to
91eb6d5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much for the PR! I just have a few questions that I have added in comments.
|
||
|Attribute|Description| | ||
|------|------| | ||
|endpoints|Endpoints to send the variables to| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would this be a list of OPs or just a list of receive OP endpoints ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a list of receive OP endpoints.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And receive OP endpoints is basically the parameter_server endpoints, right ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, they are the same.
|------|------| | ||
|RX|Input tensor to be received| | ||
|
||
|Attribute|Description| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where in this setup does the receive OP get the list of tensors as an input from send OP ?
Or would we have one receive OP per parameter ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The input of send of will be the same as the input of recv op, if there is only 1 recv op. They are all setup by the transpiler.
Currently there is one send op per trainer, and one recv op per pserver. We will move to one send/recv pair per parameter in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much, that helps!
ad47433
|
||
## Send OP | ||
|
||
The send OP sends its input tensors to one or multiple recv OPs and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The send OP sends its input tensors to one or multiple recv OPs and
I think it's not only tensor, but also SelectedRows and in fact, send op could send all kinds of variable which implement the Serialize and Deserialize interface, but now, it only supports LodTensor and SelectedRows.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have a Python example to show what are the parameters of a send and a recv operator?
For example,
X = fluid.Tensor(10, 10)
fluid.send("192.168.0.1:5505", X) # I don't know if these are the parameters required by send.
# Send and Recv OP | ||
|
||
With PaddlePaddle Fluid, the node to node communication is done | ||
through the send and recv OP. The send and recv OP will be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OP => operator
# Send and Recv OP | ||
|
||
With PaddlePaddle Fluid, the node to node communication is done | ||
through the send and recv OP. The send and recv OP will be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"will be" is passive. Please try not to use passive voices.
training `ProgramDesc`. | ||
|
||
|
||
## Send OP |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Send Op => The Send Operator
|
||
## Send OP | ||
|
||
The send OP sends its input tensors to one or multiple recv OPs and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that Send is a broadcasting operation, as you said it could send tensors to more than one recv operators? Is a broadcasting send operator what we want?
The send OP sends its input tensors to one or multiple recv OPs and | ||
then fetches its output tensors from the recv OPs. | ||
|
||
|Input|Description| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need tables here?
Fixes: #7905