-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API call to pubsub/sub does not return HTTP header until first message arrives #3304
Comments
License: MIT Signed-off-by: Jan Winkelmann <[email protected]>
…yet. fixes ipfs#3304 (ipfs#3305)" This reverts commit 68d8a29. License: MIT Signed-off-by: Jan Winkelmann <[email protected]>
Revert "http api: makes sure header is sent even when r is not ready yet. fixes ipfs#3304 (ipfs#3305)" This reverts commit 68d8a29. License: MIT Signed-off-by: Jan Winkelmann <[email protected]> License: MIT
Revert "http api: makes sure header is sent even when r is not ready yet. fixes ipfs#3304 (ipfs#3305)" This reverts commit 68d8a29. License: MIT Signed-off-by: Jan Winkelmann <[email protected]>
I guess we can reopen this... This is also related to #3367. |
@whyrusleeping can you reopen? Thanks! |
Thanks for reopening. What do you think of this? ping @haadcode |
@keks i'm open to changing the api here. the pubsub code is clearly marked as experimental, so if we need to change things to make it better, lets do it. |
Sure. If we're not going to make big changes here, I feel like this corner case should be well-documented so when someone stumbles into it they don't have to dig as deep as I did. It's cool doing it once, but already at the second time it's a waste of time. You get to know the code base though, which is good I guess :) |
haha, where do you think that documentation would best fit? |
I hoped you knew that :) Am 10. November 2016 21:49:09 MEZ, schrieb Jeromy Johnson [email protected]:
|
Maybe we should write a small guide on 'how to write a new ipfs command' ? |
Let's move this discussion to #3367 and concentrate on the the pubsub issue here. |
I looked into how the I lean towards rewriting the pubsub command code but that seems like it will be some work. What do you think @whyrusleeping? |
@keks we could do something like send a message towards the user with no topic ID set, and then catch that as a 'hello, welcome' message. That seems unintrusive enough to me. |
Yeah, but that seems clumsy from a protocol perspective. I had the idea of a "status" topic, similar to the status channel on irc. You would then receive a message saying "you are now subscribed to topic x". I think that is unintrusive and expressive, I'll proceed later today. Another problem ist that I need to import |
Sorry to be late to the game. Yeah, this has been a problem in js-ipfs-api too, so getting a confirmation message/callback would be great! |
@keks we could have a different struct defined in the floodsub commands code that is basically the same as the floodsub.Message struct, but has an extra field that would allow us to patch in that "hello". I'm hesitant to add a message that actually says something because then users have to ignore that every time. It needs to be very easily distinguishable from data (such that making the mistake of considering that message to be data becomes very difficult). |
This sounds like a commands/corehttp implementation detail leaking through. Can we fix this there before adding complexity in different place? |
@whyrusleeping We can make *-ipfs-api ignore that message @lgierth I think you mean Edit: I guess we could add a Edit2: Maybe we could add a |
By the way, why don't we just write to the output Another thing I did is dig into how the commands are called. I know this again does not seem very clean, but we simply can't do it in What do you propose how to proceed @whyrusleeping @lgierth? |
Actually, having tried to fix things on JS side, getting this fixed is required to make things work nicely in js-ipfs-api. Hope this is prioritized. |
Yeah, I'm trying to wrap my head around this. I don't come up with an elegant solution though, the command interface is really weird. |
Yeah. Agreed. @keks the reasoning for the |
Yeah, I think this is probably our best solution for now. We should be careful to make sure that api clients don't come to expect this hello message though. Once we fix the underlying problem, we will want to remove it |
Well but why don't we just expose the output writer to the command? Sure, we couldn't set the channel as output, but what is the benefit of that? Why not simply write a Marshaller from the type e.g. like this:
You would call all this in the I'd love to be convinced that the way we do things now are actually a good idea but this seems pretty over-engineered. E.g. the channel abstraction appears not to be helpful at all and you just lose information about what you are working with. I tried to find a way to do this with the current structure but failed. I made a type that passes on the Really, please. Let's just do the Marshalling and Unmarshalling "manually" (e.g. create a type that does this for us) and expose the I am pretty tired and I hope you get what I'm getting at. Let's pass the response writer to |
I have an elegant workaround for this in my (test) code: out <- string("{}") Works beautifully ;) |
I was trying to solve the more general problem as @lgierth suggested in #3304 (comment). Also, if what you post works, |
@haadcode where do you put this? I remember having tried to put something like that before this line, but the commands lib only expects Really, there is too much magic going on here. Let's keep it simple and pass the Also if I should proceed we should open a new issue that reflects the current take on things. |
Oh and also @Kubuxu |
Here is a gist that gives an example for my approach: https://gist.github.com/keks/0f2412fa616129cfc1af3ff011f67d7c Imagine the dashes in file names were slashes - gist doesn't allow directories |
@keks At first glance that seems fine to me, the only caveat is we have to make sure to handle the case where we arent using the API. In that case we arent marshaling to json, we're using the objects directly. The abstraction here is that the commands shouldnt care that they are going over an http transport, this commands lib was designed to be agnostic to how it was being run. (i'm not saying this is the best design, its just what we have and we can't break the api for users at this point) If you can make the changes you propose without having to change the API of every other command, then go right ahead. |
I put the out <- string("{}") right there where you tried it too. Dunno why it works, but it does the trick. |
Thanks for weighing in. I wasn't aware of this, but this definitely makes sense. That makes my proposal difficult, though.
I'll try to find a design that also works for sending values directly (no API) and a good transitioning plan so we don't need to change all at once.
Ohhh, now I get what's happening. I was testing with |
@keks maybe we can expose some kind of "Start Response" method through the response object that different commands can call (optionally) as needed. |
So I have a workaround ready in https://github.com/keks/go-ipfs/tree/feat/instaflush but it needs #3402 to merge first. I realized |
I tested @keks' PRs above this morning and the tests I wrote for js-ipfs-api pass with flying colors. That is to say: what is PRed here fixes the problems we've been having on the JS side! \o/ 👍 for getting these merged asap. |
* commit '6f3ae5da293f971c09e3e6fc0763aeca04ba98d3': rename DataSize to FileSize update HashOnRead validation to properly support cids http api: makes sure header is sent even when r is not ready yet. fixes ipfs#3304 (ipfs#3305) fix add/cat of small files raw dag: make raw nodes work in cat and get, add tests feat: make metrics injection log an error instead of warning test: check if metrics work in sharness Fix metrics being injected after node initalization unixfs: allow use of raw merkledag nodes for unixfs files ds-help: add helper func to convert from Cid to DsKey and the reverse cli: refactor to expose argument parsing functionality # Conflicts: # core/commands/add.go
This way calling
"go-ipfs-api".Request.Send()
blocks until the first message is received. Which is not nice.The text was updated successfully, but these errors were encountered: