Send images #69
Comments
|
@cammellos @yenda @flexsurfer we have these user requirements to build on for image sending. What's the point of discrepancy with what we already have? It would be good if we could capture pros/cons for alternative solutions, along with necessary third party services or dependencies. |
|
I think we floated a few options: IPFSIPFS in its current form requires "pinning" in order for the content not to be recycled. Link sharingThis is just normal image handling, you upload a link to any image on any domain (ipfs gateways included), and you display it to the user. Sending content through wakuThis would work just the same as how messages are currently exchanged, sent through waku likely through a sort of multi-part protocol. Other options might be possible. What I think are the pros and cons of each: IPFSPro
Cons
Link sharingPro
Cons
Sending content through wakuPro
Cons
ConclusionsI personally would go for a mix of all of them, basically how overall I would see it working is;
What I don't know is how much impact will be to send images through waku, I don't currently see a big problem with it (we can send them in an efficient format, I don't see the advantage of using IPFS exclusively, in terms of privacy is only better if we actually expect users to have their own "trusted" gateways, which I don't believe user have or do, as IPFS gateways are technologically identical to web servers so assuming that are more secure gives users a false sense of security. |
|
I think the profile image feature is related to this one. We will have to decide how to store the image in a way that doesn't leak user information. I don't think it would be desirable if by resolving ens names and profile image the user ip is revealed to a third party for instance. Otherwise that would be too easy to get someone's ip by sending them a message with an account that has an ens name. |
in a separate discussion we're thinking of adopting webp image compression for use with other image assets. With a heavier than average compression I'm thinking we could go down to about ~500kb per image uploaded. That's considerably less than the 2mb / image of the usual suspects. Another thing, not sure if it applies here, typically apps store multiple versions of the same image, a low-res thumbnail minuscule in size shown in chat and the full res image that only gets downloaded once the user specifically requests it, usually when opening the full screen preview |
|
There's also to consider encryption, we can't probably ask the user to upload an image to IPFS without encryption (for one-to-one or group chats for example).
This should be possible in case we host images somewhere else (so the waku message might contain a thumbnail). |
|
just for the record I really like the Waku only solution, that's why I floated the numbers, I could help with providing some data on average image sizes if we're thinking of running a simulation. |
|
please, I really have no idea myself :) we can put a cap as well |
|
so there are two things we can do to decrease image file size, one is size, we could downscale images so their longer side is no greater than 2000px. Typically that value is around 4000px for images captured with phone camera or stored in your gallery. The other is compression. edit: I'm also attaching how the same would look with regular JPG 60% compression which is faster computation wise and afaik wouldn't require additional frameworks
|
|
waku and webp looks really interesting especially for 1-1 and group chats |
|
Yes, to me that sounds the most obvious first step, it has low security implications, and should work out mostly out of the box, I would give this a try first. |
|
the pros of sending as markdown we don't need any implementation on protocol level then, and users could use this feature without uploading images in status, they could just send strings, not sure if we have limits for message content probably we should have so users don't send huge images at least from the status app |
|
would markdown mean that in image is placed in the regular message container, just instead of text? That's something I wouldn't like us to do, I'd prefer if the image filled the entire message bubble, without the extra padding text has.
|
|
yeah we could do the same as we do (or did?) it for emoji, if there is only one image in message show it without bubble |
|
Say we embed them in markdown, how would you handle displaying (I don't mind either approaches myself): |
|
Another issue is that for older clients this will be splat out , as they don't have a concept of images: |
|
regarding markdown, will we be able to know if a message is an image for the purposes of copying pasting, long-press selection, replies etc.? for example if we were to attach individual actions for image type messages? Also for previewing, it's critical we add the tap to preview in full screen view to get the full UX you'd come to expect |
|
Generally I would say yes, in some implementations it might be slightly more involved, but it should be fine in most cases, so I'd say yes |
|
What do we need to get the ball rolling on this one? It seems to me that there's some consensus in at least trying with images over waku, rather than IPFS, at least to start with. This would mean for the time being that images need to be below 1MB for now, which seems acceptable. Webp seems like the format we'd want to support, although I think we can support pngs as well just as easily. The points still to clarify are:
Embedded in markdownPros
Cons
As a separate message typePros
Cons
I probably would favor approach 2 for now, in terms of effort they are basically the same and I think it's a bit more flexible for the time being, although we won't be able to mix text an images (same as whatsapp for example, we can still add a caption of course). What do you think? |
|
Re larger wire size, would it make sense to general-purpose-compress all payloads with something like snappy? This is what's done in eth2 for example |
|
It would make sense to compress payloads in most cases, as definitely some will benefits from it (images probably won't as they are already highly compressed). The best way to do this is probably to compress at the RLP level, where we can negotiated the protocol between each host, because otherwise we would get into compatibility issues (in public chat we don't know the recipients, so effectively we have to always support non-compressed messages) |
just to chime in, WebP is a compression algorithm that can output to any image file container be it a PNG or JPG. The format we want to use is JPG, with or without WebP (we have to test how it performs on mobile, it's noticeably more resource intensive when I used it on an 8-core Mac, compared to regular JPEG compression, it's also not as clear if we'll get smaller file sizes than regular JPEG, see the table.) |
|
regarding webp in react native |
|
thanks for the clarification @errorists |
|
Also, there's a difference on what we send to the wire and what we display. For example is perfectly ok to send We have quite a lot of freedom here, and we can delay the exact choice of the format once we see how it behaves (changing it it's pretty straightforward), |
|
Separate message, each image container aka 'message bubble' can only have a single image inside, we shouldn't mix text and images, those would be split into two separate messages on the list : one for image, another for text |
might get some of the base64 losses back though if going with the embedded option - and indeed, doing it at a lower level makes sense - I mentioned mainly in case size was the only blocker against embedding. |
|
@arnetheduck thanks, it's a good suggestion, we'll see if we can push this forward with @errorists thanks, given those consideration is probably safer for now to have a separate content-type for messages, @flexsurfer any objection? |
|
@cammellos with regards to moving this forward. Here are some of the required actions I see. Please tell me if I'm missing any:
|
|
no objections :) |
|
|
To clarify, in what chats can Image Content type be used? Public, 1:1, Group? |
|
yes, any kind of chat |
|
For reference, these are all related actions:
|
|
i can take 4-6 if no objections |
|
Awesome @flexsurfer, thanks! Any takers for 1? I'm also happy to make a start but in that case I'll still need someone to fill in the blanks |
|
#69 (comment) should be enough for the technical specs? |
|
3 is also not needed and won't be done as part of this. Also, should we capture everything in one tasks with a user story and a clear deliverable? (for example is replying to images necessary to go live or can be done in a second etc) we can split it off in further issues if necessary, but we should encourage working together on this and not each one on separate tasks, as we can get into some issues with integration/discrepancies |
|
@cammellos can you take that and submit a PR on status-im/specs? Also, is there a way to do this work in an "adjacent" topic? ie. something users can opt-out or opt-in on? |
|
Some more specifics on what happens to images once displayed would be helpful for |
|
Are people seeing this :) is the 3rd time I post it: SPEC CHANGES #107 |
Sorry, the numbers make it look like priorities. Noted, not a requirement. Added the note Optional.
I was hoping this could serve as Epic issue. Happy to create one elsewhere if that works better. Agree, added links to next issue and the full list of epic tasks to individual tasks.
That's my bad! Sorry! Was going by the new issue in status-react. Didn't realize how extensive the doc already is |
Yes, you could post an image on a separate topic and the actual message on the main topic (or post both in the "image topic" but in that way you won't be able to show the user a placeholder for the picture) and only fetch if the user explicitly allow that. It's a bit cumbersome and you might get into a place where the image was not delivered etc, it would save bandwidth to those users who don't want to fetch images, while slightly increase bw for those who are always opted in |
|
Awesome, thank you! I'm obviously assuming this would become an issue (ie. folks wanting to opt out) so I might be making problems up where they don't exist. If we send images directly now, would that be a "one way decision?" Can that be revisited in the future and re-implemented with the adjacent topic if folks express that opt-out is important to them?
It sounds like we have functional and technical specs for this now. I'd like to see a list of issues (that reference this one) that take no longer than 3 days to complete. Basically, if we understand the work well enough, we should understand it to the point where we can plan and split up the work. |
Yes and no :) So say we go live with this, any client with this version will send images on the main topic, all the clients with this version will be able to receive this. At some point we want to move to a separate topic. What will happen is that in newer client everything will work as expected, while older client won't be able to see those images, so we could show some text saying "Upgrade to the latest version to see the image" (which is similar to what we'd show now to existing users once images in any shape or form go live, although they probably will see something like "content-type x not handled", but that we will fix for the next release) If that's ok then we can do it once there's a clear need. |
|
@cammellos @flexsurfer do you see any tasks here that would exceed 3 days to complete? #69 (comment) Disregarding item 3 and 7 for now |
|
Time estimates again 🤦 3 days is unlikely to be enough for any complex task imo if you take into account testing and QA, and this tasks will likely need manual QA. 4 is definitely more then 3 days to be conservative, disregarding the fact that some code is out already. |
|
Update regarding images: Current PoW target is set to I am on a different task this morning, but after that I will re-evaluate, see if there's a potential solution keeping images in waku, otherwise time for plan b. |
|
what is plan b? |
|
offtopic: I'm wondering how it worked for avatars, we had pretty huge images for avatars |
|
We are talking about decreasing the PoW, that's one option. It's a good point for avatars, as a matter of fact I believe many times the image was not sent (it's probabilistic, so you might get lucky every now and then a get it). For example when syncing contacts I had to remove images as they would not be dispatched (I believed at that time it was the size rather than the PoW, but likely it was the PoW instead). |
|
so if we're going waky way my suggestion is to add image sending support only for 1-1 and group chats for the first time and see how it goes, wdyt @andremedeiros @cammellos @hesterbruikman |
Agreed. Less spam sensitive as well. Is it feasible to offer link unfurling for public chats in parallel? |
This is a feature to be tracked separately, probably best to have a separate conversation, but it has no dependencies on this work. |
|
Url handling in chat epic for future reference: status-im/status-mobile#10516 |
Image Sending Specs
cc @errorists @flexsurfer :)
User Stories
As a user, I want to send media from my device in 1:1, group and public chats.
POC: status-im/status-mobile#9455
MVP: Still images only
Technical Requirements
v1, v2: UI enhancements
To be prioritized according to user feedback. Ranked here according to effort/complexity:
Technical Requirements
Future versions: Send video and other files
The text was updated successfully, but these errors were encountered: