From 829ac10b151f9f07c44b0cd669d97764c84a06f5 Mon Sep 17 00:00:00 2001 From: Jacob Heun Date: Mon, 4 Nov 2019 19:16:52 +0100 Subject: [PATCH 1/6] Add notes for core-implementations-weekly 2019-11-04 --- ...2019-11-04--core-implementations-weekly.md | 223 ++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md new file mode 100644 index 00000000..7814a90a --- /dev/null +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -0,0 +1,223 @@ +# 🚀 IPFS Core Implementations Weekly Sync 🛰 Nov 4, 2019 + +- **Lead:** @alanshaw +- **Notetaker:** @jacobheun +- **Attendees:** + - @jacobheun + - @lidel + - @aschmahmann + - @stebalien + - @vasco-santos + - @achingbrain + - @dirkmc +- **Recording:** https://youtu.be/ww4dQWtnRc8 + +## Agenda + +- Ensure a notetaker exists! +- Ask everyone to put their name into the list of attendees +- Initiatives +- Blockers (from the async status update) +- General questions. Could be things like: + - I'm stuck with something, I don't know who to ask. Who knows who to ask? + - Who can help me with xyz? +- Plan for this week + - Select issues to work on +- Review remaining parking lot topics if there is time left + +## Notes + +### Initiatives +> Update or add info on initiatives that are happening + +#### Upgraded Release Process +@stebalien / @achingbrain + +- js-ipfs 0.40 release issue created + - https://github.com/ipfs/js-ipfs/issues/2558 + - Repo migration tool, blocked on a PR for this +- Nothing new from go-ipfs + +#### Upgrade Testing Infra/Process +@jimpick / @stebalien / @alanshaw / @daviddias + +- The testground team is having a hack week. +- Working on automatically spinning up nodes, trying to spin up lots of DHT nodes +- Working on simulating latency, etc, to better mimic the network + +#### Subdomain Gateway (Base32, Origin isolation) +@lidel + +- JS land: + - js-peer-id: addressed review of [feat: support Peer ID represented as CID](https://github.com/libp2p/js-peer-id/pull/105), and it's merged! + - js-ipfs: redone [feat: js-ipfs support of CIDs in /ipns/ content paths](https://github.com/ipfs/js-ipfs/pull/2566) to make it shippable _today_, ready for review/merge + +#### Distributed Signaling (js-libp2p) +@jacobheun + +- Still on hold pending async/await + +#### IPNS (go-ipfs / js-ipfs) +@aschmahmann / @hugomrdias + +IPNS-over-PubSub + - https://github.com/libp2p/go-libp2p-pubsub/pull/184 landed 🥳 + - go-libp2p-pubsub v0.2.1 released + - in the process of being integrated into go-libp2p-pubsub-router and then IPFS + - includes resolving [go-libp2p-pubsub-router#28](https://github.com/libp2p/go-libp2p-pubsub-router/issues/28) + +#### Add Performance (go-ipfs) +@aschmahmann + +Plan is to figure out how to do asynchronous database writes for IPFS Add +Three options: + - Extend the datastore interface to allow asynchronous writes/flushes [go-datastore#137](https://github.com/ipfs/go-datastore/issues/137) + - Write/improve our asynchronous writing layer on top of a synchronous database (e.g. the BufferedDAG) + - Write an asynchronous datastore on top of a synchronous datastore with periodic flushing + +Currently going to try and improve the BufferedDAG while pushing on the datastore interface improvements + +#### Migration to multihash keys in blockstore (js-ipfs) +@alanshaw / @auhau + +- IPFS repo migrator tool integration into ipfs-repo PR in progress + - https://github.com/ipfs/js-ipfs-repo/pull/202 + +#### IPFS Mount +@djdv + +- No update + +#### Bitswap Updates +@dirkmc + - Working through code reviews for [Bitswap proof-of-concept](https://github.com/ipfs/go-bitswap/pull/189) + - [Reviews and changes to go-peertaskqueue](https://github.com/ipfs/go-peertaskqueue/pull/8) + - [Test plans](https://github.com/ipfs/testground/issues/79) for Bitswap in testground + - Parallel reads PR merged and deployed to cluster. + - Shows significant (2x?) improvement in throughput for fetching filecoin proof parameters. + +#### js-ipfs/js-libp2p/js-ipld async await refactor +@achingbrain / @jacobheun / @vasco-santos / @alanshaw + +- js-libp2p + - plaintext 2, internal crypto upgrading and pnet have been merged [js-libp2p#469](https://github.com/libp2p/js-libp2p/pull/469) + - PeerStore refactor in progress [js-libp2p#470](https://github.com/libp2p/js-libp2p/pull/470) + - Connection Management Registrar in progress [js-libp2p#471](https://github.com/libp2p/js-libp2p/pull/471) + - Pubsub PRs in review. Need the Registrar and PeerStore work to complete. + - Identify refactor underway [js-libp2p#473](https://github.com/libp2p/js-libp2p/pull/473) + - This will also include support for the [Identify Push protocol](https://github.com/libp2p/specs/tree/master/identify#identifypush) + - Circuit Relay and Discovery refactors are next on the list + +### Design Review Proposals +> You want to propose something for design review. + +- @alanshaw: (pre-review) Proposal for uniting the files APIs + - https://github.com/ipfs/interface-js-ipfs-core/blob/feat/unite-the-files-apis/SPEC/FILESv2.md +- @achingbrain: Proposal for improving the HTTP API (streaming, error handling, general conventions) + - https://github.com/ipfs/specs/pull/224, please get involved! + +### Blockers/Asks +> You are blocked and need to talk about it to help unblock. List it here! + +### Questions +> You have questions and need them answered. List them here. + +### Parking Lot +> Anything you would like to discuss that doesn't fit into any of the other categories. + +- Someone brought up an issue about adding large amounts of data and it creating memory leaks (go-ipfs) + - We might have something that's keeping pointers to things open + - Creating a nil repo, and it's not getting closed! Might be a 1 line fix + +### Week Update (for async review) +> Only post updates that are relevant to the IPFS Core Implementations Work. We know you do a ton more things but for this call, the focus are the implementations ❤️ 🌟 + +@alanshaw +- Done: + - (Almost done) Working on uniting the files APIs proposal + - https://github.com/ipfs/interface-js-ipfs-core/blob/feat/unite-the-files-apis/SPEC/FILESv2.md + - Reviewed ProtoSchool Tutorial for root level files API + - Reviewed adding automatic repo versions https://github.com/ipfs/js-ipfs-repo/pull/202#pullrequestreview-308485909 + - Reached consensus on removing `libp2p-websocket-star` and `libp2p-webrtc-star` from default `js-ipfs` bundle +- Next: + - Land IPFS repo migrations tool and start release train for 0.40 + +@lidel +- Done: + - cidv1b32: + - js-peer-id: addressing review of [feat: support Peer ID represented as CID](https://github.com/libp2p/js-peer-id/pull/105) + - js-ipfs: redone [feat: js-ipfs support of CIDs in /ipns/ content paths](https://github.com/ipfs/js-ipfs/pull/2566) to make it shippable _today_, ready for review/merge + - ipfs-companion: + - [fix: recovery quirks in Firefox and Chromium](https://github.com/ipfs-shipyard/ipfs-companion/pull/807) + - [Permanent in-depth review on Chrome Web Store](https://github.com/ipfs-shipyard/ipfs-companion/issues/808) + - Released [v2.9.0](https://github.com/ipfs-shipyard/ipfs-companion/releases/tag/v2.9.0) + - [Analysis of Manifest V3 preview shipped with Google Chrome Canary](https://github.com/ipfs-shipyard/ipfs-companion/issues/666#issuecomment-548943969) + - distributed-wikipedia + - tried to update tr.wikipedia-on-ipfs.org, identified some new pain points + - ipfs/team-mgmt: [helper for creating PRs with meeting notes](https://github.com/ipfs/team-mgmt/pull/1054) + - Reviewed + - Sync on Pinning UX PRD with Michelle + - ipfs-webui: [feat: allow urls with authentication](https://github.com/ipfs-shipyard/ipfs-redux-bundle/pull/28/files) + [fixed ci failures](https://github.com/ipfs-shipyard/ipfs-redux-bundle/pull/29#discussion_r342050199) + - js-ipfs: [replacing ws-star with ws-stardust](https://github.com/libp2p/js-libp2p-websocket-star/issues/70#issuecomment-548318235) + - ProtoSchool: [feat: Non MFS Files API tutorial](https://github.com/ProtoSchool/protoschool.github.io/pull/303#pullrequestreview-309933765) + - js-ipfs: [extending libp2p.transports/libp2p.discovery instead of overriding it](https://github.com/ipfs/js-ipfs/issues/2579#issuecomment-549399564) +- Next: + - test matrix for webui that can be run via `aegir text-external` + +@dirkmc +- Done: + - Working through code reviews for [Bitswap proof-of-concept](https://github.com/ipfs/go-bitswap/pull/189) + - [Reviews and changes to go-peertaskqueue](https://github.com/ipfs/go-peertaskqueue/pull/8) + - [Test plans](https://github.com/ipfs/testground/issues/79) for Bitswap in testground +- Next: + - Continue with code reviews for Bitswap proof-of-concept + + +@jacobheun +- Done: + - Connection crypto and pnet refactor is [complete](https://github.com/libp2p/js-libp2p/pull/469) + - Now uses plaintext 2.0, which exchanges public keys + - Fixed an issue with the async websocket transport and Buffer Lists [libp2p-websockets#97](https://github.com/libp2p/js-libp2p-websockets/pull/97) + - Started the Identify refactor, which also adds Identify Push [js-libp2p#473](https://github.com/libp2p/js-libp2p/pull/473) +- Next: + - Finish Identify refactor + - Peer Discovery and Circuit Relay refactors + - Support secio refactor + +@vasco-santos +- Done: + - PeerStore v0 to replace PeerBook + - [libp2p/js-libp2p#470](https://github.com/libp2p/js-libp2p/pull/470) + - Registrar for libp2p protocols + - [libp2p/js-libp2p#471](https://github.com/libp2p/js-libp2p/pull/471) + - Pubsub new iteration + - [libp2p/js-libp2p#467](https://github.com/libp2p/js-libp2p/pull/467) + - Draft PR for new `js-libp2p` API documentation + - [libp2p/js-libp2p#472](https://github.com/libp2p/js-libp2p/pull/472) + - Started PR for dht refactor +- Blocked: N/A +- Next: + - Finish pubsub refactor + - Continue DHT subsystem refactor + +Note: OOO 7th afternoon, 8th and 11th November + +@Stebalien: +- Done: + - Reviews + - Libp2p work & reviews +- Blocked: +- Next: + - Testlab hack week. + + +@aschmahmann: + - + +@name +- Done: + - What have you accomplished since the last Weekly? +- Blocked: + - Were there any blockers? If so, which ones? Is it still blocked? Why? +- Next: + - What is the next important thing you should focus on From 877c118f9e68a09d44d8c5d085ff3adde6160419 Mon Sep 17 00:00:00 2001 From: Henrique Dias Date: Mon, 4 Nov 2019 20:32:20 +0000 Subject: [PATCH 2/6] add async updates --- .../Q4/2019-11-04--core-implementations-weekly.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md index 7814a90a..af3c631a 100644 --- a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -209,7 +209,16 @@ Note: OOO 7th afternoon, 8th and 11th November - Blocked: - Next: - Testlab hack week. - + +@hacdias: +- Done: + - IPFS Desktop + - Re-add the feature of copying a shareable link to the clipboard when adding files to IPFS via drag'n'dropping files to the app icon ([PR](https://github.com/ipfs-shipyard/ipfs-desktop/pull/1221)) + - IPFS Web UI + - Allow basic authentication to connect to the API just by passing a url of the form http(s)://user:pass@host:port ([PR](https://github.com/ipfs-shipyard/ipfs-redux-bundle/pull/28)) + - Released 2.6.0 with peer's performance improvements and nearby + local network highlights ([release](https://github.com/ipfs-shipyard/ipfs-webui/issues/1085)) + - Cohosting + - `ipfs-cohost` 2.0 released, matching the SPEC! @aschmahmann: - From bdb3f2bba94b07c0c9b2c93ff77f78f65ed37fd1 Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Tue, 5 Nov 2019 10:03:03 +0000 Subject: [PATCH 3/6] docs: add async update --- .../2019/Q4/2019-11-04--core-implementations-weekly.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md index af3c631a..9470c0bb 100644 --- a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -220,6 +220,15 @@ Note: OOO 7th afternoon, 8th and 11th November - Cohosting - `ipfs-cohost` 2.0 released, matching the SPEC! +@achingbrain: +- Done: + - HTTP v1 Files spec: https://github.com/ipfs/specs/pull/224 + - Reviewed https://github.com/ipfs/specs/pull/223 +- Blocked: + - N/a +- Next: + - Help @pedromiguelss get ipfs/interop#86 across the line + @aschmahmann: - From 9b0b0c4e6b09cb962c69e5db0fc250fe867103dd Mon Sep 17 00:00:00 2001 From: Alan Shaw Date: Tue, 5 Nov 2019 10:05:01 +0000 Subject: [PATCH 4/6] feat: add pedro's update --- ...2019-11-04--core-implementations-weekly.md | 23 ++++++++++++------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md index 9470c0bb..44c950cd 100644 --- a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -229,13 +229,20 @@ Note: OOO 7th afternoon, 8th and 11th November - Next: - Help @pedromiguelss get ipfs/interop#86 across the line -@aschmahmann: - - - -@name +@pedrosantos - Done: - - What have you accomplished since the last Weekly? -- Blocked: - - Were there any blockers? If so, which ones? Is it still blocked? Why? + - Merged `interface-ipfs-core` docs conversion to async await + - https://github.com/ipfs/interface-js-ipfs-core/pull/542 + - Finished `interface-ipfs-core` tests conversion to async await + - https://github.com/ipfs/interface-js-ipfs-core/pull/547 + - Started to implement requested changes on `interface-ipfs-core` PR + - https://github.com/ipfs/interface-js-ipfs-core/pull/547 +- Blocked: N/A - Next: - - What is the next important thing you should focus on + - Finish `interface-ipfs-core` code review changes and merge it + - https://github.com/ipfs/interface-js-ipfs-core/pull/547 + - Merge second interop PR + - https://github.com/ipfs/interop/pull/87 + - Add `—human` flag to `bitswap.stat` and `repo.stat` + - https://github.com/ipfs/js-ipfs/issues/1996 + - https://github.com/ipfs/js-ipfs/issues/2057 From 979cbbf32d429cdeef39b7f232d3f2976b9d87c3 Mon Sep 17 00:00:00 2001 From: Alan Shaw Date: Tue, 5 Nov 2019 10:06:18 +0000 Subject: [PATCH 5/6] fix: pedro github name --- .../2019/Q4/2019-11-04--core-implementations-weekly.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md index 44c950cd..1d84a8d0 100644 --- a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -229,7 +229,7 @@ Note: OOO 7th afternoon, 8th and 11th November - Next: - Help @pedromiguelss get ipfs/interop#86 across the line -@pedrosantos +@pedromiguelss - Done: - Merged `interface-ipfs-core` docs conversion to async await - https://github.com/ipfs/interface-js-ipfs-core/pull/542 From 61804e1f92e76bd6ff3a16649f3aa0266cda5beb Mon Sep 17 00:00:00 2001 From: Henrique Dias Date: Mon, 11 Nov 2019 22:15:32 +0000 Subject: [PATCH 6/6] Update 2019-11-04--core-implementations-weekly.md --- .../Q4/2019-11-04--core-implementations-weekly.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md index 1d84a8d0..42f0aba3 100644 --- a/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md +++ b/meeting-notes/2019/Q4/2019-11-04--core-implementations-weekly.md @@ -246,3 +246,16 @@ Note: OOO 7th afternoon, 8th and 11th November - Add `—human` flag to `bitswap.stat` and `repo.stat` - https://github.com/ipfs/js-ipfs/issues/1996 - https://github.com/ipfs/js-ipfs/issues/2057 + +@hacdias +- Done: + - Cohosting + - `ipfs-cohost`: print pruned websites ([PR](https://github.com/ipfs-shipyard/ipfs-cohost/pull/12)) + - Released 2.1.0 with the change above. + - We decided to put a brake on this endeavour until the end of the quarter so we can focus on testing and other OKRs. + - IPFS Web UI + - Allow basic auth protected APIs by inserting an API URL of the type `http(s)://user:pass@example.com:port`. Related PRs/issues: + - https://github.com/ipfs-shipyard/ipfs-redux-bundle/pull/28 + - https://github.com/ipfs-shipyard/ipfs-redux-bundle/pull/36 + - https://github.com/ipfs-shipyard/ipfs-webui/pull/1296 + - https://github.com/ipfs-shipyard/ipfs-webui/issues/836