Skip to content

Conversation

@randombishop
Copy link
Contributor

@randombishop randombishop commented Nov 5, 2021

Project Abstract

Our mission is to build a public and non-intrusive blockchain registry to empower authors, search engines and recommendation systems that respect privacy and are 100% transparent.
https://www.wika.network

For which grant level are you applying?

  • Level 1: Up to $10,000, 2 approvals
  • Level 2: Up to $50,000, 3 approvals
  • Level 3: Unlimited, 5 approvals (for > $100k Web3 Foundation Council approval)

Application Checklist

  • The application template has been copied, renamed ( project_name.md) and updated.
  • A BTC or Ethereum (DAI/USDT) address for the payment of the milestones is provided inside the application.
  • I have read and acknowledged the terms and conditions.
  • The software delivered for this grant will be released under an open-source license specified in the application.
  • The initial PR contains only one commit (squash and force-push if needed).
  • The grant will only be announced once the first milestone has been accepted.

@CLAassistant
Copy link

CLAassistant commented Nov 5, 2021

CLA assistant check
All committers have signed the CLA.

@randombishop randombishop changed the title Grant Request v1.0 Web Registry DAO / Wika Network Grant Request Nov 5, 2021
@semuelle semuelle self-assigned this Nov 8, 2021
@semuelle
Copy link
Member

semuelle commented Nov 8, 2021

Hi Nabil, thank you for the application.

I appreciate the level of detail, and I think your project has merit. The first three milestones are pretty straightforward. I would just ask you to provide a bit more detail about what exactly you are delivering in milestone 1. They are mostly services, so I assume they will be deployed somewhere, but what can I expect as deliverable (config file, deployment script, ...)? I would also expect some Substrate-related deliverable there.

M4 is mainly code review, which we don't usually fund. It is also rather vague ("Add necessary pallets"). M5, as it is, should really be covered by the previous milestones.

What I would suggest is that you apply for M1-M3 now, and then, once you had someone review the Substrate part of the project and have a slightly better understanding of what's required for the governance part, apply for a follow-up grant. Would that work for you?

@semuelle semuelle added the changes requested The team needs to clarify a few things first. label Nov 8, 2021
@randombishop
Copy link
Contributor Author

randombishop commented Nov 8, 2021

Thank you very much Sebastian for the fast and excellent feedback!

  • About milestone 1, yes, it will be a mix of services and code deliverables.
    The ETL will come with a docker file and CI/CD configs so fully packaged for re-use.
    But we did not plan to automate Neo4J and ElasticSearch deployment and config. We only plan to document how we did it, would that be enough?
    Should I just add those precisions to milestone 1 or do you recommend another option?

  • You nailed it for the vagueness of milestone 4 - we still don't know what we don't know :)
    Also understood it's not fundable, will be removed from this scope.

  • The spirit of milestone 5 was to conduct a global round of hardening after previous milestones are completed, substrate code review and its consequent changes. Basically solidify everything before graduating to mainnet, which is the global success criteria we originally targeted for the grant request. Completely understand that testing should be included directly in milestones 1-2-3, just want to make sure that by removing 4 and 5, we won't be able to promise graduation to mainnet in this first round. Do you confirm?

Very looking forward to hearing back from you and will update the PR as soon as I receive your directions.

Thanks again!

@semuelle
Copy link
Member

semuelle commented Nov 8, 2021

Thanks for the quick reply, Nabil.

  • But we did not plan to automate Neo4J and ElasticSearch deployment and config. We only plan to document how we did it, would that be enough?

If you look at the FAQ I linked above, you will see that we don't usually fund activities around deployment or hosting. The main reason being that a single, centralised deployment of a service is not of much value to the ecosystem if it's not easily reproducible, as it could be shut down the day after approval. However, I'd say that, if the documentation of these is thorough enough that it is reproducible, it is acceptable as a deliverable.

Should I just add those precisions to milestone 1 or do you recommend another option?

Yes, please add details to the specification.

The spirit of milestone 5 was to conduct a global round of hardening after previous milestones are completed, substrate code review and its consequent changes.

That makes sense, and we get that often. We are just hesitant to fund activities that are impossible to scope, which is why postponing this to a follow-up grant makes the most sense here.

just want to make sure that by removing 4 and 5, we won't be able to promise graduation to mainnet in this first round. Do you confirm?

That is absolutely fine for a grant.

* Grant Request v1.1

* Update wika_network.md

* Update wika_network.md
@randombishop
Copy link
Contributor Author

Fantastic, thanks for the guidance Sebastian.
I made these changes accordingly: https://github.com/randombishop/Grants-Program/pull/2/files
Please let me know how it looks now...

@semuelle
Copy link
Member

semuelle commented Nov 8, 2021

Thanks for the update. I will share it with the rest of the team. Please note that most of the team is out of office this week, so it'll probably take a week before they have a chance to read it.

@semuelle semuelle added ready for review The project is ready to be reviewed by the committee members. and removed changes requested The team needs to clarify a few things first. labels Nov 8, 2021
@randombishop
Copy link
Contributor Author

Great, thank you @semuelle !


Because there are too many Like buttons out there!

Each one generates data in private databases that are often used to optimize for advertisment or hidden agendas.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue I see here is that, instead of allowing one company to process my likes, I now allow everyone to process them. That's not exactly an improvement. And wouldn't I have to give a website that integrates your like button access to the accounts in my wallet? That would allow them to link any other data they have of me with all the likes in your registry.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good point!
When a website integrates the plugin, it doesn't solve the data privacy problem as you point out, it even gives them access to more data, but one could argue that it's still a step towards progress.
Why? We think that the direct integration in websites is an intermediate step towards web3 to gain more mass adoption.

First of all, in that setting, the user doesn't have to be logged in anymore, or can use an anonymous account.
Secondly, we expect the widget to be first integrated by actors who actually want to promote web3 rather than very powerful web2 companies, and by doing so, will give us a user base ready to upgrade to the browser extension version, or even polkadot wallets that provide wika functionality in a seamless and protected fashion.
The ideal setting is when the browser is equipped with an extension that hides the actual address from the websites and only injects the widget and the like functionality. In that case the website will only be able to record the like action without an actual handle on the address.

My point is that by providing a web3 compliant infrastructure for registering web pages and liking, the world should progressively evolve into using it in the ideal way.
Somehow, this question applies to web3 in general: as I've been connecting my Metamask and Polkadot extension to so many websites already, it's true that I didn't share my personal credentials, but I also shared much more data. I also added my email address to opensea to receive offers, so now not only opensea has my personal info, but it also has all my wallet activity all the way to Uniswap and Aave. Not ideal from a data privacy standpoint at all, but i still believe that using opensea is a positive step towards web3 evolution.

This is actually the first time I think about the problem the way you pose it and this is just my intuitive immediate reaction. I will share this with the rest of the team and discuss further about what other ways we could come up to address it. And also please help me with any useful links on the subject or ways you think the infrastructure could be better designed. We are definitely here to learn!

| 1. | list_url_by_liker(address) | List the URLs liked by input address. |
| 2. | list_url_by_owner(address) | List the URLs owned by input address. |
| 3. | search(query) | Search URLs matching the query using domain, title and meta data |
| 4. | recommend(address) | Generate a list of recommendations for a user |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you plan on generating recommendations? Do you store any other data than addresses and liked URLs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The chain only stores addresses and liked URLs, but as specified in the ETL, Elastic Search will store one document per URL with title and other meta information pulled from the web page.
We plan on combining collaborative filtering (who liked same pages and what other pages they liked) combined with content based ML (with the data pulled by the ETL) to build a first version of the recommended function.
Please note that we really plan to make it the simplest thing that could possibly work as a proof of concept. Performance optimizations and scalability concerns will not be part of this initial scope.

semuelle
semuelle previously approved these changes Nov 15, 2021
Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the replies. Happy to proceed.

Copy link
Contributor

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the application. I have a few questions/comments: Regarding the indexer, how do you make sure it’s decentralized or how do you connect the off-chain world with the on-chain world? Also did you take a look at other indexing solutions in our ecosystem: https://github.com/w3f/Grants-Program/blob/master/docs/polkadot_stack.md#wrench-tools-apis-and-languages Furthermore, could you potentially remove the part about Sora and the bridge from the image. We generally don’t want to give the impression that we fund specific deployments or specific projects in our ecosystem. We only care about the development of open source software that is useful for our ecosystem. Finally, have you considered developing smart contracts instead of pallets? This way you could for example potentially deploy on a parachain directly without the need for the bridge.

* Remove reference to DEX
@randombishop
Copy link
Contributor Author

  • Regarding the indexer, how do you make sure it’s decentralized or how do you connect the off-chain world with the on-chain world?

We designed for rewarding the recommenders in the blockchain contract (see social contract schema.)
Agents willing to participate as recommenders will have to index the data and compete in providing the most useful recommendations and searching tools. So indexing and algorithms can be open source tools or proprietary technology services, but they would all compete in a public transparent blockchain infrastructure.
With milestones 1 and 2, we will deliver an open source code base, along with medium articles to reproduce the deployment, that will play the role of the first implementation to kick start the system and help other agents easily get started in this activity.

  • Also did you take a look at other indexing solutions in our ecosystem?

Only looked at Subquery briefly and was thinking it would probably be the library to use to implement milestone #1. But not sure as we didn't start actually working on this part. Our use case has additional data to be pulled from websites so we'll probably need either to integrate and extend, or fork existing solutions.

  • Could you potentially remove the part about Sora and the bridge from the image?

Understood, DEX integration removed from the scope.

  • Have you considered developing smart contracts instead of pallets?

Storing the sequence of like users and implementing the rewards with 0(1) complexity required some very custom data structure and logic that I wasn't sure could be done with Ink! in such an optimized fashion.
Also, the fact that we're using offchain workers to perform the website ownership verifications made me lean towards a plain Rust implementation.
But this is our first substrate project so definitely learning and maybe when we reach the audit part we might end up refactoring parts of the codebase.

Thanks for the excellent questions @Noc2 and looking forward to hearing back from you!

Copy link
Contributor

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. I’m willing to go ahead with the application.

@Noc2 Noc2 requested a review from semuelle November 17, 2021 14:00
Copy link
Contributor

@mmagician mmagician left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious to see Elasticsearch as the system to query your database acquired via ETL! I've worked with it in the past and have been quite happy with the usability. Also, for an ETL solution you might want to check you https://github.com/p2p-org/polkadot-profit-transformer
who are just finishing their grant - their solution (while complex) is fairly complete, as far as I can tell.

@mmagician mmagician merged commit 5383bc2 into w3f:master Nov 22, 2021
@github-actions
Copy link
Contributor

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at grantsPR@web3.foundation and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

@randombishop
Copy link
Contributor Author

Fantastic news, thank you W3F team, we sincerely appreciated your transparency and the quality of your feedback!
We will regularly post updates about our progress as comments in this PR...

@randombishop
Copy link
Contributor Author

Hi @semuelle @mmagician @Noc2!
Sharing our progress update here:

  • We made progress building an important foundational component: we have now scripts to populate the blockchain automatically with page ownerships and likes. This will be useful to generate data and simulations, and for testing purposes in general.
  • We have installed a Subquery instance and gained some expertise on how it works, and it seems like this tool will make milestone 1 very straightforward.
  • Took us a couple of weeks to figure out how to insert this work stream into our bandwidths, divide work and plan forward, and with that, we expect milestones 1 and 3 to be done by the end of February, with milestone 3 as fast follow.

I know this timeline is drifting from the initial grant request plan, but we had other workloads coming in at the same time, plus holidays season.

Please let me know if you have any questions or concerns, and thank you very much for your participation!

@semuelle
Copy link
Member

Thanks for the update, @randombishop. We are glad to hear things are progressing.

Don't worry about small delays; as long as we know that the project is active, we have no reason to be concerned.

@randombishop
Copy link
Contributor Author

Hey W3F team!
Happy to announce that Milestone 1 is up and running and we started the delivery process (w3f/Grant-Milestone-Delivery#365)
Looking forward to hearing your feedback and finalizing this first milestone.
General progress update:
We'll probably be responding to feedback about milestone 1 and working on 2 and 3 in parallel in the next weeks.
Milestone 2: seems easy to build now so should not take more than 3 to 4 weeks.
Milestone 3: we are trying different UX variations and we'll take the time to iterate until we converge to the simplest and finest UX we can so no precise timeframe yet.
Cheers!

@randombishop
Copy link
Contributor Author

Hello hello W3F team!
Here we are again with milestone 2 (w3f/Grant-Milestone-Delivery#423)
Looking forward to collaborating and working together like previous one, we know the drill now :)

@Noc2
Copy link
Contributor

Noc2 commented Apr 19, 2022

That’s great. Thanks for pinging us here. It might take a little bit longer at the moment, since we currently have a lot of outstanding evaluations and part of the team is sick/ooo.

@randombishop
Copy link
Contributor Author

No worries, get better!

@randombishop
Copy link
Contributor Author

Hey team,

Thanks again for giving us the opportunity to work together and the smoothness of the process so far with milestones 1 and 2. Posting the project update here for general communication:

  • Milestones 1 and 2 were delivered, and we have now a testnet, indexed databases, an API to consume them and a polkascan instance to scan the blockchain.

  • We are heavily refactoring the frontend codebases for the webapp and the browser extension into one single repo, and iterating a lot on the UX. This UX and milestone 3 integrations will be the cornerstone of the system we're building, and we really want to take the time to nail the design and make it as simple and usable as it can get. For this reason, no promises on ETA for milestone 3, but we'll deliver!

  • Also, we are trying to find partners to test the integration and UX end-to-end with real users before we call milestone 3 done. Please reach out if you have any contacts that might be interested to help us: small online news websites, bloggers, independent journalists running newsletters, etc.

Cheers!

@takahser
Copy link
Contributor

takahser commented May 3, 2022

@randombishop likewise, thanks for your deliveries, the good words and the update. I'm happy to hear that you're satisfied with our program.

For this reason, no promises on ETA for milestone 3, but we'll deliver!

That's not a problem, looking forward to M3! 👍

Please reach out if you have any contacts that might be interested to help us

Maybe we can ask our grants PR team. I'll send them an email and cc you:)

@alxs
Copy link
Contributor

alxs commented Oct 31, 2022

Checking in @randombishop, are you still planning to deliver milestone 3? If so, could you please submit an amendment to update the timeline?

@randombishop
Copy link
Contributor Author

Checking in @randombishop, are you still planning to deliver milestone 3? If so, could you please submit an amendment to update the timeline?

hey @alxs ! Yes, though we didn't make enough progress on it lately, we're still planning to deliver it.

@cwilker will make an amendment with a new timeline and take the lead for the last milestone.

@randombishop
Copy link
Contributor Author

@redben @cwilker As discussed yesterday, noting here that you guys are committed to other obligations and won't have time to work on this grant in the near future. We will terminate this grant now; but feel free to re-submit an application for milestone 3 in the future if you'd like to re-activate this project.

@takahser
Copy link
Contributor

Just for reference, termination PR: #1568 (merged)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review The project is ready to be reviewed by the committee members.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants