June::Db-milestone_1.md

[MikeMS-sys]

Milestone Delivery Checklist

• The milestone-delivery-template.md has been copied and updated.
• The invoice form 📝 has been filled out for this milestone.
• This pull request is being made by the same account as the accepted application.
• In case of acceptance, the payment will be transferred to the BTC/ETH payment address provided in the application.
• The delivery is according to the Guidelines for Milestone Deliverables.

Link to the application pull request: w3f/Grants-Program#466

[Noc2]

Thanks for the delivery. We will look into it as soon as possible.

[Noc2]

Just to link it here. You want to combine this delivery with this amendment, correct? Could you provide more details? Why did you move the current delivery to milestone 4?

[andskur]

Hi @Noc2
Yes we want to combine delivery of this Milestone with update of our initial application.

we plan to change 2nd Milestone (hash map data storage) for a new (offchain::ipfs update) and then put hash map data storage in Milestone 4 (the last).

[semuelle]

The branches/paths linked in Deliverables 0b and 2 don't exist, could you fix those?

[semuelle]

Also:

• The image on Docker Hub is either private or nonexistent
• Deliverable 2 points to your own substrate-frontend-template, but the test instructions point to substrate-developer-hub, and I don't see any changes to your fork. Are you still planning on making changes to the frontend? Otherwise creating a fork of your own seems superfluous.
• Can you add instructions how to build and run locally, with and without Docker, to your testing guide? Following the rs-ipfs instructions, both docker build and cargo build fails with type annotations needed errors.

[MikeMS-sys]

@semuelle Hello and sorry for still not answer, unfortunately we were forced to hold the development works for a week for personal reasons beyond our control.

We almost done with updating offchain::ipfs (2nd Milestone). In there are we agreed to coordinate with equilibrium.co updating offchain::ipfs. We have send a mail them, but still didn't got any answer.

And of couse will fix requested changes for Milestone 1.

[alxs]

@MikeMS-sys perhaps try opening an issue or a draft PR on their repo? If you still don't get an answer from them, we'll try to mediate. Happy to hear about your progress. By the way: you'll probably also want to bring the offchain::ipfs Manual up to date as part of the documentation for M2 if needed.

[MikeMS-sys]

@alxs Of course, we were planned it when achive the delivery of the Milestone.

[semuelle]

@MikeMS-sys, any updates on this?

[MikeMS-sys]

@semuelle
To shut down Milestone 1, we had to first update offchain :: ipfs (M2).
So, we have already updated [offchain :: ipfs] (https://github.com/rs-ipfs/substrate) and opened a PR on their repo rs-ipfs/substrate#5.

Today we are working on the remaining сlauses of the Grant-milestone-delivery, and then make update for the 1st stage and PR for the 2nd.

Remembering our conversation about the encryption module, we will most likely prepare a PR for 3 Milestone.

[semuelle]

Hi @MikeMS-sys, in order to continue with the evaluation of the milestone, could you address the issues I have raised in my two comments above?

[MikeMS-sys]

Hi @semuelle! For sure! We plan to make the update at night. Mostly it is not our style to delay the work, but objective reasons contributed to this and we will not keep you waiting long with a justification answer.

[alxs]

@MikeMS-sys

[alxs]

@MikeMS-sys any updates on Sebastian's feedback? Please note that the evaluation process is usually a lot faster than it's been the case for these deliveries - according to the terms and conditions, you have 14 days to address feedback. If you can't get back to this within the next 2 weeks, please submit an amendment to update the timeline of your grant and we'll put the delivery on hold.

[andskur]

#322 (comment)

[semuelle]

Hi @andskur & @MikeMS-sys. I had planned to evaluate both of your milestones today, but there are still issues with this delivery.

• There are no public uddug/substrate-juni or andskur/juni-db images on DockerHub as mentioned in 0c and 0d
• Deliverable 0c is described as "Complex quality assurance for all platform features. Core functions will be fully covered by unit tests to ensure functionality and robustness." The doc only describes how to call functions from the Substrate frontend template, though.

[andskur]

hi @semuelle , thanks for your reply. Would be very nice if we could evaluate 2nd milestone first. We still need to finish few issues with first milestone delivery (plan to update you about it this weekends)

[semuelle]

Hey @MikeMS-sys & @andskur, any news on this milestone?

[andskur]

Hey @MikeMS-sys & @andskur, any news on this milestone?

Hi @semuelle, sorry for late reply. We should figure out last issues (web app and unit testing) and aim to update you later this week or early next week

[andskur]

Hi @cruikshankss i have already updated it🙂

[ashlink11]

Hi @cruikshankss i have already updated it🙂

Thank you. I'm a bit backlogged right now but I plan to keep you updated and get to it as soon as possible. To speed things up, please make sure your documentation explains the ideals you've achieved and the true value of your project, and ensure the testing guide is catered to make it as easy as possible (truly step-by-step with great details) for someone new to your project to test that your deliverables all work. Talk soon! Thanks!

[ashlink11]

Good day @andskur!

I tried cloning your repo and running TRYBUILD=overwrite cargo test, but I got this output

error: failed to parse manifest at `/Users/ash/wthree/substrate/Cargo.toml`

Caused by:
  feature `named-profiles` is required

  The package requires the Cargo feature called `named-profiles`, but that feature is not stabilized in this version of Cargo (1.57.0-nightly (d56b42c54 2021-09-27)).
  Consider adding `cargo-features = ["named-profiles"]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#custom-named-profiles for more information about the status of this feature.

Do you have any insight as to what I'm doing wrong? Many thanks! -Ashley

[ashlink11]

Good day @andskur,

I'd really love to complete your evaluation this week. If you're available, I'd be super interested in meeting with you for 30 mins for a live demo video call/discussion of the software architecture so I can understand the value of your project.

Tomorrow I'm free 1-2:30pm and after 3pm CET. Friday I'm free 1-2:30pm and after 4:30 CET. My time zone is UTC−07:00 which hopefully explains my availability this week. Please feel free to choose a 30-min time window. I also plan to message the other teams who have evaluations I hope to finish this week. It's a really busy week for me which is why I'm trying to schedule these demos so I can speed up my pace; my apologies for the delay. And if a time gets taken, I plan to edit this message in GitHub so please check this convo tab on GH for the latest updates. Feel free to send a calendar invite to ashleyg@web3.foundation if you select a time and I plan to RSVP from there.

Otherwise, I'm open to scheduling an Element or Discord chat. If not, I'm totally patient if you'd rather update the testing guide whenever you have free time. There's no rush and just hope you take all the time you need to build the tech right. :) Cheers!

[andskur]

hi @cruikshankss , sorry for a long reply. This week is a super busy for us too. I will check tests asap but don't think that we have a suitable time slots for a call this week☹️

[ashlink11]

@andskur Thanks for the response! No rush. Please update the testing guide when you have the time and ping me with an update, or ping me to set up a demo. Hope you have a nice weekend. :)

[takahser]

@andskur sorry for the delay here, we're currently a bit behind in our backlog. We're going to review your delivery again as soon as possible, most probably within this week.

[takahser]

@andskur sorry again for the delay here. I have two of questions:

1. I took a look at your delivery and I'm confused about which testing guide to use. In your delivery you link The offchain::ipfs Manual while on @semuelle's evaluation I see the link to this google doc. The latter seems more complete to me, which is why I tried following this guide. Could you update delivery 0c. Testing Guide, if neccessary?
2. I didn't find the docker image you linked: uddug/substrate-juni. Could you double-check if you have uploaded it and if it's public?

[andskur]

hi @takahser . please check last updated delivery - https://github.com/w3f/Grant-Milestone-Delivery/blob/81c60850a3b95df679054f795c7005a52666aaf9/deliveries/Juni::Db-milestone_1.md

doc and testing guide - https://uddugteam.github.io/JuniDB-manual
docker image - https://hub.docker.com/repository/docker/andskur/juni-db

[takahser]

@andskur FYI - the docker image is still not public

docker image - https://hub.docker.com/repository/docker/andskur/juni-db

Edit:

• I also ran the tests and noticed that there are currently a lot of warnings. Feel free to fix them, if possible. Here are some of the warnings:

     Compiling sc-transaction-pool v4.0.0-dev (/home/seraya/repos/substrate-juni-db/client/transaction-pool)
  warning: unnecessary lifetime parameter `'a`
    --> frame/support/src/storage/bounded_vec.rs:49:25
     |
  49 | pub struct BoundedSlice<'a, T, S>(&'a [T], PhantomData<S>);
     |                         ^^
     |
     = help: you can use the `'static` lifetime directly, in place of `'a`
  
  warning: associated function `project_ref` is never used
    --> client/network/src/protocol/notifications/upgrade/collec.rs:87:1
     |
  87 | #[pin_project::pin_project]
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(dead_code)]` on by default
  
  warning: associated function `project_ref` is never used
    --> client/network/src/protocol/notifications/upgrade/notifications.rs:83:1
     |
  83 | #[pin_project::pin_project]
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  
  warning: associated function `project_ref` is never used
     --> client/network/src/protocol/notifications/upgrade/notifications.rs:107:1
      |
  107 | #[pin_project::pin_project]
      | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  
     Compiling frame-system v4.0.0-dev (/home/seraya/repos/substrate-juni-db/frame/system)
  warning: `frame-support` (lib) generated 1 warning
  
  (...)
  
  49 | / construct_runtime! {
  50 | |     pub enum Runtime where
  51 | |         Block = Block,
  52 | |         NodeBlock = Block,
  ...  |
  57 | |     }
  58 | | }
     | |_^ function or associated item not found in `pallet::Pallet<Runtime>`
     |
     = help: items from traits can only be used if the trait is implemented and in scope
     = note: the following trait defines an item `check_inherent`, perhaps you need to implement it:
             candidate #1: `ProvideInherent`
     = note: this error originates in the macro `construct_runtime` (in Nightly builds, run with -Z macro-backtrace for more info)
  
  error[E0599]: no associated item named `INHERENT_IDENTIFIER` found for struct `pallet::Pallet` in the current scope
    --> tests/construct_runtime_ui/undefined_inherent_part.rs:49:1
     |
  11 |       pub struct Pallet<T>(_);
     |       -------------------- associated item `INHERENT_IDENTIFIER` not found for this struct
  ...
  49 | / construct_runtime! {
  50 | |     pub enum Runtime where
  51 | |         Block = Block,
  52 | |         NodeBlock = Block,
  ...  |
  57 | |     }
  58 | | }
     | |_^ associated item not found in `pallet::Pallet<Runtime>`
     |
     = help: items from traits can only be used if the trait is implemented and in scope
     = note: the following trait defines an item `INHERENT_IDENTIFIER`, perhaps you need to implement it:
             candidate #1: `ProvideInherent`
     = note: this error originates in the macro `construct_runtime` (in Nightly builds, run with -Z macro-backtrace for more info)
  
  error[E0599]: no function or associated item named `is_inherent_required` found for struct `pallet::Pallet` in the current scope
    --> tests/construct_runtime_ui/undefined_inherent_part.rs:49:1
     |
  11 |       pub struct Pallet<T>(_);
     |       -------------------- function or associated item `is_inherent_required` not found for this struct
  ...
  49 | / construct_runtime! {
  50 | |     pub enum Runtime where
  51 | |         Block = Block,
  52 | |         NodeBlock = Block,
  ...  |
  57 | |     }
  58 | | }
     | |_^ function or associated item not found in `pallet::Pallet<Runtime>`
     |
     = help: items from traits can only be used if the trait is implemented and in scope
     = note: the following trait defines an item `is_inherent_required`, perhaps you need to implement it:
             candidate #1: `ProvideInherent`
     = note: this error originates in the macro `construct_runtime` (in Nightly builds, run with -Z macro-backtrace for more info)
  ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈

• Another thing to note is that after running the tests, the node starts, which is unexpected:

  test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 27.58s
  
       Running tests/common.rs (target/debug/deps/common-36e087badc34e501)
  
  running 0 tests
  
  test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
  
       Running tests/export_import_flow.rs (target/debug/deps/export_import_flow-683669886d5ceaa4)
  
  running 1 test
  2022-10-19 09:58:25 Low open file descriptor limit configured for the process. Current value: 4096, recommended value: 10000.    
  2022-10-19 09:58:25 Substrate Node    
  2022-10-19 09:58:25 ✌️  version 3.0.0-dev-208eb7818-x86_64-linux-gnu    
  2022-10-19 09:58:25 ❤️  by Parity Technologies <admin@parity.io>, 2017-2022    
  2022-10-19 09:58:25 📋 Chain specification: Development    
  2022-10-19 09:58:25 🏷  Node name: aquatic-suit-6702    
  2022-10-19 09:58:25 👤 Role: AUTHORITY    
  2022-10-19 09:58:25 💾 Database: RocksDb at /tmp/.tmpuyKzxN/chains/dev/db/full    
  2022-10-19 09:58:25 ⛓  Native runtime: node-268 (substrate-node-0.tx2.au10)    
  2022-10-19 09:58:26 [0] 💸 generated 1 npos voters, 1 from validators and 0 nominators    
  2022-10-19 09:58:30 🔨 Initializing Genesis block/state (state: 0xacac…3aa7, header-hash: 0xf2e5…93d5)    
  2022-10-19 09:58:30 👴 Loading GRANDPA authority set from genesis on what appears to be first startup.    
  2022-10-19 09:58:33 👶 Creating empty BABE epoch changes on what appears to be first startup.    
  2022-10-19 09:58:33 Using default protocol ID "sup" because none is configured in the chain specs    
  2022-10-19 09:58:33 🏷  Local node identity is: 12D3KooWMTCdrUUk5CZdEDsUeGNU76ZySRvnovfuLSSPDWMrfrSH    
  2022-10-19 09:58:33 📦 Highest known block at #0    
  2022-10-19 09:58:33 〽️ Prometheus exporter started at 127.0.0.1:9615    
  2022-10-19 09:58:33 Listening for new connections on 127.0.0.1:9944.    
  2022-10-19 09:58:33 👶 Starting BABE Authorship worker    
  2022-10-19 09:58:33 Accepted a new tcp connection from 127.0.0.1:45196.    
  2022-10-19 09:58:35 Accepted a new tcp connection from 127.0.0.1:45198.    
  2022-10-19 09:58:36 🙌 Starting consensus session on top of parent 0xf2e54f2923564d33db3d2b70e45ed5bf0bdfdeb0ac0fe76f35ba7fed0ef993d5    

[andskur]

Hi @takahser ,

Image is public... I don't understand why it private for you

Will check tests warning and why node starts

[takahser]

@andskur you can easily verify if it's publicly accessible by accessing the image without being logged into docker hub or when browsing incognito - is it still accessible then? For me it isn't, as I get redirected to the login mask:

Will check tests warning and why node starts

Ok great, thank you.

[andskur]

@takahser you right, sorry. I have sent wrong link.

Correct one - https://hub.docker.com/r/andskur/juni-db

[takahser]

@andskur thanks, this link works. Were you able to fix the other issues?

[takahser]

Ping @andskur

[andskur]

hi @takahser , sorry for long reply.

Seems like this warnings is come from original substrate v3. Should we fix it in this case?

About the node starts - I also see that this behaviour is okay for original substrate. Also node starts when tests running, not after tests.

[takahser]

@andskur sorry for the delay again and thanks for your patience. I took another look at your delivery.
I managed to:

• run the node using the docker template
• run the frontend
• run the unit tests (?)

However, when testing the frontend, I wasn't able to query the key that I previously stored a value for:

Or at least I cannot read the value (which should be 'BAR'). Instead, I only see this message:

fetching data for key ...
["5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY"]

Is this the expected behaviour? Am I missing something here?

Regarding the warnings:

Seems like this warnings is come from original substrate v3.

Are you using the latest version of Substrate? Since you seem to have copied the whole Substrate repo I'd be curious to learn on how you're planning to keep it up to date with the original Substrate repo.

[takahser]

Hi @andskur
Any update here? If there's no progress within the next 2 weeks I'm going to assume you're no longer interested in completing the grant. And we might end up terminating it, as per our T&Cs.

[MikeMS-sys]

Hi @takahser
Earlier we jointly found that the offchain::ipfs fork is the best solution for developing Juni::Db. We updated it to the latest Substrate version and opened a PR, thus our fork become one of the main forks of the original offchain::ipfs.

But you asked the right question. In the last months, we have been trying to make an acceptable version of Juni by maintaining the current version of Substrate. But now we are thinking that it can be very difficult to keep the repository up to date and achieve adequate results. Due to the development of the ecosystem and, accordingly, the Substrate, etc.

If coming back earlier, Juni was 1 of 3 main components of a slightly larger project to analyze medical data. And perhaps our initial decision will be more logical to achieve the current goal - namely, the development of our database - means as a standalone Substrate pallet without dependencies on any forks like offchain::ipfs.

And, of course, we would like to get some advice on how best to proceed with the grant.

[takahser]

@MikeMS-sys thanks for your reply.

We updated it to the latest Substrate version and opened a PR, thus our fork become one of the main forks of the original offchain::ipfs.

Which PR are you referring to? Does that mean you're currently blocked by the acceptance of said PR?

But now we are thinking that it can be very difficult to keep the repository up to date and achieve adequate results.

Does that mean that you won't proceed with this grant?

And perhaps our initial decision will be more logical to achieve the current goal - namely, the development of our database - means as a standalone Substrate pallet without dependencies on any forks like offchain::ipfs.

One solution would be to cancel this grant and apply for a new one. Or you could amend the current grant, if that makes more sense to you, depending on the overlap between the original and amended grant.

[MikeMS-sys]

@takahser

Which PR are you referring to? Does that mean you're currently blocked by the acceptance of said PR?

I refer to rs-ipfs/substrate#5 PR and No, that PR already was accepted, and don't block us.

Does that mean that you won't proceed with this grant?

No, we aim at the release of the Juni::Db but are thinking about a more logical solution.

One solution would be to cancel this grant and apply for a new one. Or you could amend the current grant, if that makes more sense to you, depending on the overlap between the original and amended grant.

Application Document https://github.com/w3f/Grants-Program/blob/master/applications/JuniDB.md



What we’ve done. First, we updated off chain::ipfs https://github.com/w3f/Grant-Milestone-Delivery/blob/master/evaluations/june-db_2_semuelle.md


We plan to make a new PR and change the solution in Milestone 2 on "The standalone Substrate pallet" without dependencies on any forks like offchain::ipfs. And then to fix the Basic database layout implementation with key-value data storage (Milestone 1) as you commented earlier https://github.com/w3f/Grant-Milestone-Delivery/pull/254#issuecomment-1330619906.


[takahser]

@MikeMS-sys

I refer to rs-ipfs/substrate#5 PR and No, that PR already was accepted, and don't block us.

I didn't look deeply into the PR, but it wasn't accepted, instead it was closed, right? Did you post the wrong PR maybe?

No, we aim at the release of the Juni::Db but are thinking about a more logical solution.

What's your estimate on coming up with a solution for it? I'm asking because this PR has been open for >1 year and I think it's time to close it, if we don't see any progress been made, nor concrete plans on what to achieve until what date.

We plan to make a new PR and change the solution in Milestone 2

What's your time schedule on that?

[takahser]

@MikeMS-sys after a thorough review of your delivery and discussion within the grants committee, we've concluded that the best solution forward would be to terminate your grant. These are the reasons that lead to that decision:

• significant delays: this delivery has been open for nearly 1.5 years now
• no clear plans on how to move forward
• no activity in the codebase for more than 3 months
• multiple warnings from W3F committee members here, here and here

Please note that this termination is in accordance to our T&C.

We would like to thank you for your interest in the grants program and for your contributions to the ecosystem. You're also welcome to apply for further grants in the future, though please note that the committee will likely take the current termination into account.