core: initialization and Repository

[Wondertan]

Context

The penultimate piece for #30. It introduces initialization flow for the embedded Core node along with repository to access Core' stored data. Basically, it implements some of the important functionality for future init command.

Others

• Implements custom serialization for core.Config(rationale is in the code)
• Starts own collection of utilities
• Provides FS, InMem and Mock repositories

Refs

Depends on #71 and #74
Moves towards done #30

[liamsi]

Can you also merge #71 and rebase this on master again?

[liamsi]

with repository to access Core' stored data

isn't the whole idea that celestia-node only communicates with core (includes accessing data) via the RPC interface (even if local)?

[liamsi]

On first sight it seems this PR breaks somewhat with the design paradigm we have settled on (namely that the the node only knows about very few RPC endpoints and that is the only thing it needs from core). The motivation for that seems to be that the future node-specific data and config are written into the same "repo" as the core node's?
I understand why this might be desirable but that is really not important for devnet.

We have to be careful to focus on the DA-specific parts!

[liamsi]

Implements custom serialization for core.Config(rationale is in the code)

Which custom serialzation (toml is not custom)? Where is the rationale?

[Wondertan]

Remember, we are treating tendermint as a black-box.

Yeeeaaah, on a high level, but the deeper you go, the less it black-boxable 🤷🏻‍♂️

[Wondertan]

The motivation for that seems to be that the future node-specific data and config are written into the same "repo" as the core node's?
I understand why this might be desirable but that is really not important for devnet.

The motivations here are:

• Make embedded Core work
• Simplify deployment of things by having everything related to initialization in one place.
  • This is a crucial part for devnet and future nets. Instead of having separate hacky initialization scripts for Core and Node, we control where it actually needs to be controlled. The PR just implements initialization complexity earlier than later.
• Thoroughly understand tendermint initialization flow and its requirement

[Wondertan]

Which custom serialzation (toml is not custom)?

Not custom serialization, but custom serialization logic.

Where is the rationale?

Next LoadConfig func

[liamsi]

Yeeeaaah, on a high level, but the deeper you go, the less it black-boxable 🤷🏻‍♂️

Let's still try to stick to that paradigm as much as possible.

[renaynay]

couple nits, I'm okay with this but I hate the idea that we need to do this much work to get core to work embedded within node. It's not that much reward per amount of work that's gone into it so far. It's fine otherwise though. We aim to remove all this regardless after Devnet.

[Wondertan]

We aim to remove all this regardless after Devnet.

@renaynay, WDYM?

[Wondertan]

According to offsite. We treat Tendermint as a black box for devnet. Later on, we will have to decouple it into more pieces, so this code is likely to stay preserving control over initialization.

[renaynay]

I mean, we will not run celestia-node and celestia-core processes together. This is a crutch for devnet. After devnet, all should be communicating via p2p layer (meaning celestia-node does not need to know / care about celestia-core ).

[renaynay]

OK Thank you!

[liamsi]

👏🏼 LGMT