From ae3d16da211eb3a24cdc59b265e080ee6923c1ac Mon Sep 17 00:00:00 2001 From: Gabriel Levcovitz Date: Thu, 13 Apr 2023 00:36:09 -0300 Subject: [PATCH 1/3] feat(events): update designs --- .../0001-high-level-design.md | 14 +----- .../0002-low-level-design.md | 43 +++++++++++++++++-- 2 files changed, 40 insertions(+), 17 deletions(-) diff --git a/projects/reliable-integration/0001-high-level-design.md b/projects/reliable-integration/0001-high-level-design.md index e36621d4..e3185185 100644 --- a/projects/reliable-integration/0001-high-level-design.md +++ b/projects/reliable-integration/0001-high-level-design.md @@ -39,18 +39,6 @@ These features will not be part of the first phase of this project: - Give users the choice to receive only a subset of events, according to some criteria. - The `--flush-events` flag. By default, all events are retained. In the future, the user could provide a `--flush-events` flag to enable the flushing of events after each event is sent to the client. -## Event generation during the full-node cycle - -Considering this full-node cycle: - -![Full-Node-Cycle drawio](./0001-images/full_node_cycle.png) - -Where: -- `Load` is the period right after the full-node is started, where the local database is read. -- `Sync` is the period where `Load` is finished and the full-node continuously receive/send txs to other peers until full node is stopped. - -By default, the events generated during load will not be emitted. If the user wants to enable them, one must provide the `--emit-load-events` flag. - ## Flow ![Event Flow drawio](./0001-images/event_flow.png) @@ -217,7 +205,7 @@ It will be triggered when the full-node is initializing and is reading locally f ### LOAD_FINISHED -It will be triggered when the full-node is ready to establish new connections, sync, and exchange transactions, at the same that when the manager state changes to `READY` [here](https://github.com/HathorNetwork/hathor-core/blob/85206cb631b609a5680e276e4db8cffbb418eb88/hathor/manager.py#L652). Other events will be triggered ONLY after this one, if the `--emit-load-events` flag is not enabled. `EmptyData` is sent. +It will be triggered when the full-node is ready to establish new connections, sync, and exchange transactions, at the same that when the manager state changes to `READY` [here](https://github.com/HathorNetwork/hathor-core/blob/85206cb631b609a5680e276e4db8cffbb418eb88/hathor/manager.py#L652). `EmptyData` is sent. ### NEW_VERTEX_ACCEPTED diff --git a/projects/reliable-integration/0002-low-level-design.md b/projects/reliable-integration/0002-low-level-design.md index 2ca7add3..471e2e4d 100644 --- a/projects/reliable-integration/0002-low-level-design.md +++ b/projects/reliable-integration/0002-low-level-design.md @@ -6,13 +6,49 @@ This low-level design intends to go deeper on the technical aspects of the proje ## Full-node execution +Considering this full-node cycle: + ![Full-Node-Cycle drawio](./0002-images/full_node_cycle.png) -### TODO: This section will be updated in a separate PR +At first, when the full node is initialized, it moves to the `LOAD` state, where data is loaded from the database, without consulting the network. Once all local data is read, the full-node continuously syncs with peers, changing to the `SYNC` state. + +In order to know which events will be generated, we will store the states (`LOAD` and `SYNC`) as metadata. So, when the full node starts, we will check the metadata. If it is empty or `LOAD`, we do not know where it stopped and will regenerate all events during load. If it is `SYNC`, we will not generate any events until `LOAD_FINISHED` is emitted. + +In the table below we describe all possible starting scenarios for the full node, and the respective desired outcome: + +| Starting scenario | Vertices in the database | Events in the database | State metadata | Desired outcome | +|-----------------------------------|--------------------------|------------------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------------| +| `Fresh start` | None | None | Empty | Generate all events, from `event_id = 0`. All events come during the `SYNC` phase. | +| `Snapshot start` | Some | None | Empty | Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. | +| `Restart from aborted LOAD phase` | Some | Some | `LOAD` | All events in the database are discarded. Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. | +| `Restart from aborted SYNC phase` | Some | Some | `SYNC` | Generate events from `event_id = last_event_id + 1`. All events come during the `SYNC` phase. | + +There's no case in which there are some events in the database, but no vertices. + +For the `Fresh start` scenario, if we consider there's a `LOAD` phase with 0 events, then the desired outcome becomes the same for the first three scenarios. This makes it possible to simplify the table into only two scenarios, only depending on `State metadata`, as described in the beginning of this section: + +| Starting scenario | State metadata | Desired outcome | +|------------------------------------------------------------------------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------| +| `Fresh start` or `Snapshot start` or `Restart from aborted LOAD phase` | Empty or `LOAD` | All events in the database are discarded. Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. | +| `Restart from aborted SYNC phase` | `SYNC` | Generate events from `event_id = last_event_id + 1`. All events come during the `SYNC` phase. | + +Note: when generating snapshots, events should not be included, as they will be discarded and regenerated anyway. + +Now, we can describe a step-by-step representing these rules: + +1. The full node begins the initialization process +2. The `EventManager` starts + 1. It retrieves the previous node state from the database and stores it + 2. If its stored previous state is empty or `LOAD`, it discards all events currently in the database +3. The full node emits the `MANAGER_ON_START` PubSub event + 1. The `EventManager` sets `LOAD` as the current node state in the database +4. The full node loads all vertices from the database +5. If the `EventManager`'s stored previous state is empty or `LOAD`, a new vertex event is generated for all vertices in the database, in topological order +6. The full node finishes loading and emits the `LOAD_FINISHED` PubSub event + 1. The `EventManager` sets `SYNC` as the current node state in the database +7. The full node starts syncing new vertices from the network -~~At first, when the full node is initiated, it moves to the `LOAD` state, where data is load from the database, without consulting the network. Once all local data is read, the full-node continuously sync with peers, changing to `SYNC` state.~~ -~~In order to know which events will be triggered, we will store the states (`LOAD` and `SYNC`) as metadata. So, when the full node starts, we will check the metadata. If it is empty or `LOAD`, we do not know where it stopped and will emit all events. If it is `SYNC`, we will skip all event triggered during full node initialization until `LOAD_FINISHED`~~ ## Flags @@ -21,7 +57,6 @@ The following flags will be provided: | Flag | Description | |----------------------|-------------------------------------------------| | `--enable-event-queue` | Enable the message broker mechanism | -| `--emit-load-events` | Enable emission of events during the load phase | ## Event From d85c03382cc887d920333fabefef4d86e9ccd1cc Mon Sep 17 00:00:00 2001 From: Gabriel Levcovitz Date: Thu, 13 Apr 2023 14:09:15 -0300 Subject: [PATCH 2/3] feat(events): remove unused image --- .../0001-images/full_node_cycle.png | Bin 8715 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 projects/reliable-integration/0001-images/full_node_cycle.png diff --git a/projects/reliable-integration/0001-images/full_node_cycle.png b/projects/reliable-integration/0001-images/full_node_cycle.png deleted file mode 100644 index b4daec13f52ffb6ff79a2009d4db83d0d7727ed1..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8715 zcmd6NcTiJn-)%xLgx-{11;v8WAyPw;A|)s)O#wxk^xh#z6$o8HI!civQlu+{UIasn zQj}h$27=U3?naNE_k8c4_kJ^%88Uk&dp~78zp~cad8(_UMoqy+0Rn-jHPlsZfj|%l zaP0|$0-vVd={6t`?|_Di;%zUJ)ePh4Gi~oX{B>v2V)vW*V(ZqM6GbvZLdcYwpOTA& zh0x7*PQ_`@Hqt*s8IP`E&%sOGX_r7PcShMchSkF8AJjNpZLu&jLqIJ>`GhZt(s<53 z%s7bJl&F92d;k5AQT_XdqvM3G)MDW8FZ`QS{BrSM0bOyeo1gWNA{S`c)(4*|n)2k{ zu6y5-Ekq}$n|cwIp(AB9`zDWzpQS4~BxwI7dzU_Qk|l*%0S?Wy_qdRZJdAMeK{n=l z7~#vQycZkBZ%5NC8}Jg=|6b2T$0$3CaPWdOHkNEBty`u+ zBDu=3yaBE%+fxwKdZT<{=^DSNLVkW#tb(8^2jVjS0yP4Iuj%`bueU7H-9WV+9VweYSU_ z4~4-)eB2)y_h#g6&S_>`jrtuB6Hc>Y2b*oS!6%q??i`_)=SP`;J*eMczs*W~KvJwG zpCCZ!@PDkWoj0N|ysSmud^@ZD{uTKaOgie|5e5BPFflJ)(Cxk8pXk#pNNF-%se=MB z*nf3`eT+HdqGHMtuED*$w^T@gN8yE12yrr?GS_mT^(rE(0%JwJg|^?y-|V2t5;1++ zH!|^Hv%}v>VN}ZA8t-PP_8b1^s)bk^O;`R;%U;ww2|9R zhHGb?KUwD`Q`sD5HBc0)9_lASvc^ZF_;#GG^{V!#vE)KU=W8Skt-OtgxQZ}@Z2@0w z|JlDs2z~l^Sgro*B5*#hZemhC2trEnsu% zlx#Ab-cNpKFBVw~^{*Wgpa+styi1>n(tIgq$RL)lGl@5e&*9;-^HqB%&v)Y`KF`1m z(s-gGaqm`Nx9g~7X8(?m;KC$c(?Y)l5&)C0vr+c)7G=Wo1YhbcILYAo&X?0Q)tqSU z)Ll@DpIY|72WTjDjY3xtA$O|f(N}5CkJo=gyI)uVHeEb;Onw6GY!of(qbKR5dRyIL z6!8_u1!Ow+`6ytNv%*wPE5~?=9$WfZ8PW^-1O1;DBe<^+zTBc^pqfk5tfbm`JjX8e znR(1QK?nn*f^ZOzebJjnqf<#nHU8)wKHON|3Ha+UDaf= zdN6|m3Y|9L+4YvW0NtWn;JLWAR%iMsao^(1@-WYtrZ)LkbgWt#dM`p!3ZCQ1> zk|^giMUABIOL+bG9`C&?ytgb*@iY52W?~gmCjy9A_6+Ze&T5Wn$-Wtk>AC(~Ar8g^ z>v(({we8#fvFd=zUY-&QgtzY1Bq5rI@%dWxkB@V(R&hh3*|Ih%my+gnsx7*i#N58? zI=Z#(61s)3xf+Re{e@?-1oy;kpCaSxo@f@4=0V7V>DH(D1qB^xg5|mQF>|*rx76L; zxx!4nf6H2{#$_&Y1mo|!c8XOg*e^(n{(XR1$**~*(=Xk2w;8XK`EDn$B+;Ho_Z;tE zhrQ5cc=>LT!2^d0%qrSL-L=5>@ncctFscnzeJW|lMvfsemRrH0J8*aX0Wp7ZWu)Bl zV0XiEwljV__r*CZf%;{N{D;CyMU!PB+0CNEfnuSy_K?BByJB|3!o7D!c&`wy6IrIdG9Y4V4FiJAOpvVR>}IzET;u+rs?`sc`scM&ye1;y|9|?)sGF zXpOUFdo-&X^r2FDX29XrP}RXuiNYR%FB5dtxaw28qQV0znCn811R9MVWIf!SiG3TR zq;DE{=q|Q?4Dr<0PiB?#srZ5~)GJ8EVB{C}#S<*|pf|O$LUUQBH7rRKrns;o)+=is$Si0O_o~8X(M@sI6)JqpfD1R1 zog)8~a87fD$7l7cgzNErY#MahURjj9nM`{$a(ASDx1^1=T z7l}|XElI%fq37n>2mDvp-9V4^NyTNA?9~Asqgi8h06b>#&%IEfo{TKP@p6H4Z*24hbWBi|#qa~jN6zgBJ+au3(K z7RZGcLY$!MP)&mh>-ZOv_NgP)j#*fUDx@({r9v-NcD0kUsoIBchR=G#8J@SNAm9mekLuw?MAc(IaJS5*+2_R&9<MhP!m0kJ}yw1`Gm6QlTaPq zd2@H9K!}hJa_>M0_JF3>N$*-)TkB3yy&{VkA6eZpL*hvrYdu_6u* z_BJyrcGkwq7uN?tJt!i0@MEID>NPEIZ84MDf-4B*^09e>M3wJQ@ekH%4cG9bx zq~oT3`d>SDF{~HR%%*<*H*aYp4G7mDYO!3EVf9oIUKx=y+uI(SW1bTu(&jvEdt1vx zGCiPxu@VEIDB7RtK^XP@Rgkh7dr{8eswrmWJ1=UjX)h#GDG;0v-uc1p42|Kq_ORrc zy<_0fo>jRemQCP3n?||c=1uji#2=*N+6i5EUjM2rq}5{U9|f}Dwq&3|N?y)%ayVp7 zWEvlM>^phfS`mMKZyu+IVf9k_lJ`*p(iY@Q9Chv8gX-~WXY^2}gISOZy;;5FQVye~ z8(vQ+g2{209xa_3giO8#AGI^4=$m-Ad7p}Mg|{;r$rd(c->~B05?@|aING2!4k&`4 zAd!L7DFV^(!kI8?_7xE~o%CV99?)PtBFmXNpQ4GUvAIb+@mIM}OrUa%%O#w)ZKzn* zEE55XHX5Lg^_*z>Cd!Vev#*ruz_Rf(993uAl)IG{koMmoPyc_^?>pr(^yFlDrL5VUL`-Zjv1kjW>y57Cs&Q+>d%6f z^nqM1XKBCCCf~We-P{-SN`HrFWMKh;1T;K~!=goO-8Cpgj8p;ZaO2bt3 zD8KUT0)x7kD=K5($SHz@gR5eOEpl&X4>_4hWGiAEfTg4vu}@cG0_obmBBM?caJ`{( z3@S|6XwW!v1j8Z1DjUa?l#!KjKk*8%nj|k%Pc-GZ8e8Ee%|Ly^E@F8~Kb)y*YBQNK zyk737nR6HF{dUdxs^Ji`VmjE~I$o#&agbt=I|66~wlNv$G(V{K6-ZYwP=Ym7H4FHf zezINCpxDU=G|m_SEkgt_$T^1=*)`%FhNW)vQZ5>H3Tx#kKOqMLTl5-c>y7`P0i5Z6 zQ4iIh)Fb#G>XC>E>*OXHiPQsf{i}uaDM^*xwKsCybOvlsU)d&A^8Tmi=!y~j) zMelsq?d1aKNEFH?_dz6G-@u}}@E`U<{zdZhn~Xq?6ZWEk0q$L@#sXA_2|O3!8c7E{ zPN)ycUAe^n0rq8|NRpTRpVUVx`7!Itmo6!X6C%rRx-yfMdq4CMa}FFM#M62n8y?G* z8g|HS2=Q~&kLrcTicKC}AOqkz_iq9gaLK+#U%J6a$-f_ceox+Ky^+J;Domo0SnaUQ zy5I~Q<(>nGPH|1dom2u?)NWy9&pf-S7N!e?;1Fo;Pt^+e&2P`Suri^ZHcqPU7{J3u;Dv z!;77azYogs!ArIuH%m07Pbg{<{*ZkrqpkI%gSGoeK-L7J!fwXY6h0LvlKEl<-s0Pq z=py(~#OWg&CG!G4WZ!Sn56Fh$KeFLO-e8+-63yLVt@Mrl&c{Z&Vm90Nmt8nWw8on5 z1pwOPDXNlOR06is;*Sd()KTO5M<>0y4Qcr7W)lifUMzuAc4!e!n({ul_P_*!VF24Q zK6*^|OUc{-l#E3e9n;obpJd)yfv8aDtcrSiI&n|wmui12en4~O{Lx&1mPx*-Z1CB5 z@qSobmLOBR|KlxH#Oqpd5l>U5Gkj*p$1Vz5VaszeB<~F1+P0*1|I%4ud|6FdLMH40 zNRJdxL`_sAed%#vE%*7BEdnRlQ1D)q1c?a)pSIoR#R*=7wQSAs%6B`_RfUSZ=)>%v zGMk@vHu?FgGG%lCPR7;14A1Lflu>;EL&pI$y7q+94%1SohySS2 zAs_;r9l4DnR5Td5K`qQd7E5uYT3bzt0fHnB@`ulCu9bg0@e;as$Ka(8!s;YMAH44mpo-DSm4lrbo3vMUZ-^`0ks?t zsGQT6L;tZG7FcE}e=djJaM{feC=Y*|0Aw_c-9ioNtZJYWRc^6IlfEX*ELF614Z25i zqQf;W6~@hDKLmR)^JwigF&r4TIa&)x_p<9M&^u418Cw~ z6zQKil!@YO@u~&4*w{k@Rx5`-wS?1V)D#zA6>}JcyAozKd^xJhkXiEfCHU{(g#r2` zsvJ%uAyoe(!LZiFLg)SRP)S<#M8LO9fL$%DN?7%zf7HN@jM$ve$uL@OhkWfcOOMsB zbGCM{Nr1vt+NegeJecpza@;&0@`$nSR9rO)!>$J^i6jH8^cU?a&RT^ncv^ioH(49KVx!Hfr)xXyOC0;&a5N&gmTL6 zjJir@I+CgLW($SHEh!PjQ~lX&+!CJTl#ZAB6Ft)MQ_*skpIBV+<8tgnLqk?G&jk%1 z#tywTwu((lO-)}ezE}Hj#z{fgoAAi@;BaT+n|abrHa#}>%_q)+N~AI#E2Yltl6Fqh zFy0Zfw#YMs^wxHE6&);;pcwkt&5&8;AE9)&!oo* zA>NAv=h%IGeC$%CCj7(}jFi;38x0WZg}SD&Liq~htC0dAq(+6iY*T>aP97b8eIe#t z_RE3ca2ie*0;$2Zn7un7*MdG>6X9x#ArDW_Z>B|;JjH~CC&eYVUEC+Wu}6N%pnJ(3 zeJUi0Vk?JrDXhe~YEW7F*KT|}eX>z$^aFvi7-dl(z1B?>r6ZNI+x zswlgT+lmDc!u3pC903AaDRHhxo$-7+=8#!1{?mOZd9sjcPwjKM1UX?7x?7h%GyjT< zZJU$P+l$}4DPP2Fv<7r^)GHg!3VE(d2)l92}rZ6qmijol*Lj?Q+J&%8 zrVhMC2;rw4uGnQ*$EHcu7cg{}G#owdVBAsb>^k@3J1OA&CYy*Z55C>Z?CJ(84hA!hHrP(R4xWoa7K7aA$*rpsyUelEZO`v*|%MCf%XU z)9%peCqUs;`5O~{?><&9VBzG*2?6cYM^z4Ek7GAjXPsx-U&O@FoOl^0o*FSJV6N*~ zkA!-Zt#F6U(&xg#Y}M;dobt6R^0e#Q4}hef|Dtay0XSbGfX|Tsw&GJPK^n5Qe3NFD z{j^Rxd@9KX=n9fFd&m^wrj@Ui-Pg#RVjGUON-P0W<-`xO9C>G9*^_?F4GQCB`WxA4 zGLIP`zs>Dch8;^6s_aG{_Hx3V8s$?|Cql;d+n7z^L!XC+(&+hAhh5b-djpSf9~nI^ z6Kw76CAb52b52SqtGr*$-b#5dhs@SwQ?Mj;I(iXTU~J5|&0p*cG2R5)TIJl6!-GL% z#}~lchIH|aj10@Jq>I~JfKV*oevHik8;X@a`pDp07kC`t8;yE`IX)s*J#Z3yW&-#i zav0Zzn2d>7)gWCUSZ$$Ds7nu|bp~)a`#tPQ0-ObD!iHF#Ok-xnHpaj*?s{AZ0G6+>ta&??0BspPec@1TNh&Hz^W(9DEEa-`z6BvIqkHWqp^GF~DH~x(%h7nx^wzhD9p7Z&g7G!Dp`hGAhw?xY+I8UqiVs2X@*x0&w!j5A&W$Z==@-(q| zn#t)XeYuE^W!Ab&%@bHG_S&8z{3O^w=A^uNO8&h30cO=X_y<1J`D8SByu`iPTFgWrnn_-ABYK`=qkodBS5$)eMK!*Vk`L&$@0c-4QUM zI`o^`-I(#P2OWipO@~oNkn@tX!+w>u1G4~y!SMI1vT&Hf!FsS9-W_VTFvU`?J~|mB zH4PNURXD`NoTCRiU3UIXIv9TtsO7IF+Z9@k~-Dd zC~(Y>iMdpud(%!$9?zno|3NctlA-YNG zQ3fne=GHj0Q704~-B4DG9%8EZ+IU&(JQF4t>we1N#J`%kBLhj3a;ysMR-ys%o-ICY z1&Vg-ucE04kP5>&CuU{dD2>Yq?FnIwLk@Y{Z3}(H~HfOSKHR^s`L~Z8ncNhmaw`{ zH1tH;%|j6^*|$%8Xjv5?A;69d!Vrr+!rV5C3>|pKL20<_v<&l8&Wkb7e2S;_k#IQw zJIMV0*0Md@idl%c-Cq!PCfs+6CBL)jU~y1mGZ_c73$!%j!a!#;HOufVR+i)OO;K$% zlDkITm)ZQf5$Qppl+62t3k*CX%ib5bca+a=6( zkK>q<=uOOl7Z-diNrbUbucS_D zJsK|z@J`8b8uE=;71-H Njq5rp#Y$#R{txT&)%5@X From 3a6c667fdbf028613e51def7b455cadf80dec784 Mon Sep 17 00:00:00 2001 From: Gabriel Levcovitz Date: Thu, 20 Apr 2023 13:56:36 -0300 Subject: [PATCH 3/3] feat(events): add notes on genesis and metadata storage --- projects/reliable-integration/0002-low-level-design.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/projects/reliable-integration/0002-low-level-design.md b/projects/reliable-integration/0002-low-level-design.md index 471e2e4d..381fec23 100644 --- a/projects/reliable-integration/0002-low-level-design.md +++ b/projects/reliable-integration/0002-low-level-design.md @@ -14,6 +14,8 @@ At first, when the full node is initialized, it moves to the `LOAD` state, where In order to know which events will be generated, we will store the states (`LOAD` and `SYNC`) as metadata. So, when the full node starts, we will check the metadata. If it is empty or `LOAD`, we do not know where it stopped and will regenerate all events during load. If it is `SYNC`, we will not generate any events until `LOAD_FINISHED` is emitted. +The state metadata will be stored in a specific RocksDB column family, used only for event-related metadata. + In the table below we describe all possible starting scenarios for the full node, and the respective desired outcome: | Starting scenario | Vertices in the database | Events in the database | State metadata | Desired outcome | @@ -32,8 +34,6 @@ For the `Fresh start` scenario, if we consider there's a `LOAD` phase with 0 eve | `Fresh start` or `Snapshot start` or `Restart from aborted LOAD phase` | Empty or `LOAD` | All events in the database are discarded. Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. | | `Restart from aborted SYNC phase` | `SYNC` | Generate events from `event_id = last_event_id + 1`. All events come during the `SYNC` phase. | -Note: when generating snapshots, events should not be included, as they will be discarded and regenerated anyway. - Now, we can describe a step-by-step representing these rules: 1. The full node begins the initialization process @@ -48,7 +48,13 @@ Now, we can describe a step-by-step representing these rules: 1. The `EventManager` sets `SYNC` as the current node state in the database 7. The full node starts syncing new vertices from the network +### Genesis + +Events for the genesis block and two transactions are always emitted when events are generated from `event_id = 0`. + +### Snapshots +When generating snapshots, events should not be included, as they will be discarded and regenerated anyway. ## Flags