Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parachain/collator): handle backed overseer message #3559

Merged
merged 94 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
2f3bb6f
feat/scale: add `BitVec` (#3253)
kanishkatn May 29, 2023
f001854
feat(erasure_coding): introduce erasure coding for PoV Distributor (#…
edwardmack Jun 8, 2023
d2c0163
Fix(lint): fixes a lint issue in bitvec-test (#3306)
kanishkatn Jun 9, 2023
710b03c
feat(parachain): add types (#3297)
kanishkatn Jun 15, 2023
be2aaea
feat(parachain): Add StatementDistributionMessage varingDataType (#3316)
axaysagathiya Jun 20, 2023
d08d7f1
feat/runtime: Add few parachain runtime calls (#3241)
kanishkatn Jun 21, 2023
4a103f4
feat(parachain): Create struct for Approval Distribution Message (#3326)
edwardmack Jun 21, 2023
4928d45
chore(parachain): improve comments of statement and statement distrib…
axaysagathiya Jun 23, 2023
50f2bdb
feat(parachain): Add CollationProtocol VaryingDataType (#3337)
axaysagathiya Jun 26, 2023
c88e72d
feat(parachain): Implement request and response message for /req_stat…
axaysagathiya Jul 6, 2023
4d103a3
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 6, 2023
b8ddd99
fix(lib/erasure): Move test assertion to non-error condition (#3371)
edwardmack Jul 7, 2023
c805896
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 7, 2023
c2d1083
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 7, 2023
1104a22
chore(lib/parachain): add tests to decode varying data types regardin…
axaysagathiya Jul 7, 2023
e7ab6e9
chore(lib/parachain): remove pov byte array and use pov struct (#3375)
axaysagathiya Jul 7, 2023
59ff8eb
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 10, 2023
b445405
feat(lib/parachain): added parachain service and registering protocol…
kishansagathiya Jul 11, 2023
2fb5189
Implemented extra parachain host runtime API calls (#3237)
kishansagathiya Jul 27, 2023
4702a7a
Fix(lib/parachain): use actual forkID in parachain network protocol (…
axaysagathiya Aug 1, 2023
9e12f3a
feat(lib/parachain): introduced parachain candidate validation (#3249)
kishansagathiya Aug 4, 2023
b571559
moved `lib/parachain` to `dot/parachain` (#3429)
kishansagathiya Aug 8, 2023
d57a322
feat(lib/parachain): Create and integrate Validation Protocol struct …
edwardmack Aug 18, 2023
9ed0b70
runtime/parachain: add ValidationCodeByHash and copy tests to `wazero…
kanishkatn Aug 22, 2023
0772943
feat(lib/erasure): implement Go binding over rust for erasure coding …
axaysagathiya Aug 24, 2023
38c0c49
feat(lib/erasure): construct a trie of chunks of erasure coded values…
axaysagathiya Aug 28, 2023
33e9f9a
chore(parachain): use wazero instead of wasmer
kishansagathiya Aug 30, 2023
788a160
chore(dot/parachain): rename statement (#3470)
axaysagathiya Sep 17, 2023
12fcb0f
feat(dot/parachain): Implement overseer skeleton (#3460)
edwardmack Sep 27, 2023
703c211
chore(dot/peerset): added parachain related reputation values and rea…
kishansagathiya Oct 2, 2023
36d91ce
feat(dot/parachain): Implement candidate backing skeleton (#3497)
axaysagathiya Oct 2, 2023
56496bc
collator protocol skeleton (#3512)
kishansagathiya Oct 5, 2023
807c876
Handle Declare message received by a collator
kishansagathiya Oct 9, 2023
5d1f2c0
temp
kishansagathiya Oct 10, 2023
2242509
added tests, need to prettify a bit though
kishansagathiya Oct 11, 2023
526c657
prettified tests
kishansagathiya Oct 11, 2023
cf84289
added one more check
kishansagathiya Oct 11, 2023
5051079
Update dot/parachain/collator-protocol/message.go
kishansagathiya Oct 12, 2023
9b5a969
Update dot/parachain/collator-protocol/message.go
kishansagathiya Oct 12, 2023
f435794
fixing lint
kishansagathiya Oct 12, 2023
58fdcba
Merge branch 'kishan/feat/parachain/collator-declare' of github.com:C…
kishansagathiya Oct 12, 2023
fafbb5e
underscore to camelcase
kishansagathiya Oct 12, 2023
c9055dd
feat(parachain): availability store skeleton (#3521)
edwardmack Oct 12, 2023
6a300dc
temp
kishansagathiya Oct 12, 2023
c9588eb
some progress
kishansagathiya Oct 13, 2023
a9c55fa
temp
kishansagathiya Oct 13, 2023
23dccd0
feat(parachain/collator): handle Declare message received by a collat…
kishansagathiya Oct 16, 2023
72a1453
more changes, some improvements in overseer
kishansagathiya Oct 18, 2023
4d4c049
more changes
kishansagathiya Oct 18, 2023
8b77ca9
more addition to collator protocol
kishansagathiya Oct 19, 2023
d1c5de8
added some tests for advertise collation
kishansagathiya Oct 19, 2023
4fb70f3
feat(parachain/overseer): added message forwarding in overseer (#3546)
kishansagathiya Oct 20, 2023
dd0d8ca
Merge branch 'feat/parachain' into kishan/feat/parachain/collator-adv…
kishansagathiya Oct 21, 2023
db47168
wrote some tests
kishansagathiya Oct 23, 2023
3d0e782
TestInsertAdvertisement
kishansagathiya Oct 25, 2023
64846c8
added TestFetchCollation
kishansagathiya Oct 25, 2023
4ba1cf2
fixing lint
kishansagathiya Oct 25, 2023
afcc501
small fix
kishansagathiya Oct 25, 2023
c90d6c8
more progress
kishansagathiya Oct 30, 2023
f56db9c
more progress
kishansagathiya Oct 30, 2023
0d9c1bd
some more tests
kishansagathiya Oct 31, 2023
0bd5c40
more tests
kishansagathiya Oct 31, 2023
368cc7d
handling backed message
kishansagathiya Oct 31, 2023
90c24f8
more progress
kishansagathiya Oct 31, 2023
e1925b6
more stuff
kishansagathiya Oct 31, 2023
6440a1d
feat(parachain/collator): handle AdvertiseCollation message received…
kishansagathiya Nov 10, 2023
bca22c9
feat(parachain/availabilitystore): implement database store/retrieve …
edwardmack Nov 20, 2023
fedb910
feat(parachain/collator): handle Seconded overseer message (#3557)
kishansagathiya Nov 21, 2023
b0a9d73
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Nov 21, 2023
a0dcc91
feat(dot/parachain): added overseer signals (#3638)
kishansagathiya Jan 9, 2024
f7bc903
feat(dot/parachain/backing): handle `statement` message coming from o…
axaysagathiya Jan 16, 2024
58b6204
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Jan 17, 2024
e03dd3f
Merge branch 'kishan/feat/parachain/collation-backed' of github.com:C…
kishansagathiya Jan 17, 2024
b131ff9
chore(dot/parachain/overseer): fix `TestHandleBlockEvents` the flaky …
axaysagathiya Jan 23, 2024
19a437a
feat(dot/parachain/backing): handle CanSecond incoming overseer messa…
axaysagathiya Jan 23, 2024
7c94a6f
feat(parachain/availabilitystore): implement database access function…
edwardmack Jan 23, 2024
c7684be
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Jan 24, 2024
bfa13b0
temp
kishansagathiya Jan 24, 2024
0db0f7c
temp
kishansagathiya Jan 25, 2024
8b82afc
temp
kishansagathiya Jan 30, 2024
a7e24d0
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Feb 12, 2024
04d8f15
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Feb 12, 2024
f603330
temp
kishansagathiya Feb 13, 2024
f4e02bd
tests for backed message
kishansagathiya Feb 15, 2024
6bafa28
Merge branch 'feat/parachain' into kishan/feat/parachain/collation-ba…
kishansagathiya Feb 15, 2024
21f97ef
removed backed_test file
kishansagathiya Feb 15, 2024
0b7e8eb
temp
kishansagathiya Feb 15, 2024
6e39ad7
fix lint
kishansagathiya Feb 15, 2024
1282033
fix lint
kishansagathiya Feb 15, 2024
cef9c2b
test fix
kishansagathiya Feb 15, 2024
5840653
temp
kishansagathiya Feb 19, 2024
21b0433
addressed reviews
kishansagathiya Feb 19, 2024
0ef4161
fix tests
kishansagathiya Feb 21, 2024
dc00ad7
temp
kishansagathiya Feb 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dot/mock_node_builder_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 20 additions & 1 deletion dot/node_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,26 @@ func TestNewNode(t *testing.T) {
return stateSrvc, nil
})

mockBlockState := &state.BlockState{}
tries := state.NewTries()
tries.SetEmptyTrie()

db := state.NewInMemoryDB(t)
prefix := []byte{98, 108, 111, 99, 107}

err = db.Put(append(prefix, []byte{104, 115, 104, 0, 0, 0, 0, 0, 0, 0, 0}...),
[]byte{98, 108, 111, 99, 107, 104, 115, 104})
require.NoError(t, err)
err = db.Put(append(prefix, []byte{104, 114, 115}...),
[]byte{98, 108, 111, 99, 107, 104, 115, 104, 98, 108, 111, 99, 107, 104, 115, 104})
require.NoError(t, err)
err = db.Put(append(prefix, []byte{102, 105, 110, 97, 108, 105, 115, 101, 100, 95, 104, 101, 97, 100, 98, 108, 111, 99, 107, 104, 115, 104, 98, 108, 111, 99, 107, 104, 115, 104}...), //nolint
[]byte{98, 108, 111, 99, 107, 104, 115, 104, 98, 108, 111, 99, 107, 104, 115, 104})
require.NoError(t, err)
err = db.Put(append(prefix, []byte{104, 100, 114, 98, 108, 111, 99, 107, 104, 115, 104, 98, 108, 111, 99, 107, 104, 115, 104, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}...), //nolint
[]byte{59, 69, 201, 194, 45, 206, 206, 117, 163, 10, 204, 156, 41, 104, 203, 49, 30, 107, 5, 87, 53, 15, 131, 180, 48, 244, 117, 89, 219, 120, 105, 117, 0, 9, 249, 202, 40, 223, 5, 96, 194, 41, 26, 161, 107, 86, 225, 94, 7, 209, 225, 146, 112, 136, 245, 19, 86, 213, 34, 114, 42, 169, 12, 167, 203, 218, 38, 220, 140, 20, 85, 248, 248, 28, 174, 18, 228, 252, 89, 226, 60, 233, 97, 178, 200, 55, 246, 211, 246, 100, 40, 58, 249, 6, 211, 68, 224, 0})

mockBlockState, err := state.NewBlockState(db, tries, nil)
require.NoError(t, err)
mockStateService := &state.Service{
Block: mockBlockState,
}
Expand Down
63 changes: 35 additions & 28 deletions dot/parachain/collator-protocol/message.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ package collatorprotocol
import (
"errors"
"fmt"
"time"

"github.com/ChainSafe/gossamer/dot/network"
"github.com/ChainSafe/gossamer/dot/parachain/backing"
collatorprotocolmessages "github.com/ChainSafe/gossamer/dot/parachain/collator-protocol/messages"
parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
"github.com/ChainSafe/gossamer/dot/peerset"
"github.com/ChainSafe/gossamer/lib/common"
Expand Down Expand Up @@ -178,10 +180,10 @@ const (
Seconded
)

// BlockedAdvertisement is vstaging advertisement that was rejected by the backing
// blockedAdvertisement is vstaging advertisement that was rejected by the backing
// subsystem. Validator may fetch it later if its fragment
// membership gets recognised before relay parent goes out of view.
type BlockedAdvertisement struct {
type blockedAdvertisement struct {
// peer that advertised the collation
peerID peer.ID
collatorID parachaintypes.CollatorID
Expand All @@ -194,26 +196,23 @@ func (cpvs CollatorProtocolValidatorSide) canSecond(
candidateRelayParent common.Hash,
candidateHash parachaintypes.CandidateHash,
parentHeadDataHash common.Hash,
) bool {
) (bool, error) {

canSecondRequest := backing.CanSecondMessage{
CandidateParaID: candidateParaID,
CandidateRelayParent: candidateRelayParent,
CandidateHash: candidateHash,
ParentHeadDataHash: parentHeadDataHash,
ResponseCh: make(chan bool),
}

responseChan := make(chan bool)

cpvs.SubSystemToOverseer <- struct {
responseChan chan bool
canSecondRequest backing.CanSecondMessage
}{
responseChan: responseChan,
canSecondRequest: canSecondRequest,
cpvs.SubSystemToOverseer <- canSecondRequest
select {
case canSecondResponse := <-canSecondRequest.ResponseCh:
return canSecondResponse, nil
case <-time.After(parachaintypes.SubsystemRequestTimeout):
return false, parachaintypes.ErrSubsystemRequestTimeout
}

// TODO: Add timeout
return <-responseChan
}

// Enqueue collation for fetching. The advertisement is expected to be
Expand Down Expand Up @@ -297,10 +296,6 @@ func (cpvs *CollatorProtocolValidatorSide) fetchCollation(pendingCollation Pendi

func (cpvs *CollatorProtocolValidatorSide) handleAdvertisement(relayParent common.Hash, sender peer.ID,
prospectiveCandidate *ProspectiveCandidate) error {
// TODO:
// - tracks advertisements received and the source (peer id) of the advertisement
// - accept one advertisement per collator per source per relay-parent

perRelayParent, ok := cpvs.perRelayParent[relayParent]
if !ok {
cpvs.net.ReportPeer(peerset.ReputationChange{
Expand Down Expand Up @@ -366,26 +361,38 @@ func (cpvs *CollatorProtocolValidatorSide) handleAdvertisement(relayParent commo
}

/*NOTE:---------------------------------------Matters only in V2----------------------------------------------*/
isSecondingAllowed := !perRelayParent.prospectiveParachainMode.IsEnabled || cpvs.canSecond(
collatorParaID,
relayParent,
prospectiveCandidate.CandidateHash,
prospectiveCandidate.ParentHeadDataHash,
)
var isSecondingAllowed bool
if !perRelayParent.prospectiveParachainMode.IsEnabled {
isSecondingAllowed = true
} else {
isSecondingAllowed, err = cpvs.canSecond(
collatorParaID,
relayParent,
prospectiveCandidate.CandidateHash,
prospectiveCandidate.ParentHeadDataHash,
)
if err != nil {
return fmt.Errorf("checking if seconding is allowed: %w", err)
}
}

if !isSecondingAllowed {
logger.Infof("Seconding is not allowed by backing, queueing advertisement,"+
" relay parent: %s, para id: %d, candidate hash: %s",
relayParent, collatorParaID, prospectiveCandidate.CandidateHash)

blockedAdvertisements := append(cpvs.BlockedAdvertisements, BlockedAdvertisement{
backed := collatorprotocolmessages.Backed{
ParaID: collatorParaID,
ParaHead: prospectiveCandidate.ParentHeadDataHash,
}

blockedAd := blockedAdvertisement{
peerID: sender,
collatorID: peerData.state.CollatingPeerState.CollatorID,
candidateRelayParent: relayParent,
candidateHash: prospectiveCandidate.CandidateHash,
})

cpvs.BlockedAdvertisements = blockedAdvertisements
}
cpvs.BlockedAdvertisements[backed.String()] = []blockedAdvertisement{blockedAd}
return nil
}
/*--------------------------------------------END----------------------------------------------------------*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
package messages

import (
"fmt"

parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
"github.com/ChainSafe/gossamer/lib/common"
)
Expand Down Expand Up @@ -35,6 +37,10 @@ type Backed struct {
ParaHead common.Hash
}

func (b Backed) String() string {
return fmt.Sprintf("para_id:_%d,_para_head:_%s", b.ParaID, b.ParaHead.String())
}

// Invalid represents an invalid candidata.
// We recommended a particular candidate to be seconded, but it was invalid; penalise the collator.
type Invalid struct {
Expand Down
92 changes: 92 additions & 0 deletions dot/parachain/collator-protocol/mock_blockstate_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

103 changes: 103 additions & 0 deletions dot/parachain/collator-protocol/mock_subsystem_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions dot/parachain/collator-protocol/mocks_generate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
package collatorprotocol

//go:generate mockgen -destination=mocks_test.go -package=$GOPACKAGE . Network
//go:generate mockgen -destination=mock_blockstate_test.go -package=$GOPACKAGE github.com/ChainSafe/gossamer/dot/parachain/overseer BlockState
//go:generate mockgen -destination=mock_subsystem_test.go -package=$GOPACKAGE github.com/ChainSafe/gossamer/dot/parachain/overseer Subsystem
Loading
Loading