-
Notifications
You must be signed in to change notification settings - Fork 534
network: connection deduplication #4695
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
algorandskiy
merged 96 commits into
algorand:master
from
AlgoAxel:feature/connection-deduplication
Feb 17, 2023
Merged
Changes from 15 commits
Commits
Show all changes
96 commits
Select commit
Hold shift + click to select a range
e0781b9
Initial Commit - Basic Identity Challenge Passing
AlgoAxel 3988f26
Add round trip data signing for ID Challenges
AlgoAxel a3c3632
Remove identity mappings on peer disconnect
AlgoAxel 38a063f
Add Websocket based final Identification Challenge Verification
AlgoAxel 80c8758
Address linting, tests, initial comments
AlgoAxel 64c08dc
Add Protocol Version branching to Identity Challenge Exchange
AlgoAxel 0ae8dbd
Merge branch 'master' into feature/connection-deduplication
AlgoAxel 22041a1
Fix Merge and cleanup
AlgoAxel e191df9
Add Unit Tests for Identity Challenges
AlgoAxel 29c18da
Change string format to %s
AlgoAxel 8cb84b0
Fix Warnf for lint
AlgoAxel 0a76944
Enforce Identity Challenge Verification and return if it fails
AlgoAxel 507bb1f
Fix lint
AlgoAxel f2d1d96
Refactor Locking; Refactor B64 Challenge Header Creation
AlgoAxel b49d4e2
Change Protocol Tag Name for Network Identity Challenge
AlgoAxel c4929a1
Address Shant's comments pt 1
AlgoAxel 871a8bc
Merge remote-tracking branch 'origin/master' into feature/connection-…
AlgoAxel f1db433
Redesign: Optimistic Identity; Refactor
AlgoAxel 47e309e
Add Config value "ConnectionDeduplicationName"
AlgoAxel c84066f
comment fixup
AlgoAxel f7f0dd5
Remove unused unverifiedPeersByID
AlgoAxel 65b20d6
comment fixup
AlgoAxel f2817ba
comment fixup
AlgoAxel 3e00de6
comment fixup
AlgoAxel a1dd736
comment fixup
AlgoAxel 9818f14
comment fixup
AlgoAxel 4c38928
Fix Lock Usage
AlgoAxel 23ee484
Corrections from Manual Testing
AlgoAxel b5ec229
Improve config comment
AlgoAxel 9d8fe0d
Unit Tests and Related Fixes
AlgoAxel b735407
Spelling fix
AlgoAxel 28a9135
Add partitionTest to new tests
AlgoAxel 56da467
Add addPeerLockless for consistent peersByID checking
AlgoAxel 87ac231
Use Lock instead of RLock
AlgoAxel 09d045d
Add wsNetwork Unit Tests for IdentityChallenge
AlgoAxel 5d9da11
Split Start() to allow tests to inject overloaded values to wsNetwork
AlgoAxel 321ebbe
Move ConnectionDeduplicationName to v27 config
AlgoAxel 8ea163f
Rearrange use of startListener and startRoutines in unit tests
AlgoAxel cb33262
Add delay after tryConnect in unit tests
AlgoAxel 31e593c
Add Counters for Identity Challenge Disconnect
AlgoAxel 9192b6d
Remove unintended files
AlgoAxel aa7ed0f
Address Comments from cce
AlgoAxel 6f7916c
Move Identity Tracking into an identityTracker struct
AlgoAxel 0e102aa
Change Identity Challenge "Address" field to use addr
AlgoAxel 0d48753
call init before addPeer and peer.Close if addPeer fails
AlgoAxel 462ca38
Refactor: Move Identity management into a scheme
AlgoAxel c828cc0
Reviewdog fix
AlgoAxel 7c1aa66
Reviewdog fix
AlgoAxel 35d7c79
Add explainer comment
AlgoAxel b4a6255
Move comment
AlgoAxel 14b33f5
Add more unit tests
AlgoAxel 0fabd8a
Change ConnectionDeduplication config check to use PublicAddress
AlgoAxel 8785251
Merge remote-tracking branch 'origin/master' into feature/connection-…
AlgoAxel e7f9e74
Add msgp to network directory; Add autoconfig for PublicAddress
AlgoAxel 775de78
Make IdentityChallengeScheme an interface
AlgoAxel a9bb175
emit errors from IdentityChallengeScheme when verification fails
AlgoAxel de504c7
Exapnd Code Coverage; Add Better Package Comment; Add Disconnect to
AlgoAxel 6cfafe9
make msgp
AlgoAxel 8183555
Expand Test Coverage
AlgoAxel 49a1135
Fix expanded test coverage
AlgoAxel e8227fa
Testing cleanup
AlgoAxel b671728
A little more testing
AlgoAxel 976594c
Add Disconnect after Unverified Timeout
AlgoAxel 66ad453
Reviewdog fix; Extend wait for TryConnect
AlgoAxel 1777e6c
Remove flaky test assert
AlgoAxel 5e9cb35
Remove Redundant Copyright Headers
AlgoAxel a9182f5
Use host:port format for auto PublicAddress
AlgoAxel 84b776d
Fix AutoPublicAddress Test
AlgoAxel 4237762
Update network/netidentity.go
AlgoAxel ce56938
Merge branch 'master' into feature/connection-deduplication
AlgoAxel beffba4
Fix lint from merge
AlgoAxel 2c76fe3
Fix for newly merged Unit Test
AlgoAxel 95c274f
use protocol/hash prefixes and MsgSigned structs for netidentity
cce 75f308d
Merge branch 'feature/connection-deduplication' into conn-dedup-hashids
cce d5891f6
Expand logging, use .With()
AlgoAxel f1b9657
Improve Logging
AlgoAxel bcef256
Fix double defined localAddr
AlgoAxel da93fcb
add extra case to TestPeeringWithBadIdentityVerification
cce 74a99b0
incorporate closeEarly; Amend tests
AlgoAxel fea7d63
rename VerifyRequestAndAttachResponse and update a few comments
cce 240b11f
Update network/netidentity.go
cce 0dcdac3
Update network/netidentity.go
cce ef353bb
Address comments
AlgoAxel 9950d72
like prioScheme, only initialize identityScheme if it is enabled
cce 2f81650
Move metric increments
AlgoAxel a8cba7d
Merge branch 'feature/connection-deduplication' into nil-identityScheme
cce 95d487f
Merge pull request #3 from cce/nil-identityScheme
AlgoAxel b67c1dd
rename parameter attach to attachTo
AlgoAxel 74bf8ee
have mockIdentityTracker wrap real identityTracker
cce f3afd4a
flip meaning of isOccupied
cce b565751
Merge pull request #4 from cce/test-real-tracker
AlgoAxel 97d4c80
correct mocked identityTracker isOccupied behavior
AlgoAxel 60a1f2b
Increase identity peering test timeouts
AlgoAxel 3b40e63
touch gitignore
AlgoAxel c26f448
restore gitignore
AlgoAxel 5d9c380
add disconnectBadIdentityData
cce File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,6 +48,7 @@ crypto/libs | |
| *~ | ||
| *.swp | ||
| *.swo | ||
| *.swn | ||
|
|
||
| # Mac | ||
| .DS_Store | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,219 @@ | ||
| // Copyright (C) 2019-2022 Algorand, Inc. | ||
| // This file is part of go-algorand | ||
| // | ||
| // go-algorand is free software: you can redistribute it and/or modify | ||
| // it under the terms of the GNU Affero General Public License as | ||
| // published by the Free Software Foundation, either version 3 of the | ||
| // License, or (at your option) any later version. | ||
| // | ||
| // go-algorand is distributed in the hope that it will be useful, | ||
| // but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| // GNU Affero General Public License for more details. | ||
| // | ||
| // You should have received a copy of the GNU Affero General Public License | ||
| // along with go-algorand. If not, see <https://www.gnu.org/licenses/>. | ||
|
|
||
| package network | ||
|
|
||
| import ( | ||
| "bytes" | ||
| "context" | ||
| "encoding/base64" | ||
| "fmt" | ||
| "time" | ||
|
|
||
| "github.com/algorand/go-algorand/crypto" | ||
| "github.com/algorand/go-algorand/protocol" | ||
| ) | ||
|
|
||
| // ProtocolConectionIdentityChallengeHeader is used to exchange IdentityChallenges | ||
| const ProtocolConectionIdentityChallengeHeader = "X-Algorand-IdentityChallenge" | ||
|
algonautshant marked this conversation as resolved.
Outdated
|
||
|
|
||
| // minimumProtocolVersion is used to evaluate if a peer's supported protocol | ||
| // should include identityChallenge exchange. {2, 3} represents version 2.3 | ||
| var minimumProtocolVersion = [2]int64{2, 3} | ||
|
|
||
| func shouldSupportIdentityChallenge(v string) bool { | ||
| maj, min, err := versionToMajorMinor(v) | ||
| if err != nil { | ||
| return false | ||
| } | ||
| if maj < minimumProtocolVersion[0] { | ||
| return false | ||
| } | ||
| if maj > minimumProtocolVersion[0] { | ||
| return true | ||
| } | ||
| if min < minimumProtocolVersion[1] { | ||
| return false | ||
| } | ||
| return true | ||
| } | ||
|
|
||
| type identityChallenge struct { | ||
| Key crypto.PublicKey `codec:"pk"` | ||
| Challenge [32]byte `codec:"c"` | ||
| Signature crypto.Signature `codec:"s"` | ||
| } | ||
|
|
||
| type identityChallengeResponse struct { | ||
| identityChallenge | ||
| ResponseChallenge [32]byte `codec:"rc"` | ||
| } | ||
|
|
||
| // NewIdentityChallenge creates an IdentityChallenge with randomized 32byte Challenge | ||
| func NewIdentityChallenge(p crypto.PublicKey) identityChallenge { | ||
| c := identityChallenge{ | ||
| Key: p, | ||
| Challenge: [32]byte{}, | ||
| } | ||
| crypto.RandBytes(c.Challenge[:]) | ||
| return c | ||
| } | ||
|
|
||
| func (i identityChallenge) signableBytes() []byte { | ||
| return bytes.Join([][]byte{ | ||
|
AlgoAxel marked this conversation as resolved.
Outdated
|
||
| []byte("IC"), | ||
| i.Challenge[:], | ||
| i.Key[:], | ||
| }, | ||
| []byte(":")) | ||
| } | ||
|
|
||
| func (i identityChallenge) sign(s *crypto.SignatureSecrets) crypto.Signature { | ||
| return s.SignBytes(i.signableBytes()) | ||
| } | ||
|
|
||
| func (i identityChallenge) verify() error { | ||
| b := i.signableBytes() | ||
| verified := i.Key.VerifyBytes(b, i.Signature) | ||
| if !verified { | ||
| return fmt.Errorf("included signature does not verify identity challenge") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // SignAndEncodeB64 signs the identityChallenge, attaches a signature, and converts | ||
| // the structure to a b64 and msgpk'd string to be included as a header | ||
| func (i *identityChallenge) SignAndEncodeB64(s *crypto.SignatureSecrets) string { | ||
| i.Signature = i.sign(s) | ||
| enc := protocol.EncodeReflect(i) | ||
| b64enc := base64.StdEncoding.EncodeToString(enc) | ||
| return b64enc | ||
| } | ||
|
|
||
| // IdentityChallengeFromB64 will return an Identity Challenge from the B64 header string | ||
| func IdentityChallengeFromB64(i string) identityChallenge { | ||
| msg, err := base64.StdEncoding.DecodeString(i) | ||
| if err != nil { | ||
| return identityChallenge{} | ||
|
algonautshant marked this conversation as resolved.
Outdated
|
||
| } | ||
| ret := identityChallenge{} | ||
| err = protocol.DecodeReflect(msg, &ret) | ||
| if err != nil { | ||
| return identityChallenge{} | ||
| } | ||
| return ret | ||
| } | ||
|
|
||
| // SignedEncodedB64IdentityChallenge creates an encoded header string for identityChallenges, | ||
| // provided signing keys. It simply wraps the process of creating and encoding the object. | ||
| // Also returns the challenge generated | ||
| func SignedEncodedB64IdentityChallenge(keys *crypto.SignatureSecrets) ([32]byte, string) { | ||
|
algonautshant marked this conversation as resolved.
Outdated
|
||
| c := NewIdentityChallenge(keys.SignatureVerifier) | ||
| return c.Challenge, c.SignAndEncodeB64(keys) | ||
| } | ||
|
|
||
| // NewIdentityChallengeResponse creates an IdentityChallengeResponse from a received identityChallenge | ||
| func NewIdentityChallengeResponse(p crypto.PublicKey, id identityChallenge) identityChallengeResponse { | ||
| c := identityChallengeResponse{ | ||
| identityChallenge: identityChallenge{ | ||
| Key: p, | ||
| Challenge: id.Challenge, | ||
| }, | ||
| ResponseChallenge: [32]byte{}, | ||
| } | ||
| crypto.RandBytes(c.ResponseChallenge[:]) | ||
| return c | ||
| } | ||
|
|
||
| func (i identityChallengeResponse) signableBytes() []byte { | ||
| return bytes.Join([][]byte{ | ||
|
AlgoAxel marked this conversation as resolved.
Outdated
|
||
| []byte("ICR"), | ||
| i.Challenge[:], | ||
| i.ResponseChallenge[:], | ||
| i.Key[:], | ||
| }, | ||
| []byte(":")) | ||
| } | ||
|
|
||
| func (i identityChallengeResponse) sign(s *crypto.SignatureSecrets) crypto.Signature { | ||
| return s.SignBytes(i.signableBytes()) | ||
| } | ||
|
|
||
| func (i identityChallengeResponse) verify() error { | ||
| b := i.signableBytes() | ||
| verified := i.Key.VerifyBytes(b, i.Signature) | ||
| if !verified { | ||
| return fmt.Errorf("included signature does not verify identity challenge") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // IdentityChallengeResponseFromB64 will return an Identity Challenge Response from the B64 header string | ||
| func IdentityChallengeResponseFromB64(i string) identityChallengeResponse { | ||
| msg, err := base64.StdEncoding.DecodeString(i) | ||
| if err != nil { | ||
| return identityChallengeResponse{} | ||
| } | ||
| ret := identityChallengeResponse{} | ||
| err = protocol.DecodeReflect(msg, &ret) | ||
| if err != nil { | ||
| return identityChallengeResponse{} | ||
| } | ||
| return ret | ||
| } | ||
|
|
||
| func (i *identityChallengeResponse) SignAndEncodeB64(s *crypto.SignatureSecrets) string { | ||
| i.Signature = i.sign(s) | ||
| enc := protocol.EncodeReflect(i) | ||
| b64enc := base64.StdEncoding.EncodeToString(enc) | ||
| return b64enc | ||
| } | ||
|
|
||
| // SignedEncodedB64IdentityChallengeResponse will create a new IdentityChallengeResponse, | ||
| // and will sign and encode to B64 for header inclusion. The challenge generated is also returned | ||
| func SignedEncodedB64IdentityChallengeResponse(keys *crypto.SignatureSecrets, c identityChallenge) ([32]byte, string) { | ||
|
algonautshant marked this conversation as resolved.
Outdated
|
||
| r := NewIdentityChallengeResponse(keys.SignatureVerifier, c) | ||
|
algonautshant marked this conversation as resolved.
Outdated
|
||
| return r.ResponseChallenge, r.SignAndEncodeB64(keys) | ||
| } | ||
|
|
||
| // SendIdentityChallengeVerification will send a signaturure of a challenge which was assigned | ||
| // by the wsPeer | ||
| func SendIdentityChallengeVerification(wp *wsPeer, sig crypto.Signature) error { | ||
| mbytes := append([]byte(protocol.NetIDVerificationTag), sig[:]...) | ||
| sent := wp.writeNonBlock(context.Background(), mbytes, true, crypto.Digest{}, time.Now()) | ||
| if !sent { | ||
| return fmt.Errorf("could not send identity challenge verification") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // identityVerificationHandler processes identity challenge verification messages, | ||
| // which are websocket messages containing a signed signature which should match with the | ||
| // peer's assigned challenge | ||
| func identityVerificationHandler(message IncomingMessage) OutgoingMessage { | ||
| peer := message.Sender.(*wsPeer) | ||
| sig := crypto.Signature{} | ||
| copy(sig[:], message.Data[:64]) | ||
| verified := peer.identity.VerifyBytes(peer.identityChallenge[:], sig) | ||
| if verified { | ||
| peer.IdentityVerified() | ||
|
AlgoAxel marked this conversation as resolved.
Outdated
|
||
| } | ||
| return OutgoingMessage{} | ||
| } | ||
|
|
||
| var identityHandlers = []TaggedMessageHandler{ | ||
| {protocol.NetIDVerificationTag, HandlerFunc(identityVerificationHandler)}, | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.