Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peer store refacto #700

Merged
merged 12 commits into from
May 25, 2022
Merged

Peer store refacto #700

merged 12 commits into from
May 25, 2022

Conversation

Menduist
Copy link
Contributor

@Menduist Menduist commented Mar 9, 2022

Improves the Peer Store:

There is now a global PeerStore structure (instead of having one for libp2p, one for waku, etc)
To create a PeerBook:

type StateBook = ref object of PeerBook[MagicState]
switch.peerStore[StateBook][peerId] = Rocking

The operators have been nim-ified ([], []= & del)

And a pruner system, that will only keep a certain amount of dead peers in memory

Also, switch to seqs instead of HashSet for the inner books (eg PeerBook[seq[MultiAddress]]) because we generally don't have enough elements to justify the overhead of a HashSet

LMKWYT

@Menduist Menduist requested review from dryajov and jm-clius March 9, 2022 17:29
@Menduist Menduist marked this pull request as ready for review March 10, 2022 11:56
peerStore: PeerStore,
peerId: PeerId) =

if peerStore.capacity <= 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usually -1 would mean infinite capasity, seems to be reversed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this,

0: regular capacity
0: no capacity
< 0: infinite

@@ -307,6 +307,9 @@ proc peerCleanup(c: ConnManager, conn: Connection) {.async.} =
await c.triggerConnEvent(
peerId, ConnEvent(kind: ConnEventKind.Disconnected))
await c.triggerPeerEvents(peerId, PeerEvent(kind: PeerEventKind.Left))

if not(c.peerStore.isNil):
c.peerStore.cleanup(peerId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to clean up peers that were disconnected. If Waku were to use the native switch peer store, we'd need to change some peer persistence logic though - we do currently track disconnected peers in our peer store (a DisconnectBook), mostly because it's a convenient way to determine a back off period before attempting to reconnect to such peers after a restart. I think we need to do something else here in any case, though, and can't think of any other good reasons to keep peers in the store after disconnection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, currently this will keep a certain amount of disconnected peers (peerStore.capacity which the app can choose), and then remove them from every book.

I was wondering if you needed more sophisticated logic (eg, have a per-book settings on how much dead peer to save)
Happy to brainstorm a bit on your needs :)

jm-clius
jm-clius previously approved these changes Mar 17, 2022
Copy link
Contributor

@jm-clius jm-clius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Thanks!

@dryajov
Copy link
Contributor

dryajov commented Apr 8, 2022

Whats missing here?

@Menduist Menduist mentioned this pull request May 11, 2022
@@ -187,6 +192,9 @@ proc build*(b: SwitchBuilder): Switch
if isNil(b.rng):
b.rng = newRng()

if isNil(b.peerStore):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If PeerStore is required now, we don't need a withPeerStore anymore, at least not one that takes an instance only the non-default params needed to construct it? Otherwise, this construction flow doesn't make much sense.

@dryajov
Copy link
Contributor

dryajov commented May 17, 2022

LGTM, besides the convoluted construction flow. Needs to be changed before merging.

@Menduist Menduist force-pushed the peerstorerefacto branch from 1734add to f0c0789 Compare May 20, 2022 10:10
@Menduist Menduist enabled auto-merge (squash) May 24, 2022 16:23
@Menduist Menduist merged commit 60becad into unstable May 25, 2022
@Menduist Menduist deleted the peerstorerefacto branch May 25, 2022 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants