New builders PoC, take 3

[Menduist]

Alternative to #646

This PR adds a generic builder system:

import builder3
proc setup(i: int, x: float, res: var string) {.setupproc.} =
  res = $(i.float + x)

proc setup(i: var float) {.setupproc.} =
  i = 10.12

proc setup(i: string) {.setupproc.} = discard

let builder = Builder()
builder.with(5.int)
builder.with(float)
echo builder.build(string) # outputs '15.12'

The builder will read the setupprocs of the various input type, and try to create the string from it. In this case:

• float is required to create int, and int will output a string
• float & string doesn't require anything
• So, the float will be created, and his setup procedure (which sets it to 10.12) will be called
• Now that we have the float, the int can be created from it, and will add itself (which in this case is 5, as given to with). And finally, output the string
• The string setup will be called, and is a noop

I really like this system, because the ordering is done automatically depending on the arguments of setup, and it's fully generic.
Applied to the switch:

  let rng = newRng()
  var switchBuilder = Builder.new()
  switchBuilder.with(PeerInfo, key=PrivateKey.random(rng[]).tryGet(), addrs=[ma])
  switchBuilder.with(PeerStore)
  switchBuilder.with(MultistreamSelect)
  switchBuilder.with(Identify) 
  switchBuilder.with(rng)
  switchBuilder.with(Noise)
  switchBuilder.with(Mplex)
  switchBuilder.with(ConnManager)
  switchBuilder.with(TcpTransport)
  switchBuilder.with(Dialer)
  switchBuilder.with(TcpTransport)
  
  return switchBuilder.build(Switch)

This is very similar to #646, except that we don't have to order the withs. Also, it can be used on any type (rng for instance)

The setup is however quite different:

proc setup*(p: ConnManager, peerStore: PeerStore) {.setupproc.} =
  p.peerStore = peerStore

The most basic kind of setup (beside no-ops), we have a dependency on PeerStore, so we'll be setup once PeerStore is available. And we can then do whatever we want with it

proc setup*(n: Mplex, muxers: var Table[string, MuxerProvider]) {.setupproc.} =
  proc newMuxer(conn: Connection): Muxer =
    Mplex.new(conn)
  muxers[MplexCodec] = MuxerProvider.new(newMuxer, MplexCodec)

Here, var is used to signal that we will modify muxers, so any setup dependent on Table[string, MuxerProvider] shouldn't be called before us.

proc setup*(
  p: Switch,
  peerInfo: PeerInfo,
  ms: MultistreamSelect,
  transports: seq[Transport],
  connManager: ConnManager,
  peerStore: PeerStore,
  dialer: Dialer,
  identify: Identify
  ) {.setupproc, raises: [Defect, LPError].} =

  p.peerInfo = peerInfo
  p.ms = ms
  p.transports = transports
  p.connManager = connManager
  p.peerStore = peerStore
  p.dialer = dialer
  p.mount(identify)

Lot of dependencies for the switch, but this should be straightforward to understand.

The builder system itself could be moved to another package, since it's completely standalone.
More details on how it works:

• At the core of it, we have a context, which is a type -> value table, similar to how Peer store refacto #700 works. For instance, context[Switch] = Switch.new()
• Then, the setupproc will read the parameters, and build a structure from the parameters, example:

proc setup(i: int, x: float, res: var string) {.setupproc.} =
  res = $(i.float + x)

will become

proc setup(i: int; x: float; res: var string) =
  res = $(i.float + x)

proc getBuildDeps*(T: type[int]; val: int): BuildDep =
  result = BuildDep()
  result.outs.add("int")
  result.deps.add("float")
  result.outs.add("string")
  result.run = proc (b: Builder) =
    var valCopy = val
    var p0 = b.context[string]
    setup(valCopy, b.context[float], p0)
    b.context[string] = p0
    b.context[int] = valCopy

• Finally, with will collect every BuildDep. When build is called, we try to build everything if the dependency graph is not cyclic.

This is a rough PoC, please don't look too much at builder3.nim or your retina may burn

However, I find the API quite nice: if you have a dep, just take it as a parameter, and everything work automagically. This system can be improved upon in a few powerful way, for instance default values:

# if the user didn't setup the PeerStore manually with `with`, create the default one
proc setup*(p: ConnManager, peerStore: PeerStore = PeerStore.new()) {.setupproc.} =
  p.peerStore = peerStore

Multiple parameters to build:

  let (switch, discoveryInterface) = builder.build(Switch, DiscoveryInterface)

And probably other fun stuff.

Also note that the setup procedure is usable without the entire system:

let connManager = ConnManager.new()
connManager.setup(PeerStore.new())

And that you can build anything with the system, for instance in tests:

let builder = Builder.new()
builder.with(PeerStore, maxSize=100)
let conmanager = builder.build(ConnManager)

which will become even more powerful with default values (you could build anything without worrying about it's deps)