Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure C# SpacetimeDB runtime #1455

Merged
merged 1 commit into from
Jul 12, 2024
Merged

Conversation

RReverser
Copy link
Contributor

Description of Changes

This is a large refactor moving things around to different namespaces and utils. I meant to do this for a while, but it became particularly urgent as we're moving towards API stability.

This PR has 3 goals:

  1. Simplify usings that users need to work on C# modules - now using SpacetimeDB; is all you need, without separate using SpacetimeDB.Module; and using static SpacetimeDB.Runtime;.
  2. Split out and "hide" private APIs that are public only because they're used in the code emitted by codegen, but are not meant to be called by users and are not guaranteed to be stable. Before this PR, they lived in the abovementioned SpacetimeDB.Module namespace and SpacetimeDB.Runtime static class alongside real public APIs, but after this PR they're all consolidated under the SpacetimeDB.Internal namespace. This should prevent them from appearing in IDE autocomplete and make it clear they're not meant for public use.
  3. Reuse SpacetimeDB.Address and SpacetimeDB.Identity between client and module SDKs. Before this PR, they were already very similar, but couldn't be reused because they lived under SpacetimeDB.Runtime class in the server-side API and under SpacetimeDB namespace in the client-side API. After this PR, they both live in the SpacetimeDB namespace in the BSATN.Runtime package which is shared between the SDKs. I've also converted them to records to simplify equality operations, added a base helper they both inherit from, as well as added some extra checks - length check + conversion to record to prevent null byte[] contents (which are too easy to get with structs but not with class or record).

Closes #1454.

API and ABI breaking changes

Very much so.

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

3 - I caught a lot of issues via both dotnet test and the SDK tests, but this is a lot of renaming and minor API changes, so it's still possible that something slipped by and the codegen references APIs which are not there anymore.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!

  • dotnet test for Roslyn shapshots + extended it to make sure the output can be compiled successfully.
  • SDK C# tests.
  • Would appreciate reviewers playing with creating a new project, adding bunch of types / tables and checking that all works as expected.

@RReverser RReverser added breaks library compatibility This change breaks the SpacetimeDB library interface api-break labels Jun 21, 2024
@RReverser RReverser requested review from bfops, kazimuth and lcodes June 21, 2024 11:22
@RReverser RReverser force-pushed the ingvar/unnest-csharp-apis branch from f6dd63a to 84aee09 Compare June 21, 2024 18:38
@bfops
Copy link
Collaborator

bfops commented Jun 21, 2024

@RReverser The description SGTM! Do you have recommendations for how to review the diff? How much of the added/removed code is just moved around vs genuinely changed? At first glance it seems a bit overwhelming.

@RReverser
Copy link
Contributor Author

Almost all of the changes are genuine move around / rename / splitting out into helpers from a functional point of view.

From functional pov the only few differences that I can think of are:

  • Change to the usings / lifting things to the top SpacetimeDB namespace, as the main goal of this PR.
  • Making Identity and Address into records,so now users have to instantiate them via new() - previously they were structs, and compiler was happy to silently leave them with default null contents which can lead to some bad errors down the line.
  • A drive by fix to iter_advance in RawBindings - it had a missing attribute that was commented out in the recent recv style ABI PR and my IDE was complaining about it. Users don't have to do anything about this one.
  • Hiding the error code field from the base STDB Exception class. I don't think we want to expose it for unknown errors - it's still part of the message - so this is technically a part of the "making things private" goal.

I could split this up into a few smaller PRs, but after the experience with doing that for the previous refactor to share BSATN with the client C# SDK, having lots of small isolated PRs drags the entire review out for months. And this is not even the last refactor I want to do, there is one final one for the Codegen package as well (that one is lower priority because it would not be a breaking user-visible change)...

If you'd prefer it, I can still give it a go, but personally I'd say instead of reviewing the entire diff, just look at 1) changes to the Roslyn generated code, you can find it in Codegen.Tests/snapshots and 2) changes to user code like sdk-test-cs/Lib.cs and such.

If those look reasonable and if you can init + publish + generate client code from a C# module without issues, it should be fine.

@bfops bfops self-assigned this Jun 24, 2024
@RReverser RReverser force-pushed the ingvar/unnest-csharp-apis branch from 84aee09 to ab4eb48 Compare June 24, 2024 21:18
@RReverser
Copy link
Contributor Author

I completely rewrote Git history of this PR, each individual commit is now formatted, passes tests, and does one thing at a time with clear description.

It's a silly amount of work for something that will be squashed anyway, but should make it much easier to review on commit-per-commit basis compared to the previous mess.

Copy link
Contributor

@kazimuth kazimuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh shoot, I forgot to post these. In general this looks good.

@kazimuth
Copy link
Contributor

kazimuth commented Jul 5, 2024

It turns out that review above had lost some comments so I've re-reviewed. I really don't have many comments though because this has been broken apart into commits so nicely, it generally makes sense and is clear.

@bfops
Copy link
Collaborator

bfops commented Jul 8, 2024

Moving this back to 0.11 since it's blocking some 0.11 PRs

@bfops bfops assigned kazimuth and unassigned bfops Jul 10, 2024
@bfops bfops removed their request for review July 10, 2024 19:34
Specify & fix root namespace for each project

Move BSATN internals into a subfolder

Rename RawBindings -> FFI

Move Filter into Internal namespace

Restore buffer FFI annotation

This was (accidentally?) commented out in the recv style ABI PR.

It worked sort of by accident regardless, but those are important for FFI + GC integration, so uncommenting back.

Move Module into Internal namespace

Add Builtins.cs with Unit, Identity, and Address

Moves builtin types to a new file under a top-level SpacetimeDB namespace and refactors Identity and Address to be more like those used in the client C# SDK, so that they could be shared between the two SDKs.

Verify that codegen result compiles in `dotnet test`

Accept raw index directly in IndexDef constructor

Accessing Internal.FFI namespace just to wrap-unwrap the struct doesn't make much sense, as in this context we should know what we're passing even without a newtype.

Extract exceptions to a separate file

Move Marshallers under corresponding classes

Remove old dependency on LINQ optimiser

NFC: simplify field extraction in Filter

Extract table utils into ITable interface

Extract reducer definition to a separate file

Move IndexType and Consume helper to internals

Refactor reducer/table/type registration

Just hiding internals and splitting up helper functions.

Remove `SpacetimeDB.Runtime` usings in samples
@kazimuth kazimuth force-pushed the ingvar/unnest-csharp-apis branch from 7293c0a to a039bdc Compare July 12, 2024 16:57
Copy link
Contributor

@kazimuth kazimuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@kazimuth kazimuth added this pull request to the merge queue Jul 12, 2024
Merged via the queue into master with commit 6f0f20b Jul 12, 2024
7 checks passed
RReverser added a commit to clockworklabs/com.clockworklabs.spacetimedbsdk that referenced this pull request Jul 25, 2024
## Description of Changes

- Roslyn codegen won't need to be added as a separate dependency
anymore.
- ByteArrayComparer, Identity, and Address will now live in
BSATN.Runtime so that they're reused between the client and module SDKs.

## API

 - [ ] This is an API breaking change to the SDK

*If the API is breaking, please state below what will break*


## Requires SpacetimeDB PRs

- [x] clockworklabs/SpacetimeDB#1440
- [ ] clockworklabs/SpacetimeDB#1455

---------

Co-authored-by: Zeke Foppa <github.com/bfops>
@bfops bfops added the api-break A PR that makes an API breaking change label Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-break A PR that makes an API breaking change breaks library compatibility This change breaks the SpacetimeDB library interface release-0.11
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split out APIs used by C# codegen into separate namespace
3 participants