Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better thought out caching mechanism that actually works #802

Merged
merged 1 commit into from
Mar 9, 2025

Conversation

brandur
Copy link
Contributor

@brandur brandur commented Mar 9, 2025

This one follows up #794 again. That change included a template cache,
with the idea being that we could reuse a rendered sqlc query when all
input values were expected to be stable. For example, when replacing
something like a schema name, we'd presumably always be replacing the
template value with the same schema over and over again, so it'd be
better to save on the work.

But that caching system hadn't adequately been thought through, and
wouldn't really work. I realized when I was putting #798 (explicit
schemas) together that if you injected two schema values from two
different clients then you'd end up using the same cache value, which is
wrong.

Here, we tool out the cache layer a little more so that it considers
input values and named args, which all must be consistent for a cached
value to be returned. We also add tests to make sure of it.

Building a cache key unfortunately has to rely on concatenating some
strings together and presorting map keys for stability, but even with
the extra work involved, a cache hit still comes out to be quite a bit
faster than a miss (~2.5x faster), so it seems worth doing:

$ go test ./rivershared/sqlctemplate -bench=.
goos: darwin
goarch: arm64
pkg: github.com/riverqueue/river/rivershared/sqlctemplate
cpu: Apple M4
BenchmarkReplacer/WithCache-10           1626988               735.7 ns/op
BenchmarkReplacer/WithoutCache-10         695517              1710 ns/op
PASS
ok      github.com/riverqueue/river/rivershared/sqlctemplate    3.419s

@brandur brandur force-pushed the brandur-proper-cache branch from 3617481 to 934b142 Compare March 9, 2025 05:25
@brandur brandur requested a review from bgentry March 9, 2025 05:28
@brandur brandur force-pushed the brandur-proper-cache branch from 934b142 to 0f115df Compare March 9, 2025 05:39
This one follows up #794 again. That change included a template cache,
with the idea being that we could reuse a rendered sqlc query when all
input values were expected to be stable. For example, when replacing
something like a schema name, we'd presumably always be replacing the
template value with the same schema over and over again, so it'd be
better to save on the work.

But that caching system hadn't adequately been thought through, and
wouldn't really work. I realized when I was putting #798 (explicit
schemas) together that if you injected two schema values from two
different clients then you'd end up using the same cache value, which is
wrong.

Here, we tool out the cache layer a little more so that it considers
input values and named args, which all must be consistent for a cached
value to be returned. We also add tests to make sure of it.

Building a cache key unfortunately has to rely on concatenating some
strings together and presorting map keys for stability, but even with
the extra work involved, a cache hit still comes out to be quite a bit
faster than a miss (~2.5x faster), so it seems worth doing:

    $ go test ./rivershared/sqlctemplate -bench=.
    goos: darwin
    goarch: arm64
    pkg: github.com/riverqueue/river/rivershared/sqlctemplate
    cpu: Apple M4
    BenchmarkReplacer/WithCache-10           1626988               735.7 ns/op
    BenchmarkReplacer/WithoutCache-10         695517              1710 ns/op
    PASS
    ok      github.com/riverqueue/river/rivershared/sqlctemplate    3.419s
@brandur brandur force-pushed the brandur-proper-cache branch from 0f115df to 90e6499 Compare March 9, 2025 05:43
Copy link
Contributor

@bgentry bgentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid, good catch on this.

@brandur
Copy link
Contributor Author

brandur commented Mar 9, 2025

ty man.

@brandur brandur merged commit 7e17903 into master Mar 9, 2025
10 checks passed
@brandur brandur deleted the brandur-proper-cache branch March 9, 2025 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants