Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove jsonwire.ParseFloat fast path #139

Merged
merged 1 commit into from
Jan 21, 2025

Conversation

huww98
Copy link
Contributor

@huww98 huww98 commented Jan 20, 2025

Calling strconv.ParseFloat() does not allocate now, so the necessity of fast path is reduced. This fast path actually slows down the process when it fails.

If overflows, return infinity as all other libs, don't give surprise to user. Since we now always treat overflow as error, this may not be very important, but still a breaking change.

Simple benchmark for your reference

func BenchmarkParseFloat_NoPoint(b *testing.B) {
	data := []byte("123456789")
	for range b.N {
		ParseFloat(data, 64)
	}
}

func BenchmarkParseFloat_WithPoint(b *testing.B) {
	data := []byte("1234567.89")
	for range b.N {
		ParseFloat(data, 64)
	}
}

func BenchmarkParseFloat_Overflow(b *testing.B) {
	data := []byte("1e1000")
	for range b.N {
		ParseFloat(data, 64)
	}
}

before

goos: darwin
goarch: arm64
pkg: github.com/go-json-experiment/json/internal/jsonwire
cpu: Apple M2 Pro
BenchmarkParseFloat_NoPoint-12          152670760                7.743 ns/op           0 B/op          0 allocs/op
BenchmarkParseFloat_WithPoint-12        37303958                31.28 ns/op            0 B/op          0 allocs/op
BenchmarkParseFloat_Overflow-12         16789585                69.25 ns/op           56 B/op          2 allocs/op
PASS
ok      github.com/go-json-experiment/json/internal/jsonwire    4.774s

after

goos: darwin
goarch: arm64
pkg: github.com/go-json-experiment/json/internal/jsonwire
cpu: Apple M2 Pro
BenchmarkParseFloat_NoPoint-12          48795962                24.33 ns/op            0 B/op          0 allocs/op
BenchmarkParseFloat_WithPoint-12        45615379                25.68 ns/op            0 B/op          0 allocs/op
BenchmarkParseFloat_Overflow-12         17682922                65.94 ns/op           56 B/op          2 allocs/op
PASS
ok      github.com/go-json-experiment/json/internal/jsonwire    4.707s

Float overflow behavior of other libs:

python3

Python 3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> json.loads("1e1000")
inf

Chrome

> JSON.parse("1e1000")
< Infinity

@dsnet
Copy link
Collaborator

dsnet commented Jan 20, 2025

Thanks.

//
// If the number overflows the finite representation of a float,
// then we return MaxFloat since any finite value will always be infinitely
// more accurate at representing another finite value than an infinite value.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment still stands as a valid argument. This is especially more relevant in v2 since we do provide opt-in support for actual infinity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, reverted most changes.

we do provide opt-in support for actual infinity

Sorry, but I don't understand this. I see the only option related to overflow is removed at #130 .

I don't want this behavior because (almost) all others just return infinity, including Go strconv. I think they have good reason for that. For example, if I get this json:

{
    "total": 1e1000
    "used": 1e500
}

I definitely don't want total - used to be 0. Infinite value is inaccurate, but it records the inaccuracy, e.g. "1e1000 - 1e500" should be NaN.

And for Token.Float(), we don't return error, so we even can't know we got overflow.

But we can defer this discussing to another PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out that example.

Unfortunately, I don't believe there are any good answers here.

With regard to Javascript and Python. While they are both consistent with each other in that they parse 1e1000 as Infinity without error, they are inconsistent with round trip serialization:

  • JavaScript: JSON.stringify(JSON.parse("1e1000")) returns null.
  • Python: json.dumps(json.loads("1e1000")) returns Infinity (which isn't even valid JSON).

Neither of them report an error and also inconsistent with each other for roundtrip serialization. In my opinion, this is the worst position to take since it leads to silent data loss.

In Go, we have decided that a float overflow is an error. This is arguably the most significant divergence from JS and Python, and whether we still return Inf and or MaxFloat is a minor detail.

  • The reason for using max float is to ensure the JSON kind does not abruptly change due to a round trip. Thus, unmarshaling 1e1000 and marshaling it should still resulting a JSON number (e.g., 1.7976931348623157e+308), not a JSON string (e.g., "Inf").
  • I can see how total - use becoming 0 could be a bug, but I should note that we are still reporting a Go error for the overflow. Thus, in order for this to be a logical bug, it would have to be after an error that was ignored.

fv, err := strconv.ParseFloat(string(b), bits)
if math.IsInf(fv, 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep this behavior.

Calling strconv.ParseFloat() does not allocate now, so the necessity of fast
path is reduced. This fast path actually slows down the process when it fails.
@huww98 huww98 force-pushed the cleanup-parse-float branch from 1b19926 to e9fc080 Compare January 20, 2025 05:42
@huww98 huww98 changed the title Cleanup jsonwire.ParseFloat to simply wrap strconv Remove jsonwire.ParseFloat fast path Jan 20, 2025
@dsnet
Copy link
Collaborator

dsnet commented Jan 21, 2025

Thank you for the cleanup!

We can continue to discuss whether Inf or MaxFloat64 is the right value to use in the event of overflow. Should we decide to change behavior, that can be a separate PR.

@dsnet dsnet merged commit 402e8b6 into go-json-experiment:master Jan 21, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants