Round trip floats

[dtolnay]

Ideally this test would pass.

extern crate serde_json;

#[macro_use]
extern crate quickcheck;

quickcheck! {
    fn floats(n: f64) -> bool {
        let j = serde_json::to_string(&n).unwrap();
        serde_json::from_str::<f64>(&j).unwrap() == n
    }
}

On the printing side grisu2 guarantees that the f64 closest to the string representation is identical to the original input, so the inaccuracy is somewhere on the parsing side.

[Mark-Simulacrum]

I'm not sure the conclusion that the issue is with parsing is entirely correct (although there do seem to be problems in that area as well), since this test fails in the dtoa crate: test_write(17.922589369123898, "17.922589369123898");.

    thread 'test_f64' panicked at 'assertion failed: `(left == right)` (left: `"17.922589369123899"`, right: `"17.922589369123898"`)', tests/test.rs:36

[dtolnay]

@Mark-Simulacrum the f64 that is closest to 17.922589369123899 is identical to the f64 that is closest to 17.922589369123898, so dtoa (grisu2) is correct here.

assert_eq!(17.922589369123898, 17.922589369123899);

As I wrote:

grisu2 guarantees that the f64 closest to the string representation is identical to the original input

[dtolnay]

In the extreme, you probably wouldn't expect this test to pass:

test_write(17.92258936912389800000000000000001f64, "17.92258936912389800000000000000001");

[Mark-Simulacrum]

Hmm, I wonder how Javascript's JSON.parse/JSON.stringify deal with this. JSON.stringify(JSON.parse('{ "test": 17.922589369123898 }')) returns the expected (by me, at least) 17.922589369123898 despite assert_eq!(17.922589369123898, 17.922589369123899); being true in JS. Well, I somewhat expect that decode(encode(any f64)) == the same f64; but I guess that isn't the case at least for serde-json and json-rust: perhaps this isn't a good requirement.

[dtolnay]

@Mark-Simulacrum using f64 you cannot have both of these return "the expected":

JSON.stringify(JSON.parse('{ "test": 17.922589369123898 }'))
JSON.stringify(JSON.parse('{ "test": 17.922589369123899 }'))

because those are the same f64.

[dtolnay]

@Mark-Simulacrum

Well, I somewhat expect that decode(encode(any f64)) == the same f64; but I guess that isn't the case at least for serde-json and json-rust: perhaps this isn't a good requirement.

Actually that is a good requirement. That is what this issue is asking for. That is what grisu2 guarantees should work if your parsing is good. This issue is for making our parsing good.

The JS code you gave is a different (not good) requirement of encode(decode(str)) == str.

[Mark-Simulacrum]

Okay. Well, I'd be glad to help out, but I'm not sure where to start--or if you'd even like help :).

[dtolnay]

We would love help as always! The number-parsing code starts here: de.rs#L224. Currently if it sees something like 17.922589369123899 then it computes 17922589369123899u64 as f64 / 10e14 which is pretty good but we need a better algorithm. A place to start would be looking at how RadidJSON handles this case.

[Mark-Simulacrum]

I've started looking into it, but haven't had any luck yet. I don't want to block anyone else from working on it, especially since I probably won't have much luck with it--I'm not very proficient at debugging parsing/serializing code.

My initial findings seem to point to u64 as f64 not being a safe cast (we lose ~3 digits of precision there). I'm not sure how to work around that as of yet though.

[dtolnay]

Looks like RapidJSON does the same thing we do. Here is the cast and here is the division.

[dtolnay]

Let's follow up with a value -> string -> value roundtripping quickcheck once this is implemented. #138 has most of the code.

[mthiesen]

This is what V8 uses for parsing strings to floats. Supposedly this guarantees correct round tripping of IEEE floats.

[donbright]

can we have the option to parse into some kind of decimal type? (as in, base 10 floating point)

[demurgos]

Hi,
What is the current state of support for floats?

I saw that ryu is a dependency but floats are still not round-tripping when using the examples from the README. I cannot find how to enable precise floats.

Edit: Following more tests, I may be wrong and round-trip works. It would still be nice to have an update for this issue.

[dtolnay]

The ryu crate is for printing floats to string, so it wouldn't help with the parsing side which is where loss of accuracy happens.

I am closing this issue and will open a new issue to hopefully make it clear that the remaining work is about parsing.