recent_history absorbing accumulation result r derived from ${\bf C}$ prematurely

[arjanz]

When importing 1_004 of assurances we notice an inconsistency with the assurances 'current' statistics:

JAM Duna Data

[{'assurances': 1, 'blocks': 1, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 3}, {'assurances': 0, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 0, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 0, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 0, 'blocks': 3, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 6}, {'assurances': 0, 'blocks': 1, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 3}]

[{PyJAMaz}] data

[{'assurances': 1, 'blocks': 1, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 3}, {'assurances': 1, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 1, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 1, 'blocks': 0, 'guarantees': 1, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 0}, {'assurances': 1, 'blocks': 3, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 6}, {'assurances': 1, 'blocks': 1, 'guarantees': 0, 'pre_images': 0, 'pre_images_size': 0, 'tickets': 3}]

We expect the current statistics for assurances to be updated for the validator index in each assurance extrinsic, according to GP 13.4, but it seems there is only one update for the author of the block.

[sourabhniyogi]

Thank you, our fault -- See here for all 6 assurances going to "1" and then here for all going to "2" and then here for the epoch change. Let us know if this appears correct (or not!) --

[arjanz]

Confirmed, the assurances statistics in 1_004 matches ours now.

However, our state trie root (0x15ce8da3...9ab911ac) still doesn't match the one in the trace file, we identified the following difference:

{'values_changed': {"root['recent_history']['recent_history'][4]['mmr']['peaks'][0]": {'pyjamaz': '0x0000000000000000000000000000000000000000000000000000000000000000', 'trace': '0x85c18c1d0fd636dd782a34e81ac2a24ba2bdd6c28d46f3646b260eadd424c99d'}}}

Not 100% sure, but could this be same H_0 issue we encountered before?

[sourabhniyogi]

Will shortly provide a derivation for C3 in [assurances/state_transitions/1_004.json](https://github.com/jam-duna/jamtestnet/blob/f3c9fdb14cf32cdcad36a9beacf6f1f94fe6703c/data/assurances/state_transitions/1_004.json#L225) specifically the `0x85c18c1d0fd636dd782a34e81ac2a24ba2bdd6c28d46f3646b260eadd424c99d` value in [assurances/state_transitions/1_004.json](https://github.com/jam-duna/jamtestnet/blob/f3c9fdb14cf32cdcad36a9beacf6f1f94fe6703c/data/assurances/state_snapshots/1_004.json#L244C22-L244C88) which I think you compute as `0x0000000000000000000000000000000000000000000000000000000000000000` instead?

[sourabhniyogi]

Ok after realizing that 0x85c18c1d0fd636dd782a34e81ac2a24ba2bdd6c28d46f3646b260eadd424c99d is the accumulation result $r$ from ${\bf C}$ (as described in GP Sec 7 here), we see that we prematurely used $r$ derived from ${\bf C}$ immediately after accumulation but GP Eq 4.7 shows that (like the state root inclusion ) we have to wait until the next state transition to use that value of $r$!

Its not that the accumulate root $r$ (derived from ${\bf C}$) is0x0000..0000 -- instead its that after the very first accumulate in this assurances/state_transitions/1_004.json resulting in a non-0x0000...0000 $r$ will affect the NEXT state C3/Beta -- which will absorb this new non-0x0000..0000 value of $r$ (again, derived from ${\bf C}$ from the accumulation in the PREVIOUS state transition.

Thank you for pointing out the problem, we will provide a derivation for ${\bf C}$ and thus $r$ to cover this very tricky two-state footwork in the hopes that we can fully get on the same page.

Stay tuned!

[sourabhniyogi]

We now have 0x0000...0000 in C3 here and believe it is necessary to add a new accumulation root parameter $r$ to StateTransition to concisely summarize ${\bf C}$

We are improving our fib/trib/megatron services to output actual 32-byte outputs so that this $r$ value actually changes.

But in the meantime you can check this!