Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 6ab7d28

Browse files
nenadmilosevic95josef-widdersergio-menacason
authoredFeb 15, 2023
Nenad/150 abci spec fixes (celestiaorg#210)
* Text fixed -"Methods" section fixed to consider locked value. * File with extra scenarios added * Text fixed to not mention locked value * Grammar fixed * Update spec/abci/abci++_extra.md * Update spec/abci/abci++_extra.md * Update spec/abci/abci++_extra.md * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md Co-authored-by: Sergio Mena <[email protected]> * Update spec/abci/abci++_extra.md * Update spec/abci/abci++_methods.md * file with scenarios integrated * renaming Tendermint->CometBFT done * Apply suggestions from code review Co-authored-by: Daniel <[email protected]> * Fixed PR's comments: -fixed daniel's and lasaro's comments -renamed abci++_tmint_expected_behavior.md to abci++_comet_expected_behavior.md --------- Co-authored-by: Josef Widder <[email protected]> Co-authored-by: Sergio Mena <[email protected]> Co-authored-by: Daniel <[email protected]>
1 parent 36827cb commit 6ab7d28

5 files changed

+177
-11
lines changed
 

‎spec/abci/README.md

+4-2
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,10 @@ This specification is split as follows:
3232
- [Requirements for the Application](./abci++_app_requirements.md) - formal requirements
3333
on the Application's logic to ensure CometBFT properties such as liveness. These requirements define what
3434
CometBFT expects from the Application; second part on managing ABCI application state and related topics.
35-
- [CometBFT's expected behavior](./abci++_tmint_expected_behavior.md) - specification of
35+
- [CometBFT's expected behavior](./abci++_comet_expected_behavior.md) - specification of
3636
how the different ABCI++ methods may be called by CometBFT. This explains what the Application
3737
is to expect from CometBFT.
38+
- [Example scenarios](./abci++_example_scenarios.md) - specific scenarios showing why the Application needs to account
39+
for any CometBFT's behaviour prescribed by the specification.
3840
- [Client and Server](abci++_client_server.md) - for those looking to implement their
39-
own ABCI application servers
41+
own ABCI application servers.

‎spec/abci/abci++_basic_concepts.md

+5-4
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,8 @@ call sequences of these methods.
7878
proposer to perform application-dependent work in a block before proposing it.
7979
This enables, for instance, batch optimizations to a block, which has been empirically
8080
demonstrated to be a key component for improved performance. Method `PrepareProposal` is called
81-
every time CometBFT is about to broadcast a Proposal message, but no previous proposal has
82-
been locked at consensus level. CometBFT gathers outstanding transactions from the
81+
every time CometBFT is about to broadcast a Proposal message and _validValue_ is `nil`.
82+
CometBFT gathers outstanding transactions from the
8383
mempool, generates a block header, and uses them to create a block to propose. Then, it calls
8484
`RequestPrepareProposal` with the newly created proposal, called *raw proposal*. The Application
8585
can make changes to the raw proposal, such as modifying the set of transactions or the order
@@ -90,8 +90,9 @@ call sequences of these methods.
9090
- [**ProcessProposal:**](./abci++_methods.md#processproposal) It allows a validator to
9191
perform application-dependent work in a proposed block. This enables features such as immediate
9292
block execution, and allows the Application to reject invalid blocks.
93-
CometBFT calls it when it receives a proposal and the consensus algorithm has not locked on a
94-
value. The Application cannot modify the proposal at this point but can reject it if it is
93+
94+
CometBFT calls it when it receives a proposal and _validValue_ is `nil`.
95+
The Application cannot modify the proposal at this point but can reject it if it is
9596
invalid. If that is the case, the consensus algorithm will prevote `nil` on the proposal, which has
9697
strong liveness implications for CometBFT. As a general rule, the Application
9798
SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of

‎spec/abci/abci++_tmint_expected_behavior.md ‎spec/abci/abci++_comet_expected_behavior.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ consensus-exec = (inf)consensus-height
6161
consensus-height = *consensus-round decide commit
6262
consensus-round = proposer / non-proposer
6363
64-
proposer = prepare-proposal process-proposal
64+
proposer = [prepare-proposal process-proposal]
6565
non-proposer = [process-proposal]
6666
decide = begin-block *deliver-txs end-block
6767
@@ -161,8 +161,10 @@ Let us now examine the grammar line by line, providing further details.
161161
>consensus-round = proposer / non-proposer
162162
>```
163163
164-
* For every round, if the local process is the proposer of the current round, CometBFT starts by
165-
calling `PrepareProposal`, followed by `ProcessProposal`.
164+
* For every round, if the local process is the proposer of the current round, CometBFT calls `PrepareProposal`, followed by `ProcessProposal`.
165+
These two always come together because they reflect the same proposal that the process
166+
also delivers to itself.
167+
166168
<!--
167169
168170
Then, optionally, the Application is
@@ -172,7 +174,7 @@ Let us now examine the grammar line by line, providing further details.
172174
-->
173175
174176
>```abnf
175-
>proposer = prepare-proposal process-proposal
177+
>proposer = [prepare-proposal process-proposal]
176178
>```
177179
178180
* Also for every round, if the local process is _not_ the proposer of the current round, CometBFT

‎spec/abci/abci++_example_scenarios.md

+161
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
order: 6
3+
title: ABCI++ extra
4+
---
5+
# Introduction
6+
7+
In the section [CometBFT's expected behaviour](./abci++_comet_expected_behavior.md#valid-method-call-sequences),
8+
we presented the most common behaviour, usually referred to as the good case.
9+
However, the grammar specified in the same section is more general and covers more scenarios
10+
that an Application designer needs to account for.
11+
12+
In this section, we give more information about these possible scenarios. We focus on methods
13+
introduced by ABCI++: `PrepareProposal` and `ProcessProposal`. Specifically, we concentrate
14+
on the part of the grammar presented below.
15+
16+
```abnf
17+
consensus-height = *consensus-round decide commit
18+
consensus-round = proposer / non-proposer
19+
20+
proposer = [prepare-proposal process-proposal]
21+
non-proposer = [process-proposal]
22+
```
23+
24+
We can see from the grammar that we can have several rounds before deciding a block. The reasons
25+
why one round may not be enough are:
26+
* network asynchrony, and
27+
* a Byzantine process being the proposer.
28+
29+
If we assume that the consensus algorithm decides on block $X$ in round $r$, in the rounds
30+
$r' <= r$, CometBFT can exhibit any of the following behaviours:
31+
32+
1. Call `PrepareProposal` and/or `ProcessProposal` for block $X$.
33+
1. Call `PrepareProposal` and/or `ProcessProposal` for block $Y \neq X$.
34+
1. Does not call `PrepareProposal` and/or `ProcessProposal`.
35+
36+
In the rounds when it is the proposer, CometBFT's `PrepareProposal` call is always followed by the
37+
`ProcessProposal` call. The reason is that the process always delivers the proposal to itself, which
38+
triggers the `ProcessProposal` call.
39+
40+
As the number of rounds the consensus algorithm needs to decide in a given run is a priori unknown, the
41+
application needs to account for any number of rounds, where each round can exhibit any of these three
42+
behaviours. Recall that the application is unaware of the internals of consensus and thus of the rounds.
43+
44+
# Possible scenarios
45+
The unknown number of rounds we can have when following the consensus algorithm yields a vast number of
46+
scenarios we can expect. Listing them all is unfeasible. However, here we give several of them and draw the
47+
main conclusions. Specifically, we will show that before block $X$ is decided:
48+
49+
1. On a correct node, `PrepareProposal` may be called multiple times and for different blocks ([**Scenario 1**](#scenario-1)).
50+
1. On a correct node, `ProcessProposal` may be called multiple times and for different blocks ([**Scenario 2**](#scenario-2)).
51+
1. On a correct node, `PrepareProposal` and `ProcessProposal` for block $X$ may not be called ([**Scenario 3**](#scenario-3)).
52+
1. On a correct node, `PrepareProposal` and `ProcessProposal` may not be called at all ([**Scenario 4**](#scenario-4)).
53+
54+
55+
## Basic information
56+
57+
Each scenario is presented from the perspective of a process $p$. More precisely, we show what happens in
58+
each round's $step$ of the [Tendermint consensus algorithm](https://arxiv.org/pdf/1807.04938.pdf). While in
59+
practice the consensus algorithm works with respect to voting power of the validators, in this document
60+
we refer to number of processes (e.g., $n$, $f+1$, $2f+1$) for simplicity. The legend is below:
61+
62+
### Round X:
63+
64+
1. **Propose:** Describes what happens while $step_p = propose$.
65+
1. **Prevote:** Describes what happens while $step_p = prevote$.
66+
1. **Precommit:** Describes what happens while $step_p = precommit$.
67+
68+
## Scenario 1
69+
70+
$p$ calls `ProcessProposal` many times with different values.
71+
72+
### Round 0:
73+
74+
1. **Propose:** The proposer of this round is a Byzantine process, and it chooses not to send the proposal
75+
message. Therefore, $p$'s $timeoutPropose$ expires, it sends $Prevote$ for $nil$, and it does not call
76+
`ProcessProposal`. All correct processes do the same.
77+
1. **Prevote:** $p$ eventually receives $2f+1$ $Prevote$ messages for $nil$ and starts $timeoutPrevote$.
78+
When $timeoutPrevote$ expires it sends $Precommit$ for $nil$.
79+
1. **Precommit:** $p$ eventually receives $2f+1$ $Precommit$ messages for $nil$ and starts $timeoutPrecommit$.
80+
When it expires, it moves to the next round.
81+
82+
### Round 1:
83+
84+
1. **Propose:** A correct process is the proposer in this round. Its $validValue$ is $nil$, and it is free
85+
to generate and propose a new block $Y$. Process $p$ receives this proposal in time, calls `ProcessProposal`
86+
for block $Y$, and broadcasts a $Prevote$ message for it.
87+
1. **Prevote:** Due to network asynchrony less than $2f+1$ processes send $Prevote$ for this block.
88+
Therefore, $p$ does not update $validValue$ in this round.
89+
1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
90+
block and send $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
91+
92+
### Round 2:
93+
94+
1. **Propose:** Same as in [**Round 1**](#round-1), just another correct process is the proposer, and it
95+
proposes another value $Z$. Process $p$ receives the proposal on time, calls `ProcessProposal` for new block
96+
$Z$, and broadcasts a $Prevote$ message for it.
97+
1. **Prevote:** Same as in [**Round 1**](#round-1).
98+
1. **Precommit:** Same as in [**Round 1**](#round-1).
99+
100+
101+
Rounds like these can continue until we have a round in which process $p$ updates its $validValue$ or until
102+
we reach round $r$ where process $p$ decides on a block. After that, it will not call `ProcessProposal`
103+
anymore for this height.
104+
105+
## Scenario 2
106+
107+
$p$ calls `PrepareProposal` many times with different values.
108+
109+
### Round 0:
110+
111+
1. **Propose:** Process $p$ is the proposer in this round. Its $validValue$ is $nil$, and it is free to
112+
generate and propose new block $Y$. Before proposing, it calls `PrepareProposal` for $Y$. After that, it
113+
broadcasts the proposal, delivers it to itself, calls `ProcessProposal` and broadcasts $Prevote$ for it.
114+
1. **Prevote:** Due to network asynchrony less than $2f+1$ processes receive the proposal on time and send
115+
$Prevote$ for it. Therefore, $p$ does not update $validValue$ in this round.
116+
1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this
117+
block and send non-$nil$ $Precommit$ message. As a consequence, $p$ does not decide on $Y$.
118+
119+
After this round, we can have multiple rounds like those in [Scenario 1](#scenario-1). The important thing
120+
is that process $p$ should not update its $validValue$. Consequently, when process $p$ reaches the round
121+
when it is again the proposer, it will ask the mempool for the new block again, and the mempool may return a
122+
different block $Z$, and we can have the same round as [Round 0](#round-0-1) just for a different block. As
123+
a result, process $p$ calls `PrepareProposal` again but for a different value. When it reaches round $r$
124+
some process will propose block $X$ and if $p$ receives $2f+1$ $Precommit$ messages, it will decide on this
125+
value.
126+
127+
128+
## Scenario 3
129+
130+
$p$ calls `PrepareProposal` and `ProcessProposal` for many values, but decides on a value for which it did
131+
not call `PrepareProposal` or `ProcessProposal`.
132+
133+
In this scenario, in all rounds before $r$ we can have any round presented in [Scenario 1](#scenario-1) or
134+
[Scenario 2](#scenario-2). What is important is that:
135+
- no proposer proposed block $X$ or if it did, process $p$, due to asynchrony, did not receive it in time,
136+
so it did not call `ProcessProposal`, and
137+
138+
- if $p$ was the proposer it proposed some other value $\neq X$.
139+
140+
### Round $r$:
141+
142+
1. **Propose:** A correct process is the proposer in this round, and it proposes block $X$.
143+
Due to asynchrony, the proposal message arrives to process $p$ after its $timeoutPropose$
144+
expires and it sends $Prevote$ for $nil$. Consequently, process $p$ does not call
145+
`ProcessProposal` for block $X$. However, the same proposal arrives at other processes
146+
before their $timeoutPropose$ expires, and they send $Prevote$ for this proposal.
147+
1. **Prevote:** Process $p$ receives $2f+1$ $Prevote$ messages for proposal $X$, updates correspondingly its
148+
$validValue$ and $lockedValue$ and sends $Precommit$ message. All correct processes do the same.
149+
1. **Precommit:** Finally, process $p$ receives $2f+1$ $Precommit$ messages, and decides on block $X$.
150+
151+
152+
153+
## Scenario 4
154+
155+
[Scenario 3](#scenario-3) can be translated into a scenario where $p$ does not call `PrepareProposal` and
156+
`ProcessProposal` at all. For this, it is necessary that process $p$ is not the proposer in any of the
157+
rounds $0 <= r' <= r$ and that due to network asynchrony or Byzantine proposer, it does not receive the
158+
proposal before $timeoutPropose$ expires. As a result, it will enter round $r$ without calling
159+
`PrepareProposal` and `ProcessProposal` before it, and as shown in Round $r$ of [Scenario 3](#scenario-3) it
160+
will decide in this round. Again without calling any of these two calls.
161+

‎spec/abci/abci++_methods.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -550,7 +550,7 @@ When a node _p_ enters consensus round _r_, height _h_, in which _q_ is the prop
550550
4. Upon reception of Proposal message, along with all the block parts, for round _r_, height _h_
551551
from _q_, _p_ follows the validators' algorithm to check whether it should prevote for the
552552
proposed block, or `nil`.
553-
5. If the validators' consensus algorithm indicates _p_ should prevote for the proposed block:
553+
5. If the validators' consensus algorithm indicates _p_ should prevote non-nil:
554554
1. CometBFT calls `RequestProcessProposal` with the block. The call is synchronous.
555555
2. The Application checks/processes the proposed block, which is read-only, and returns
556556
`ACCEPT` or `REJECT` in the `ResponseProcessProposal.status` field.

0 commit comments

Comments
 (0)
Please sign in to comment.