|
| 1 | +--- |
| 2 | +sidebar_position: 1 |
| 3 | +--- |
| 4 | + |
| 5 | +# System Tests |
| 6 | + |
| 7 | +System tests provide a framework to write and execute black box tests against a running chain. This adds another level |
| 8 | +of confidence on top of unit, integration, and simulations tests, ensuring that business-critical scenarios |
| 9 | +(like double signing prevention) or scenarios that can't be tested otherwise (like a chain upgrade) are covered. |
| 10 | + |
| 11 | +## Vanilla Go for Flow Control |
| 12 | + |
| 13 | +System tests are vanilla Go tests that interact with the compiled chain binary. The `test runner` component starts a |
| 14 | +local testnet of 4 nodes (by default) and provides convenient helper methods for accessing the |
| 15 | +`system under test (SUT)`. |
| 16 | +A `CLI wrapper` makes it easy to access keys, submit transactions, or execute operations. Together, these components |
| 17 | +enable the replication and validation of complex business scenarios. |
| 18 | + |
| 19 | +Here's an example of a double signing test, where a new node is added with the same key as the first validator: |
| 20 | +[double signing test example](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/tests/systemtests/fraud_test.go) |
| 21 | + |
| 22 | +The [getting started tutorial](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/tests/systemtests/getting_started.md) |
| 23 | +contains a step-by-step guide to building and running your first system test. It covers setting chain state via genesis |
| 24 | +or |
| 25 | +transactions and validation via transaction response or queries. |
| 26 | + |
| 27 | +## Design Principles and Guidelines |
| 28 | + |
| 29 | +System tests are slower compared to unit or integration tests as they interact with a running chain. Therefore, certain |
| 30 | +principles can guide their usage: |
| 31 | + |
| 32 | +- **Perspective:** Tests should mimic a human interacting with the chain from the outside. Initial states can be set via |
| 33 | + genesis or transactions to support a test scenario. |
| 34 | +- **Roles:** The user can have multiple roles such as validator, delegator, granter, or group admin. |
| 35 | +- **Focus:** Tests should concentrate on happy paths or business-critical workflows. Unit and integration tests are |
| 36 | + better suited for more fine-grained testing. |
| 37 | +- **Workflows:** Test workflows and scenarios, not individual units. Given the high setup costs, it is reasonable to |
| 38 | + combine multiple steps and assertions in a single test method. |
| 39 | +- **Genesis Mods:** Genesis modifications can incur additional time costs for resetting dirty states. Reuse existing |
| 40 | + accounts (node0..n) whenever possible. |
| 41 | +- **Framework:** Continuously improve the framework for better readability and reusability. |
| 42 | + |
| 43 | +## Errors and Debugging |
| 44 | + |
| 45 | +All output is logged to `systemtests/testnet/node{0..n}.out`. Usually, `node0.out` is very noisy as it receives the CLI |
| 46 | +connections. Prefer any other node's log to find stack traces or error messages. |
| 47 | + |
| 48 | +Using system tests for state setup during debugging has become very handy: |
| 49 | + |
| 50 | +- Start the test with one node only and verbose output: |
| 51 | + |
| 52 | + ```sh |
| 53 | + go test -v -tags=system_test ./ --run TestAccountCreation --verbose --nodes-count=1 |
| 54 | + ``` |
| 55 | + |
| 56 | +- Copy the CLI command for the transaction and modify the test to stop before the command |
| 57 | +- Start the node with `--home=<project-home>/tests/systemtests/testnet/node0/<binary-name>/` in debug mode |
| 58 | +- Execute CLI command from shell and enter breakpoints |
0 commit comments