Skip to content

Commit

Permalink
📝 Update docs structure, design
Browse files Browse the repository at this point in the history
  • Loading branch information
lukacan committed Aug 20, 2024
1 parent 41066bd commit c3c8a95
Show file tree
Hide file tree
Showing 32 changed files with 377 additions and 230 deletions.
10 changes: 10 additions & 0 deletions documentation/docs/fuzzing/execute/debug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Debug

To debug your program with values from a crash file:

```bash
# fuzzer will run the <TARGET_NAME> with the specified <CRASH_FILE_PATH>
trident fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH>
# for example:
trident fuzz run-debug fuzz_0 trident-tests/fuzz_tests/fuzzing/fuzz_0/cr1.fuzz
```
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Run and Debug
# Run

## Run
Once you have finished the implementation of the Fuzz Test, you can run the Test as follows:
To execute the desired fuzz test, run:

```bash
# Replace <TARGET_NAME> with the name of particular
Expand Down Expand Up @@ -54,31 +53,17 @@ allow_duplicate_txs = false
fuzzing_with_stats = true
```

Or you can pass any parameter via [environment variables](https://github.com/rust-fuzz/honggfuzz-rs#environment-variables).
??? note

A list of hongfuzz parameters can be found in honggfuzz [usage documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md#cmdline---help). The parameters passed via **environment variables** have **higher priority**. For example:
Or you can pass any parameter via [environment variables](https://github.com/rust-fuzz/honggfuzz-rs#environment-variables).

```bash
# Time-out: 10 secs
# Number of concurrent fuzzing threads: 1
# Number of fuzzing iterations: 10000
# Display Solana logs in the terminal
HFUZZ_RUN_ARGS="-t 10 -n 1 -N 10000 -Q" trident fuzz run <TARGET_NAME>
```

### Fuzzing statistics
Sometimes, it's useful to know how often a particular instruction has been invoked and how many times it has succeeded or failed. To display these statistics when fuzzing is finished or interrupted, set the `fuzzing_with_stats` option to `true` in the `[fuzz]` section of the Trident.toml configuration file. Please note that this option is disabled by default because it impacts performance.

The statistics show the total number of invocations for each instruction, which is the sum of successful and failed invocations. Successful invocations are those that return an `Ok()` result. Failed invocations are those that return an `Err()` result. Additionally, the statistics also show as `Check Failed` the number of successful invocations that did not pass the user-defined invariants check. Note that unhandled panics are currently logged only as crashes and are not displayed in the fuzzing statistics table.
A list of hongfuzz parameters can be found in honggfuzz [usage documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md#cmdline---help). The parameters passed via **environment variables** have **higher priority**. For example:

Keep in mind that the number of fuzz iterations does not directly correspond to the total number of invocations. In one fuzz iteration, the fuzzer might be unable to deserialize fuzz data into instructions, causing the entire iteration to be skipped.
```bash
# Time-out: 10 secs
# Number of concurrent fuzzing threads: 1
# Number of fuzzing iterations: 10000
# Display Solana logs in the terminal
HFUZZ_RUN_ARGS="-t 10 -n 1 -N 10000 -Q" trident fuzz run <TARGET_NAME>
```

## Debug
To debug your program with values from a crash file:

```bash
# fuzzer will run the <TARGET_NAME> with the specified <CRASH_FILE_PATH>
trident fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH>
# for example:
trident fuzz run-debug fuzz_0 trident-tests/fuzz_tests/fuzzing/fuzz_0/cr1.fuzz
```
25 changes: 25 additions & 0 deletions documentation/docs/fuzzing/execute/statistics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Fuzzing statistics

!!! tip

In order to show statistics of fuzzing session (statistics are displayed after the fuzzing session ended), set `fuzzing_with_stats` within the Trident.toml to `true`.

```toml
[fuzz]
# ...
fuzzing_with_stats = true
# ...
```


Currently exported statistics from the Fuzzing Session

#### Simple

- Number of invocations of each instruction during the fuzzing session.
- Number of successful invocations of each instruction during the fuzzing session.
- Number of failed invariants checks for each instruction during the fuzzing session.

??? note

Keep in mind that the number of fuzz iterations does not directly correspond to the total number of invocations. In one fuzz iteration, the fuzzer might be unable to deserialize fuzz data into instructions, causing the entire iteration to be skipped.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Fuzzing Examples
# Trident Examples

### Hello World! Example
- [hello_world](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/hello_world)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Fuzz test-only
# Initialize Fuzz Test

If you are interested only in generating templates for fuzz tests run
For initialization of workspace for fuzz tests, call:
```bash
trident init fuzz
```
Expand All @@ -21,3 +21,10 @@ project-root
├── Trident.toml
└── ...
```

## Add new Fuzz Test

If you wish to generate template for a new fuzz test, call:
```bash
trident fuzz add
```
30 changes: 30 additions & 0 deletions documentation/docs/fuzzing/first-steps/writing-fuzz-test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Writing Fuzz Test

!!! tip

At the current development stage, there are some manual steps required to start the Fuzzing Session. In principle:

**Prerequisites:**

- Add dependencies specific to your program to `trident-tests/fuzz_tests/Cargo.toml` (such as anchor-spl etc.).
- Add necessary `use` statements into `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/accounts_snapshots.rs` to import missing types.

**Writing Fuzz Tests**

1. Specify pseudo-random accounts to re-use [Accounts to re-use](./accounts.md).
2. Specify instruction data [Instruction Data](./instruction-data.md).
3. Specify instruction accounts [Instruction Accounts](./instruction-accounts.md).

**Run and Debug**

1. Execute desired fuzz test [Run](../execute/run.md)
2. See the found crash with more details [Debug](../execute/debug.md)

!!! note

For better fuzzing results and experience you can also manually adjust the following:

1. Define Invariants checks [Invariants Checks](../writing-fuzz-test-extra/invariants-checks.md).
2. Specify instruction sequences[Instruction sequences](../writing-fuzz-test-extra/instruction-sequences.md).
3. Specify custom data types[Custom Data types](../writing-fuzz-test-extra/custom-data-types.md).
4. Well structured data[Arbitrary](../writing-fuzz-test-extra/arbitrary.md).
22 changes: 0 additions & 22 deletions documentation/docs/fuzzing/fuzzing-introduction.md

This file was deleted.

6 changes: 0 additions & 6 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p0.md

This file was deleted.

21 changes: 0 additions & 21 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p1.md

This file was deleted.

20 changes: 0 additions & 20 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p2.md

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# AccountsSnapshots

TBD
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# How to use Arbitrary crate
# Structured Data

The [Arbitrary](https://docs.rs/arbitrary/latest/arbitrary/) crate in Rust is used for generating well-typed, structured instances of data from raw byte buffers, making it useful for fuzzing by producing random but structured data for tests.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# How to use Custom Data Types
# Custom Data Types

If you use Custom Types as Instruction data arguments, you may encounter a problem that the Custom Type does not implement

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Customize instructions generation
# Instruction sequences

It is possible to customize how the instructions are generated and which instructions will be executed at the beginning (`pre_ixs`), in the middle (`ixs`) and at the end (`post_ixs`) of each fuzz iteration. This can be useful for example if your program needs an initialization or you want to fuzz some specific program state.

- Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/test_fuzz.rs` and implement the corresponding optional method of the `FuzzDataBuilder<FuzzInstruction>` trait. For example, in order to always call the `initialize` instruction, modify the trait's implementation as follows:
!!! tip

- Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/test_fuzz.rs` and implement the corresponding optional method of the `FuzzDataBuilder<FuzzInstruction>` trait. For example, in order to always call the `initialize` instruction, modify the trait's implementation.

```rust
impl FuzzDataBuilder<FuzzInstruction> for MyFuzzData {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Define invariants checks
# Invariants checks

After each successful instruction execution, the `check()` method is called to check the account data invariants.

For each instruction, you can compare the account data **before** and **after** the instruction execution such as:
!!! tip

For each instruction, you can compare the account data **before** and **after** the instruction execution.

```rust
fn check(
Expand Down
31 changes: 31 additions & 0 deletions documentation/docs/fuzzing/writing-fuzz-test/accounts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Accounts to reuse

{{ config.site_name }} helps you to generate only a limited amount of pseudo-random accounts and reuse them in the instructions.

??? note

**Details:**

Always generating only random accounts would **in most cases lead to a situation where the fuzzer would be stuck because the accounts would be almost every time rejected by your Anchor program**. Therefore it is necessary to specify, what accounts should be used and also limit the number of newly created accounts to reduce the space complexity.


!!! tip

Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/fuzz_instructions.rs` file and complete the pre-generated `FuzzAccounts` structure. It contains all accounts used in your program. You have to determine if the account is a:

- Signer
- PDA
- Token Account
- Program account

Then use the corresponding AccountsStorage.


```rust
pub struct FuzzAccounts {
signer: AccountsStorage<Keypair>,
some_pda: AccountsStorage<PdaStore>,
token_vault: AccountsStorage<TokenStore>,
mint: AccountsStorage<MintStore>,
}
```
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Specify instruction accounts
# Instruction accounts

{{ config.site_name }} fuzzer generates random indexes of accounts to use in each instruction. Each created account is saved in the global `FuzzAccounts` structure which helps you to reuse already existing accounts across all instructions.
{{ config.site_name }} generates random indexes of accounts to use in each instruction. Each created account is saved in the global FuzzAccounts structure which helps you to reuse already existing accounts across all instructions.

You are required to define, how these accounts should be created and which accounts should be passed to an instruction. It is done using the `IxOps` trait and its method `get_accounts`.
!!! tip

You are required to define, how these accounts should be created and which accounts should be passed to an instruction. It is done using the `IxOps` trait and its method `get_accounts`.

- Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/fuzz_instructions.rs` file and complete the pre-generated `get_accounts` methods for each instruction.

- Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/fuzz_instructions.rs` file and complete the pre-generated `get_accounts` methods for each instruction such as:

```rust
fn get_accounts(
Expand Down Expand Up @@ -35,7 +38,10 @@ fn get_accounts(
Ok((signers, acc_meta))
}
```
Notice especially the helper method `fuzz_accounts.<account_name>.get_or_create_account` that is used to create a Keypair or retrieve the Public key of the already existing account.

!!! note

Notice especially the helper method `fuzz_accounts.<account_name>.get_or_create_account` that is used to create a Keypair or retrieve the Public key of the already existing account.

## Create an arbitrary account
The `AccountsStorage<T>` type provides an implementation of the `get_or_create_account` method that helps you create new or read already existing accounts. There are different implementations for different types of storage (`Keypair`, `TokenStore`, `MintStore`, `PdaStore`) to simplify the creation of new accounts.
Expand Down
22 changes: 22 additions & 0 deletions documentation/docs/fuzzing/writing-fuzz-test/instruction-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Instruction data

{{ config.site_name }} generates random instruction data for you.

!!! tip

Currently, it is however required, that you manually assign the random fuzzer data to the instruction data. It is done using the `IxOps` trait and its method `get_data`.

- Go to the `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/fuzz_instructions.rs` file and complete the pre-generated `get_data` methods for each instruction.

```rust
fn get_data(
&self,
_client: &mut impl FuzzClient,
_fuzz_accounts: &mut FuzzAccounts,
) -> Result<Self::IxData, FuzzingError> {
let data = fuzz_example1::instruction::Invest {
amount: self.data.amount,
};
Ok(data)
}
```
3 changes: 3 additions & 0 deletions documentation/docs/fuzzing/writing-fuzz-test/programs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Include Programs

TBD
Loading

0 comments on commit c3c8a95

Please sign in to comment.