Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📝 Update docs structure, design #193

Merged
merged 1 commit into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions documentation/docs/fuzzing/execute/debug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Debug

To debug your program with values from a crash file:

```bash
# fuzzer will run the <TARGET_NAME> with the specified <CRASH_FILE_PATH>
trident fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH>
# for example:
trident fuzz run-debug fuzz_0 trident-tests/fuzz_tests/fuzzing/fuzz_0/cr1.fuzz
```


## Debug Output

!!! important

The debug output is at current development stage really verbose and contains lldb parts. We are working on improving this experience. In the picture below you can see an example of provided debug output.

1. Series of Transaction Logs
2. Structures of data send within the Instructions
3. **Panic** or **Crash**, based on if the Fuzzing paniced within the Solana Program or Invariant Check failed.



![alt text](../../images/run-debug.png)
Original file line number Diff line number Diff line change
@@ -1,14 +1,42 @@
# Run and Debug
# Run

## Run
Once you have finished the implementation of the Fuzz Test, you can run the Test as follows:
To execute the desired fuzz test, run:

```bash
# Replace <TARGET_NAME> with the name of particular
# fuzz test (for example: "fuzz_0")
trident fuzz run <TARGET_NAME>
```

## Trident output

!!! important

The output provided by Honggfuzz is as follows

1. Number of Fuzzing Iterations.
2. Feedback Driven Mode = Honggfuzz generates data based on the feedback (i.e. feedback based on Coverage progress).
3. Average Iterations per second
4. Number of crashes it found (**panics** or failed **invariant checks**)


```bash
------------------------[ 0 days 00 hrs 00 mins 01 secs ]----------------------
Iterations : 688 (out of: 1000 [68%]) # -- 1. --
Mode [3/3] : Feedback Driven Mode # -- 2. --
Target : trident-tests/fuzz_tests/fuzzing.....wn-linux-gnu/release/fuzz_0
Threads : 16, CPUs: 32, CPU%: 1262% [39%/CPU]
Speed : 680/sec [avg: 688] # -- 3. --
Crashes : 1 [unique: 1, blocklist: 0, verified: 0] # -- 4. --
Timeouts : 0 [10 sec]
Corpus Size : 98, max: 1048576 bytes, init: 0 files
Cov Update : 0 days 00 hrs 00 mins 00 secs ago
Coverage : edge: 10345/882951 [1%] pc: 163 cmp: 622547
---------------------------------- [ LOGS ] ------------------/ honggfuzz 2.6 /-
```


## Customize Fuzzing
Under the hood {{ config.site_name }} uses [honggfuzz-rs](https://github.com/rust-fuzz/honggfuzz-rs).

You can pass [supported parameters](https://github.com/Ackee-Blockchain/trident/blob/develop/examples/fuzz-tests/hello_world/Trident.toml) via the **{{ config.site_name }}.toml** configuration file:
Expand Down Expand Up @@ -54,31 +82,16 @@ allow_duplicate_txs = false
fuzzing_with_stats = true
```

Or you can pass any parameter via [environment variables](https://github.com/rust-fuzz/honggfuzz-rs#environment-variables).
??? note

A list of hongfuzz parameters can be found in honggfuzz [usage documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md#cmdline---help). The parameters passed via **environment variables** have **higher priority**. For example:
Or you can pass any parameter via [environment variables](https://github.com/rust-fuzz/honggfuzz-rs#environment-variables).

```bash
# Time-out: 10 secs
# Number of concurrent fuzzing threads: 1
# Number of fuzzing iterations: 10000
# Display Solana logs in the terminal
HFUZZ_RUN_ARGS="-t 10 -n 1 -N 10000 -Q" trident fuzz run <TARGET_NAME>
```

### Fuzzing statistics
Sometimes, it's useful to know how often a particular instruction has been invoked and how many times it has succeeded or failed. To display these statistics when fuzzing is finished or interrupted, set the `fuzzing_with_stats` option to `true` in the `[fuzz]` section of the Trident.toml configuration file. Please note that this option is disabled by default because it impacts performance.

The statistics show the total number of invocations for each instruction, which is the sum of successful and failed invocations. Successful invocations are those that return an `Ok()` result. Failed invocations are those that return an `Err()` result. Additionally, the statistics also show as `Check Failed` the number of successful invocations that did not pass the user-defined invariants check. Note that unhandled panics are currently logged only as crashes and are not displayed in the fuzzing statistics table.
A list of hongfuzz parameters can be found in honggfuzz [usage documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md#cmdline---help). The parameters passed via **environment variables** have **higher priority**. For example:

Keep in mind that the number of fuzz iterations does not directly correspond to the total number of invocations. In one fuzz iteration, the fuzzer might be unable to deserialize fuzz data into instructions, causing the entire iteration to be skipped.

## Debug
To debug your program with values from a crash file:

```bash
# fuzzer will run the <TARGET_NAME> with the specified <CRASH_FILE_PATH>
trident fuzz run-debug <TARGET_NAME> <CRASH_FILE_PATH>
# for example:
trident fuzz run-debug fuzz_0 trident-tests/fuzz_tests/fuzzing/fuzz_0/cr1.fuzz
```
```bash
# Time-out: 10 secs
# Number of concurrent fuzzing threads: 1
# Number of fuzzing iterations: 10000
# Display Solana logs in the terminal
HFUZZ_RUN_ARGS="-t 10 -n 1 -N 10000 -Q" trident fuzz run <TARGET_NAME>
```
27 changes: 27 additions & 0 deletions documentation/docs/fuzzing/execute/statistics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Fuzzing statistics

!!! important

In order to show statistics of fuzzing session (statistics are displayed after the fuzzing session ended), set `fuzzing_with_stats` within the Trident.toml to `true`.

```toml
[fuzz]
# ...
fuzzing_with_stats = true
# ...
```


Currently exported statistics from the Fuzzing Session

#### Simple

- Number of invocations of each instruction during the fuzzing session.
- Number of successful invocations of each instruction during the fuzzing session.
- Number of failed invariants checks for each instruction during the fuzzing session.

??? note

Keep in mind that the number of fuzz iterations does not directly correspond to the total number of invocations. In one fuzz iteration, the fuzzer might be unable to deserialize fuzz data into instructions, causing the entire iteration to be skipped.

On the other hand this is expected behavior as the underlying data are randomly (with coverage feedback) generated, so the Honggfuzz will not necessarily find appropriate data each iteration.
53 changes: 53 additions & 0 deletions documentation/docs/fuzzing/extra/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
hide:
- toc
---

# Trident by Examples

<div class="grid cards" markdown>

- :material-hand-wave:{ .lg .middle } __Hello World!__

---

Hello World example with Trident.

[Hello World!](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/hello_world)

- :octicons-bug-16:{ .lg .middle } __Possible vulnerabilities and bugs__

---

Check the possible attack vectors and bugs that can be detected using Trident.

[Unchecked Arithmetic](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/unchecked-arithmetic-0)

[Incorrect Instruction Sequence](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/incorrect-ix-sequence-1)

[Unauthorized Access](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/unauthorized-access-2)

[Incorrect Integer Arithmetic](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/incorrect-integer-arithmetic-3)

- :octicons-tools-24:{ .lg .middle } __Customize with Arbitrary__

---

You can use Arbitrary crate to your advantage and limit or customize the data that are sent to the instructions.

[Custom Data Types](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/arbitrary-custom-types-4)


[Limiting Instructions Inputs](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/arbitrary-limit-inputs-5)

- :material-call-made:{ .lg .middle } __Cross-Program Invocation__

---

Trident supports Cross-Program Invocation, you can fuzz your programs and create NFTs at the same time.

[Simple CPI](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/simple-cpi-6)

[CPI with Metaplex Metadata Program](https://github.com/Ackee-Blockchain/trident/tree/master/examples/fuzz-tests/cpi-metaplex-7)

</div>
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Fuzz test-only
# Initialize Fuzz Test

If you are interested only in generating templates for fuzz tests run
For initialization of workspace for fuzz tests, call:
```bash
trident init fuzz
```
Expand All @@ -21,3 +21,10 @@ project-root
├── Trident.toml
└── ...
```

## Add new Fuzz Test

If you wish to generate template for a new fuzz test, call:
```bash
trident fuzz add
```
32 changes: 32 additions & 0 deletions documentation/docs/fuzzing/first-steps/writing-fuzz-test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Writing Fuzz Test

!!! important

At the current development stage, there are some manual steps required to start the Fuzzing Session. In principle:

**Prerequisites:**

- Add dependencies specific to your program to `trident-tests/fuzz_tests/Cargo.toml` (such as anchor-spl etc.).
- Add necessary `use` statements into `trident-tests/fuzz_tests/<FUZZ_TEST_NAME>/accounts_snapshots.rs` to import missing types.

**Writing Fuzz Tests**

1. Include desired Programs [Include Programs](../writing-fuzz-test/programs.md).
2. Specify pseudo-random accounts to re-use [Accounts to re-use](../writing-fuzz-test/accounts.md).
3. Specify instruction data [Instruction Data](../writing-fuzz-test/instruction-data.md).
4. Specify instruction accounts [Instruction Accounts](../writing-fuzz-test/instruction-accounts.md).

**Run and Debug**

1. Execute desired fuzz test [Run](../execute/run.md)
2. See the found crash with more details [Debug](../execute/debug.md)

!!! note

For better fuzzing results and experience you can also manually adjust the following:

1. Define Invariants checks [Invariants Checks](../writing-fuzz-test-extra/invariants-checks.md).
2. Specify instruction sequences[Instruction sequences](../writing-fuzz-test-extra/instruction-sequences.md).
3. Specify custom data types[Custom Data types](../writing-fuzz-test-extra/custom-data-types.md).
4. Well structured data[Arbitrary](../writing-fuzz-test-extra/arbitrary.md).
4. AccountsSnapshots macro[AccountsSnapshots](../writing-fuzz-test-extra/accounts-snapshots.md).
23 changes: 0 additions & 23 deletions documentation/docs/fuzzing/fuzzing-examples.md

This file was deleted.

22 changes: 0 additions & 22 deletions documentation/docs/fuzzing/fuzzing-introduction.md

This file was deleted.

6 changes: 0 additions & 6 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p0.md

This file was deleted.

21 changes: 0 additions & 21 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p1.md

This file was deleted.

20 changes: 0 additions & 20 deletions documentation/docs/fuzzing/howto/fuzzing-howto-p2.md

This file was deleted.

This file was deleted.

This file was deleted.

Loading
Loading