Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more fuzzers and libFuzzer harness #553

Merged
merged 32 commits into from
Jan 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0277ce5
Add LLVM fuzzer harness
Dec 22, 2021
1c8ec2e
Add AFL++ test case generator
Dec 22, 2021
b89c76b
Fuzz more gridDisk functions
Dec 22, 2021
79d2e2c
add fuzzerH3SetToLinkedGeo
Dec 22, 2021
a28a807
Add more fuzzers
Dec 22, 2021
9445cbb
Additional fuzzers
Dec 22, 2021
f971dcf
add fuzzerVertexes
Dec 22, 2021
e0c8841
Add test-fuzzer script
Dec 22, 2021
007b7c3
Fix linux build
Dec 22, 2021
02abb99
Fix fuzzerIndexIO
Dec 22, 2021
caedc90
test-fuzzer use subshell for ls
isaacbrodsky Dec 22, 2021
de5a2c3
Update test-fuzzer again
isaacbrodsky Dec 22, 2021
8bbc36a
Fix test-fuzzer again
isaacbrodsky Dec 22, 2021
021c994
fuzzerCompact
Dec 23, 2021
2195863
Update readme
Dec 23, 2021
4a65123
libFuzzer tests
Dec 23, 2021
1e7066e
reformat header
Dec 23, 2021
0ede718
README updates
Dec 23, 2021
65b4ef3
fuzzerDirectedEdge
Dec 23, 2021
79b1f44
fuzzerLocalIj
Dec 23, 2021
25360ef
fix fuzzerDirectedEdge build
Dec 24, 2021
ac4b918
Fix fuzzer programs
isaacbrodsky Dec 27, 2021
1145bce
remove logging
isaacbrodsky Dec 27, 2021
cd14266
remove h3Println
isaacbrodsky Dec 27, 2021
76532d8
add fuzzerPoylgonToCells
isaacbrodsky Jan 3, 2022
84bc4e9
Update per review
Jan 3, 2022
323d9e9
Merge branch 'master' into llvm-fuzzer-harness
Jan 3, 2022
e04c62c
Add comment on memcpy per review
Jan 3, 2022
0016f1c
Fix potential crash in vertexRotations
Jan 3, 2022
7807131
Merge branch 'master' into llvm-fuzzer-harness
Jan 3, 2022
4b4e623
Catch possible failure in getIcosahedronFaces
Jan 3, 2022
d501e51
Don't assert specific error in testVertex
Jan 3, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .github/workflows/test-fuzzer.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: test-fuzzer

on:
push:
branches: [master, stable-*]
pull_request:
branches: [master, stable-*]

jobs:
afl-tests:
name: Fuzzer Compilation ${{ matrix.compiler }}
runs-on: ubuntu-latest
env:
CC: ${{ matrix.compiler }}

strategy:
matrix:
compiler: [clang, gcc]

steps:
- uses: actions/[email protected]

- name: Configure build
run: |
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..

- name: Build
run: |
cd build
make fuzzers

- name: Run fuzzers once
run: |
cd build
for fuzzer in bin/fuzzer*; do
echo $fuzzer
$fuzzer --generate inputData
$fuzzer inputData
done

libfuzzer-test:
name: LibFuzzer Compilation ${{ matrix.compiler }}
runs-on: ubuntu-latest
env:
CC: ${{ matrix.compiler }}

strategy:
matrix:
compiler: [clang]

steps:
- uses: actions/[email protected]

- name: Configure build
run: |
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_LIBFUZZER=ON ..

- name: Build
run: |
cd build
make fuzzers

- name: Run fuzzers once
run: |
cd build
for fuzzer in bin/fuzzer*; do
echo $fuzzer
# TODO: Increase run count
$fuzzer -runs=1000
done
37 changes: 37 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ option(BUILD_BENCHMARKS "Build benchmarking applications." ON)
option(BUILD_FUZZERS "Build fuzzer applications (for use with afl)." ON)
option(BUILD_FILTERS "Build filter applications." ON)
option(BUILD_GENERATORS "Build code generation applications." ON)
# If ON, libfuzzer settings are used to build the fuzzer harnesses. If OFF, a frontend
# for afl++ is provided instead.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the frontend part of this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

frontend in this context means an implementation of main that accepts arguments in a way that AFL can integrate with. Should I expand on the comments for this option?

option(ENABLE_LIBFUZZER "Build fuzzers with libFuzzer support." OFF)

if(WIN32)
# Use bash (usually from Git for Windows) for piping results
Expand Down Expand Up @@ -89,6 +92,10 @@ if(NOT WIN32)
# to fully enable coverage.
list(APPEND H3_LINK_FLAGS $<$<CONFIG:Debug>:--coverage>)
endif()
if(ENABLE_LIBFUZZER)
list(APPEND H3_COMPILE_FLAGS -fsanitize=fuzzer,address,undefined)
list(APPEND H3_LINK_FLAGS -fsanitize=fuzzer,address,undefined)
endif()
Comment on lines +95 to +98
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that these settings don't really work for the other binaries since main gets redefined.


option(WARNINGS_AS_ERRORS "Warnings are treated as errors" OFF)
if(WARNINGS_AS_ERRORS)
Expand Down Expand Up @@ -143,6 +150,7 @@ set(APP_SOURCE_FILES
src/apps/applib/include/benchmark.h
src/apps/applib/include/utility.h
src/apps/applib/include/args.h
src/apps/applib/include/aflHarness.h
src/apps/applib/lib/kml.c
src/apps/applib/lib/utility.c
src/apps/applib/lib/args.c)
Expand Down Expand Up @@ -219,6 +227,19 @@ set(OTHER_SOURCE_FILES
src/apps/fuzzers/fuzzerLatLngToCell.c
src/apps/fuzzers/fuzzerCellToLatLng.c
src/apps/fuzzers/fuzzerGridDisk.c
src/apps/fuzzers/fuzzerH3SetToLinkedGeo.c
src/apps/fuzzers/fuzzerDistances.c
src/apps/fuzzers/fuzzerCellArea.c
src/apps/fuzzers/fuzzerExactEdgeLength.c
src/apps/fuzzers/fuzzerCellProperties.c
src/apps/fuzzers/fuzzerIndexIO.c
src/apps/fuzzers/fuzzerResolutions.c
src/apps/fuzzers/fuzzerHierarchy.c
src/apps/fuzzers/fuzzerVertexes.c
src/apps/fuzzers/fuzzerCompact.c
src/apps/fuzzers/fuzzerDirectedEdge.c
src/apps/fuzzers/fuzzerLocalIj.c
src/apps/fuzzers/fuzzerPolygonToCells.c
src/apps/benchmarks/benchmarkPolygonToCells.c
src/apps/benchmarks/benchmarkPolygon.c
src/apps/benchmarks/benchmarkH3SetToLinkedGeo.c
Expand Down Expand Up @@ -635,12 +656,28 @@ if(BUILD_FUZZERS)

macro(add_h3_fuzzer name srcfile)
add_h3_executable(${name} ${srcfile} ${APP_SOURCE_FILES})
if(ENABLE_LIBFUZZER)
target_compile_definitions(${name} PRIVATE H3_USE_LIBFUZZER)
endif()
add_dependencies(fuzzers ${name})
endmacro()

add_h3_fuzzer(fuzzerLatLngToCell src/apps/fuzzers/fuzzerLatLngToCell.c)
add_h3_fuzzer(fuzzerCellToLatLng src/apps/fuzzers/fuzzerCellToLatLng.c)
add_h3_fuzzer(fuzzerGridDisk src/apps/fuzzers/fuzzerGridDisk.c)
add_h3_fuzzer(fuzzerH3SetToLinkedGeo src/apps/fuzzers/fuzzerH3SetToLinkedGeo.c)
add_h3_fuzzer(fuzzerDistances src/apps/fuzzers/fuzzerDistances.c)
add_h3_fuzzer(fuzzerCellArea src/apps/fuzzers/fuzzerCellArea.c)
add_h3_fuzzer(fuzzerExactEdgeLength src/apps/fuzzers/fuzzerExactEdgeLength.c)
add_h3_fuzzer(fuzzerCellProperties src/apps/fuzzers/fuzzerCellProperties.c)
add_h3_fuzzer(fuzzerIndexIO src/apps/fuzzers/fuzzerIndexIO.c)
add_h3_fuzzer(fuzzerResolutions src/apps/fuzzers/fuzzerResolutions.c)
add_h3_fuzzer(fuzzerHierarchy src/apps/fuzzers/fuzzerHierarchy.c)
add_h3_fuzzer(fuzzerVertexes src/apps/fuzzers/fuzzerVertexes.c)
add_h3_fuzzer(fuzzerCompact src/apps/fuzzers/fuzzerCompact.c)
add_h3_fuzzer(fuzzerDirectedEdge src/apps/fuzzers/fuzzerDirectedEdge.c)
add_h3_fuzzer(fuzzerLocalIj src/apps/fuzzers/fuzzerLocalIj.c)
add_h3_fuzzer(fuzzerPolygonToCells src/apps/fuzzers/fuzzerPolygonToCells.c)
endif()

if(BUILD_BENCHMARKS)
Expand Down
76 changes: 76 additions & 0 deletions src/apps/applib/include/aflHarness.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Copyright 2021 Uber Technologies, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/** @file aflHarness.h
* @brief Adapter from LLVM fuzzer to AFL++
*/
#ifndef AFLHARNESS_H
#define AFLHARNESS_H

#include <string.h>

#include "utility.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);

#ifndef H3_USE_LIBFUZZER

/**
* Generate a AFL++ test case file of the right size initialized to all zeroes.
*
* @param filename
* @param expectedSize
* @return int
*/
int generateTestCase(const char *filename, size_t expectedSize) {
FILE *fp = fopen(filename, "wb");
uint8_t zero = 0;
if (fwrite(&zero, sizeof(zero), expectedSize, fp) != expectedSize) {
error("Error writing\n");
}
fclose(fp);
return 0;
}

#define AFL_HARNESS_MAIN(expectedSize) \
int main(int argc, char *argv[]) { \
if (argc == 3) { \
if (strcmp(argv[1], "--generate") != 0) { \
error( \
"Invalid option (should be --generate, otherwise look at " \
"aflHarness.h to see options)"); \
} \
return generateTestCase(argv[2], expectedSize); \
} \
if (argc != 2) { \
error("Should have one argument (test case file)\n"); \
} \
const char *filename = argv[1]; \
FILE *fp = fopen(filename, "rb"); \
uint8_t data[expectedSize]; \
if (fread(&data, expectedSize, 1, fp) != 1) { \
error("Error reading\n"); \
} \
fclose(fp); \
return LLVMFuzzerTestOneInput(data, expectedSize); \
}

#else

#define AFL_HARNESS_MAIN(expectedSize)

#endif // H3_USE_LIBFUZZER

#endif // AFLHARNESS_H
107 changes: 101 additions & 6 deletions src/apps/fuzzers/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,102 @@
# Fuzzer harnesses for H3

This directory contains helper programs for testing the H3 library using the
''[American fuzzy lop](https://lcamtuf.coredump.cx/afl/)'' fuzzer.
''[American fuzzy lop](https://lcamtuf.coredump.cx/afl/)''/
''[AFL++](https://github.com/AFLplusplus/AFLplusplus)'' or
''[libFuzzer](https://www.llvm.org/docs/LibFuzzer.html)'' fuzzers.
Fuzzing is a technique for discovering crashes and other edge cases in code
such as the H3 core library.

# Installation
# Function coverage

| Function | File or status
| -------- | --------------
| latLngToCell | [fuzzerLatLngToCell](./fuzzerLatLngToCell.c)
| cellToLatLng | [fuzzerCellToLatLng](./fuzzerCellToLatLng.c)
| cellToBoundary | [fuzzerCellToLatLng](./fuzzerCellToLatLng.c)
| gridDisk | [fuzzerGridDisk](./fuzzerGridDisk.c)
| gridDiskDistances | [fuzzerGridDisk](./fuzzerGridDisk.c)
| gridRingUnsafe | [fuzzerGridDisk](./fuzzerGridDisk.c)
| polygonToCells | [fuzzerPoylgonToCells](./fuzzerPolygonToCells.c)
| h3SetToMultiPolygon | [fuzzerH3SetToLinkedGeo](./fuzzerH3SetToLinkedGeo.c)
| degsToRads | Trivial
| radsToDegs | Trivial
| distance | [fuzzerDistances](./fuzzerDistances.c)
| getHexagonAreaAvg | [fuzzerResolutions](./fuzzerResolutions.c)
| cellArea | [fuzzerCellArea](./fuzzerCellArea.c)
| getHexagonEdgeLengthAvg | [fuzzerResolutions](./fuzzerResolutions.c)
| exactEdgeLength | [fuzzerExactEdgeLength](./fuzzerExactEdgeLength.c)
| getNumCells | [fuzzerResolutions](./fuzzerResolutions.c)
| getRes0Cells | Trivial
| getPentagons | [fuzzerResolutions](./fuzzerResolutions.c)
| getResolution | [fuzzerCellProperties](./fuzzerCellProperties.c)
| getBaseCellNumber | [fuzzerCellProperties](./fuzzerCellProperties.c)
| stringToH3 | [fuzzerIndexIO](./fuzzerIndexIO.c)
| h3ToString | [fuzzerIndexIO](./fuzzerIndexIO.c)
| isValidCell | [fuzzerCellProperties](./fuzzerCellProperties.c)
| cellToParent | [fuzzerHierarchy](./fuzzerHierarchy.c)
| cellToChildren | [fuzzerHierarchy](./fuzzerHierarchy.c)
| cellToCenterChild | [fuzzerHierarchy](./fuzzerHierarchy.c)
| compactCells | [fuzzerCompact](./fuzzerCompact.c)
| uncompactCells | [fuzzerCompact](./fuzzerCompact.c)
| isResClassIII | [fuzzerCellProperties](./fuzzerCellProperties.c)
| isPentagon | [fuzzerCellProperties](./fuzzerCellProperties.c)
| getIcosahedronFaces | [fuzzerCellProperties](./fuzzerCellProperties.c)
| areNeighborCells | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| cellsToDirectedEdge | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| isValidDirectedEdge | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| getDirectedEdgeOrigin | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| getDirectedEdgeDestination | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| directedEdgeToCells | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| originToDirectedEdges | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| directedEdgeToBoundary | [fuzzerDirectedEdge](./fuzzerDirectedEdge.c)
| cellToVertex | [fuzzerVertexes](./fuzzerVertexes.c)
| cellToVertexes | [fuzzerVertexes](./fuzzerVertexes.c)
| vertexToLatLng | [fuzzerVertexes](./fuzzerVertexes.c)
| isValidVertex | [fuzzerVertexes](./fuzzerVertexes.c)
| gridDistance | [fuzzerLocalIj](./fuzzerLocalIj.c)
| gridPathCells | [fuzzerLocalIj](./fuzzerLocalIj.c)
| experimentalH3ToLocalIj | [fuzzerLocalIj](./fuzzerLocalIj.c)
| experimentalLocalIjToH3 | [fuzzerLocalIj](./fuzzerLocalIj.c)

# libFuzzer Usage

libFuzzer is one of the supported fuzzing drivers.

This is the fuzzer used in [oss-fuzz](https://github.com/google/oss-fuzz/tree/master/projects/h3).

## Build

You must build H3 with Clang and enable support for libFuzzer.

```
CC=clang cmake -DENABLE_LIBFUZZER=ON .
make fuzzers
```

## Run

To begin fuzzing, execute the fuzzer binary:

```
fuzzerLatLngToCell
```

For command line options including how to specify a test corpus, consult the [libFuzzer documentation](https://www.llvm.org/docs/LibFuzzer.html#options).

# AFL Usage

AFL/AFL++ are supported fuzzing drivers.

## Installation

```
apt install afl-clang
```

(There is also an afl-cov which looks interesting but isn't necessary.)

# Usage
## Build

You must compile with the instrumented compiler:

Expand All @@ -20,15 +105,25 @@ CXX=afl-clang++ CC=afl-clang cmake .
make fuzzers
```

Generate a blank (zeroed) test case file. This will not be very a interesting test case but is usedful
for having files of the right size.

```
fuzzerLatLngToCell --generate bytes24
```

An individual fuzzer run is invoked as follows. The argument is a file containing the number of bytes needed.

```
fuzzerGeoToH3 bytes24
fuzzerLatLngToCell bytes24
```

## Run

To begin running the fuzzer, run the following. The testcase directory (`testcase_dir`) should contain a file
with at least the right number of bytes that the fuzzer will read (such as 16 for fuzzerKRing.)
with at least the right number of bytes that the fuzzer will read (this can be generated using the `--generate`
option above.)

```
afl-fuzz -i testcase_dir -o findings_dir -- fuzzerGeoToH3 @@
afl-fuzz -i testcase_dir -o findings_dir -- fuzzerLatLngToCell @@
```
Loading