Skip to content

Commit

Permalink
Package documentation (#123)
Browse files Browse the repository at this point in the history
* Create initial documentation framework

* Use separate documentation test/deploy workflows

* GHA fixes

* Fix doctests

* Change docs deploy job name

* Documentation overhaul

* Refactor randdev example

* Recommendation on where to place `Mocking.activate`

* Update doctests in workflow

* Update Documenter workflow

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
omus and github-actions[bot] authored Jul 15, 2024
1 parent afa879c commit f7a0ee0
Show file tree
Hide file tree
Showing 14 changed files with 387 additions and 132 deletions.
37 changes: 37 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,40 @@ jobs:
- uses: codecov/codecov-action@v4
with:
files: lcov.info

doc-tests:
name: Doctests
runs-on: ubuntu-latest

# These permissions are needed to:
# - Checkout the repo
# - Delete old caches: https://github.com/julia-actions/cache#usage
# - Deploy the docs to the `gh-pages` branch: https://documenter.juliadocs.org/stable/man/hosting/#Permissions
permissions:
actions: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: "1"
- uses: julia-actions/cache@v2
- name: Configure docs environment
shell: julia --project=docs --color=yes {0}
run: |
using Pkg
Pkg.develop(PackageSpec(path=pwd()))
Pkg.instantiate()
- name: Run Doctests
shell: julia --project=docs --color=yes {0}
run: |
using Documenter: DocMeta, doctest
using Mocking: Mocking
setup = quote
using Mocking: @mock, @patch, activate, apply
activate()
end
DocMeta.setdocmeta!(Mocking, :DocTestSetup, setup; recursive=true)
doctest(Mocking; manual=false)
44 changes: 44 additions & 0 deletions .github/workflows/Documenter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Documenter
on:
push:
branches: ["main"]
tags: ["*"]
paths:
- "docs/**"
- "src/**"
- "Project.toml"
- ".github/workflows/Documenter.yml"
pull_request:
paths:
- "docs/**"
- "src/**"
- "Project.toml"
- ".github/workflows/Documenter.yml"

jobs:
deploy:
runs-on: ubuntu-latest

# These permissions are needed to:
# - Checkout the repo
# - Delete old caches: https://github.com/julia-actions/cache#usage
# - Deploy the docs to the `gh-pages` branch: https://documenter.juliadocs.org/stable/man/hosting/#Permissions
permissions:
actions: write
contents: write
statuses: write
steps:
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: "1"
- uses: julia-actions/cache@v2
- name: Configure docs environment
shell: julia --project=docs --color=yes {0}
run: |
using Pkg
Pkg.develop(PackageSpec(path=pwd()))
Pkg.instantiate()
- uses: julia-actions/julia-docdeploy@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
/Manifest.toml
/docs/Manifest.toml
/docs/build
*.jl.cov
*.jl.*.cov
*.jl.mem
/Manifest.toml
106 changes: 3 additions & 103 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,111 +1,11 @@
Mocking
=======

[![Stable Documentation](https://img.shields.io/badge/docs-stable-blue.svg)](https://juliatesting.github.io/Mocking.jl/stable)
[![Dev Documentation](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliatesting.github.io/Mocking.jl/dev)
[![CI](https://github.com/JuliaTesting/Mocking.jl/workflows/CI/badge.svg)](https://github.com/JuliaTesting/Mocking.jl/actions?query=workflow%3ACI+branch%3Amain)
[![codecov](https://codecov.io/gh/JuliaTesting/Mocking.jl/graph/badge.svg?token=BkilUame8F)](https://codecov.io/gh/JuliaTesting/Mocking.jl)
[![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle)
[![ColPrac: Contributor Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor's%20Guide-blueviolet)](https://github.com/SciML/ColPrac)


Allows Julia function calls to be temporarily overloaded for purpose of testing.

Contents
--------

- [Usage](#usage)
- [Gotchas](#gotchas)
- [Overhead](#overhead)

Usage
-----

Suppose you wrote the function `randdev` (UNIX only). How would you go about writing tests
for it?

```julia
function randdev(n::Integer)
open("/dev/urandom") do fp
reverse(read(fp, n))
end
end
```

The non-deterministic behaviour of this function makes it hard to test but we can write some
tests dealing with the deterministic properties of the function:

```julia
using Test
using ...: randdev

n = 10
result = randdev(n)
@test eltype(result) == UInt8
@test length(result) == n
```

How could we create a test that shows the output of the function is reversed? Mocking.jl
provides the `@mock` macro which allows package developers to temporarily overload a
specific calls in their package. In this example we will apply `@mock` to the `open` call
in `randdev`:

```julia
using Mocking

function randdev(n::Integer)
@mock open("/dev/urandom") do fp
reverse(read(fp, n))
end
end
```

With the call site being marked as "mockable" we can now write a testcase which allows
us to demonstrate the reversing behaviour within the `randdev` function:

```julia
using Mocking
using Test
using ...: randdev

Mocking.activate() # Need to call `activate` before executing `apply`

n = 10
result = randdev(n)
@test eltype(result) == UInt8
@test length(result) == n

# Produces a string with sequential UInt8 values from 1:n
data = unsafe_string(pointer(convert(Array{UInt8}, 1:n)))

# Generate an alternative method of `open` which call we wish to mock
patch = @patch open(fn::Function, f::AbstractString) = fn(IOBuffer(data))

# Apply the patch which will modify the behaviour for our test
apply(patch) do
@test randdev(n) == convert(Array{UInt8}, n:-1:1)
end

# Outside of the scope of the patched environment `@mock` is essentially a no-op
@test randdev(n) != convert(Array{UInt8}, n:-1:1)
```

Gotchas
-------

Remember to:

- Use `@mock` at desired call sites
- Run `Mocking.activate()` before executing any `apply` calls

Overhead
--------

The `@mock` macro uses a conditional check of `Mocking.activated()` which only allows
patches to be utilized only when Mocking has been activated. By default, Mocking starts as
disabled which should result conditional being optimized away allowing for zero-overhead.
Once activated via `Mocking.activate()` the `Mocking.activated` function will be
re-defined, causing all methods dependent on `@mock` to be recompiled.

License
-------

Mocking.jl is provided under the [MIT "Expat" License](LICENSE.md).
Allows Julia function calls to be temporarily overloaded for the purpose of testing.
3 changes: 3 additions & 0 deletions docs/Project.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
Mocking = "78c3b35d-d492-501b-9361-3d52fe80e533"
30 changes: 30 additions & 0 deletions docs/make.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
using Documenter
using Mocking: Mocking

setup = quote
using Mocking: @mock, @patch, activate, apply
activate()
end

DocMeta.setdocmeta!(Mocking, :DocTestSetup, setup; recursive=true)

makedocs(;
modules=[Mocking],
authors="Curtis Vogt and contributors",
sitename="Mocking.jl",
format=Documenter.HTML(;
canonical="https://juliatesting.github.io/Mocking.jl",
edit_link="main",
assets=String[],
prettyurls=get(ENV, "CI", nothing) == "true", # Fix links in local builds
),
pages=[
"Home" => "index.md",
"FAQ" => "faq.md",
"API" => "api.md",
# format trick: using this comment to force use of multiple lines
],
warnonly=[:missing_docs],
)

deploydocs(; repo="github.com/JuliaTesting/Mocking.jl", devbranch="main")
19 changes: 19 additions & 0 deletions docs/src/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# API

```@meta
CurrentModule = Mocking
```

```@index
```

---

```@docs
Mocking.activate
Mocking.activated
Mocking.nullify
Mocking.@mock
Mocking.@patch
Mocking.apply
```
30 changes: 30 additions & 0 deletions docs/src/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# FAQ

```@meta
CurrentModule = Mocking
```

## What kind of overhead does `@mock` add?

The [`@mock`](@ref) macro is a no-op and has zero overhead when mocking has not been activated via
[`Mocking.activate()`](@ref activate). Users can use `@code_llvm` on their code with and without `@mock` to
confirm the macro has no effect.

When `Mocking.activate` is called Mocking.jl will re-define a function utilized by `@mock`
which results in invalidating any functions using the macro. The result of this is that when
running your tests will cause those functions to be recompiled the next time they are called
such that the alternative code path provided by patches can be executed.

## Why isn't my patch being called?

When your patch isn't being applied you should remember to check for the following:

- [`Mocking.activate`](@ref activate) is called before the [`apply`](@ref) call.
- Call sites you want to patch are using [`@mock`](@ref).
- The patch's argument types are supertypes the values passed in at the call site.

## Where should I add `Mocking.activate()`?

We recommend putting the call to [`Mocking.activate`](@ref activate) in your package's
`test/runtests.jl` file after all of your import statements. The only true requirement is
that you call `Mocking.activate()` before the first [`apply`](@ref) call.
86 changes: 86 additions & 0 deletions docs/src/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Mocking

Allows Julia function calls to be temporarily overloaded for the purpose of testing.

## `randdev` Example

Suppose you wrote the function `randdev` (UNIX only). How would you go about writing tests
for it?

```jldoctest randdev; output=false
function randdev(n::Integer)
open("/dev/urandom") do fp
reverse(read(fp, n))
end
end
# output
randdev (generic function with 1 method)
```

The non-deterministic behaviour of this function makes it hard to test but we can write some
tests dealing with the deterministic properties of the function such as:

```jldoctest randdev; output=false
using Test
# using ...: randdev
n = 10
result = randdev(n)
@test eltype(result) == UInt8
@test length(result) == n
# output
Test Passed
```

How could we create a test that shows the output of the function is reversed? Mocking.jl
provides the `@mock` macro which allows package developers to temporarily overload a
specific calls in their package. In this example we will apply `@mock` to the `open` call
in `randdev`:

```jldoctest randdev_mock; output=false
using Mocking: @mock
function randdev(n::Integer)
@mock open("/dev/urandom") do fp
reverse(read(fp, n))
end
end
# output
randdev (generic function with 1 method)
```

With the call site being marked as "mockable" we can now write a testcase which allows
us to demonstrate the reversing behaviour within the `randdev` function:

```jldoctest randdev_mock; output=false
using Mocking
using Test
# using ...: randdev
Mocking.activate() # Need to call `activate` before executing `apply`
n = 10
result = randdev(n)
@test eltype(result) == UInt8
@test length(result) == n
# Produces a string with sequential UInt8 values from 1:n
data = unsafe_string(pointer(convert(Array{UInt8}, 1:n)))
# Generate an alternative method of `open` which call we wish to mock
patch = @patch open(fn::Function, f::AbstractString) = fn(IOBuffer(data))
# Apply the patch which will modify the behaviour for our test
apply(patch) do
@test randdev(n) == convert(Array{UInt8}, n:-1:1)
end
# Outside of the scope of the patched environment `@mock` is essentially a no-op
@test randdev(n) != convert(Array{UInt8}, n:-1:1)
# output
Test Passed
```
Loading

0 comments on commit f7a0ee0

Please sign in to comment.