"borrow checker invariants" section of the "leveraging the type system" chapter (#2867)

Adds materials on the "leveraging the type system/borrow checker
invariants" subject.

I'm still calibrating what's expected subject-and-style wise, so do
spell out things where I've drifted off mark.

---------

Co-authored-by: tall-vase <[email protected]>
Co-authored-by: Dmitri Gribenko <[email protected]>

Author: 3 people

src/SUMMARY.md

@@ -455,6 +455,14 @@
       - [Serializer: implement Struct](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/struct.md)
       - [Serializer: implement Property](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/property.md)
       - [Serializer: Complete implementation](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/complete.md)
+  - [Borrow checking invariants](idiomatic/leveraging-the-type-system/borrow-checker-invariants.md)
+    - [Lifetimes and Borrows: the Abstract Rules](idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md)
+    - [Single-use values](idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md)
+    - [Mutually Exclusive References / "Aliasing XOR Mutability"](idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md)
+    - [PhantomData and Types](idiomatic/leveraging-the-type-system/borrow-checker-invariants/phantomdata-01-types.md)
+    - [PhantomData and Types (implementation)](idiomatic/leveraging-the-type-system/borrow-checker-invariants/phantomdata-02-types-implemented.md)
+    - [PhantomData: Lifetimes for External Resources](idiomatic/leveraging-the-type-system/borrow-checker-invariants/phantomdata-03-lifetimes.md)
+    - [PhantomData: OwnedFd & BorrowedFd](idiomatic/leveraging-the-type-system/borrow-checker-invariants/phantomdata-04-borrowedfd.md)
   - [Token Types](idiomatic/leveraging-the-type-system/token-types.md)
     - [Permission Tokens](idiomatic/leveraging-the-type-system/token-types/permission-tokens.md)
     - [Token Types with Data: Mutex Guards](idiomatic/leveraging-the-type-system/token-types/mutex-guard.md)

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants.md

@@ -0,0 +1,116 @@
+---
+minutes: 15
+---
+
+# Using the Borrow checker to enforce Invariants
+
+The borrow checker, while added to enforce memory ownership, can model other
+problems and prevent API misuse.
+
+```rust,editable
+/// Doors can be open or closed, and you need the right key to lock or unlock
+/// one. Modelled with a Shared key and Owned door.
+pub struct DoorKey {
+    pub key_shape: u32,
+}
+pub struct LockedDoor {
+    lock_shape: u32,
+}
+pub struct OpenDoor {
+    lock_shape: u32,
+}
+
+fn open_door(key: &DoorKey, door: LockedDoor) -> Result<OpenDoor, LockedDoor> {
+    if door.lock_shape == key.key_shape {
+        Ok(OpenDoor { lock_shape: door.lock_shape })
+    } else {
+        Err(door)
+    }
+}
+
+fn close_door(key: &DoorKey, door: OpenDoor) -> Result<LockedDoor, OpenDoor> {
+    if door.lock_shape == key.key_shape {
+        Ok(LockedDoor { lock_shape: door.lock_shape })
+    } else {
+        Err(door)
+    }
+}
+
+fn main() {
+    let key = DoorKey { key_shape: 7 };
+    let closed_door = LockedDoor { lock_shape: 7 };
+    let opened_door = open_door(&key, closed_door);
+    if let Ok(opened_door) = opened_door {
+        println!("Opened the door with key shape '{}'", key.key_shape);
+    } else {
+        eprintln!(
+            "Door wasn't opened! Your key only opens locks with shape '{}'",
+            key.key_shape
+        );
+    }
+}
+```
+
+<details>
+
+- We've seen the borrow checker prevent memory safety bugs (use-after-free, data
+  races).
+
+- We've also used types to shape and restrict APIs already using
+  [the Typestate pattern](../leveraging-the-type-system/typestate-pattern.md).
+
+- Language features are often introduced for a specific purpose.
+
+  Over time, users may develop ways of using a feature in ways that were not
+  predicted when they were introduced.
+
+  Java 5 introduced Generics in 2004 with the
+  [main stated purpose of enabling type-safe collections](https://jcp.org/en/jsr/detail?id=14).
+
+  Adoption was slow at first, but some new projects began designing their APIs
+  around generics from the beginning.
+
+  Since then, users and developers of the language expanded the use of generics
+  to other areas of type-safe API design:
+  - Class information can be held onto via Java's `Class<T>` or Guava's
+    `TypeToken<T>`.
+  - The Builder pattern can be implemented using Recursive Generics.
+
+  We aim to do something similar here: Even though the borrow checker was
+  introduced to prevent use-after-free and data races, we treat it as just
+  another API design tool.
+
+  It can be used to model program properties that have nothing to do with
+  preventing memory safety bugs.
+
+- To use the borrow checker as a problem solving tool, we will need to "forget"
+  that the original purpose of it is to prevent mutable aliasing in the context
+  of preventing use-after-frees and data races.
+
+  We should imagine working within situations where the rules are the same but
+  the meaning is slightly different.
+
+- This example uses ownership and borrowing are used to model the state of a
+  physical door.
+
+  `open_door` **consumes** a `LockedDoor` and returns a new `OpenDoor`. The old
+  `LockedDoor` value is no longer available.
+
+  If the wrong key is used, the door is left locked. It is returned as an `Err`
+  case of the `Result`.
+
+  It is a compile-time error to try and use a door that has already been opened.
+
+- Similarly, `lock_door` consumes an `OpenDoor`, preventing closing the door
+  twice at compile time.
+
+- The rules of the borrow checker exist to prevent memory safety bugs, but the
+  underlying logical system does not "know" what memory is.
+
+  All the borrow checker does is enforce a specific set of rules of how users
+  can order operations.
+
+  This is just one case of piggy-backing onto the rules of the borrow checker to
+  design APIs to be harder or impossible to misuse.
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md

@@ -0,0 +1,108 @@
+---
+minutes: 15
+---
+
+# Mutually Exclusive References / "Aliasing XOR Mutability"
+
+We can use the mutual exclusion of `&T` and `&mut T` references to prevent data
+from being used before it is ready.
+
+```rust,editable
+pub struct QueryResult;
+pub struct DatabaseConnection {/* fields omitted */}
+
+impl DatabaseConnection {
+    pub fn new() -> Self {
+        Self {}
+    }
+    pub fn results(&self) -> &[QueryResult] {
+        &[] // fake results
+    }
+}
+
+pub struct Transaction<'a> {
+    connection: &'a mut DatabaseConnection,
+}
+
+impl<'a> Transaction<'a> {
+    pub fn new(connection: &'a mut DatabaseConnection) -> Self {
+        Self { connection }
+    }
+    pub fn query(&mut self, _query: &str) {
+        // Send the query over, but don't wait for results.
+    }
+    pub fn commit(self) {
+        // Finish executing the transaction and retrieve the results.
+    }
+}
+
+fn main() {
+    let mut db = DatabaseConnection::new();
+
+    // The transaction `tx` mutably borrows `db`.
+    let mut tx = Transaction::new(&mut db);
+    tx.query("SELECT * FROM users");
+
+    // This won't compile because `db` is already mutably borrowed by `tx`.
+    // let results = db.results(); // ❌🔨
+
+    // The borrow of `db` ends when `tx` is consumed by `commit()`.
+    tx.commit();
+
+    // Now it is possible to borrow `db` again.
+    let results = db.results();
+}
+```
+
+<details>
+
+- Motivation: In this database API queries are kicked off for asynchronous
+  execution and the results are only available once the whole transaction is
+  finished.
+
+  A user might think that queries are executed immediately, and try to read
+  results before they are made available. This API misuse could make the app
+  read incomplete or incorrect data.
+
+  While an obvious misunderstanding, situations such as this can happen in
+  practice.
+
+  Ask: Has anyone misunderstood an API by not reading the docs for proper use?
+
+  Expect: Examples of early-career or in-university mistakes and
+  misunderstandings.
+
+  As an API grows in size and user base, a smaller percentage of users has deep
+  knowledge of the system the API represents.
+
+- This example shows how we can use Aliasing XOR Mutability to prevent this kind
+  of misuse.
+
+- The code might read results before they are ready if the programmer assumes
+  that the queries execute immediately rather than kicked off for asynchronous
+  execution.
+
+- The constructor for the `Transaction` type takes a mutable reference to the
+  database connection, and stores it in the returned `Transaction` value.
+
+  The explicit lifetime here doesn't have to be intimidating, it just means
+  "`Transaction` is outlived by the `DatabaseConnection` that was passed to it"
+  in this case.
+
+  The reference is mutable to completely lock out the `DatabaseConnection` from
+  other usage, such as starting further transactions or reading the results.
+
+- While a `Transaction` exists, we can't touch the `DatabaseConnection` variable
+  that was created from it.
+
+  Demonstrate: uncomment the `db.results()` line. Doing so will result in a
+  compile error, as `db` is already mutably borrowed.
+
+- Note: The query results not being public and placed behind a getter function
+  lets us enforce the invariant "users can only look at query results if there
+  is no active transactions."
+
+  If the query results were placed in a public struct field, this invariant
+  could be violated.
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md

@@ -0,0 +1,72 @@
+---
+minutes: 10
+---
+
+# Lifetimes and Borrows: the Abstract Rules
+
+```rust,editable
+// An internal data type to have something to hold onto.
+pub struct Internal;
+// The "outer" data.
+pub struct Data(Internal);
+
+fn shared_use(value: &Data) -> &Internal {
+    &value.0
+}
+fn exclusive_use(value: &mut Data) -> &mut Internal {
+    &mut value.0
+}
+fn deny_future_use(value: Data) {}
+
+fn demo_exclusive() {
+    let mut value = Data(Internal);
+    let shared = shared_use(&value);
+    // let exclusive = exclusive_use(&mut value); // ❌🔨
+    let shared_again = &shared;
+}
+
+fn demo_denied() {
+    let value = Data(Internal);
+    deny_future_use(value);
+    // let shared = shared_use(&value); // ❌🔨
+}
+
+# fn main() {}
+```
+
+<details>
+
+- This example re-frames the borrow checker rules away from references and
+  towards semantic meaning in non-memory-safety settings.
+
+  Nothing is being mutated, nothing is being sent across threads.
+
+- In rust's borrow checker we have access to three different ways of "taking" a
+  value:
+
+  - Owned value `T`. Value is dropped when the scope ends, unless it is not
+    returned to another scope.
+
+  - Shared Reference `&T`. Allows aliasing but prevents mutable access while
+    shared references are in use.
+
+  - Mutable Reference `&mut T`. Only one of these is allowed to exist for a
+    value at any one point, but can be used to create shared references.
+
+- Ask: The two commented-out lines in the `demo` functions would cause
+  compilation errors, Why?
+
+  `demo_exclusive`: Because the `shared` value is still aliased after the
+  `exclusive` reference is taken.
+
+  `demo_denied`: Because `value` is consumed the line before the
+  `shared_again_again` reference is taken from `&value`.
+
+- Remember that every `&T` and `&mut T` has a lifetime, just one the user
+  doesn't have to annotate or think about most of the time.
+
+  We rarely specify lifetimes because the Rust compiler allows us to _elide_
+  them in most cases. See:
+  [Lifetime Elision](../../../lifetimes/lifetime-elision.md)
+
+</details>

src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/phantomdata-01-types.md

@@ -0,0 +1,48 @@
+---
+minutes: 5
+---
+
+# PhantomData 1/4: De-duplicating Same Data & Semantics
+
+The newtype pattern can sometimes come up against the DRY principle, how do we
+solve this?
+
+<!-- dprint-ignore-start -->
+```rust,editable,compile_fail
+pub struct UserId(u64);
+impl ChatUser for UserId { /* ... */ }
+
+pub struct PatronId(u64);
+impl ChatUser for PatronId { /* ... */ }
+
+pub struct ModeratorId(u64);
+impl ChatUser for ModeratorId { /* ... */ }
+impl ChatModerator for ModeratorId { /* ... */ }
+
+pub struct AdminId(u64);
+impl ChatUser for AdminId { /* ... */ }
+impl ChatModerator for AdminId { /* ... */ }
+impl ChatAdmin for AdminId { /* ... */ }
+
+// And so on ...
+fn main() {}
+```
+<!-- dprint-ignore-end -->
+
+<details>
+
+- Problem: We want to use the newtype pattern to differentiate permissions, but
+  we're having to implement the same traits over and over again for the same
+  data.
+
+- Ask: Assume the details of each implementation here are the same between
+  types, what are ways we can avoid repeating ourselves?
+
+  Expect:
+  - Make this an enum, not distinct data types.
+  - Bundle the user ID with permission tokens like
+    `struct Admin(u64, UserPermission, ModeratorPermission, AdminPermission);`
+  - Adding a type parameter which encodes permissions.
+  - Mentioning `PhantomData` ahead of schedule (it's in the title).
+
+</details>