Add forc-migrate tool (#6790)

## Description

This PR introduces `forc-migrate`, a Forc tool for migrating Sway
projects to the next breaking change version of Sway.

The tool addresses two points crucial for code updates caused by
breaking changes:
- it informs developers about the breaking changes and **assists in
planing and executing** the migration.
- it **automatically changes source code** where possible, reducing the
manual effort needed for code changes.

Besides adding the `forc-migrate` tool, the PR:
- extends `Diagnostic` to support migration diagnostics aside with
errors and warnings.
- changes `swayfmt` to support generating source code from arbitrary
lexed trees. The change is a minimal one done only in the parts of
`swayfmt` that are executed by migration steps written in this PR.
Adapting `swayfmt` to fully support arbitrary lexed trees will be done
in #6779.

The migration for the `references` feature, migrating `ref mut` to
`&mut`, is developed only partially, to demonstrate the development and
usage of automatic migrations that alter the original source code.

The intended usage of the tool is documented in detail in the "forc
migrate" chapter of The Sway Book: _Forc reference > Plugins >
forc_migrate_. (The generated documentation has issues that are caused
by the documentation generation bug explained in #6792. These issues
will be fixed in a separate PR that will fix it for all the plugins.)

We expect the `forc-migrate` to evolve based on the developer's
feedback. Some of the possible extensions of the tool are:
- adding additional CLI options, e.g., for executing only specific
migration steps, or ignoring them.
- passing parameters to migration steps from the CLI.
- not allowing updates by default, if the repository contains modified
or untracked files.
- migrating workspaces.
- migrating other artifacts, e.g., Forc.toml files or contract IDs.
- migrating between arbitrary versions of Sway.
- migrating SDK code.
- etc.
 
`forc-migrate` also showed a clear need for better infrastructure for
writing static analyzers and transforming Sway code. The approach used
in the implementation of this PR should be seen as a pragmatic
beginning, based on the reuse of what we currently have. Some future
options are discussed in #6836.

## Demo

### `forc migrate show`

Shows the breaking change features and related migration steps. This
command can be run anywhere and does not require a Sway project.

```
Breaking change features:
  - storage_domains    (#6701)
  - references         (#5063)

Migration steps (1 manual and 1 semiautomatic):
storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)

references
  [S] Replace `ref mut` function parameters with `&mut`

Experimental feature flags:
- for Forc.toml:  experimental = { storage_domains = true, references = true }
- for CLI:        --experimental storage_domains,references
```

### `forc migrate check`

Performs a dry-run of the migration on a concrete Sway project. It
outputs all the occurrences in code that need to be reviewed or changed,
as well as the migration time effort:

```
info: [storage_domains] Review explicitly defined slot keys in storage declarations (`in` keywords)
  --> /home/kebradalaonda/Desktop/M Forc migrate tool/src/main.sw:19:10
   |
...
19 |     y in b256::zero(): u64 = 0,
   |          ------------
20 |     z: u64 = 0,
21 |     a in calculate_slot_address(): u64 = 0,
   |          ------------------------
22 |     b in 0x0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20: u64 = 0,
   |          ------------------------------------------------------------------
   |
   = help: If the slot keys used in `in` keywords represent keys generated for `storage` fields
   = help: by the Sway compiler, those keys might need to be recalculated.
   = help:  
   = help: The previous formula for calculating storage field keys was: `sha256("storage.<field name>")`.
   = help: The new formula is:                                          `sha256((0u8, "storage.<field name>"))`.
   = help:  
   = help: For a detailed migration guide see: #6701
____

Migration effort:

storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)
      Occurrences:     3    Migration effort (hh::mm): ~00:06

references
  [S] Replace `ref mut` function parameters with `&mut`
      Occurrences:     0    Migration effort (hh::mm): ~00:00

Total migration effort (hh::mm): ~00:06
``` 

### `forc migrate run`

Runs the migration steps and guides developers through the migration
process.

## Checklist

- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.

.github/workflows/ci.yml

@@ -239,6 +239,7 @@ jobs:
           cargo install --locked --debug --path ./forc-plugins/forc-doc
           cargo install --locked --debug --path ./forc-plugins/forc-tx
           cargo install --locked --debug --path ./forc-plugins/forc-crypto
+          cargo install --locked --debug --path ./forc-plugins/forc-migrate
           cargo install --locked --debug forc-explore
       - name: Install mdbook-forc-documenter
         run: cargo install --locked --debug --path ./scripts/mdbook-forc-documenter

Cargo.toml

@@ -9,6 +9,7 @@ members = [
     "forc-plugins/forc-doc",
     "forc-plugins/forc-fmt",
     "forc-plugins/forc-lsp",
+    "forc-plugins/forc-migrate",
     "forc-plugins/forc-tx",
     "forc-test",
     "forc-tracing",

docs/book/spell-check-custom-words.txt

@@ -231,4 +231,9 @@ fmt
 deallocated
 deallocate
 destructors
-destructor
+destructor
+semiautomatically
+FuelLabs
+github
+toml
+hardcoded

docs/book/src/SUMMARY.md

@@ -105,3 +105,4 @@
     - [forc explore](./forc/plugins/forc_explore.md)
     - [forc fmt](./forc/plugins/forc_fmt.md)
     - [forc lsp](./forc/plugins/forc_lsp.md)
+    - [forc migrate](./forc/plugins/forc_migrate.md)

docs/book/src/forc/plugins/forc_migrate.md

@@ -0,0 +1 @@
+# forc migrate

forc-plugins/forc-doc/src/cli.rs

@@ -8,7 +8,7 @@ forc_util::cli_examples! {
         [ Build the docs for a project in the current path and open it in the browser => "forc doc --open" ]
         [ Build the docs for a project located in another path => "forc doc --path {path}" ]
         [ Build the docs for the current project exporting private types => "forc doc --document-private-items" ]
-        [ Build the docs offline without downloading any dependency from the network => "forc doc --offline" ]
+        [ Build the docs offline without downloading any dependencies => "forc doc --offline" ]
     }
 }
 
@@ -35,11 +35,8 @@ pub struct Command {
     /// Meaning it will only try to use previously downloaded dependencies.
     #[clap(long = "offline")]
     pub offline: bool,
-    /// Silent mode. Don't output any warnings or errors to the command line.
-    #[clap(long = "silent", short = 's')]
-    pub silent: bool,
     /// Requires that the Forc.lock file is up-to-date. If the lock file is missing, or it
-    /// needs to be updated, Forc will exit with an error
+    /// needs to be updated, Forc will exit with an error.
     #[clap(long)]
     pub locked: bool,
     /// Do not build documentation for dependencies.
@@ -50,10 +47,11 @@ pub struct Command {
     /// Possible values: PUBLIC, LOCAL, <GATEWAY_URL>
     #[clap(long)]
     pub ipfs_node: Option<IPFSNode>,
-
     #[cfg(test)]
     pub(crate) doc_path: Option<String>,
-
     #[clap(flatten)]
     pub experimental: sway_features::CliFields,
+    /// Silent mode. Don't output any warnings or errors to the command line.
+    #[clap(long = "silent", short = 's')]
+    pub silent: bool,
 }

forc-plugins/forc-fmt/src/main.rs

@@ -7,7 +7,7 @@ use forc_pkg::{
     WorkspaceManifestFile,
 };
 use forc_tracing::{init_tracing_subscriber, println_error, println_green, println_red};
-use forc_util::fs_locking::PidFileLocking;
+use forc_util::fs_locking::is_file_dirty;
 use prettydiff::{basic::DiffOp, diff_lines};
 use std::{
     default::Default,
@@ -101,15 +101,6 @@ fn run() -> Result<()> {
     Ok(())
 }
 
-/// Checks if the specified file is marked as "dirty".
-/// This is used to prevent formatting files that are currently open in an editor
-/// with unsaved changes.
-///
-/// Returns `true` if a corresponding "dirty" flag file exists, `false` otherwise.
-fn is_file_dirty<X: AsRef<Path>>(path: X) -> bool {
-    PidFileLocking::lsp(path.as_ref()).is_locked()
-}
-
 /// Recursively get a Vec<PathBuf> of subdirectories that contains a Forc.toml.
 fn get_sway_dirs(workspace_dir: PathBuf) -> Vec<PathBuf> {
     let mut dirs_to_format = vec![];

forc-plugins/forc-migrate/Cargo.toml

@@ -0,0 +1,25 @@
+[package]
+name = "forc-migrate"
+version.workspace = true
+description = "Migrate Sway projects to the next breaking change version of Sway."
+authors.workspace = true
+edition.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+
+[dependencies]
+anyhow.workspace = true
+clap = { workspace = true, features = ["derive"] }
+forc-pkg.workspace = true
+forc-tracing.workspace = true
+forc-util.workspace = true
+itertools.workspace = true
+num-bigint.workspace = true
+sha2.workspace = true
+sway-ast.workspace = true
+sway-core.workspace = true
+sway-error.workspace = true
+sway-features.workspace = true
+sway-types.workspace = true
+swayfmt.workspace = true

forc-plugins/forc-migrate/src/cli/commands/check.rs

@@ -0,0 +1,88 @@
+use clap::Parser;
+
+use crate::{
+    cli::{
+        self,
+        shared::{
+            compile_package, create_migration_diagnostic, print_features_and_migration_steps,
+        },
+    },
+    get_migration_steps_or_return,
+    migrations::{DryRun, MigrationStepKind},
+};
+use anyhow::{Ok, Result};
+use forc_util::format_diagnostic;
+use itertools::Itertools;
+use sway_core::Engines;
+
+forc_util::cli_examples! {
+    crate::cli::Opt {
+        [ Check the project in the current path => "forc migrate check"]
+        [ Check the project located in another path => "forc migrate check --path {path}" ]
+    }
+}
+
+/// Check the project for code that needs to be migrated.
+///
+/// Dry-runs the migration steps and prints places in code that need to be reviewed or changed.
+#[derive(Debug, Parser)]
+pub(crate) struct Command {
+    #[clap(flatten)]
+    pub check: cli::shared::Compile,
+}
+
+pub(crate) fn exec(command: Command) -> Result<()> {
+    let migration_steps = get_migration_steps_or_return!();
+    let engines = Engines::default();
+    let build_instructions = command.check;
+
+    let mut program_info = compile_package(&engines, &build_instructions)?;
+
+    // Dry-run all the migration steps.
+    let mut check_result = vec![];
+    for (feature, migration_steps) in migration_steps.iter() {
+        for migration_step in migration_steps.iter() {
+            let migration_point_spans = match migration_step.kind {
+                MigrationStepKind::Instruction(instruction) => instruction(&program_info)?,
+                MigrationStepKind::CodeModification(modification, _) => {
+                    modification(&mut program_info.as_mut(), DryRun::Yes)?
+                }
+                MigrationStepKind::Interaction(instruction, _, _) => instruction(&program_info)?,
+            };
+
+            check_result.push((feature, migration_step, migration_point_spans));
+        }
+    }
+
+    // For every migration step, display the found occurrences in code that require migration effort, if any.
+    for (feature, migration_step, occurrences_spans) in check_result.iter() {
+        if let Some(diagnostic) =
+            create_migration_diagnostic(engines.se(), feature, migration_step, occurrences_spans)
+        {
+            format_diagnostic(&diagnostic);
+        }
+    }
+
+    // Display the summary of the migration effort.
+    let features_and_migration_steps = check_result
+        .iter()
+        .chunk_by(|(feature, _, _)| feature)
+        .into_iter()
+        .map(|(key, chunk)| {
+            (
+                **key,
+                chunk
+                    .map(|(_, migration_step, migration_point_spans)| {
+                        (*migration_step, Some(migration_point_spans.len()))
+                    })
+                    .collect::<Vec<_>>(),
+            )
+        })
+        .collect::<Vec<_>>();
+
+    println!("Migration effort:");
+    println!();
+    print_features_and_migration_steps(&features_and_migration_steps);
+
+    Ok(())
+}

forc-plugins/forc-migrate/src/cli/commands/mod.rs

@@ -0,0 +1,3 @@
+pub(crate) mod check;
+pub(crate) mod run;
+pub(crate) mod show;