Support the JSON key file format that solana-keygen uses

[mikemaccana]

Motivation

The Solana CLI and docs use a file format which has a single secret key inside a JSON file. web3.js doesn't currently directly support these files.

This enhancement adds Keypair.fromFile() to load the Solana CLI secret key files.

Example use case

A user is using the solana CLI to perform a task. The user has created a secret key, by default, in ~/.config/solana/id.json and used this via Solana CLI. They then wish to use this same keypair in web3.js.

Right now the user has to work out how to do this themselves.

This proposal makes web3.js support the Solana CLI JSON files itself.

Details

This would work:

import { Keypair } from "@solana/web3.js";
import { readFile } from "fs/promises";
import path from "path";

// @ts-ignore (just because we haven't updated types yet)
Keypair.fromFile = async (filepath?: string) => {
  if (!filepath) {
    // Default value from Solana CLI
    filepath = "~/.config/solana/id.json";
  }
  if (filepath[0] === "~") {
    const home = process.env.HOME || null;
    if (home) {
      filepath = path.join(home, filepath.slice(1));
    }
  }

  const fileContents = (await readFile(filepath)).toString();
  return Keypair.fromSecretKey(Uint8Array.from(JSON.parse(fileContents)));
};

Then to use, just:

const keyPair = await Keypair.fromFile();

or:

const keyPair = await Keypair.fromFile('somefile.json');

or:

const keyPair = await Keypair.fromFile('~/code/solana/demos/steve.json');

[steveluscher]

One reason not to do this is that @solana/web3.js is supposed to be multiplatform (web/server/mobile) and this only works on the server.

Two thoughts:

1 – This already sort of works in Node:

const SECRET_KEY_BYTES = require('/home/me/.config/solana/id.json');

const { Keypair } = require('@solana/web3.js');

console.log(Keypair.fromSecretKey(new Uint8Array(SECRET_KEY_BYTES)));

2 – We could offer a separate npm package of server utilities that did this kind of stuff, as wrappers around @solana/web3.js, so that we don't create confusing APIs that only work on a subset of web3.js' target platforms.

[mikemaccana]

Understood re: making sure web3.js works everywhere (web/server/mobile). The second suggestion - having server-specific functions in a separate package - sounds good. I personally have a bunch of server-specific helpers, I know @nickfrosty does too, I wouldn't be surprised if many Solana users are effectively rewriting the same things, and we should save them the effort.

Having this kind of thing in a separate server package means:

• We support the file format the CLI tools use, out of the box
• People don't know to know that it's possible to import JSON (which may not be common knowledge) or learn how to import JSON in ES6 (import * as SECRET_KEY_BYTES from '/home/me/.config/solana/id.json'; does it), handle homedir expansion, etc.

[mcintyre94]

I definitely agree that it'd be nice to have these sorts of helpers in a package somewhere. Would be useful for teaching too! Maybe a new repo under solana-developers org?

[steveluscher]

I like that idea. My hope is that many such packages will spring up around the new web3.js, because its API is so flexible. It would be difficult to do such a thing without hacking or subclassing the old Keypair class, but it's very easy to do when everything is a function that operates over some data.

[mikemaccana]

Actually making this (we have a lot of noise right now in some courses due to adding functions that should be in a package somewhere). Will link here once I get solana-developers access.

[mikemaccana]

@steveluscher Done. Want to review? solana-developers/helpers#1

[nickfrosty]

@steveluscher or @mikemaccana I recommend closing this issue since the @solana-developers/node-helpers package is live

[github-actions]

Because there has been no activity on this issue for 7 days since it was closed, it has been automatically locked. Please open a new issue if it requires a follow up.