Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli/unstable): add parse() with descriptive schema #4362

Closed
wants to merge 31 commits into from

Conversation

timreichen
Copy link
Contributor

@timreichen timreichen commented Feb 20, 2024

ref: #4272

This PR adds parse() that takes a descriptive schema for parsing args.
This is inspired by rusts clap, yargs and commanderjs.

This needs more tests for edge cases and some discussions about features like

  • Option.value.accepted?: T[] for allowed values as a shortcut for
    `Option.fn: (value) => { if (!accepted.includes(value)) { throw new Error("...")}
  • Option.value.min?: number for a minimum of arguments taken
  • Option.value.max?: number for a maximum of arguments taken

The type generics also need some rework to imply correct types based on Option.default and Option.type for returned result and inside Option.fn(value: T)
Feedback and suggestions are very welcome. @iuioiua @ngdangtu-vn

@github-actions github-actions bot added the cli label Feb 20, 2024
@iuioiua iuioiua added the feedback welcome We want community's feedback on this issue or PR label Feb 20, 2024
@kt3k
Copy link
Member

kt3k commented Feb 24, 2024

This is the 2nd set of API for parsing cli arguments. It feels confusing to have 2 different sets of APIs for the same purpose.

I'd recommend you develop this tool as a 3rd party tool. If that tool got popularity and adoption in the ecosystem, we would be able to consider adopting it in std

@timreichen
Copy link
Contributor Author

timreichen commented Feb 24, 2024

This is the 2nd set of API for parsing cli arguments. It feels confusing to have 2 different sets of APIs for the same purpose.

I agree, two wouldn't make too much sense, though the one currently implemented is based on minimist and is not capable of some use cases, especially comparing it to other poplar tools like cliffy, nor compatible with such a schema based approach (as pointed out in #4272)

I'd recommend you develop this tool as a 3rd party tool. If that tool got popularity and adoption in the ecosystem, we would be able to consider adopting it in std

There are plenty of popular 3rd party tools out there (commander, yargs, cliffy, etc.) that have this declarative approach implemented and are far more popular than minimist. So I think publishing another 3rd party tool wouldn't make much sense to prove that point.

The declarative approach this implements avoids the function call chains and replaces them with native objects and arrays to make it less framework-ish, but is similar to the named modules.
Example:

...
.option("--foo")
.option("--bar")

=>

{ ...
  options: [
    { name: "foo", },
    { name: "bar", },
  ]
}

@andrewthauer
Copy link
Contributor

Deno is fantastic tool for writing scripts and CLIs in general. I've been using it a lot lately and find that I can get a lot of mileage using just Deno + @std. The only other libraries I typically reach for are dax & cliffy.

For parsing args, the @std parseArgs works in a pinch for very basic usage. However, I don't find the API overly intuitive and quite simplistic for anything but very basic scripts where I don't feel the need to provide a nice DX on top.

The direction @timreichen is taking here would be much more preferred imo then the existing parseArgs. I believe it would help bridge the gap between very simplistic scripts/CLIs and a more advanced CLI which I'd typically reach for cliffy to implement parsing with.

@kt3k
Copy link
Member

kt3k commented Apr 30, 2024

There are plenty of popular 3rd party tools out there (commander, yargs, cliffy, etc.) that have this declarative approach implemented and are far more popular than minimist.

How do you measure the popularity of packages? yargs has 72M weekly downloads, and minimist has 44M. I think these are similarly popular.

Also I don't see what declarative approach means. minimist (and therefore parseArgs of std/cli) accepts the entire parameter all at once as an option object. There's nothing procedural here. I would say minimist also has 'declarative' API.

So I think publishing another 3rd party tool wouldn't make much sense to prove that point.

Without being tested as 3rd party library, how can we be convinced that this design is better than minimist?

@timreichen
Copy link
Contributor Author

timreichen commented May 1, 2024

Sidenote: What I mean by declarative approach

I mean having all declarations in one object structured in a certain way:

Option properties

While minimist declares options in an object, it is done so by splitting the declaration by property.

property->objectName

parseArgs(Deno.args, {
  boolean: ["color"], // type is declared here
  default: { color: true }, // default value is declared here
  negatable: ["color"], // negatable is declared here
});

commanderjs, yargs and friends do it the other way around, where all properties of an option are gathered in one object (or in commanderjs case in one command).

object->properties

parse(Deno.args, {
  options: [
    { name: "color", type: Boolean, default: true, negatable: true } // all option properties declared in one object
  ]
});

Subcommands

As explored here, minimist does not support subcommand parsing and a possible implementation would lead to messy code due to the nature of property->objectName.

Named values

Since there is no option object but a collection of properties declared, it is not possible to have named values for options. This is a problem, when one wants to be able to print help for a cli.

property->objectName

???

object->properties

parse(Deno.args, {
  options: [
    { name: "foo", value: { name: "VALUE" } }
  ],
});

How do you measure the popularity of packages? yargs has 72M weekly downloads, and minimist has 44M. I think these are similarly popular.

Yes, but commanderjs has 141M weekly downloads. So I would say together with yargs, that this kind of approach is much more popular.

Without being tested as 3rd party library, how can we be convinced that this design is better than minimist?

I was talking about the fact that commanderjs and other libs that follow such an implementation are already used extensively and are more popular than minimist (by weekly downloads).
Some functionality like subcommands are apparently not possible with minimist.

@kt3k
Copy link
Member

kt3k commented May 1, 2024

Since there is no option object but a collection of properties declared, it is not possible to have named values for options. This is a problem, when one wants to be able to print help for a cli.

I don't see this argument very well. --help is usually done like the below with the current parseArgs:

import { parseArgs } from "jsr:@std/cli/parse-args";

const args = parseArgs(Deno.args, {
  boolean: ["help"],
});
// This becomes { _: [], help: true }, { _: [], help: false }, etc.

if (args.help) {
  console.log("Usage");
  Deno.exit(0);
}

Isn't { _: [], help: true } named option?

Also I don't see well what { name: "VALUE" } part does in your example..

I was talking about the fact that commanderjs and other libs that follow such an implementation are already used extensively and are more popular than minimist (by weekly downloads).

But this suggested API design isn't similar to commander nor yargs at all. It's completely an invented design, which is not tested anywhere.

@kt3k
Copy link
Member

kt3k commented May 1, 2024

@timreichen BTW what do you think about util.parseArgs of Node.js, which is also available in Deno via node:util module

@timreichen
Copy link
Contributor Author

@timreichen BTW what do you think about util.parseArgs of Node.js, which is also available in Deno via node:util module

I think it is very bare-bone. But I like how one defines the options as an object with properties.

Copy link

codecov bot commented May 13, 2024

Codecov Report

Attention: Patch coverage is 86.78414% with 30 lines in your changes missing coverage. Please review.

Project coverage is 96.20%. Comparing base (d93b33a) to head (a4a7566).

Files Patch % Lines
cli/parse.ts 86.78% 29 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4362      +/-   ##
==========================================
- Coverage   96.26%   96.20%   -0.06%     
==========================================
  Files         470      471       +1     
  Lines       38243    38470     +227     
  Branches     5546     5612      +66     
==========================================
+ Hits        36813    37011     +198     
- Misses       1389     1417      +28     
- Partials       41       42       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@denoland denoland deleted a comment May 31, 2024
@kt3k kt3k mentioned this pull request Jun 10, 2024
6 tasks
Comment on lines +376 to +384
await t.step("strip single quoted value", () => {
const expected = { options: { foo: "bar" }, arguments: {} };
const actual = parse(["--foo", "'bar'"], {
options: [
{ name: "foo", value: { name: "VALUE" } },
],
});
assertEquals(actual, expected);
});
Copy link

@phaux phaux Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Does the current parseArgs do this? Bash will already remove the quotes. You can just leave them if someone passes them anyway by escaping them.

It reminds me of PHP automatically escaping and unescaping random strings without my consent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, fair point.

Comment on lines +143 to +146
const expected = { options: { foo: ["bar", "baz"] }, arguments: {} };
const actual = parse(["--foo", "bar", "baz"], {
options: [{ name: "foo", value: { name: "VALUE", multiple: true } }],
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect to require passing --foo bar --foo baz here. The current parseArgs works that way.

Same with all the other cases with multiple.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point.

Deno.test("parse() handles commands", async (t) => {
await t.step("fn() is called", () => {
let called = false;
parse(["run", "--foo"], {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think parse with command should return which command was called.

This parse should return { command: ["run"], options: {...}, arguments: {...} }.

Other examples:

  • [] - no command was called.
  • ["foo"] - foo command was called.
  • ["foo", "bar"] - a nested command was called.

You would be able to discriminate on this property to get the correct types of options for a particular command.

Comment on lines +15 to +20
value?: {
name: string;
optional?: boolean;
multiple?: boolean;
requireEquals?: boolean;
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this nesting is necessary and the value's name field can be removed (does it even do anything?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below.

Comment on lines +24 to +30
export interface Argument<T = unknown> {
name: string;
description?: string;
multiple?: boolean;
optional?: boolean;
fn?: (value: T) => T | void;
}
Copy link

@phaux phaux Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 things:

  1. Do the positional arguments need a name? I guess it's useful for generating help later, but they might as well be returned as a tuple. I think it's simpler.

  2. Maybe instead of multiple and optional flags on arguments, have separate lists of requiredArguments, optionalArguments, and a single restArgument property on Command. That way it can be enforced that required arguments always come first and there's at most one rest (multiple) argument at the end. It doesn't make sense to have them in any other order.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. That is exactly right, the name property as well as the nesting is intended for the (future) stringification of an option.
  2. That gets more complicated, if one intends to extend the functionality with lets for example standalone options or conflictingWith options. So I think it is good to have these properties in the option object itself.

Comment on lines +32 to +39
export interface Command<T> {
name: string;
description?: string;
options?: ReadonlyArray<Option<T>>;
commands?: ReadonlyArray<Command<T>>;
arguments?: ReadonlyArray<Argument>;
fn?: (result: ParseResult) => void;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe consider options and commands to be a Record where keys are names?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have considered that, but if an option doesn't have properties, They would need to be declared with an empty object, which seems strange:

parse(["--foo"], { options: { foo: {} } })
parse(["run", "--foo"], {
  commands: {
    run: { options: { foo: {} } }
  }
});

Or is there a better way?

@kt3k kt3k changed the title feat(cli): add parse() with descriptive schema feat(cli/unstable): add parse() with descriptive schema Aug 8, 2024
@kt3k
Copy link
Member

kt3k commented Aug 8, 2024

types.ts file seems unnecessary to me as the types in it are not shared with other scripts. I think that should be merged in parse.ts

Can you also write some short usage guide which showcases 2, 3 typical usages of parse

@iuioiua
Copy link
Contributor

iuioiua commented Aug 8, 2024

I'll be closing this without merge. parseArgs() is one of the most widely used functions in the Standard Library. Swapping it out with a new implementation presents issues centered around disruption (I'm sure they're obvious, so I won't list them). We're currently focusing on Deno 2, including polishing the Standard Library up for it. So we need to be quite selective on major additions.

I think your PR is heading in the right direction, and the design seems superior to the current implementation. However, the case for the addition must be sufficiently strong for the Deno core team to accept. So I strongly recommend porting this implementation to your own package, and making the design, implementation, documentation, and testing as best as possible. Users should feel a tangible benefit from migrating too. It's probably best done accomplished with others. Then, once ready, present it at sometime in the future as a PR.

That all said, we greatly appreciate the thought and effort you've gone through with this, and other, PRs. Thank you very much.

@iuioiua iuioiua closed this Aug 8, 2024
@timreichen timreichen deleted the cli_parse branch December 19, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli feedback welcome We want community's feedback on this issue or PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants