Generate getops string from docopt

[jasonkarns]

I love docopt but there are certain scenarios where I can't leverage docopt for the parsing, but still use docopt for documenting a script.

In these scenarios, I frequently need to use getopts for actual option parsing. It would be awesome if one could generate the getopts string from docopt. (Or vice versa.)

This way I could generate the string and either paste it into my getopts usage, or at least compare and verify that my docopt is documenting correctly per my getopts string.

Of course, getopts doesn't support long flags. So attempting to generate getopts string from docopt which uses long flags would be an error.

[CristianCantoro]

(My2cents, I am not the maintainer of this projects)
There is a part of me that thinks that this would be out of scope for docopts. I mean, it's a cool project and definitely useful, but I am not entirely convinced by the idea that this should be part of docopts and not a separate tool, say doc2getops.

[jasonkarns]

Yeah, I could see it as a separate tool. My main rationale for wanting it built in is that 99% of the work for such a feature is going to be parsing the docopt. Generating a getopts string from the "known/parsed" options structure is near trivial at that point (considering how limited the getopts features are)

And I'd worry that a completely separate tool would deviate from proper docopt parsing too much and cause problems. It might also be worthwhile as a tiny feature for the website itself. That would scratch my itch, hopefully keep the docopt parsing where it belongs, and not expand the scope of any one docopt tool. 🤷‍♂️

[CristianCantoro]

@jasonkarns, I agree entirely, but then understanding which are the right boundaries for a tool is an art.

[Sylvain303]

@jasonkarns, yes interesting. Could you provide some fake bash scrips that would contain your expected code usage?

My oldstyle getopts scripts looks like (the example may not work, it's just for a sample):

usage() {
  cat << EOT
Usage:
  banip [-w|-r|-s|-u] IP
  banip -l

Options:
    -h  display this help
    -s  display IP status
    -u  unban given IP
    -w  add IP to whitelist
    -r  remove IP from whitelist
    -l  list all banned IPs
EOT
}

while getopts "hs:u:w:r:l" OPTION
do
  case $OPTION in
    h)
      usage
      exit 0
      ;;
    s)
      status=true
      the_ip=$OPTARG
      ;;
    u)
      unbanip=true
      the_ip=$OPTARG
      ;;
    w)
      whitelist_ip=$OPTARG
      ;;
    r)
      unwhitelist_ip=$OPTARG
      ;;
    l)
      list_ips=true
      ;;
    *)
      exit 1
      ;;
    
  esac
done
shift $(($OPTIND - 1))
# do actions after

How would you think to use docopts as getopts string generator?
I mean, your are talking about using docopts as an helper, no more as part of the code?
or is it something else?

[jasonkarns]

@Sylvain303 I mean completely outside the code.

If it were part of a CLI, my ideal usage might be something like:

$ docopt <my_bash_script>
hs:u:w:r:l

So it would read in the bash script, parse out the docopt, and then print the corresponding getopts string to STDOUT. That way I could just copy/paste the getopts string into the script.

Or alternatively, I could just paste my docopt into the http://try.docopt.org/ website, and expect to see the corresponding getopts string printed on the site somewhere. (Where I'd just copy/paste the getopts string into my script.)

My rationale for this is in scenarios where including docopt itself directly is difficult (or limited for other reasons). So it wouldn't be generating the getopts string "live" as part of the code. If I could use docopt in my script directly, then I wouldn't have to use getopts at all. I'd be able to use docopt for the arg parsing itself.

[Sylvain303]

@jasonkarns OK. So we are in the case where docopts can't be directly copied to the server or for limited reason you cannot use it on the server. I don't speak about workstation I see no reason... freespace, no?

The first reason shouldn't be one, due to the golang version which should be able to run anywhere, and cross-compile from every serious compile environment, or VM.

For the second, I guess it's a concern about disagree to use docopt parsing behavior: you don't have the right or the will to change the actual options parsing behavior inside a legacy script. Or the laziness to script a deployment to a old server or something about the permission to do it (falling back to case 1)?

docopt lib has some parsing "misbehavior" that I need to script and provide a bug some where. May you have reached those too?

If we (I) code the idea to provide an helper for outputting getopts string, it could be:

docopts --gen-getopts  --auto-parse my_bash_script
# or
docopts --gen-getopts -h "$my_help_msg"

given the actual supported options.

But, may be we should encourage to use docopts and fix docopt parser instead of using our time to provide workaround?

Could you provide example of the limited case, please?

[Sylvain303]

@jasonkarns any news about your request?

Could you explain more?

My rationale for this is in scenarios where including docopt itself directly is difficult (or limited for other reasons). So it wouldn't be generating the getopts string "live" as part of the code. If I could use docopt in my script directly, then I wouldn't have to use getopts at all. I'd be able to use docopt for the arg parsing itself.

What scenarios make « including docopt itself directly is difficult »?

When you mention docopt, without S, is it really docopts binary?

Or are you talking about the docopt lib python or other thing?

For me, docopt without the S, stand for the library implementing docopt parsing. Which is available in many languages. And docopts with as S names the binary program designed for GNU/bash in order to bring docopt parsing available to bash.

See also #9 for more exchange on the standalone discussion.

If you think that your request is no more necessary, could you close this issue, please.

[jasonkarns]

My use cases are the following:

development-related scripts (ie, those that live in script/ sub directory for all kinds of libraries/modules for versioning, releasing, running tests, etc). Since only maintainers will need to run these scripts, it is acceptable to require the docopts be available. However, a good chunk of the apps/libs in question are managed via npm (they aren't JS, but leverage npm as the script runner and for dep management). In this case, if I could have docopts binary added as a dep via npm, then 👍

rbenv-* and nodenv-* (the core tools as well as a handful of plugins to each). These are installed by users and run on their workstations. No dependency manager can be assumed and these tools need to be self contained. It's not acceptable to require the users to first install docopts binary. But this is the case where the option parsing is typically done with getopts

last case is understandably rare... I maintain a few bats related libraries that contain helpers (mocking, assertion helpers, etc) for the BATS bash testing tool. Most of these libraries are bash functions. I leverage docopt to describe the function signature. (It's a fantastic way to conform to conventional docs.) But these functions are sourced into BATS and I'm not happy with also requiring docopts binary being available to the user's test suite. (Again, for dep management concerns. Users of bats might be testing bash utilities; but they may also use bats for testing ruby or js clis. In those cases, they're probably using npm or rubygems for their deps. Getting docopts binary available to bats outside of the user's existing dep management tool is not trivial.) But being able to document the helper functions' signature with docopt, and then automatically generate the getopts string, to paste into the function would be 🔥

[Sylvain303]

@jasonkarns thanks for the detailed use case.

I'm not convinced by the argument "No dependency manager can be assumed and these tools", because I come from sysadmin world, and I think the every tool should be packaged. So I feel installing a software with curl URL | bash - is an security abomination, especially if you add sudo too.

I must admit that docopts as no package yet.

That said, I also understand the need to go "without" dependency. But, it tends to produce strange code.

I will work on the docopt lib parser internal, in a near future, so I will be more accurate on parsing.
I will probably produce some debug output suitable for your usage. I will keep you informed.

I also created #22 for packaging this tool, so it would become easy to install it as part of other tools. ;-)

Thanks for your contribution.

[jasonkarns]

So I feel installing a software with curl URL | bash - is an security abomination, especially if you add sudo too.

Agree. Mac users typically install the rbenv/nodenv stuff with homebrew. Most linux installs are by a git clone + install.sh script. Rbenv and others are available via other means. But as the maintainers of the tool, we can't assume any particular method of installation.

That said, I also understand the need to go "without" dependency. But, it tends to produce strange code.

Indeed. But for my purposes, most of my usecases aren't "big enough" to warrant the dep. But at the same time, I like the clarity and consistency of docopt style. So I continue to use docopt for documentation/usage/help. But hand-code the parsing. (As you can imagine since the parsing is typically done with getopts, the options are fairly limited so docopts wouldn't really be providing that much value for these scenarios. A bigger options surface area would increase the value of docopts and therefore make it worth the cost of adding a dep.