Consider removal of project cli.py

[antonymilne]

Just some rough notes for now; details to be added when I get a chance...

• does anyone use cli.py instead of writing a plugin? Does anyone do anything other than add new commands or override run?
• consider deployment runners and the pattern they follow, which would much more conveniently be done using Universal Kedro deployment (Part 3) - Add the ability to extend and distribute the project running logic #1041
• needs to be done relatively soon as would need to add deprecation warnings
• this would make achieving Improve kedro run as a package #1423 much simpler

My ultimate hope here is that we can remove the project cli.py and implement something like #1041, while making #1423 much easier.

[datajoely]

If you want to extend the run command with extra click inputs I think this is the only way. What I would like is if we go down the kedro init route we could generate one if people need it.

I think from a user point of view copy and pasting something that works is much less intimidating that creating a plugin. By that logic, I'd also be open to kedro init plugin as a different way of people doing this.

[antonymilne]

Currently it's the only way without writing a plugin, but how common actually is it to add extra click arguments to run? I consider it kind of niche/advanced functionality. And, out of those cases, how many times do you want to make the change to just one project using cli.py rather than several projects using a plugin? So personally I think it's probably fine to require a plugin to do this. But this issue exists to provoke some varying opinions and discussion and user research on the topic 👍 I think @noklam brought this up originally actually so am very interested in his thoughts here!

Totally agree that whatever we do we should make it easier to actually add those extra click arguments though e.g. through sort of kedro init plugin. The current model of copying and pasting from docs is not great in both the case of making a plugin and how to override cli.py. There's many gotchas, it's hard to keep up to date, and it's very easy to get tripped by both routes at the moment.

[datajoely]

niche/advanced functionality

Until we natively provide a way for people to inject / override catalog entries, we're going to see people doing the same thing

[noklam]

Adding a data point of use of cli.py (feel free to edit)

• Allow injecting data into a KedroSession run #2169 (comment)
• A while ago I also got request for doing a kedro run --exclude_node, which the easiest way to do is just create the cli.py - https://github.com/noklam/kedro_gallery/blob/master/kedro_cli_extension/src/kedro_cli_extension/cli.py

I think this is a pretty advanced use case, but I think you will find it interesting @AntonyMilneQB

[noklam]

AFIAK this topic comes out of #1423 as there is a subtle difference between running kedro as a package VS running kedro via KedroSession, the CLI entry point is one of the differences.

I am closer to @datajoely on this, I don't think we offer a better way to achieve this.

• A file is less intimating, adding one minor argument seems too much overhead for a need of a plugin (In the above case where you actually do a custom runner to launch CLI, it may be worth the effort to maintain it as a separate plugin)
• Distributing a plugin is tricky. Practically you may install from a Git repo directly, but there are many restricted environments doesn't allow you to do so, or you need to go through a lot of steps to set up your JForg etc.
• I quite like the idea of kedro init to generate these template files

How would removing the cli.py make #1423 easier?

[datajoely]

Another vote for kedro init