Error: error reading configuration: open PROJECT: no such file or directory

[b1zzu]

Bug Report

What did you do?

I'm trying to follow the Generating Manifests and Metadata tutorial, and I've executed the kustomize generator cmd in an empty directory:

operator-sdk generate kustomize manifests

What did you expect to see?

I expected the cmd to ask for the missing information and generate the kustomization.yaml and the .clusterserviceversion.yaml files

What did you see instead? Under which circumstances?

The cmd returned this error:

Error: error reading configuration: open PROJECT: no such file or directory

Environment

Operator type:

/language helm

Kubernetes cluster type:

$ operator-sdk version

operator-sdk version: "v1.0.1", commit: "4169b318b578156ed56530f373d328276d040a1b", kubernetes version: "v1.18.2", go version: "go1.13.15 linux/amd64", GOOS: "linux", GOARCH: "amd64"

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.0+b3b92b2", GitCommit:"b3b92b2", GitTreeState:"clean", BuildDate:"2020-07-15T09:27:21Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"linux/amd64"}

[camilamacedo86]

Hi @b1zzu,

Shows that you are trying to use SDK CLI version 1.0+ with a project which was not migrated to the new layout. You need to migrate your projects see: https://sdk.operatorframework.io/docs/building-operators/helm/migration/

[camilamacedo86]

Closing it as solved. However, please feel free to ping us to re-open if you need it.

[joelanford]

Re-opening for a bit more discussion because I've seen this question come up a few times.

@camilamacedo86 Can we investigate why that command needs a PROJECT file? I wonder if there's a workaround we could suggest or a change we could make to relax this requirement.

[camilamacedo86]

Hi @joelanford,

It is fine. See that the PROJECT file is a requirement for the CLI and then, IMO it should NOT change to keep the consistency across the types as well. It has the project config that is used for example to run the commands such as:

• The project layout: Without it, we do not know if is Helm/Ansible/Go and the plugin version
• The project name: required for the features to integrate the project with OLM.
• The version (project): to allow what plugins are supported or not and how the tool should work with. See that we can have diff behaviours based on the layout of the PROJECT version as well.

In the f2f, we spoke about re-create the projects based on the PROJECT file which could bring facilities such as provide a tool to make easier the process to migrate the versions. By storing all data in the PROJECT file it could be easily achieved.

PS.: Change it would probably mean no longer be able to use the CLI implementation from kb/upstream.

[joelanford]

Some SDK subcommands don't require the PROJECT file (e.g. operator-sdk bundle validate and operator-sdk run bundle). I'm curious what specifically the generate kustomize manifests subcommand needs it for?

Is there a workaround we can suggest? (e.g. create a PROJECT file and add some minimal set of info that is required).

One of the goals of operator-sdk v1.0 was to make many of the extra subcommands usable outside the standard project structure, so I'm wondering if we think this command is inextricably tied to the project structure or if it could be run standalone. Looking at the flags exposed by the generate kustomize manifests subcommand leaves the impression that it is intended for it to be able to be used in other project structures.

[camilamacedo86]

Hi @b1zzu,

I got the @joelanford point. If you are looking for only run operator-sdk generate kustomize manifests to generate the CSV for your project then the workaround so far would create a file in the root dir of your project like:

$ cat PROJECT 
version: 2

However, if you would like to use all commands/features described in the docs you still need to migrate your project for the new layout: https://sdk.operatorframework.io/docs/building-operators/helm/migration/.

Also, after chatting with @joelanford we might want to facilitate it for no longer be required the workaround so let's use this issue to track this need and see how we can approve it.

[estroz]

The config's projectName and resources fields are used by all generate commands in different capacities. This doesn't need to be the case. @anmol372 is working on making this dependency optional.

/assign @anmol372
/language go
/language ansible
/kind feature

[openshift-ci-robot]

@estroz: GitHub didn't allow me to assign the following users: anmol372.

Note that only operator-framework members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

The config's projectName and resources fields are used by all generate commands in different capacities. This doesn't need to be the case. @anmol372 is working on making this dependency optional.

/assign @anmol372
/language go
/language ansible
/kind feature

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[anmol372]

Hi,
I am working on refactoring the generate subcommands to support building a CSV, outside of a project.
Please assign this issue to me.

[camilamacedo86]

Hi @estroz,

Just to clarifies based on the tests made so far:

• If we add version: 2 the command is getting the projectName by getting the dir directory name. The same is not true when the version is v3-alpha.
• The resources have been not used in helm to gen the CSV. Should it be used? If yes, then shows that we have a bug.
• Also, if teh version is v3-alpha than the layout is requested.

Regards the solution, I have a concern and a point to raise.

See that if the layout is not in the PROJECT file the stamp used will be : operators.operatorframework.io.metrics.project_layout: go

In this way, one of the possible solutions that I'd like to suggest is we ask for the user input the type when the PROJECT file is not found such as we do to ask if the users would like to create the controller and the resource when we run create api:

$ operator-sdk generate kustomize manifests
Informe your project language [helm/ansible/go/other]

c/c @joelanford

[jmrodri]

@joelanford one thing I noticed is that just about everything looks at the PROJECT file first, like operator-sdk version. The version command code does not, I think the cli that configures the plugins is causing this and it is done before all the commands (unless I misread it):

https://github.com/operator-framework/operator-sdk/blob/master/internal/cmd/operator-sdk/cli/cli.go#L50
https://github.com/operator-framework/operator-sdk/blob/master/internal/cmd/operator-sdk/cli/cli.go#L57-L87

I created a PROJECT directory in my homedir. Then ran the following:

$ operator-sdk version
FATA[0000] failed to read config: read PROJECT: is a directory 

version doesn't use anything from PROJECT.

[joelanford]

@jmrodri That error is slightly different than the one in this issue.

If you don't have a PROJECT file at all, the SDK commands that don't actually require anything from it should work. That's what this issue is about.

If you have a PROJECT file, it looks like SDK (via Kubebuilder) expects it to be valid. I don't know if this behavior was intentional or not. If not, we could probably look into ignoring the PROJECT file altogether for ExtraCommands.

[jmrodri]

@jmrodri That error is slightly different than the one in this issue.

If you don't have a PROJECT file at all, the SDK commands that don't actually require anything from it should work. That's what this issue is about.

If you have a PROJECT file, it looks like SDK (via Kubebuilder) expects it to be valid. I don't know if this behavior was intentional or not. If not, we could probably look into ignoring the PROJECT file altogether for ExtraCommands.

So running operator-sdk version always tries to read the PROJECT file if there is one. And if it happens to be a directory which is not a valid PROJECT file, then you get the error I described. And it happens fairly early in the process.

When we call operator-sdk version I see the following happening:

operator-sdk.cli.Run() is called and calls GetPluginsCLIAndRoot() https://github.com/operator-framework/operator-sdk/blob/master/internal/cmd/operator-sdk/cli/cli.go#L49-L72
which calls cli.New: https://github.com/kubernetes-sigs/kubebuilder/blob/master/pkg/cli/cli.go#L86
which in turn calls cli.initialize(): https://github.com/kubernetes-sigs/kubebuilder/blob/master/pkg/cli/cli.go#L99
cli.initialize() fails with the following: https://github.com/kubernetes-sigs/kubebuilder/blob/master/pkg/cli/cli.go#L201

That all happens BEFORE we call cli.Run(): https://github.com/operator-framework/operator-sdk/blob/master/internal/cmd/operator-sdk/cli/cli.go#L51

So I think it's not just "extracommands" that are affected. EVERY command looks at the PROJECT file before it's even run.

[jmrodri]

The TL;DR is that I believe my comment #3998 (comment) is completely related because it's a command that does NOT need to look at the PROJECT file. Or at least it shouldn't FAIL if it's invalid.

[joelanford]

The kubebuilder CLI library needs to read the project file to load the plugin which defines the command. So it makes sense for that file to be read for init and create api, for example.

I agree that we should look to see if we can skip that part for ExtraCommands.

[estroz]

@jmrodri can you file a kubebuilder issue for skipping PROJECT reads for extra commands, and link it here?

[jmrodri]

kubebuilder issue to skip PROJECT read for extra commands: kubernetes-sigs/kubebuilder#1755

[camilamacedo86]

@jmrodri,

I was checking this one further and shows that Kubebuilder does not require the PROJECT file for extra commands at all. See that it will not raise an issue when the file do not exist: https://github.com/kubernetes-sigs/kubebuilder/blob/76e8d21fd0efb89eb7de53bd7b48a1c70c3771d7/pkg/cli/cli.go#L187-L191

The error Error: error reading configuration: open PROJECT: no such file or directory faced when we run operator-sdk generate kustomize manifests came from

operator-sdk/internal/cmd/operator-sdk/generate/kustomize/manifests.go

Lines 97 to 100 in bbc2f25

	cfg, err := projutil.ReadConfig()
	if err != nil {
	return fmt.Errorf("error reading configuration: %v", err)
	}

.

PS.: I am unable to reproduce

$ operator-sdk version
FATA[0000] failed to read config: read PROJECT: is a directory

IMO what we can do here to solve this issue is as described in #3998 (comment) is:

• In the above line when the PROJECT file was not found we can ask for the user what is the type of the project via command line.

$ operator-sdk generate kustomize manifests
Informe your project language [helm/ansible/go/other]

OR

We can fill the stamp with a generic value which means undefined. Then, we do not need the PROJECT file at all.

[estroz]

Closed by #4074 and #4282

[kyguy]

Running into the same issue in operator-sdk version: v1.4.0:

When running

operator-sdk generate kustomize manifests

it returns

Error: error reading configuration: open PROJECT: no such file or directory

Am I doing something incorrectly? Or was it decided that we should create a PROJECT file in the root dir to get around the error? (e.g. echo "version: 2" > PROJECT).

operator-sdk version

operator-sdk version: "v1.4.0", commit: "67f9c8b888887d18cd38bb6fd85cf3cf5b94fd99", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"

kubectl version

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

[estroz]

This was supposed to have been fixed by #4074 but on further inspection was not. I'll patch this.

/reopen
/assign

[openshift-ci-robot]

@estroz: Reopened this issue.

In response to this:

This was supposed to have been fixed by #4074 but on further inspection was not. I'll patch this.

/reopen
/assign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[estroz]

On second thought:

• Refactoring generate subcommands for to support older layouts #4074 did not actually intend to fix this problem for generate kustomize manifests, just for generate bundle and generate packagemanifests
• generate kustomize manifests probably shouldn't work outside of a project directory for a few reasons:
  1. It was originally intended to work within a project since it reads a bunch of metadata from the PROJECT config
  2. It requires API types for certain project types (Go in particular) be present
  3. The other generate commands now build upon an empty CSV manifest, so you can call them outside of a project if you'd like

I'm going to re-close this since generate kustomize manifests is now working as intended. @kyguy feel free to file a feature request if you'd like to see changes to, or have concerns about, what I said above, so discussion can continue.