Bug: Paths for deploy command flags conflict

[AlexBurkey]

Description:

I'm attempting to specify both a --config-file and --template-file in different directories. Both files exist, but if the template file is in a different directory, that seems to become the working directory.

Working commands:
sam deploy --config-file samconfig.foo.toml --template-file foo-template.yaml
sam deploy --config-file ../../samconfig.foo.toml --template-file path/to/foo-template.yaml

Command fails:
sam deploy --config-file samconfig.foo.toml --template-file path/to/foo-template.yaml

Error message: Error: Config file samconfig.foo.toml does not exist or could not be read!

Steps to reproduce:

1. Create a config file in your working directory
2. Create a template file in a different directory
3. Attempt to run a sam deploy command specifying the locations of both

Observed result:

2024-10-16 10:26:28,535 | Config file samconfig.foo.toml does not exist or could not be read!                                                                                                                                     
Error: Config file samconfig.foo.toml does not exist or could not be read!

Expected result:

The deploy is able advance with the proper configs. I was unable to find any info in the docs saying that specifying the --template-file command would change the working directory.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

sam --info
{
  "version": "1.98.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-14.6.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "Not available",
    "aws_cdk": "Not available",
    "terraform": "1.9.7"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Add --debug flag to command you are running

[mndeveci]

Thanks for reporting this issue.

samconfig.toml file was always designed to be used in a relative folder with the template.yaml file. This can be seen in the comment below;
https://github.com/aws/aws-sam-cli/blob/develop/samcli/commands/_utils/options.py#L90-L92

So when you define your template file location, SAM CLI will look for samconfig.toml (or the overriden config file parameter) relative to that folder. I will communicate with our documentation team to add this information here; https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

[AlexBurkey]

Interesting, I wasn't aware it shouldn't be set to .aws-sam/build either.

This behavior makes sense to me when running sam build but the deploy command is causing us some fits. We're using both python and java. Using the --template-file flag works just fine for the python services since they aren't compiled. Is the intention that we would run a command like sam build --config-file samconfig.foo.toml --template-file path/to/template.yaml and then run sam deploy --config-file samconfig.foo.toml without the --template-file flag so that it uses the one in .aws-sam/build?

I also find it confusing because sam deploy --help says that the flag will use template.yaml by default but that's not the case, it uses .aws-sam/build/template.yaml by default.

[mndeveci]

My recommendation would be, moving template.yaml and samconfig.toml (or samconfig.yaml) files in the root folder of your application. When running any SAM CLI command, you need to change your directory to the root folder of the project. With that, you don't need to provide the location of the --template-file or --config-file unless you want to use a different one in the same folder.

If we change this behavior, we will be breaking existing customers, who already adopted using this way.