Replace vagrant-based testing with docker-based testing

[RaghavaAlajangi]

This PR aims to replace vagrant-based testing with docker-based testing. Docker-based testing is useful because it is faster to execute, easier to manage dependencies, and better integrated with CI/CD.

To do:

• Make changes in the CICD file
• Add docker-run-test.sh and docker-compose.ci.yml files
• Tests pass locally using docker
• Update CHANGELOG
• Passed CICD

[RaghavaAlajangi]

Hi @paulmueller
please review my changes and let me know if they need any changes.

[paulmueller]

Congratulations, this is great!

The only thing that I would like to change is the environment file name.

[paulmueller]

I would propose to name this file to docker-env.ci and consequently use it in the entire workflow, i.e.

• remove the "Create env file" step in check.yml
• use that file in docker-compose.ci.yml's env_file key.

The reason is that .env is too generic which might lead to ambiguities.

[RaghavaAlajangi]

Ok, I will change it.

[RaghavaAlajangi]

Aha, this approach has failed. Maybe it is compulsory to have a .env extension!

[RaghavaAlajangi]

Ok, I tested this file (docker-env.ci) manually. The variables are loaded when I run docker compose up but some of them are still set to empty strings. I don't know why.

[RaghavaAlajangi]

We need both .env and env_file variables for the pipeline to be passed. It is explained here. We need .env file for docker-compose and env_file to send variables to the ckan container. The name of the file doesn't matter.

[paulmueller]

You can specify the env file via docker command line: docker-compose --env-file .my-env up -d

From the thread you linked, I read that there is a difference between .env and env_file. Does it mean you have to split the docker configuration from the ckan/postgresql/etc configuration? In that case I would name them docker-compose.ci.env and docker-dcor.env.

[RaghavaAlajangi]

If I understand correctly, you want me to split the docker-ci.env that I have into two files?

But, we don't have to split this file. We have to keep the Create env file step in check.yml. that's the solution because .env is used to set the environment variables for all the container used in compose file. Whereas, env_file is used to pass the variables into the dcor-ckan container. So, basically, we use docker-ci.env file to set both .env and env_file

[paulmueller]

If I understand correctly, you want me to split the docker-ci.env that I have into two files?

This is the question that I asked you 😸

But, we don't have to split this file. We have to keep the Create env file step in check.yml. that's the solution because .env is used to set the environment variables for all the container used in compose file. Whereas, env_file is used to pass the variables into the dcor-ckan container. So, basically, we use docker-ci.env file to set both .env and env_file

You could remove the "Create env file" step and instead specify the env file in docker compose (docker-compose --env-file docker-ci.env up ...), correct?

[RaghavaAlajangi]

sorry for the confusion. Yes, we can set the global variables with docker-compose --env-file docker-ci.env up and remove the "Create env file" step. And, we can use the same file (docker-ci.env) to set the variables for dcor-ckan container as well. Is that ok, or do you still want me to create two different files?

[paulmueller]

Sounds good to me 👍

[RaghavaAlajangi]

Hi @paulmueller ,
I made the changes. Can you have a look?