make-vagrant package fails to build upgrade boxes

[emkll]

Description

Steps to Reproduce

• git checkout 1.0.0
• make vagrant-package

Expected Behavior

The vagrant box should be built successfully

Actual Behavior

Fails at common : Remove generic kernel packages :

failed: [app-staging] (item=linux-image-.*generic) => {"changed": true, "cmd": ["apt-get", "remove", "-y", "linux-image-.*generic"], "delta": "0:00:06.382642", "end": "2019-09-20 17:19:32.600485", "item": "linux-image-.*generic", "msg": "non-zero return code", "rc": 100, "start": "2019-09-20 17:19:26.217843", "stderr": "E: Sub-process /usr/bin/dpkg returned an error code (1)", "stderr_lines": ["E: Sub-process /usr/bin/dpkg returned an error code (1)"]

Comments

This is because the common role is run before the grsecurity role in the upgrade boc building playbook:
https://github.com/freedomofpress/securedrop/blob/develop/molecule/vagrant-packager/playbook.yml#L17

The production and staging playbooks run grsecurity before common.
https://github.com/freedomofpress/securedrop/blob/develop/install_files/ansible-base/securedrop-prod.yml#L54

inverting the order in vagrant-packager resolves locally

[emkll]

Further issues when building with v3 onion services:

    TASK [tor-hidden-services : Generate Onion v3 keys if required the Tails admin system] ***
    changed: [mon-staging -> localhost]
    changed: [app-staging -> localhost]
    
    TASK [tor-hidden-services : Get the v3 keys locally from the Tails admin system] ***
 [WARNING]: Unable to find '/tmp/molecule/securedrop/vagrant-packager/sd-
orig/install_files/ansible-base/roles/tor-hidden-
services/../../tor_v3_keys.json' in expected paths (use -vvvvv to see paths)

    fatal: [mon-staging]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
 [WARNING]: Unable to find '/tmp/molecule/securedrop/vagrant-packager/sd-
orig/install_files/ansible-base/roles/tor-hidden-
services/../../tor_v3_keys.json' in expected paths (use -vvvvv to see paths)

    fatal: [app-staging]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}

I suspect this is because the role path is overriden [1] for in the vagrant-builder scenario, and this path is used to lookup the keys in the playbook when setting up v3 services[2]:

[1]

securedrop/molecule/vagrant-packager/molecule.yml

Line 45 in 9f32d46

ANSIBLE_ROLES_PATH: ".molecule/roles"

[2]

securedrop/install_files/ansible-base/roles/tor-hidden-services/tasks/configure_tor_hidden_services.yml

Line 69 in 9f32d46

v3_local_key_info: "{{ lookup('file', role_path+'/../../tor_v3_keys.json')|from_json }}"

whereas the tor hidden services files are generated in the securedrop root directory:

securedrop/install_files/ansible-base/roles/tor-hidden-services/tasks/configure_tor_hidden_services.yml

Line 56 in 9f32d46

command: "python {{ sd_root_dir }}/admin/securedrop_admin/__init__.py --root {{ sd_root_dir }} generate_v3_keys"