Run IC as non-root

[Rulox]

Proposed changes

Closes #529

This PR makes the changes in all the Dockerfiles (and IC code) to run IC and NGINX as non root.

Changes are:

• Containers will use nginx as user instead of root
• IC and NGINX will run using the nginx user
• Changed the pid, opentracing config and unix sockets to /var/lib/nginx
• Used setcap so we can still use ports 80 and 443 without being root
• Removed user directive in NGINX conf (not needed anymore as we are running NGINX as nginx user)
• Added security policies to IC Pod in all manifests (and helm manifests)

PS: I have arranged the Dockerfiles to keep consistency (order, number of layers, etc).

Thanks to @LorcanMcVeigh as he started working on this.

[Rulox]

Checking CI :)

[pleshakov]

@Rulox thanks. please see my comments

[pleshakov]

👍

[tellet]

Looks good