Fixing bug where the very first lifecycle pipeline may fail when there is no associated applicaiton repo in the hld (#537)

Co-authored-by: Dennis Seah <[email protected]>

Author: mtarng

docs/commands/data.json

@@ -335,7 +335,7 @@
     "alias": "r",
     "description": "Reconcile a HLD with the services tracked in bedrock.yaml.",
     "options": [],
-    "markdown": "## Description\n\nThe reconcile feature scaffolds a HLD with the services in the `bedrock.yaml`\nfile at the root level of the application repository. Recall that in a\nmono-repo, `spk service create` will add an entry into the `bedrock.yaml`\ncorresponding to all tracked services. When the service has been merged into\n`master` of the application repository, a pipeline (see `hld-lifecycle.yaml`,\ncreated by `spk project init`) runs `spk hld reconcile` to add any _new_\nservices tracked in `bedrock.yaml` to the HLD.\n\nThis command is _intended_ to be run in a pipeline (see the generated\n`hld-lifecycle.yaml` created from `spk project init`), but can be run by the\nuser in a CLI for verification.\n\nFor a `bedrock.yaml` file that contained within the\n`https://dev.azure.com/foo/bar/_git` repository, that has the following\nstructure:\n\n```yaml\nrings:\n  master:\n    isDefault: true\nservices:\n  - path: ./services/fabrikam\n    displayName: \"fabrikam\"\n    k8sBackendPort: 8001\n    k8sBackend: \"fabrikam-k8s-svc\"\n    pathPrefix: \"fabrikam-service\"\n    pathPrefixMajorVersion: \"v1\"\n    helm:\n      chart:\n        branch: master\n        git: \"https://dev.azure.com/foo/bar/_git\"\n        path: stable/fabrikam-application\n    middlewares:\n      - \"\"\nvariableGroups:\n  - fabrikam-vg\n```\n\nA HLD is produced that resembles the following:\n\n```\n├── component.yaml\n└── fabrikam\n    ├── access.yaml\n    ├── component.yaml\n    ├── config\n    │   └── common.yaml\n    └── fabrikam\n        ├── component.yaml\n        ├── config\n        │   └── common.yaml\n        └── master\n            ├── component.yaml\n            ├── config\n            │   └── common.yaml\n            └── static\n                ├── ingress-route.yaml\n                └── middlewares.yaml\n```\n\nWith the `ingress-route.yaml` representing a\n[Traefik2 Ingress Route](https://docs.traefik.io/routing/providers/kubernetes-crd/#kind-ingressroute)\nbacked by a Kubernetes Service, and the `middlewares.yaml` representing a\n[Traefik2 Middleware](https://docs.traefik.io/routing/providers/kubernetes-crd/#kind-middleware)\nthat strips path prefixes.\n\nFor the `bedrock.yaml` shown above, the `ingress-route.yaml` produced is:\n\n```yaml\napiVersion: traefik.containo.us/v1alpha1\nkind: IngressRoute\nmetadata:\n  name: fabrikam-master\nspec:\n  routes:\n    - kind: Rule\n      match: \"PathPrefix(`/v1/fabrikam-service`) && Headers(`Ring`, `master`)\"\n      middlewares:\n        - name: fabrikam-master\n      services:\n        - name: fabrikam-k8s-svc-master\n          port: 8001\n```\n\nAnd the `middlewares.yaml` produced is:\n\n```yaml\napiVersion: traefik.containo.us/v1alpha1\nkind: Middleware\nmetadata:\n  name: fabrikam-master\nspec:\n  stripPrefix:\n    forceSlash: false\n    prefixes:\n      - /v1/fabrikam-service\n```\n\nNote that there exists a third generated file, `access.yaml`. For the above\n`bedrock.yaml`, `access.yaml` contains a single line, which represents a\n[Fabrikate access.yaml definition](https://github.com/microsoft/fabrikate/blob/master/docs/auth.md#accessyaml),\nallowing Fabrikate to pull Helm Charts that are contained within the same\napplication repository:\n\n```yaml\n\"https://dev.azure.com/foo/bar/_git\": ACCESS_TOKEN_SECRET\n```\n\nWhen `fabrikate` is invoked in the HLD to Manifest pipeline, it will utilize the\n`ACCESS_TOKEN_SECRET` environment variable injected at pipeline run-time as a\nPersonal Access Token to pull any referenced helm charts from the application\nrepository.\n"
+    "markdown": "## Description\n\nThe reconcile feature scaffolds a HLD with the services in the `bedrock.yaml`\nfile at the root level of the application repository. Recall that in a\nmono-repo, `spk service create` will add an entry into the `bedrock.yaml`\ncorresponding to all tracked services. When the service has been merged into\n`master` of the application repository, a pipeline (see `hld-lifecycle.yaml`,\ncreated by `spk project init`) runs `spk hld reconcile` to add any _new_\nservices tracked in `bedrock.yaml` to the HLD.\n\nAs of version v0.6.0 and the addition of `spk ring` commands,\n`spk hld reconcile` will add rings to the corresponding application repository\ndirectory in the HLD repository. Additionally, as of v0.6.1, the reconcile\ncommand will also remove rings from the corresponding application repository\ndirectory in the HLD repository.\n\nThis command is _intended_ to be run in a pipeline (see the generated\n`hld-lifecycle.yaml` created from `spk project init`), but can be run by the\nuser in a CLI for verification.\n\nFor a `bedrock.yaml` file that contained within the\n`https://dev.azure.com/foo/bar/_git` repository, that has the following\nstructure:\n\n```yaml\nrings:\n  master:\n    isDefault: true\nservices:\n  - path: ./services/fabrikam\n    displayName: \"fabrikam\"\n    k8sBackendPort: 8001\n    k8sBackend: \"fabrikam-k8s-svc\"\n    pathPrefix: \"fabrikam-service\"\n    pathPrefixMajorVersion: \"v1\"\n    helm:\n      chart:\n        branch: master\n        git: \"https://dev.azure.com/foo/bar/_git\"\n        path: stable/fabrikam-application\n    middlewares:\n      - \"\"\nvariableGroups:\n  - fabrikam-vg\n```\n\nA HLD is produced that resembles the following:\n\n```\n├── component.yaml\n└── fabrikam\n    ├── access.yaml\n    ├── component.yaml\n    ├── config\n    │   └── common.yaml\n    └── fabrikam\n        ├── component.yaml\n        ├── config\n        │   └── common.yaml\n        └── master\n            ├── component.yaml\n            ├── config\n            │   └── common.yaml\n            └── static\n                ├── ingress-route.yaml\n                └── middlewares.yaml\n```\n\nWith the `ingress-route.yaml` representing a\n[Traefik2 Ingress Route](https://docs.traefik.io/routing/providers/kubernetes-crd/#kind-ingressroute)\nbacked by a Kubernetes Service, and the `middlewares.yaml` representing a\n[Traefik2 Middleware](https://docs.traefik.io/routing/providers/kubernetes-crd/#kind-middleware)\nthat strips path prefixes.\n\nFor the `bedrock.yaml` shown above, the `ingress-route.yaml` produced is:\n\n```yaml\napiVersion: traefik.containo.us/v1alpha1\nkind: IngressRoute\nmetadata:\n  name: fabrikam-master\nspec:\n  routes:\n    - kind: Rule\n      match: \"PathPrefix(`/v1/fabrikam-service`) && Headers(`Ring`, `master`)\"\n      middlewares:\n        - name: fabrikam-master\n      services:\n        - name: fabrikam-k8s-svc-master\n          port: 8001\n```\n\nAnd the `middlewares.yaml` produced is:\n\n```yaml\napiVersion: traefik.containo.us/v1alpha1\nkind: Middleware\nmetadata:\n  name: fabrikam-master\nspec:\n  stripPrefix:\n    forceSlash: false\n    prefixes:\n      - /v1/fabrikam-service\n```\n\nNote that there exists a third generated file, `access.yaml`. For the above\n`bedrock.yaml`, `access.yaml` contains a single line, which represents a\n[Fabrikate access.yaml definition](https://github.com/microsoft/fabrikate/blob/master/docs/auth.md#accessyaml),\nallowing Fabrikate to pull Helm Charts that are contained within the same\napplication repository:\n\n```yaml\n\"https://dev.azure.com/foo/bar/_git\": ACCESS_TOKEN_SECRET\n```\n\nWhen `fabrikate` is invoked in the HLD to Manifest pipeline, it will utilize the\n`ACCESS_TOKEN_SECRET` environment variable injected at pipeline run-time as a\nPersonal Access Token to pull any referenced helm charts from the application\nrepository.\n"
   },
   "infra generate": {
     "command": "generate",

src/commands/hld/reconcile.md

@@ -8,6 +8,12 @@ corresponding to all tracked services. When the service has been merged into
 created by `spk project init`) runs `spk hld reconcile` to add any _new_
 services tracked in `bedrock.yaml` to the HLD.
 
+As of version v0.6.0 and the addition of `spk ring` commands,
+`spk hld reconcile` will add rings to the corresponding application repository
+directory in the HLD repository. Additionally, as of v0.6.1, the reconcile
+command will also remove rings from the corresponding application repository
+directory in the HLD repository.
+
 This command is _intended_ to be run in a pipeline (see the generated
 `hld-lifecycle.yaml` created from `spk project init`), but can be run by the
 user in a CLI for verification.

src/commands/hld/reconcile.test.ts

@@ -173,7 +173,7 @@ describe("createAccessYaml", () => {
 });
 
 describe("purgeRepositoryComponents", () => {
-  const fsSpy = jest.spyOn(fs, "unlink");
+  const fsUnlinkSpy = jest.spyOn(fs, "unlink");
 
   beforeEach(() => {
     mockFs({
@@ -214,12 +214,26 @@ describe("purgeRepositoryComponents", () => {
   afterEach(() => {
     mockFs.restore();
     jest.clearAllMocks();
-    fsSpy.mockClear();
+    fsUnlinkSpy.mockClear();
   });
 
   const hldPath = "hld-repo";
   const repositoryName = "bedrock-project-repo";
 
+  it("should return immediately if the repository directory does not exist in the hld", async () => {
+    mockFs({
+      "hld-repo": {
+        config: {
+          "common.yaml": "someconfigfile",
+        },
+        "component.yaml": "somecomponentfile",
+      },
+    });
+    purgeRepositoryComponents(hldPath, repositoryName);
+
+    expect(fs.unlink).not.toHaveBeenCalled();
+  });
+
   it("should invoke fs.unlink for each file in project repository except config files and access.yaml", async () => {
     purgeRepositoryComponents(hldPath, repositoryName);
 
@@ -267,7 +281,7 @@ describe("purgeRepositoryComponents", () => {
   });
 
   it("should throw an error if fs fails", async () => {
-    fsSpy.mockImplementationOnce(() => {
+    fsUnlinkSpy.mockImplementationOnce(() => {
       throw Error("some error");
     });
 

src/commands/hld/reconcile.ts

@@ -115,6 +115,13 @@ export const purgeRepositoryComponents = (
   assertIsStringWithContent(absHldPath, "hld-path");
   assertIsStringWithContent(repositoryName, "repository-name");
 
+  // On very first run of the lifecycle for this repository, the repository directory will not exist. No need to purge.
+  if (!fs.existsSync(path.join(absHldPath, repositoryName))) {
+    logger.info(
+      `repository: ${repositoryName} not found in ${absHldPath}. Will skip deletion step of reconcile.`
+    );
+    return;
+  }
   const filesToDelete = getAllFilesInDirectory(
     path.join(absHldPath, repositoryName)
   ).filter(