Dynamic Variables only calculated once if "sh:" is identical

[is-jonreeves]

• Task version: 3.6
• Operating System: Windows 10, Alpine Linux 3.13

It seems like Dynamic Variables that share the same verbatim sh: line will not get recomputed when run in their new context. For example if a global var is set to the current dir, but then a task with dir: ../ re-sets it, this value does not update for that task. If you make a slight tweak to the sh: command, it does work.

Example Taskfile showing the issue

version: '3'
silent: true

vars:
  PROJECT_ROOT:
    sh: echo $PWD

tasks:

  # Use Global Dynamic Var
  example1:
    cmds:
      - echo "Example 1"
      - echo "{{.PROJECT_ROOT}}"

  # Redefine Global Dynamic Var as Local Dynamic Var (with own dir context)
  example2:
    dir: ../
    vars:
      PROJECT_ROOT:
        sh: echo $PWD
    cmds:
      - echo "Example 2"
      - echo "{{.PROJECT_ROOT}}"

  # Create a new Local Dynamic Var (with own dir context)
  example3:
    dir: ../../
    vars:
      REPO_ROOT:
        sh: echo $PWD
    cmds:
      - echo "Example 3"
      - echo "{{.REPO_ROOT}}"

Expected:

Example 1
C:\Users\jon\Projects\taskfile-test
Example 2
C:\Users\jon\Projects
Example 3
C:\Users\jon

Actual:

Example 1
C:\Users\jon\Projects\taskfile-test
Example 2
C:\Users\jon\Projects\taskfile-test
Example 2
C:\Users\jon\Projects\taskfile-test

Workaround

If you tweak the sh: command to be unique each time its used, then it appears to force it to rerun correctly. So something like this will allow it to work:

sh: echo $PWD
sh: echo $PWD''
sh: echo $PWD''''  

Side-Note/Feature on Directories

I'll probably open a feature request for this, but the reason that I am jumping through the above hoops is that there doesn't seem to be an official "special variable" to get the Current Working Directory, which would be really handy. For commands that requires a Abs paths like Docker, obtaining these in a cross-platform manner can be annoyingly repetitive to mange. If there was a "special variable" with the current working directory and it was correctly affected by the dir: within each task, this would help a lot especially with the "slim-sprig" functions.

[weeping-somnambulist]

I'm of the opinion that the second example should work, and that re-initializing a global var in the local scope should be something that is supported.

@andreynering would you agree that a global variable re-initialized as a local variable should be a supported feature? If so I can create a new, simpler issue that we can create a PR against.

@is-jonreeves I agree that some special variables would be desirable, as mentioned in issue #525.

[andreynering]

Hi @is-jonreeves, and thanks for opening these issues!

@SwitchedToGitlab's suggestions make sense, but may not be easy to implement.

---

Note to myself: Another way to fix this would be to allow cache: false setting on vars. This likely wouldn't be a good idea, though, as these vars could run many times without cache, more than once per task, given the current implementation. This would certainly be unexpected to users.

[Porges]

At the very least, dynamic variable caching must take into account the working directory. We have two tasks with different working directories which both use realpath XXXX to resolve file paths. These are both cached to the same value so whichever one runs first sets it and the other one reads it, so one task will always end up with an incorrect value.

[Porges]

See this commit for example problem/fix: Azure/azure-service-operator@f83921d

[ghostsquad]

I kind of like the idea of an "uncachable" variable, because it would act kind of like a function.

Though the solution to this problem may be lazy variables discussed in V4. Basically a variable doesn't get evaluated (nor cached) until the first time it's actually used. Thus, you can define a variable as a global, and it won't even run until it's referenced elsewhere.

Though, this example does bring to light that with lazy variables, you may still want eager evaluation, such as in the example below, where you want to know the PWD of where the taskfile lives, thus, defining it as a global makes the most sense.

vars:
   PROJECT_DIR:
      sh: echo $PWD

In V4, I can look at supporting both eager & lazy functionality. I can also look into "uncachable" variables, which would likely not be a default behavior, but interesting none the less.

[megakoresh]

This issue is still quite prevalent and annoying but discussion around it seems to have died down. Is there solution in sight, or do we need to wait for V4 for this to be fixed?

[mgbowman]

We just ran into this and we found an acceptable workaround. Sharing here in case others come across this ...

    vars:
      ENV_JSON:
        # add the _comment_ string '# {{now}}' to bust dynamic var cache
        # https://github.com/go-task/task/issues/524
        sh: 'task --offline env # {{now}}'