Strange behavior of looper check

[nsheff]

I'm trying to configure a pipeline to use looper and pipestat. When I run looper check, it does very weird things:

looper check
Looper version: 1.7.0
Command: check
Using looper config (.looper.yaml).
Config file does not have version key. Defaulting to 2.1.0
Config file does not have version key. Defaulting to 2.1.0
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initializing results file '/home/nsheff/code/seqcolapi/analysis/demo/results/pipeline_stats.yaml'
File does not exist, but create_file is true. Creating...
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Initialized looper config file: /home/nsheff/code/seqcolapi/analysis/demo/looper_pipestat_config.yaml
No schema supplied.
Initialize PipestatBackend
Initialize FileBackend
Traceback (most recent call last):
  File "/home/nsheff/.local/bin/looper", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/nsheff/.local/lib/python3.11/site-packages/looper/cli_looper.py", line 782, in main
    return Checker(prj)(args)
           ^^^^^^^^^^^^^^^^^^
  File "/home/nsheff/.local/lib/python3.11/site-packages/looper/looper.py", line 119, in __call__
    for status_id in psm.status_schema.keys():
                     ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'keys'

There are a several issues here:

1. it's not working (there's no 'check' happening)
2. Why is it initialzing pipestat backends once per sample?
3. why is it initializing looper config files once per sample?
4. why is it initializing looper config file at all
5. this is actually creating a new file called "looper_pipestat_config.yaml", which makes absolutely no sense to me.
6. Why is it initializing the looper config file 7 times in a row before it does anything else?
7. can we clarify the 'No schema supplied" error message? No Schema Supplied should offer more information pipestat#157

[donaldcampbelljr]

There are a several issues here:

1. it's not working (there's no 'check' happening)
2. Why is it initialzing pipestat backends once per sample?
3. why is it initializing looper config files once per sample?
4. why is it initializing looper config file at all
5. this is actually creating a new file called "looper_pipestat_config.yaml", which makes absolutely no sense to me.
6. Why is it initializing the looper config file 7 times in a row before it does anything else?
7. can we clarify the 'No schema supplied" error message?

1. Currently, I cannot reproduce this when using hello_looper example (pipestat folder).

2-6. Looper checks for each sample if Pipestat has been configured before proceeding with anything else. If pipestat can be successfully configured, it generates a configuration file to be used by pipestat looper_pipestat_config.yaml which looks something like this:

results_file_path: /home/drc/GITHUB/hello_looper/hello_looper/pipestat/./results.yaml
flag_file_dir: /home/drc/GITHUB/hello_looper/hello_looper/pipestat/./results/flags
record_identifier: frog_2
output_dir: /home/drc/GITHUB/hello_looper/hello_looper/pipestat/./results
schema_path: /home/drc/GITHUB/hello_looper/hello_looper/pipestat/./pipeline_pipestat/pipestat_output_schema.yaml
pipeline_name: test_pipe
pipeline_type: sample

The message output to the terminal is a bit confusing; it should probably be Initialized pipestat configuration file. Originally, it was checking to see if one already existed. However, this caused issues during testing where you had to manually delete the file when changing your pipestat attributes in your looper config file. Basically, I could re-enable this check but then we would still get terminal output for every sample e.g. `Pipestat configuration file already exists".

For point 5 specifically, here is ~ the commit where we added that functionality in Sep: cd31351

7. I'll make an issue in pipestat to offer clarity around this.

[donaldcampbelljr]

I added a small commit to update the message output to be more accurate when initializing the pipestat configuration file.

[donaldcampbelljr]

I believe this issue is going to require quite a bit of refactoring work and thinking about pipestat/looper integration. It should probably be pushed to 1.9.0 or even 2.0.0. This refactoring would also solve #425.

For now, I could suppress the looper_pipestat_config.yaml message printing to terminal. In tandem, I can also apply a fix to pipestat so that pipestat just tells the user it creates the specific backend, e.g.

Initialize PipestatBackend
Initialize FileBackend

vs

Initialize FileBackend

[donaldcampbelljr]

I made those minor changes to Looper and Pipestat, so that the terminal is less verbose when using Looper + Pipestat.

[donaldcampbelljr]

With the work done in #496, this issue should be solved. I've cut down on pipestat's verbosity as well in the upcoming v0.9.2 release.