True graph skeleton as optional algorithm input

[dmachlanski]

In other words, create an undirected graph from the true DAG (or any graph), and pass it to an algorithm as input. Sampled data should be passed as input as well (as per usual). It's important to note that the passed skeleton is the true undirected graph, not an estimate.

The reason is to be able to test pairwise algorithms, such as these.

Usually, pairwise methods are tested only on (X, Y) datasets. But testing them on bigger graphs (nodes > 2) is arguably more interesting and challenging. This, however, requires to provide the algorithms with a starting point in the form of a graph's skeleton. The task then boils down to orient the edges. The final product is a fully oriented graph (can have cycles), so most, if not all, of the existing metrics can be used without issues.

For an example, see [1] section 5 (5.2 and 5.4 specifically).

[1] O. Goudet, D. Kalainathan, P. Caillou, D. Lopez-Paz, I. Guyon, and M. Sebag, ‘Learning Functional Causal Models with Generative Neural Networks’, Springer International Publishing, 2018. doi: 10.1007/978-3-319-98131-4.

[felixleopoldo]

That sounds like a good idea! The true graph is already part of, the output.adjmat using the function

benchpress/workflow/rules/filename_patterns.py

Line 37 in 2396976

def alg_output_adjmat_path(algorithm):

The data wildcard pattern matches everything that was used to generate the data, i.e. graph, parameters, and data modules. However this should be splitted as is done in e.g. this function

benchpress/workflow/rules/filename_patterns.py

Line 59 in 2396976

def time_path(algorithm):

At least the {data} wildcard will have to be renamed so that it doesn't clash with the current one. Once the {adjmat} field is accessible as a wildcard, you use it to create is as a new input variable on the format in e.g.

benchpress/workflow/rules/graph/pcalg_randdag/rule.smk

Line 3 in 2396976

adjmat = "{output_dir}/adjmat/" + pattern_strings["pcalg_randdag"] + "/seed={replicate}.csv"

where pattern_strings["pcalg_randdag"] is replaced by {adjmat} (I think that is better than taking the raw wildcard {adjmat} string in the script).