Minor fig up. Fixed dep_troubl. Conda in readme

fercrcode · fercrcode · commit d6d804378ed2 · 2021-01-14T10:53:24.000Z
diff --git a/.gitignore b/.gitignore
@@ -158,3 +158,4 @@ Preprocessed_Data/PDO21_UNTrep2/Pro_20200109__EGF__Test__P21_0ng_2.M__Phase.txt
 Preprocessed_Data/PDO21_UNTrep2/Pro_20200109__EGF__Test__P21_0ng_2.S__Phase.txt
 Preprocessed_Data/maria_bothtxtfcs.7z
 Analysis/EMD_output/UNNAMED/
+Allpackages_includingbaselibs.csv
diff --git a/README.md b/README.md
@@ -1,29 +1,30 @@
 [![Documentation Status](https://readthedocs.org/projects/cytof-dataanalysis/badge/?version=latest)](https://cytof-dataanalysis.readthedocs.io/en/latest/?badge=latest)
 # **Cy**TOF Si**gn**alling An**al**ysis (*CyGNAL*)
 
-Repository of the [Cell Communication Lab](http://tape-lab.com/) at UCL's Cancer Institute. The Cell Communication Lab studies how oncogenic mutations communicate with stromal and immune cells in the colorectal cancer (CRC) tumour microenvironment (TME). By understanding how mutations regulate all cell types within a tumour, we aim to uncover novel approaches to treat cancer.
-
-In this repo we present *CyGNAL*, a pipeline for analysing mass cytometry data similar to that used in our *Nature Methods* paper: [Cell-type-specific signaling networks in heterocellular organoids](https://www.nature.com/articles/s41592-020-0737-8). With code in both Python and R, CyGNAL assumes some preliminary and inter-step processing through the platform [Cytobank](https://cytobank.org/) (although the user could in theory use any other solution for this and the gating steps).
+In this repository we present *CyGNAL*, a pipeline for analysing mass cytometry data similar to that used in our *Nature Methods* paper: [Cell-type-specific signaling networks in heterocellular organoids](https://www.nature.com/articles/s41592-020-0737-8). With code in both Python and R, CyGNAL assumes some preliminary and inter-step processing through the platform [Cytobank](https://cytobank.org/) (although the user could in theory use any other solution for this and the gating steps).
 
-Overview of the current workflow:
+Overview of CyGNAL (dashed blue line) within a standard mass cytometry analysis:
 ![alt text][Overview]
 
-[Overview]: https://github.com/TAPE-Lab/CyGNAL/blob/master/figs/flowchart_v1.png "Overview of cell identification"
+[Overview]: https://github.com/TAPE-Lab/CyGNAL/blob/master/figs/flowchart_v1.2.png "Overview of CyGNAL"
 
 ## How to use
 
 Main steps in code folder. Various utilities can be found in code/utils.
 
-Raw data contains sample dataset files. Pipeline can take in both FCS and .txt files (as tab-separated dataframes).
+### Input data
 
+Raw data contains sample dataset files. Pipeline can take in both FCS and .txt files (as tab-separated dataframes).
 
-*NOTE*: The dataset used in this tutorial is a down-sampled version (5,000 cells per time point, EpCAM/Pan-CK gated) of the small intestinal organoid time-course experiment described in Figure 4 of our [paper](https://www.nature.com/articles/s41592-020-0737-8). The full dataset is available through [Cytobank Community](https://community.cytobank.org/cytobank/experiments/81059). The users will need to register a free Cytobank Community account to access the project and are encouraged to clone the experiments and explore the data in further details.
+*NOTE*: The toy dataset used in this tutorial is a down-sampled version (5,000 cells per time point, EpCAM/Pan-CK gated) of the small intestinal organoid time-course experiment described in Figure 4 of our [paper](https://www.nature.com/articles/s41592-020-0737-8). The full dataset is available through [Cytobank Community](https://community.cytobank.org/cytobank/experiments/81059). The users will need to register a free Cytobank Community account to access the project and are encouraged to clone the experiments and explore the data in further details.
 
 ### A Brief Step-by-Step Tutorial
 
+Brief tutotorial to run all main steps in CyGNAL with a sequential order. All console commands given assume the user is in the tool's root directory (.../CyGNAL/).
 <!-- (Refer to the Nature Protocols paper for more in-depth instructions) -->
 
 1. **(SETUP):** Clone the repository and ensure you have all necessary software and dependencies.
+    * We strongly encourage using [conda](https://docs.conda.io/en/latest/miniconda.html) to setup an environment from 'conda_env.yml' using `conda create -f conda_env.yml`.
 
 2. **Pre-process:** Copy all the data files to the 'Raw_Data' folder and run `1-data_preprocess.py`. The output files with their antibody panel processed (i.e. measured channels decluttered, empty channels deleted, cell-index assigned) will be saved in the 'Preprocessed_Data' folder, together with a *'panel_markers.csv'* file listing all the markers measured in the given experiment.
 
@@ -47,20 +48,20 @@ Raw data contains sample dataset files. Pipeline can take in both FCS and .txt f
 
 ## Dependencies
 
-We strongly encourage using [conda](https://docs.conda.io/en/latest/miniconda.html) to setup an environment from 'conda_env.yml'.
-
-* Python: Tested with Python v3.6 and v3.7. Used in the backbone of the workflow and most computational steps.
+* Python: Tested with Python v3.6, v3.7, and v3.8. Used in the backbone of the workflow and most computational steps.
     * `fcsparser`
     * `fcswrite`
     * `numpy`
     * `pandas`
     * `plotly`
+    * `pynndescent`
     * `rpy2`
     * `scprep`
     * `sklearn`
     * `umap-learn`
 
-* R: Tested with R v3.6.1 and RStudio v1.2.5001. Mostly used for visualisation, but also for computing the PCA.
+* R: Tested with v3.6.1 < R <= v4. Mostly used for visualisation, but also for computing the PCA.
+    * `ComplexHeatmap`
     * `DT`
     * `factoextra`
     * `FactoMineR`
@@ -82,3 +83,9 @@ We strongly encourage using [conda](https://docs.conda.io/en/latest/miniconda.ht
 
 The work here is actively being developed by Ferran Cardoso ([@FerranC96](https://github.com/FerranC96)) and Dr. Xiao Qin ([@qinxiao1990](https://github.com/qinxiao1990)). 
 Based also on original work by Pelagia Kyriakidou.
+
+We acknowledge the work of all third-parties whose packages are used in CyGNAL.
+
+## About the group
+
+Repository of the [Cell Communication Lab](http://tape-lab.com/) at UCL's Cancer Institute. The Cell Communication Lab studies how oncogenic mutations communicate with stromal and immune cells in the colorectal cancer (CRC) tumour microenvironment (TME). By understanding how mutations regulate all cell types within a tumour, we aim to uncover novel approaches to treat cancer.
diff --git a/dependency_troubleshoot.py b/dependency_troubleshoot.py
@@ -1,40 +1,52 @@
 #Script to test if all necessary dependencies have been installed:
-import sys
-
-python_packages =["fcsparser","fcswrite","rpy2","numpy","OpenSSL.version","os",
-                    "pandas","scprep","subprocess","umap","warnings"]
+import sys, importlib
 
+python_packages =["copy","fcsparser","fcswrite","itertools","numpy","pandas",
+                    "plotly","pynndescent","re","rpy2","scprep","sklearn",
+                    "subprocess","umap"]
+#copy, itertools, re, subprocess -> come with Python
+count=0
 for i in python_packages:
     try:
-        import i
+        importlib.import_module(i)
+        count +=1
     except:
         print ("WARNING!: Python package ",i, " is not installed properly. Please install it manually")
+if len(python_packages)==count:
+    print("All Python packages are installed!")
 
 try:
     from rpy2.robjects import r
 except:
     sys.exit("ERROR: rpy2 python package is missing and we can not test for missing R packages")
 
 
-
 r('''
     #Packages to use:
-    list.of.packages <- c("DT", 
+    list.of.packages <- c(
+                            "ComplexHeatmap", 
+                            "DT",
+                            "factoextra",
+                            "FactoMineR",
+                            "flowCore",
                             "GGally",
-                            "psych",
                             "Hmisc",
                             "MASS",
+                            "matrixStats",
+                            "plotly",
+                            "psych",
                             "RColorBrewer",
                             "shiny",
-                            "tidyverse",
-                            "FactoMineR",
-                            "factoextra",
-                            "matrixStats",
-                            "plotly"
-                            )
+                            "tidyverse"
+                        )
     # check if pkgs are installed already, if not, install automatically:
     new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
-    if(length(new.packages)) install.packages(new.packages, repos = "http://cran.us.r-project.org")
-    #Load packages
-    lapply(list.of.packages, require, character.only = TRUE)    
-    ''')
+    if(length(new.packages)) {
+        print(paste0("WARNING!: Missing R package(s): ", new.packages))
+        print("Attempting to install missing R package(s)...")
+        install.packages(new.packages, repos = "http://cran.us.r-project.org")
+        lapply(list.of.packages, require, character.only = TRUE) #Load packages
+    } else {
+        print("All R packages are installed!")
+    }
+    ''')
diff --git a/figs/flowchart_v1.2.png b/figs/flowchart_v1.2.png
