Skip to content

Commit 3a37a36

Browse files
bricaudbricaud
committed
first commit
0 parents  commit 3a37a36

File tree

3 files changed

+488
-0
lines changed

3 files changed

+488
-0
lines changed

‎README.md

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# BXD project
2+
3+
The BXD project aims at revealing and exploring the complex relationship between mice genes, phenotypes, protein expression in tissues and more. The main goal of this repository is to provide a tutorial on the data and different machine learning approaches used for investigating the mice data. It is coded in python with jupyter notebooks. Each notebook focus on a particular aspect of the data or a particular ML method.
4+
5+
## List of notebooks
6+
7+
Here is the list of notebooks with a short description.
8+
9+
* `data_exploration.ipynb`: introduction to the dataset with a decription of the different files and what they contain.
10+
* `random_forest-phenotypes-genotypes.ipynb`: A simple implementation of Random Forest to find complex combination of genes that could influence phenotypes.
11+
12+
## How to run the tutorials
13+
14+
There are several possible ways to run the notebooks.
15+
* clone the repository on a local machine with python and jupyter installed, along with several data mining and machine learning modules. The simplest is to install Anaconda.
16+
* The nobebooks can be run online for example using Binder. Everything will run online without the need to install anything on the personal machine. However it may be slower and the user can experience disconnections during long period of inactivity.
17+
18+
## BXD data
19+
20+
The dataset for the experiments is open and stored on an server from EPFL. It is a s3 storage at `'endpoint_url':'https://os.unil.cloud.switch.ch'` that can be accessed using the url `'s3://lts2-graphnex/BXDmice/`. See the notebooks for access examples.

‎data_exploration.ipynb

+271
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": null,
6+
"id": "a7c22dcf-5a2e-48e0-9c84-a11651e9b025",
7+
"metadata": {},
8+
"outputs": [],
9+
"source": [
10+
"import pandas as pd\n",
11+
"import numpy as np\n",
12+
"import os"
13+
]
14+
},
15+
{
16+
"cell_type": "code",
17+
"execution_count": null,
18+
"id": "5cd30231-1d5b-4ca3-ae3d-728372852aef",
19+
"metadata": {},
20+
"outputs": [],
21+
"source": [
22+
"# Config for accessing the data on the s3 storage\n",
23+
"storage_options = {'anon':True, 'client_kwargs':{'endpoint_url':'https://os.unil.cloud.switch.ch'}}\n",
24+
"s3_path = 's3://lts2-graphnex/BXDmice/'"
25+
]
26+
},
27+
{
28+
"cell_type": "markdown",
29+
"id": "2ab1e6b6-c958-4461-8dc1-f9e173e558e3",
30+
"metadata": {
31+
"tags": []
32+
},
33+
"source": [
34+
"## Genotype\n",
35+
"The genotype file contains a list of differences in the genome of the different mice. These differences are at the scale of a nucleotide. In the data table, each row is an `SNP` [Single-nucleotide polymorphism](https://en.wikipedia.org/wiki/Single-nucleotide_polymorphism). It can be inherited from one of the initial ancestors or the other. This is encoded as a binary value -1 or 1. The initial ancestors have a zero value."
36+
]
37+
},
38+
{
39+
"cell_type": "code",
40+
"execution_count": null,
41+
"id": "589de0ca-5aae-4025-85d0-836d12de47fb",
42+
"metadata": {},
43+
"outputs": [],
44+
"source": [
45+
"# Load the data\n",
46+
"# Genotype\n",
47+
"genotype_path = os.path.join(s3_path, 'genotype_BXD.txt.gz')\n",
48+
"genotype = pd.read_csv(genotype_path, sep='\\t', storage_options=storage_options)\n",
49+
"print('File {} Opened.'.format(genotype_path))"
50+
]
51+
},
52+
{
53+
"cell_type": "code",
54+
"execution_count": null,
55+
"id": "7ad624ce-63fe-452e-a612-cc3ea685a3dd",
56+
"metadata": {},
57+
"outputs": [],
58+
"source": [
59+
"genotype.head()"
60+
]
61+
},
62+
{
63+
"cell_type": "code",
64+
"execution_count": null,
65+
"id": "f12c0d44-a09a-451a-a727-c8f7ba7f160d",
66+
"metadata": {},
67+
"outputs": [],
68+
"source": [
69+
"# Gene postion in the genome\n",
70+
"geno_map_path = os.path.join(s3_path, 'map_BXD.txt.gz')\n",
71+
"geno_map = pd.read_csv(geno_map_path, sep='\\t', storage_options=storage_options)\n",
72+
"print('File {} Opened.'.format(geno_map_path))"
73+
]
74+
},
75+
{
76+
"cell_type": "code",
77+
"execution_count": null,
78+
"id": "9655b2f5-6604-4e38-a15d-f17c79847b80",
79+
"metadata": {},
80+
"outputs": [],
81+
"source": [
82+
"geno_map.head()"
83+
]
84+
},
85+
{
86+
"cell_type": "markdown",
87+
"id": "03dfacf9-da9a-4dbe-a320-58b766316c30",
88+
"metadata": {},
89+
"source": [
90+
"## Tissues\n",
91+
"During or after experiments, the expression of proteins in different tissues of the mice has been measured.\n",
92+
"The measurements have been recorded in a file per tissue. The data are in a large table with proteins as rows and mice as columns. The expression is a float number.\n",
93+
"\n",
94+
"For each mouse, only a subset of the tissues have been measured. Therefore, not all mice are present in each tissue data and different group of mice are found in the different tissue files."
95+
]
96+
},
97+
{
98+
"cell_type": "code",
99+
"execution_count": null,
100+
"id": "724eca66-ded1-4cee-a872-db1692a4bdf5",
101+
"metadata": {},
102+
"outputs": [],
103+
"source": [
104+
"# Tissue\n",
105+
"tissue_name = 'Muscle_CD'\n",
106+
"#organ = 'Lung'\n",
107+
"#organ = 'Hippocampus'\n",
108+
"#organ = 'Gastrointestinal'\n",
109+
"tissue_path = os.path.join(s3_path, 'expression data', tissue_name + '.txt.gz')\n",
110+
"tissue = pd.read_csv(tissue_path, sep='\\t', storage_options=storage_options)\n",
111+
"print('File {} Opened.'.format(tissue_path))"
112+
]
113+
},
114+
{
115+
"cell_type": "code",
116+
"execution_count": null,
117+
"id": "c04eff0b-b998-4d0b-af7f-53b7ca3160f0",
118+
"metadata": {},
119+
"outputs": [],
120+
"source": [
121+
"tissue.head()"
122+
]
123+
},
124+
{
125+
"cell_type": "code",
126+
"execution_count": null,
127+
"id": "70abc242-b56a-4c5f-b83a-f503ae156445",
128+
"metadata": {},
129+
"outputs": [],
130+
"source": [
131+
"df_genotype = pd.read_csv(path_bxd + file_genotype, sep='\\t')\n",
132+
"df_map = pd.read_csv(path_bxd + 'map_BXD.txt', sep='\\t')\n",
133+
"df_genotype.insert(0, 'Chr', df_map['Chr'].values)\n",
134+
"df_genotype.insert(2, 'Pos', df_map['Pos'].values)"
135+
]
136+
},
137+
{
138+
"cell_type": "markdown",
139+
"id": "a976b942-5433-4725-b9a3-b8593baca597",
140+
"metadata": {},
141+
"source": [
142+
"## Phenotype\n",
143+
"The phenotype data corresponds to the results of different experiments. It is made of 2 files, one file contains the results and the other contain the description of the experiment (experiment type, authors,...).\n",
144+
"In the result table, rows correspond to phenotypes and columns to mouse strains. The entries are float numbers. The table contains a large number of missing values as not all the mouse strains have been involved in all the experiments."
145+
]
146+
},
147+
{
148+
"cell_type": "code",
149+
"execution_count": null,
150+
"id": "70697a87-b2bf-4107-a3bf-441ed7cf6cef",
151+
"metadata": {},
152+
"outputs": [],
153+
"source": [
154+
"# Load the data\n",
155+
"# Phenotype\n",
156+
"phenotype_path = os.path.join(s3_path, 'Phenotype.txt.gz')\n",
157+
"phenotype = pd.read_csv(phenotype_path, sep='\\t', storage_options=storage_options)\n",
158+
"print('File {} Opened.'.format(phenotype_path))\n",
159+
"# Phenotype description\n",
160+
"phenotypeinfo_path = os.path.join(s3_path, 'phenotypes_id_aligner.txt.gz')\n",
161+
"phenotypeinfo = pd.read_csv(phenotypeinfo_path, sep='\\t', storage_options=storage_options)\n",
162+
"print('File {} Opened.'.format(phenotypeinfo_path))"
163+
]
164+
},
165+
{
166+
"cell_type": "code",
167+
"execution_count": null,
168+
"id": "0fcea224-df5e-4255-9850-766eaf7f6445",
169+
"metadata": {},
170+
"outputs": [],
171+
"source": [
172+
"phenotype.head()"
173+
]
174+
},
175+
{
176+
"cell_type": "code",
177+
"execution_count": null,
178+
"id": "ae45cbb7-4cc2-4571-9db7-ec852d80444d",
179+
"metadata": {},
180+
"outputs": [],
181+
"source": [
182+
"phenotypeinfo.head()"
183+
]
184+
},
185+
{
186+
"cell_type": "code",
187+
"execution_count": null,
188+
"id": "411d02c2-fac9-4351-b321-0d6952946ea6",
189+
"metadata": {},
190+
"outputs": [],
191+
"source": [
192+
"phenotypeinfo[phenotypeinfo['RecordID']==12894]"
193+
]
194+
},
195+
{
196+
"cell_type": "markdown",
197+
"id": "3eef0f3f-3550-4b39-91cc-7b0692a1f642",
198+
"metadata": {},
199+
"source": [
200+
"## Data cleaning"
201+
]
202+
},
203+
{
204+
"cell_type": "markdown",
205+
"id": "fb3c6e29-4ab2-4e9e-8b40-c41ce665c85c",
206+
"metadata": {},
207+
"source": [
208+
"### Drop duplicate genes in the dataset\n",
209+
"Some lines in the genotype DataFrame are identical and we will drop them to reduce the number of features and the computation."
210+
]
211+
},
212+
{
213+
"cell_type": "code",
214+
"execution_count": null,
215+
"id": "ce653493-b42d-4952-8256-d5dd91e77371",
216+
"metadata": {},
217+
"outputs": [],
218+
"source": [
219+
"# drop duplicate genes in the dataset\n",
220+
"geno_merge = pd.merge(geno_map, genotype, on='SNP')\n",
221+
"print('Size of the data before dropping duplicates',geno_merge.shape)\n",
222+
"# define a duplicate SNP as: \n",
223+
"# 1) an SNP where all the entries corresponding to BXD mice are identical to another SNP and\n",
224+
"# 2) both SNPs are on the same chromosome.\n",
225+
"col_to_search_duplicates = ['Chr'] + list(genotype.columns.values[5:])\n",
226+
"geno_reduced = geno_merge.drop_duplicates(subset=col_to_search_duplicates)\n",
227+
"print('Size of the data after dropping duplicates',geno_reduced.shape)"
228+
]
229+
},
230+
{
231+
"cell_type": "code",
232+
"execution_count": null,
233+
"id": "60080f9d-022f-4ae1-b330-97478efa39f5",
234+
"metadata": {},
235+
"outputs": [],
236+
"source": [
237+
"# Optionally, save the result as a compressed csv file, to be used by other notebooks\n",
238+
"geno_reduced.to_csv('geno_reduced.csv.gz')"
239+
]
240+
},
241+
{
242+
"cell_type": "code",
243+
"execution_count": null,
244+
"id": "3b25fbbe-55dc-48e5-a33d-bf613a342b9e",
245+
"metadata": {},
246+
"outputs": [],
247+
"source": []
248+
}
249+
],
250+
"metadata": {
251+
"kernelspec": {
252+
"display_name": "venv",
253+
"language": "python",
254+
"name": "venv"
255+
},
256+
"language_info": {
257+
"codemirror_mode": {
258+
"name": "ipython",
259+
"version": 3
260+
},
261+
"file_extension": ".py",
262+
"mimetype": "text/x-python",
263+
"name": "python",
264+
"nbconvert_exporter": "python",
265+
"pygments_lexer": "ipython3",
266+
"version": "3.8.0"
267+
}
268+
},
269+
"nbformat": 4,
270+
"nbformat_minor": 5
271+
}

0 commit comments

Comments
 (0)