Submodule_05_Bonus_Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6eb2c6fb-13d9-4461-ad74-44262079211c",
   "metadata": {},
   "source": [
    "# MDIBL Transcriptome Assembly Learning Module\n",
    "# Bonus notebook: Using TransPi on a new dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c38bba56-40d9-4ca4-b58b-b9733b424b1f",
   "metadata": {},
   "source": [
    "In this notebook, we are going to explore how to run this module with a new dataset. These submodules provide a great framework for running a rigorous and scalable transcriptome assembly, but there are some considerations that must be made in order to run this with your own data. We will walk through that process here so that hopefully, you are able to take these notebooks to your research group and use them for your own analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80044322-a021-4fdc-ad83-504961bd1919",
   "metadata": {},
   "source": [
    "The data we are using here comes from SRA. In this example, we are using data from an experiment that compared RNA sequences in honeybees with and without viral infections. The BioProject ID is [PRJNA274674](https://www.ncbi.nlm.nih.gov/bioproject/PRJNA274674). This experiment includes 6 RNA-seq samples and 2 methylation-seq samples. We are only considering the RNA-seq data here. Additionally, we have subsampled them to about 2 millions reads collectively accross all of the samples. In a real analysis this would not be a good idea, but to keep costs and runtimes low we will use the down-sampled files in this demonstration. If you want to explore the full dataset, we recommend pulling the fastq files using the [STRIDES tutorial on SRA downloads](https://github.com/STRIDES/NIHCloudLabGCP/blob/main/notebooks/SRADownload/SRA-Download.ipynb). As with the original example in this module, we have concatenated all 6 files into one set of combined fastq files called joined_R{1,2}.fastq.gz We have stored the subsampled fastq files in this module's cloud storage bucket."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcf2a2d0-bc91-4a2a-9db0-62f1eee91f92",
   "metadata": {},
   "source": [
    "Before we start any analysis, let's set up the environment just like we did in Submodule_01 and Submodule_02 where we move to the correct directory and install software."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "795e3329-21d8-4de0-91c4-58c81371a712",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd /home/jupyter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9701541-9c05-4a02-abe7-e1126826efe3",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d0c3012-27ca-4eb7-a106-6532f62cbc82",
   "metadata": {},
   "outputs": [],
   "source": [
    "#update java\n",
    "! sudo apt update\n",
    "! sudo apt-get install default-jdk -y\n",
    "! java -version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d6ffb05-19bb-42c4-a628-6d874c5d3517",
   "metadata": {},
   "outputs": [],
   "source": [
    "# install mamba and dependencies\n",
    "! curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh\n",
    "! bash Mambaforge-$(uname)-$(uname -m).sh -b -p $HOME/mambaforge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7df1c464-b102-4e34-8f5f-57dbfc7f43b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# add mamba to your path\n",
    "import os\n",
    "os.environ[\"PATH\"] += os.pathsep + os.environ[\"HOME\"]+\"/mambaforge/bin\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39bb00de-3481-4cb0-a2fe-098cfdae51a6",
   "metadata": {},
   "source": [
    "Use mamba to install: sra-tools perl-dbd-sqlite perl-dbi from channel bioconda\n",
    "\n",
    "<details>\n",
    "  <summary>Click for help</summary>\n",
    "\n",
    "```\n",
    "mamba install -c bioconda sra-tools perl-dbd-sqlite perl-dbi -y\n",
    "```\n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1b80a47-89c9-469c-a1f8-9ee3c1817fa7",
   "metadata": {},
   "outputs": [],
   "source": [
    "! <YOUR COMMAND HERE>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21ae7ca3-c9e1-4160-a29f-37a53b8265ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "#install Nextflow\n",
    "! curl https://get.nextflow.io | bash\n",
    "! chmod +x nextflow\n",
    "! ./nextflow self-update"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7d6f314-9af1-428f-85ea-6d02e527a6a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy the software from gs://nigms-sandbox/nosi-inbremaine-storage/TransPi\n",
    "! <YOUR COMMAND HERE>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbb26d8b-21ac-4907-b25a-3bd39b853d1b",
   "metadata": {},
   "source": [
    "<details>\n",
    "  <summary>Click for help</summary>\n",
    "\n",
    "```\n",
    "gsutil -m cp -r gs://nigms-sandbox/nosi-inbremaine-storage/TransPi ./```\n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5bdc3fa-7b78-4329-ae17-49ffce2085bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy the data from gs://nigms-sandbox/nosi-inbremaine-storage/resources\n",
    "! <YOUR COMMAND HERE>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d73e5d6b-9b3f-46e2-9c5c-24713a2ad55c",
   "metadata": {},
   "source": [
    "<details>\n",
    "  <summary>Click for help</summary>\n",
    "\n",
    "```\n",
    "gsutil -m cp -r gs://nigms-sandbox/nosi-inbremaine-storage/resources ./\n",
    "```\n",
    "  \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65350b31-588b-42d9-90b4-849a523be021",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make the program executable\n",
    "!chmod -R +x ./TransPi/bin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15d79bb0-9f4c-469a-b2e6-3379e68f8f73",
   "metadata": {},
   "source": [
    "Let's have a look at what we've downloaded to make sure it's there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c5606e2-5c0c-423c-9e32-40bd550c19cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "! ls ./resources/seq2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08ae572a-fe6d-4852-a11c-5a2449c1d6b2",
   "metadata": {},
   "source": [
    "You should see the joined fastq files alongside the others that we use in the previous submodules. Now let's adjust the workflow to run on them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "331a3857-7734-41e4-819a-de3603b9c95b",
   "metadata": {},
   "source": [
    "One of the great benefits of using a workflow manager like Nextflow is that it allows easy swapping of input samples without drastic changes to the code. In the true spirit of reproducible workflows, the only change necessary in order to run the joined samples is to adjust the `reads` line in the `nextflow.config` file `params` section to point to the new reads location. In the line below, write the updated reads path that you would add to the config file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a7bb7e0-f7eb-4166-96a9-b25b0885479c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# <Your path here>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c22622e3-56a7-42bb-9884-abd39c72d6e3",
   "metadata": {},
   "source": [
    "<details>\n",
    "  <summary>Click for help</summary>\n",
    "    \n",
    "\n",
    "\n",
    "```\n",
    "// Directory for reads\n",
    "reads=\"/home/jupyter/resources/seq2/joined*R[1,2].fastq.gz\"\n",
    "```\n",
    "    \n",
    "    \n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd94aacc-2fb8-4a99-8378-2883b723253a",
   "metadata": {},
   "source": [
    "After this change, you should be able to run the same Nextflow command as you did in Submodule_02 and everything will progress automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24663040-553b-4434-83a8-93fb9d2fd58f",
   "metadata": {},
   "outputs": [],
   "source": [
    "! NXF_VER=22.10.1 ./nextflow run \\\n",
    "    ./TransPi/TransPi.nf \\\n",
    "    -profile docker \\\n",
    "    --k 17,25,43 \\\n",
    "    --maxReadLen 50 \\\n",
    "    --all "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57dc8622-fb61-4ce6-ad40-48fc710a4713",
   "metadata": {},
   "source": [
    "With the subsampled reads, the assembly should complete in about 2 hours using a n1-highmem-16 machine."
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}