-
Notifications
You must be signed in to change notification settings - Fork 7
/
README.html
1200 lines (1182 loc) · 65.6 KB
/
README.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<p><img
src="https://img.shields.io/docker/automated/tomkellygenetics/universc?label=docker%20local%20build"
alt="Docker Manual build" /> <img
src="https://img.shields.io/docker/cloud/automated/tomkellygenetics/universc?label=docker%20cloud%20build"
alt="Docker Cloud Build" /> <img
src="https://img.shields.io/docker/cloud/build/tomkellygenetics/universc?label=cloud%20build"
alt="Docker Cloud Status" /> <img
src="https://img.shields.io/docker/stars/tomkellygenetics/universc"
alt="Docker Stars" /> <img
src="https://img.shields.io/docker/pulls/tomkellygenetics/universc"
alt="Docker Pulls" /></p>
<p><img
src="https://img.shields.io/docker/v/tomkellygenetics/universc/1.2.7"
alt="Docker Image Version (tag latest semver)" /> <img
src="https://img.shields.io/microbadger/layers/tomkellygenetics/universc/latest?label=%[email protected]%22"
alt="MicroBadger Layers (latest)" /> <img
src="https://img.shields.io/docker/image-size/tomkellygenetics/universc/1.2.7?label=%22image%[email protected]"
alt="Docker Image Size (v1.2.7)" /> <img
src="https://img.shields.io/docker/v/tomkellygenetics/universc/latest"
alt="Docker Image Version (latest by date)" /> <img
src="https://img.shields.io/microbadger/layers/tomkellygenetics/universc/latest"
alt="MicroBadger Layers (latest)" /> <img
src="https://img.shields.io/docker/image-size/tomkellygenetics/universc/latest"
alt="Docker Image Size (latest)" /></p>
<p><img
src="https://img.shields.io/github/checks-status/minoda-lab/universc/master?label=GitHub%20checks"
alt="GitHub branch checks state" /> <img
src="https://img.shields.io/github/release-date/minoda-lab/universc"
alt="GitHub Release Date" /> <img
src="https://img.shields.io/github/last-commit/minoda-lab/universc/master"
alt="GitHub last commit (branch)" /> <img
src="https://img.shields.io/github/issues/minoda-lab/universc"
alt="GitHub issues" /> <img
src="https://img.shields.io/github/issues-pr/minoda-lab/universc"
alt="GitHub pull requests" /></p>
<p><a href="http://hits.dwyl.com/minoda-lab/universc"><img
src="http://hits.dwyl.com/minoda-lab/universc.svg"
alt="GitHub Views" /></a> <a
href="http://hits.dwyl.com/tomkellygenetics/universc"><img
src="http://hits.dwyl.com/tomkellygenetics/universc.svg"
alt="GitHub Views" /></a> <img
src="https://img.shields.io/github/search/minoda-lab/universc/master"
alt="GitHub search hit counter" /> <img
src="https://img.shields.io/github/forks/minoda-lab/universc?style=social"
alt="GitHub forks" /> <img
src="https://img.shields.io/github/stars/minoda-lab/universc?style=social"
alt="GitHub Repo stars" /> <img
src="https://img.shields.io/github/watchers/minoda-lab/universc?style=social"
alt="GitHub watchers" /></p>
<p><img
src="https://img.shields.io/github/languages/code-size/minoda-lab/universc"
alt="GitHub code size in bytes" /> <img
src="https://img.shields.io/github/repo-size/minoda-lab/universc"
alt="GitHub repo size" /> <img
src="https://img.shields.io/github/languages/top/minoda-lab/universc"
alt="GitHub top language" /> <img
src="https://img.shields.io/github/languages/count/minoda-lab/universc"
alt="GitHub language count" /> <a
href="https://doi.org/10.5281/zenodo.7116956"><img
src="https://zenodo.org/badge/DOI/10.5281/zenodo.7116956.svg"
alt="Zenodo DOI" /></a></p>
<p><img
src="https://img.shields.io/github/downloads/minoda-lab/universc/total?label=GitHub%20downloads"
alt="GitHub all releases" /> <img
src="https://img.shields.io/github/v/release/minoda-lab/universc?label=GitHub%20release"
alt="GitHub release (latest by date)" /> <img
src="https://img.shields.io/github/downloads/minoda-lab/universc/1.2.7/total"
alt="GitHub release (latest by date)" /> <img
src="https://img.shields.io/github/downloads/minoda-lab/universc/1.2.7/total"
alt="GitHub release (by tag)" /></p>
<p><img
src="https://github.com/minoda-lab/universc/workflows/CI%20to%20Docker%20hub/badge.svg"
alt="Docker CI" /> <img
src="https://github.com/minoda-lab/universc/workflows/Docker%20compose%20build/badge.svg"
alt="Docker compose" /> <img
src="https://github.com/minoda-lab/universc/workflows/Docker%20container%20tests/badge.svg"
alt="Actions Build" /> <img
src="https://github.com/minoda-lab/universc/workflows/Docker%20build%20image/badge.svg"
alt="Actions Call" /></p>
<p><img
src="https://github.com/minoda-lab/universc/workflows/Run%20all%20tests%20in%20Docker/badge.svg"
alt="Actions Tests" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%2010x%20Genomics/badge.svg"
alt="Test 10x Genomics" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%20DropSeq%20%2F%20Nadia/badge.svg"
alt="Test DropSeq" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%20ICELL8/badge.svg"
alt="Test ICELL8" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%20SCI%2DSeq/badge.svg"
alt="Test SCI-Seq" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%20inDrops%20v3/badge.svg"
alt="Test inDrops v3" /> <img
src="https://github.com/minoda-lab/universc/workflows/Test%20Smart%2DSeq3/badge.svg"
alt="Test Smart-Seq3" /></p>
<h1 id="universc">UniverSC</h1>
<p><strong>Single-cell processing across technologies</strong></p>
<hr />
<p><strong>Summary</strong></p>
<p>Single-cell RNA-sequencing analysis to quantify RNA molecules in
individual cells has become popular owing to the large amount of
information one can obtain from each experiment. UniverSC is a universal
single-cell processing tool that supports any UMI-based platform. Our
command-line tool enables consistent and comprehensive integration,
comparison, and evaluation across data generated from a wide range of
platforms. Here we provide a guide to install and use this tool to
process single-cell RNA-Seq data from FASTQ format.</p>
<p><strong>Package</strong></p>
<p>UniverSC version 1.2.7</p>
<p><strong>Maintainers</strong></p>
<p>Tom Kelly<sup>†</sup> (RIKEN IMS) and Kai Battenberg<sup>†</sup>
(RIKEN CSRS/IMS)</p>
<p>† These authors contributed equally to this work</p>
<p>Contact: <first name>.<family name>[at]riken.jp</p>
<hr />
<p><strong>Disclaimer</strong>: we are third party developers not
affiliated with 10X Genomics or any other vendor of single-cell
technologies. We are releasing this code on an open-source <a
href="#licensing">license</a> which calls Cell Ranger™ as an external
dependency.</p>
<hr />
<h2 id="getting-started">Getting Started</h2>
<h3 id="advanced-users">Advanced users</h3>
<p>If you have <code>cellranger</code> already installed, then all you
need to do is clone or download this git repository. You can then run
the script in this directory or add it your <code>PATH</code>. See the
<a href="#quick-start">Quick Start</a> guide below.</p>
<p>If you wish to install <code>cellranger</code> and configure this
script to run on a Linux environment, we provide details on <a
href="#installation">installation</a> below. Note that
<code>launch_universc.sh</code> requires write-access a Cell Ranger
installation so it needs to be installed in a user’s “home” directory on
a server. No admin powers needed!</p>
<p>Note that <code>cellranger</code> installations that are pre-compiled
on Linux will not run on Mac or Windows. Note that Mac OS and some Linux
distributions also have different version of sed and rename. It is
possible to compile an open-source version of Cell Ranger but it is
tricky to install the dependencies so we recommend using our docker <a
href="#Docker">image</a> if you wish to do this.</p>
<h3 id="command-line-beginners">Command-line Beginners</h3>
<p>If you are a beginner bioinformatician or wish to run this on a local
computer (Mac or Windows), no problem! We provide a “docker” image
containing everything needed to run it without installing the software
needed. All you need to do is install <a
href="https://docs.docker.com/desktop/">docker</a> and follow our guide
to use the <a href="#Docker">image</a>. This comes bundled with all the
compatible versions needed to run it.</p>
<p>Note that you need to run the shell commands given in a unix-like
command-line interface (the “Terminal” application on Mac or Linux
systems). Many shells are supported but we recommend the “bash” shell
for beginners (this is the default on most systems). Windows 10 includes
a <a
href="https://docs.microsoft.com/en-us/windows/wsl/install-win10">subsystem</a>
to run <code>bash</code>. If this is too complicated, you can open a
Linux environment (Ubuntu) in docker by following our instructions. Then
you can enter bash commands into the terminal opened by docker.</p>
<p>If you run into problems installing or running
<code>launch_universc.sh</code> please don’t hesistate to contact us via
email or GitHub.</p>
<h3 id="graphical-application-users">Graphical application Users</h3>
<p>We also provide a graphical user interface (GUI) based application to
run the Docker <a href="#Docker">image</a>. Please install Docker as
described above and pull our
<code>tomkellygenetics/universc:latest</code> image using either the
docker command-line interface (CLI) or the docker graphical
application.</p>
<p>Once you have a docker image installed on your system, you can run
the applicable binary available here:</p>
<p><a
href="https://genomec.gsc.riken.jp/gerg/UniverSC/UniverSC_app_release/">https://genomec.gsc.riken.jp/gerg/UniverSC/UniverSC_app_release/</a></p>
<h3 id="nextflow-users">Nextflow users</h3>
<p>A <a href="https://www.nextflow.io/">nextflow</a> custom <a
href="https://nf-co.re/modules">module</a> is has been developed for the
<a href="https://nf-co.re">nf-core</a> community. It will be release
open-source to use in nf-core <a
href="https://nf-co.re/pipelines">pipelines</a>.</p>
<p>See the issue and pull request for more details:</p>
<p><a
href="https://github.com/nf-core/modules/issues/1644">https://github.com/nf-core/modules/issues/1644</a></p>
<p><a
href="https://github.com/nf-core/modules/pull/1706">https://github.com/nf-core/modules/pull/1706</a></p>
<p>We welcome feedback on these tools. If you are interested in using it
or contributing to development of nextflow pipelines, please contact the
maintainers. Thank you!</p>
<h2 id="purpose">Purpose</h2>
<p>We’ve developed a bash script that will run Cell Ranger on FASTQ
files for these technologies. See below for details on how to use
it.</p>
<p>If you use this tool, please <a href="#Citation">cite</a> to
acknowledge the efforts of the authors. You can report problems and
request new features to the maintainers with and <a
href="#Issues">issue</a> on GitHub. Details on how to <a
href="#Install">install</a> and <a href="#Usage">run</a> are provided
below. Please see the <a href="#Help">help</a> and <a
href="#Examples">examples</a> to try solve your problem before
submitting an issue.</p>
<p>Details on the <a href="#Docker">Docker image</a> are given below. We
recommend using Docker unless you have a server environment with Cell
Ranger installed already.</p>
<h3 id="supported-technologies">Supported Technologies</h3>
<p>In principle, any technology with a cell barcode and unique molecular
identifier (UMI) can be supported.</p>
<p>The following technologies have been tested to ensure that they give
the expected results: 10x Genomics, Nadia (DropSeq), ICELL8 version
3</p>
<p>We provide the following preset configurations for convenience based
on published data and configurations used by other pipelines (e.g,
DropSeqPipe and Kallisto/Bustools). To add further support for other
technologies or troubleshoot problems, please submit an Issue to the
GitHub repository: <a
href="https://github.com/minoda-lab/universc/issues">minoda-lab/universc</a>
as described in <a href="#Issues">Bug Reports</a> below.</p>
<p>Some changes to the Cell Ranger install are required to run other
technologies. Therefore we provide settings for 10x Genomics which
restores settings for the Chromium instrument. We therefore recommend
using UniverSC for processing all data from different technologies as
the tool manages these changes. Please note that on a single install of
Cell Ranger, multiple technologies or multiple samples of the same
technology with different whitelist barcodes cannot be run cannot be run
simultaneousely (the tool will also check for this to avoid causing
problems with existing runs). Multiple samples of the same technology
with the same barcode whitelist can be run simultaneously.</p>
<p>If you are using <code>UniverSC</code> you should also do so to run
10x Genomics data. If you wish to restore Cell Ranger to default
settings, see the <a href="#Uninstalling">installation</a> or <a
href="#Debugging">troubleshooting</a> sections below.</p>
<h4 id="pre-set-configurations">Pre-set configurations</h4>
<ul>
<li>10x Genomics (version automatically detected): 10x, chromium
<ul>
<li>10x Genomics version 2 (16 bp barcode, 10 bp UMI): 10x-v2,
chromium-v2</li>
<li>10x Genomics version 3 (16 bp barcode, 12 bp UMI): 10x-v3,
chromium-v3</li>
</ul></li>
<li>Aligent Bravo B (16 bp barcode, No UMI): aligent, bravo</li>
<li>BD Rhapsody
<ul>
<li>BD Rhapsody v1 (27 bp barcode, 8 bp UMI): bd-rhapsody</li>
<li>BD Rhapsody v2 enhanced beads (27 bp barcode, 8 bp UMI):
bd-rhapsody-v2</li>
</ul></li>
<li>C1
<ul>
<li>C1 Fluidigm (16 bp barcode, No UMI): c1, fluidgm-c1</li>
<li>C1 CAGE (16 bp, No UMI): c1-cage</li>
<li>C1 RamDA-Seq (16 bp, No UMI): c1-ramda-seq</li>
</ul></li>
<li>CEL-Seq
<ul>
<li>CEL-Seq (8 bp barcode, 4 bp UMI): celseq</li>
<li>CEL-Seq2 (6 bp UMI, 6 bp barcode): celseq2</li>
</ul></li>
<li>Drop-Seq (12 bp barcode, 8 bp UMI): dropseq</li>
<li>ICELL8
<ul>
<li>ICELL8 version 2 (11 bp barcode, No UMI): icell8-non-umi,
icell8-v2</li>
<li>ICELL8 version 3 (11 bp barcode, 14 bp UMI): icell8 or custom</li>
<li>ICELL8 5′ scRNA with TCR OR kit (10bp barcode, NO bp UMI):
icell8-5-prime</li>
<li>ICELL8 full-length scRNA with Smart-Seq (16 bp barcode, No UMI):
icell8-full-length</li>
</ul></li>
<li>inDrops
<ul>
<li>inDrops version 1 (19 bp barcode, 6 bp UMI): indrops-v1,
1cellbio-v1</li>
<li>inDrops version 2 (19 bp barcode, 6 bp UMI): indrops-v2,
1cellbio-v2</li>
<li>inDrops version 3 (16 bp barcode, 6 bp UMI): indrops-v3,
1cellbio-v3</li>
</ul></li>
<li>MARS-Seq
<ul>
<li>MARS-Seq (6 bp barcode, 10 bp UMI): marsseq, marsseq-v1</li>
<li>MARS-Seq2 (7 bp barcode, 8 bp UMI): marsseq2, marsseq-v2<br />
</li>
</ul></li>
<li>Microwell-Seq (18 bp barcode, 6 bp UMI): microwell</li>
<li>Nadia (12 bp barcode, 8 bp UMI): nadia, dropseq</li>
<li>PIP-Seq
<ul>
<li>PIP-Seq version 0 (24 bp barcode, 8 bp UMI): pip-seq-v0</li>
<li>PIP-Seq version 1 (16 bp barcode, 6 bp UMI): pip-seq-v1</li>
<li>PIP-Seq version 2 (24 bp barcode, 12 bp UMI): pip-seq-v2</li>
<li>PIP-Seq version 3 (28 bp barcode, 12 bp UMI): pip-seq-v3</li>
<li>PIP-Seq version 4 (28 bp barcode, 12 bp UMI): pip-seq-v4</li>
</ul></li>
<li>Quartz-Seq
<ul>
<li>QuartzSeq (6 bp index, no UMI): quartz-seq</li>
<li>Quartz-Seq2 (14 bp barcode, 8 bp UMI): quartzseq2-384</li>
<li>Quartz-Seq2 (15 bp barcode, 8 bp UMI): quartzseq2-1536</li>
</ul></li>
<li>RamDA-Seq (6 bp index, no UMI): ramda-seq</li>
<li>Single-cell combinatorial indexing (SCI-RNA-Seq)
<ul>
<li>SCI-Seq 2-level indexing (30 bp barcode, 8 bp UMI): sciseq2</li>
<li>SCI-Seq 3-level indexing (40 bp barcode, 8 bp UMI): sciseq3</li>
<li>SCIFI-Seq (27 bp barcode, 8 bp UMI): scifiseq</li>
</ul></li>
<li>SCRB-Seq (6 bp barcode, 10 bp UMI): scrbseq, mcscrbseq</li>
<li>SeqWell (12 bp barcode, 8 bp UMI): plexwell, seqwell, seqwells3</li>
<li>Smart-seq
<ul>
<li>Smart-Seq (16 bp barcode, No UMI): smartseq</li>
<li>Smart-Seq2 (16 bp barcode, No UMI): smartseq2</li>
<li>Smart-Seq2-UMI, Smart-seq3 (16 bp barcode, 8 bp UMI): smartseq3</li>
</ul></li>
<li>SPLiT-Seq
<ul>
<li>SPLiT-Seq v1.0 (8 bp UMI, 24 bp barcode): splitseq</li>
<li>SPLiT-Seq v2.1 (10 bp UMI, 24 bp barcode): splitseq2</li>
</ul></li>
<li>STRT-Seq
<ul>
<li>STRT-Seq (6 bp barcode, no UMI): strt-seq</li>
<li>STRT-Seq-C1 (8 bp barode, 5 bp UMI): strt-seq-c1</li>
<li>STRT-Seq-2i (13 bp barcode, 6 bp UMI): strt-seq-2i</li>
<li>STRT-Seq-2018 (8 bp barcode, 8 bp UMI): strt-seq-2018</li>
</ul></li>
<li>SureCell (18 bp barcode, 8 bp UMI): surecell, ddseq, biorad</li>
<li>VASA-Seq
<ul>
<li>VASA-plate (6 bp UMI, 6 bp barcode): vasa-plate</li>
<li>VASA-drop (6 bp UMI, 16 bp barcode): vasa-drop</li>
</ul></li>
</ul>
<h4 id="chemistry-settings-available">Chemistry settings available</h4>
<p>All technologies support 3′ single-cell RNA-Seq. Barcode adjustments
and whitelists are changed automatically. For 5′ single-cell RNA-Seq,
this is only supported for 10x Genomics version 2 chemistry, ICELL8,
Smart-Seq, and STRT-Seq. For 10x Genomics, this is detected
automatically but can be configured with the <code>--chemistry</code>
argument. For other technologies, the template switching oligonucleotide
is automatically converted to the match the 10x sequence.</p>
<h4 id="support-for-umi-based-and-non-umi-technologies">Support for
UMI-based and non-UMI technologies</h4>
<p>By default, UMIs are supported where available so with the following
exceptions for non-UMI technologies: ICELL8 v2, RamDA-Seq, Quartz-Seq,
Smart-Seq, Smart-Seq2. While using UMI is recommended we provide a mock
UMI for counting reads for these technologies (and data from previous
versions).</p>
<p>Other techniques can be forced to replace the UMI with a mock
sequence for counting reads only with <code>--non-umi</code> or
<code>--read-only</code> arguments. Forcing non-UMI techniques is
<em>not recommended</em> unless you are integrating non-UMI and
UMI-based technologies. It is not necessary to specific
<code>--non-umi</code> for non-UMI techniques as these will be used
automatically when applicable. For ICELL8 and Smart-Seq where both
non-UMI (icell8-v2, smartseq2) and UMI-based (icell8-v3, smartseq3)
techniques are available it is possible to specify which to use.</p>
<h4 id="single-and-dual-indexed-technologies">Single and dual indexed
technologies</h4>
<p>Where needed the cell barcode can be detected in the index I1 or I2
file. Single indexes are supported for STRT-Seq and Quartz-Seq. Dual
indexes are supported for Fluidigm C1, ICELL8 full-length, inDrops-v3,
RamDA-Seq, SCI-RNA-Seq, scifi-seq, and Smart-Seq. Combinatorial indexing
technologies have linkers between barcodes removed automatically to
match the barcode whitelist.</p>
<h4 id="demultiplexing-for-dual-indexing">Demultiplexing for
dual-indexing</h4>
<p>For dual-indexed technologies such as Fluidigm C1, inDrops-v3,
Sci-Seq, SmartSeq3 it is advised to use “bcl2fastq” before calling
UniverSC:</p>
<pre><code> /usr/local/bin/bcl2fastq -v --runfolder-dir "/path/to/illumina/bcls" --output-dir "./Data/Intensities/BaseCalls"\
--sample-sheet "/path/to/SampleSheet.csv" --create-fastq-for-index-reads\
--use-bases-mask Y26n,I8n,I8n,Y50n --mask-short-adapter-reads 0\
--minimum-trimmed-read-length 0</code></pre>
<p>Please adjust the lengths for <code>--use-bases-mask</code>
accordingly for read 1, index 1 (i7), index 2 (i5), and read 2. Ensure
that <code>--create-fastq-for-index-read</code> is used where possible.
Using <code>--no-lane-splitting</code> is optional as UniverSC can
process an arbirtary number of lanes. There is no need to specify index
sequences in the same sheet for cell barcodes, using “NNNNNNNN” will
match all samples and the cell barcodes will be distinguished by the
single-cell processing pipeline. Index sequences should only be used to
demultiplex samples and replicates (not cells).</p>
<h4 id="missing-index-sequences">Missing index sequences</h4>
<p>If a sequencing facility has demultiplexed the samples for you
without this, UniverSC will attempt to extract index sequences from
FASTQ headers in read 1. If index sequences are not stored in the file
headers and samples have already been demultiplexed, a dummy index file
of the same number of reads as R1 and R2 will be required. As a
workaround, you can generate this by copying the R1 and R2 files and
replacing the sequences with the first barcode in the relevant
whitelist. For example:</p>
<pre><code>index1="TAAGGCGA"
index2="AAGGAGTA"
# create new files
cp R1_file.fastq I1_file.fastq
cp R2_file.fastq I2_file.fastq
# replace sequences
sed -i "2~4s/^.*$/${index1}/g" I1_file.fastq
sed -i "2~4s/^.*$/${index2}/g" I2_file.fastq
# replace quality scores
sed -i "4~4s/^.*$/IIIIIIII/g" I1_file.fastq I2_file.fastq</code></pre>
<p>This results in a new “sample index” for each demultiplexed sample.
To combine demultiplexed sampls for dual indexed techniques use the
following:</p>
<pre><code># for fastq files
cat Sample1_R1_file.fastq Sample2_R1_file.fastq Sample3_R1_file.fastq > Combined_R1_file.fastq
cat Sample1_R2_file.fastq Sample2_R2_file.fastq Sample3_R2_file.fastq > Combined_R2_file.fastq
cat Sample1_I1_file.fastq Sample2_I1_file.fastq Sample3_I1_file.fastq > Combined_I1_file.fastq
cat Sample1_I2_file.fastq Sample2_I2_file.fastq Sample3_I2_file.fastq > Combined_I2_file.fastq
# for compressed files (not need to uncompress)
cat Sample1_R1_file.fastq.gz Sample2_R1_file.fastq.gz Sample3_R1_file.fastq.gz > Combined_R1_file.fastq.gz
cat Sample1_R2_file.fastq.gz Sample2_R2_file.fastq.gz Sample3_R2_file.fastq.gz > Combined_R2_file.fastq.gz
cat Sample1_I1_file.fastq.gz Sample2_I1_file.fastq.gz Sample3_I1_file.fastq.gz > Combined_I1_file.fastq.gz
cat Sample1_I2_file.fastq.gz Sample2_I2_file.fastq.gz Sample3_I2_file.fastq.gz > Combined_I2_file.fastq.gz</code></pre>
<p>As this needs to done on a case-by-case basis it has not been
implemented by the UniverSC core functions. We provide this workaround
for using published data and data already processed by sequencing
facilities. Please contact the maintainers or file an issue on GitHub if
you are having problems with this case.</p>
<h4 id="custom-inputs">Custom inputs</h4>
<p>Custom inputs are also supported by giving the name “custom” and
length of barcode and UMI separated by a “_” character.</p>
<p>e.g., Custom (16bp barcode, 10bp UMI): <code>custom_16_10</code></p>
<p>Custom barcode files are also supported for preset technologies.
These are particularly useful for well-based technologies to demutliplex
based on the wells.</p>
<p>Note that custom inputs do not remove linker or adapter sequences for
combinatorial indexng technologies. These must be removed from the Read
1 file before running UniverSC. To request a preset technology setting
instead, please submit a feature request on GitHub as described
below.</p>
<h2 id="release">Release</h2>
<p>This tool will be released open-source (see <a
href="#licensing">legal stuff</a> below). We welcome any feedback on it
and any contributions to improve it. Hopefully it will save people time
by making it easier to compare technologies.</p>
<p>We have tested it on several technologies but we need users like you
to let us know how we can improve it. We hope that it will save you time
by handing tedious parts of data formatting so that you can focus on the
results.</p>
<h3 id="citation">Citation <span id="Citation"><span></h3>
<p>Please cite our publication when you use our software as follows:</p>
<p>Battenberg, K., Kelly, S.T., Ras, R.A., Hetherington, N.A., Hayashi,
K., and Minoda, A. (2022) A flexible cross-platform single-cell data
processing pipeline. <em>Nat Commun</em> <strong>13</strong>(1): 1-7.
https://doi.org/10.1038/s41467-022-34681-z</p>
<pre><code>@Article{pmid36369450,
author="Battenberg, K. and Kelly, S. T. and Ras, R. A. and Hetherington, N. A. and Hayashi, M. and Minoda, A. ",
title="{{A} flexible cross-platform single-cell data processing pipeline}",
journal="Nat Commun",
year="2022",
volume="13",
number="1",
pages="1-7",
month="Nov",
note = {https://github.com/minoda-lab/universc package version 1.2.7},
URL = {https://doi.org/10.1038/s41467-022-34681-z}
}</code></pre>
<p>The preprint can also be found here:</p>
<p>Kelly, S.T., Battenberg K., Hetherington, N.A., Hayashi, K., and
Minoda, A. (2021) UniverSC: a flexible cross-platform single-cell data
processing pipeline. bioRxiv 2021.01.19.427209; doi: <a
href="https://doi.org/10.1101/2021.01.19.427209">https://doi.org/10.1101/2021.01.19.427209</a>
package version 1.2.7. <a
href="https://github.com/minoda-lab/universc">https://github.com/minoda-lab/universc</a></p>
<pre><code>@article {Kelly2021.01.19.427209,
author = {Kelly, S. Thomas and Battenberg, Kai and Hetherington, Nicola A. and Hayashi, Makoto and Minoda, Aki},
title = {{UniverSC}: a flexible cross-platform single-cell data processing pipeline},
elocation-id = {2021.01.19.427209},
year = {2021},
doi = {10.1101/2021.01.19.427209},
publisher = {Cold Spring Harbor Laboratory},
abstract = {Single-cell RNA-sequencing analysis to quantify RNA molecules in individual cells has become popular owing to the large amount of information one can obtain from each experiment. We have developed UniverSC (https://github.com/minoda-lab/universc), a universal single-cell processing tool that supports any UMI-based platform. Our command-line tool enables consistent and comprehensive integration, comparison, and evaluation across data generated from a wide range of platforms.Competing Interest StatementThe authors have declared no competing interest.},
eprint = {https://www.biorxiv.org/content/early/2021/01/19/2021.01.19.427209.full.pdf},
journal = {{bioRxiv}},
note = {package version 1.2.7},
URL = {https://github.com/minoda-lab/universc},
}
</code></pre>
<p>The software can also be cited directly as a manual:</p>
<pre><code>@Manual{,
title = {{UniverSC}: a flexible cross-platform single-cell data processing pipeline},
author = {S. Thomas Kelly, Kai Battenberg, Nicola A. Hetherington, Makoto Hayashi, and Aki Minoda},
year = {2023},
note = {package version 1.2.7},
url = {https://github.com/minoda-lab/universc},
}</code></pre>
<h3 id="bug-reports">Bug Reports <span id="Issues"><span></h3>
<h4 id="reporting-issues">Reporting issues</h4>
<p>To add further support for other technologies or troubleshoot
problems, please submit an Issue to the GitHub repository:
https://github.com/minoda-lab/universc/issues</p>
<p>Please submit <a
href="https://github.com/minoda-lab/universc/issues">issues</a> on
GitHub to report problems or suggest features. <a
href="https://github.com/minoda-lab/universc/pulls">Pull requests</a> to
the <code>dev</code> branch on GitHub are also welcome to add features
or correct problems. Please see the <a
href="CONTRIBUTING.md">contributor guide</a> for more details.</p>
<h3 id="requesting-new-technologies">Requesting new technologies</h3>
<p>Where possible, please provide an minimal example of the first few
lines of each FASTQ file for testing purposes.</p>
<p>It is also helpful to describe the technology, such as:</p>
<ul>
<li>length of barcode</li>
<li>length of UMI</li>
<li>which reads they’re on</li>
<li>whether there is a known barcode whitelist available</li>
<li>whether adapters or linkers are required</li>
<li>whether a preprint, publication, or company specifications are
available</li>
</ul>
<p>Technologies that may be difficult to support are those with:</p>
<ul>
<li>barcodes longer than 16bp</li>
<li>barcodes with phase blocks or varying length</li>
<li>UMIs longer than 12bp</li>
<li>technologies that do not have UMI</li>
<li>combinatorial indexing</li>
<li>dual indexing</li>
</ul>
<p>Please bear this in mind when submitting requests. We will consider
to add further technologies but it could take significant resources to
add support for techniques with these designs. Note that updates to the
tool have added support for several examples of these.</p>
<h2 id="installation">Installation <span id="installation"><span></h2>
<p>This script requires Cell Ranger to be installed and exported to the
PATH (version 3.0.0 or higher recommended). The script itself is
exectuable and does not require installation to run but you can put it
in your PATH or bin of your Cell Ranger install if you wish to do so. We
provide scripts to do this for your convenience.</p>
<p>See the details below on how set up Cell Ranger and
launch_universc.sh.</p>
<h4 id="download-universc">Download UniverSC</h4>
<p>To download UniverSC open a terminal prompt and enter the following
commands.</p>
<pre><code>cd $HOME/Downloads
git clone https://github.com/minoda-lab/universc.git
cd universc</code></pre>
<h3 id="quick-start">Quick Start</h3>
<p>If you already have Cell Ranger installed, then you can run the
script without installing it.</p>
<pre><code>bash launch_universc.sh</code></pre>
<p>You can call it in another directory by giving the path to the
script.</p>
<pre><code>cd $/HOME/my_project
bash $HOME/Downloads/universc/launch_universc.sh</code></pre>
<p>See the details below on how to install Cell Ranger and
launch_universc.sh add them to the PATH so that
<code>launch_universc.sh</code> can be run from any directory.</p>
<h3 id="runnning-in-a-git-repository">Runnning in a git repository</h3>
<p>If you are running code in a git repository you can add UniverSC as a
submodule.</p>
<pre><code>cd $/HOME/my_git_repo
git submodule add https://github.com/minoda-lab/universc.git
bash universc/launch_universc.sh</code></pre>
<h3 id="system-requirements">System Requirements</h3>
<p>In principle, the script can run on any Unix systems with Cell Ranger
installed. You can check whether Cell Ranger is already availble by
running:</p>
<pre><code>whereis cellranger</code></pre>
<p>You can see which Cell Ranger installation will run as follows:</p>
<pre><code>which cellranger
cellranger count --version</code></pre>
<p>If Cell Ranger is already installed on your system, you can add it to
your $PATH as follows:</p>
<pre><code>export PATH=/home/username/path/to/cellranger-x.x.x:$PATH </code></pre>
<h4 id="installing-dependencies">Installing dependencies</h4>
<p>If Cell Ranger is not installed on your system, you must install it
before running launch_universc.sh.</p>
<p>Please see the <a
href="https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger">manual
for Cell Ranger</a> on the 10x Genomics website for more details on how
to use it. We provide support for passing various options to Cell Ranger
and sensible defaults for each technology.</p>
<p>This script is compatible with the installation of Cell Ranger that
you can <a
href="https://support.10xgenomics.com/single-cell-gene-expression/software/downloads/latest">download</a>
from the 10x Genomics website and gives the same output formats.</p>
<p>However, we recommend to use the open-source release of Cell Ranger
on GitHub. This is release on an MIT License and is not subject to the
10x Genomics End User License Agreement.</p>
<p>We provide open-source repositories with minor updates for
compatibility with current versions of dependencies.</p>
<p>The code is available here:</p>
<p><a
href="https://github.com/minoda-lab/cellranger/releases">https://github.com/minoda-lab/cellranger/releases</a></p>
<p>We also provide Docker images for Cell Ranger versions 2.0.2, 2.1.0,
2.1.1, and 3.0.2:</p>
<p><a
href="https://github.com/minoda-lab/cellranger_clean/packages">https://github.com/minoda-lab/cellranger_clean/packages</a></p>
<p><a
href="https://hub.docker.com/r/tomkellygenetics/cellranger_clean/tags">https://hub.docker.com/r/tomkellygenetics/cellranger_clean/tags</a></p>
<h4 id="cluster-mode-configuration">Cluster Mode configuration</h4>
<h4 id="software-requirements">Software Requirements</h4>
<p>These have been pre-installed in the Docker image described
above.</p>
<p>A full example of installation is available in the <a
href="https://github.com/minoda-lab/cellranger">GitHub repository</a>
and on <a
href="https://hub.docker.com/r/tomkellygenetics/cellranger_clean/dockerfile">DockerHub</a>.</p>
<ul>
<li>Python 2.7.13</li>
<li>rust 1.28.0</li>
<li>clang 6.0</li>
<li>go 1.11</li>
<li>node 8.11.4</li>
<li>Cython 0.28.0</li>
<li>STAR 2.5.1b</li>
<li>bcl2fastq 2.19.1.403</li>
<li>tsne 0.15</li>
</ul>
<p>The following additional shell utilities are required. Mac OS and
most Linux distributions come with these pre-installed.</p>
<ul>
<li>make 3.81</li>
<li>git 2.20.1</li>
<li>sed (GNU sed) 4.4 (gsed)</li>
<li>tar 2.8.3</li>
<li>rename 0.20 (perl-rename)</li>
<li>perl 5.26.1</li>
<li>rsync 2.6.9</li>
</ul>
<p>Note that rename is installed by default on Mac, Ubuntu and Debian
but a different version must be used on other Linux distrubutions.</p>
<p>CentOS and Fedora:</p>
<pre><code>sudo yum install prename</code></pre>
<pre><code>sudo dnf install prename</code></pre>
<p>Red Hat Linux:</p>
<pre><code>sudo rpm install prename</code></pre>
<p>Arch Linux:</p>
<pre><code>yay perl-rename</code></pre>
<h5 id="recommended-software">Recommended software</h5>
<ul>
<li>git-lfs 2.10.0</li>
</ul>
<h4 id="hardware-requirements">Hardware requirements</h4>
<ul>
<li>8-core Intel or AMD processor (16 cores recommended)</li>
<li>64GB RAM (128GB recommended)</li>
<li>1TB free disk space</li>
<li>64-bit CentOS/RedHat 6.0 or Ubuntu 12.04</li>
</ul>
<h4 id="ensuring-write-access-to-cell-ranger">Ensuring write-access to
Cell Ranger</h4>
<p>The conversion process requires write-access to to the Cell Ranger
install directory so an install on your user directory is
recommended.</p>
<p>You can check where Cell Ranger is installed with:</p>
<pre><code>which cellranger</code></pre>
<p>If calling the script gives the help menu, launch_universc.sh has
sucessfully run with access to the directories that it needs. It will
give an error message if the Cell Ranger directory is not writeable.</p>
<pre><code>bash launch_universc.sh</code></pre>
<p>This script requires Cell Ranger (version 3.0.0 or higher
recommended) to be installed and have write-access to the Cell Ranger
install directory, so an install on your user directory is recommended.
This script also requires Cell Ranger to be exported to the PATH. The
script itself is exectuable and does not require installation to run but
you can put it in your PATH or bin of your Cell Ranger install if you
wish to do so.</p>
<p>This script will run in bash on any OS (but it has only been tested
on Linux Debian). Running Cell Ranger with this configuration requires a
lot of memory (40Gb) so running on server is recommended. SGE job modes
are supported to run Cell Ranger with multiple threads.</p>
<p>This is required because launch_universc.sh will make changes to the
Cell Ranger install to ensure compatibility with the technology running.
A local install in you user home directory is needed to make these
changes. This ensures that these changes do not affect jobs run by other
users and allows launch_universc.sh to change the whitelist and source
code as needed.</p>
<p>These changes are reversible but mean that only one technology can be
run at the same time. You can restore original configurations with:</p>
<pre><code>bash launch_universc.sh -t "10x" --setup</code></pre>
<h5 id="local-install">Local install</h5>
<p>If Cell Ranger is not already installed we recommend installing it in
a directory that you have write access to such as
<code>$HOME/local</code>.</p>
<h5 id="importing-an-installed-version-of-cell-ranger">Importing an
installed version of Cell Ranger</h5>
<p>If Cell Ranger has been installed by a system administrator, you will
only have read-access to that installation. You can still use rather
than installing a new version but you will need to copy it to your home
directory and add this version to your PATH.</p>
<pre><code>mkdir -p $HOME/local
cd ~/local
installed_version=$(echo $(which cellranger) | rev | cut -d"/" -f2- | rev)
cp -rv $installed_version .
installed_directory=$(echo $(which cellranger) | rev | cut -d"/" -f2 | rev)
cd $installed_directory
new_version=$(pwd)
#remove previous version from PATH
export PATH=$(echo $PATH | sed "s;$installed_version:;;g")
#add new version to PATH
eval $(echo export PATH=$new_version:\$PATH)
cd ..</code></pre>
<p>You should be able to see that the locally installed version can be
called as follows:</p>
<pre><code>echo $PATH
which cellranger
bash launch_universc.sh</code></pre>
<h4 id="installing-launch_universc.sh-to-the-path">Installing
launch_universc.sh to the PATH</h4>
<h5 id="running-the-script">Running the script</h5>
<p>Adding the script to the PATH is not absolutely neccessary, it can be
called as follows from the directory that it is downloaded in.</p>
<pre><code>cd $HOME/Downloads
git clone https://github.com/minoda-lab/universc.git
cd universc
bash launch_universc.sh</code></pre>
<h5 id="automated-configuration">Automated configuration</h5>
<p>We provide a Makefile with all necessary configurations to
automatically check whether launch_universc.sh is installed
correctly.</p>
<p>You can specify any directory to install as a “prefix”. This will
create a directory “$HOME/local/universc-0.3” where the files needed
will be stored.</p>
<pre><code>cd $HOME/Downloads
git clone https://github.com/minoda-lab/universc.git
make
make install prefix=$HOME/local</code></pre>
<p>It is also possible to add the current working directory as the
installed directory.</p>
<pre><code>make install prefix="."</code></pre>
<p>In this case <em>do not</em> delete the installed directory after you
install it or the script will fail to run.</p>
<p>By default it will be installed in a root directory with read-only
access. This requires administrator priviledges. Note that the manual
can onlly be installed with root priviledges.</p>
<pre><code>sudo make install
sudo make manual</code></pre>
<p>You can verify that launch_universc.sh has been added to the
PATH.</p>
<pre><code>echo $PATH
which launch_universc.sh</code></pre>
<p>You can then run <code>launch_universc.sh</code> from any working
directory.</p>
<h5 id="updating">Updating</h5>
<p>If launch_universc.sh is already installed and you wish to update it,
first you need to pull the changes from GitHub from the universc
directory.</p>
<pre><code>cd universc
git pull origin master
make</code></pre>
<p>Then you can update to the directory of your choice using the same
options for <code>--prefix</code> as to install.</p>
<pre><code>make install prefix=$HOME/local</code></pre>
<p>This is remove previous versions of launch_universc.sh and install
the latest version.</p>
<p>The manual can be updated with:</p>
<pre><code>sudo make manual</code></pre>
<h5 id="uninstalling">Uninstalling</h5>
<p>Before uninstalling UniverSC please ensure that any versions of Cell
Ranger used are restored to their default configuration:</p>
<pre><code>export PATH=/Users/tom/Downloads/cellranger-x.y.z:$PATH
bash launch_universc.sh -t "10x" --setup</code></pre>
<p>We provide an automated script to reverse the changes above.</p>
<pre><code>make uninstall</code></pre>
<p>This is will automatically detect the installation of
launch_universc.sh.</p>
<p>If multiple versions of Cell Ranger are present, you can specify
which to remove with.</p>
<pre><code>make remove prefix=$HOME/local</code></pre>
<h4 id="custom-shell">Custom shell</h4>
<p>Make will install the script with <code>bash</code> by default but
alternative shells are supported. You will need to run the install
script or run it with your shell of choice.</p>
<pre><code>sh launch_universc.sh
sh inst/INSTALL --prefix $HOME/local
launch_universc.sh</code></pre>
<pre><code>ksh launch_universc.sh
ksh inst/INSTALL --prefix $HOME/local
launch_universc.sh</code></pre>
<pre><code>zsh launch_universc.sh
zsh inst/INSTALL --prefix $HOME/local
launch_universc.sh</code></pre>
<pre><code>fish launch_universc.sh
fish inst/INSTALL --prefix $HOME/local
launch_universc.sh</code></pre>
<p>The help menu should reflect the shell used to run it.</p>
<p>To update, similarly run the <code>inst/UPGRADE</code> script with
your chosen shell.</p>
<h3 id="docker-image">Docker image <span id="Docker"><span></h3>
<p>We provide a docker image with all software needed to run
UniverSC.</p>
<p>This requires “docker” to be installed and a valid DockerHub
account.</p>
<p>You can check whether docker is available by running:</p>
<pre><code>which docker
docker run hello-world </code></pre>
<p>This may require you to login to your account.</p>
<pre><code>docker login -u "myusername"</code></pre>
<p>If you cannot run docker on a remote server, contact your systems
administrator.</p>
<h4 id="pulling-from-remote-dockerhub-repository">Pulling from remote
DockerHub repository</h4>
<p>We provide a docker image for UniverSC version 1.2.7.</p>
<p>You can import it if you have docker installed.</p>
<pre><code>docker pull tomkellygenetics/universc:latest</code></pre>
<h4 id="running-processes-in-a-docker-container">Running processes in a
docker container</h4>
<p>Then you can run UniverSC with:</p>
<pre><code>docker run -it tomkellygenetics/universc:latest launch_universc.sh</code></pre>
<p>You can open a shell in the docker image with:</p>
<pre><code>docker run -it tomkellygenetics/universc:latest /bin/bash</code></pre>
<pre><code>docker run -it tomkellygenetics/universc:latest /bin/zsh </code></pre>
<p>Either of these shells are supported.</p>
<p>Note that Docker containers run with a default of 2 GB of memory. It
is recommended to use at least 8 GB of memory as this is often
insufficient for running UniverSC. Ideally, 16 GB of memory should be
used if it is available on your system.</p>
<pre><code>docker run --memory 16g -it tomkellygenetics/universc:latest /bin/bash</code></pre>
<h5 id="building-the-docker-image-locally">Building the Docker image
locally</h5>
<p>The Dockerfile is provided in the repository so it can be built from
source. This will build a Docker image with the latest version of
universc provided that updates to dependencies on GitHub are still
compatible.</p>
<pre><code>git clone https://github.com/minoda-lab/universc.git
docker build -t universc:latest . </code></pre>
<p>Please bear mind that it can take considerable time to install all
necessary dependencies. A stable internet connection is required.</p>
<h3 id="manual-configuration">Manual configuration</h3>
<p>You can manually add the script here to the PATH, for example:</p>
<pre><code>PATH=$HOME/Downloads/universc:$PATH</code></pre>
<p>This means that the directory where the script is can be found from
the shell.</p>
<pre><code>echo $PATH
cd ~
launch_universc.sh</code></pre>
<p>Add the following line to the <code>~/.bashrc</code> file and use
<code>source ~/.bashrc</code> to load a new session. This means that you
do not need to add the sript to the PATH in future sessions.</p>
<pre><code>export PATH=$HOME/Downloads/universc:$PATH</code></pre>
<h3 id="setting-up-cell-ranger-references">Setting up Cell Ranger
references</h3>
<p>This repository comes with almost all necessary files to run test
jobs. Test data and Cell Ranger references are available with Git large
file storage (LFS).</p>
<p>To install git LFS run:</p>
<pre><code>curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install</code></pre>
<p>To import large files from Github change to the “universc” directory
and run:</p>
<pre><code>git lfs pull origin</code></pre>
<p>This provides almost all files required. The STAR index and reference
need to be generated or imported from an existing reference. The
following code detects whether the references are available in an
existing cellranger installation.</p>
<pre><code>cellrangerversion=`cellranger count --version | head -n 2 | tail -n 1 | cut -f2 -d'(' | cut -f1 -d')'`
cellrangerpath=`which cellranger`
# set up cellranger reference
if [[ ! -f test/cellranger_reference/cellranger-tiny-ref/3.0.0/star/SA ]] && [[ -f $(dirname $cellrangerpath)/cellranger-tiny-ref/3.0.0/star/SA ]]; then
rsync $(dirname $cellrangerpath)/cellranger-tiny-ref/3.0.0/star/SA test/cellranger_reference/cellranger-tiny-ref/3.0.0/star/SA
fi
if [[ ! -f test/cellranger_reference/cellranger-tiny-ref/1.2.0/star/SA ]] && [[ -f $(dirname $cellrangerpath)/cellranger-tiny-ref/1.2.0/star/SA ]]; then
rsync $(dirname $cellrangerpath)/cellranger-tiny-ref/1.2.0/star/SA test/cellranger_reference/cellranger-tiny-ref/1.2.0/star/SA
fi</code></pre>
<p>This creates a reference for Cell Ranger here:</p>
<ul>
<li><p>test/cellranger_reference/cellranger-tiny-ref/1.2.0</p></li>
<li><p>test/cellranger_reference/cellranger-tiny-ref/3.0.0</p></li>
</ul>
<h4 id="automated-references">Automated references</h4>
<p>You can reset the references with the automated settings here:</p>
<pre><code>cd test/cellranger_reference/cellranger-tiny-ref/
make clean
make reference
cd ../../..</code></pre>
<h4 id="pre-generated-references">Pre-generated References</h4>
<p>For convenience we provide pre-generated references for the human
genome and various model species available for download:</p>
<p><a
href="https://genomec.gsc.riken.jp/gerg/UniverSC/Premade_references/">https://genomec.gsc.riken.jp/gerg/UniverSC/Premade_references/</a></p>
<h4 id="custom-cell-ranger-references">Custom Cell Ranger
references</h4>
<p>It is also possible to generate a custom reference for any genome
provided you have a FASTQ genome reference file and a GTF/GFF3
annotation file. Please ensure that the chromosomes match between the
FASTA headers and the chromosome column (1st) of the GTF/GFF3 file.</p>
<p>The <code>gffread</code> function includes with the <a
href="http://cole-trapnell-lab.github.io/cufflinks/file_formats/#the-gffread-utility">cufflinks</a>
utility can convert to gtf. For example:</p>
<pre><code>gffread test/cellranger_reference/cellranger-tiny-ref/genes-1.2.0.gff3 -T -o test/cellranger_reference/cellranger-tiny-ref/genes-1.2.0.gtf </code></pre>
<p>To generate new references we first remove the references
imported.</p>
<pre><code>rm -rf test/cellranger_reference/cellranger-tiny-ref/1.2.0 test/cellranger_reference/cellranger-tiny-ref/3.0.0</code></pre>
<p>We then generate references from the FASTA and GTF files as shown in
the following examples:</p>
<pre><code>cellranger mkref --genome=test/cellranger_reference/cellranger-tiny-ref/1.2.0 \
--fasta=test/cellranger_reference/cellranger-tiny-ref/genome-1.2.0.fa \
--genes=test/cellranger_reference/cellranger-tiny-ref/ genes-1.2.0.gtf
cellranger mkref --genome=test/cellranger_reference/cellranger-tiny-ref/3.0.0 \
--fasta=test/cellranger_reference/cellranger-tiny-ref/genome-3.0.0.fa \
--genes=test/cellranger_reference/cellranger-tiny-ref/ genes-3.0.0.gtf</code></pre>
<p>See the Cell Ranger manuals for more <a
href="https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references">details
on references</a>.</p>
<h2 id="usage">Usage <span id="Usage"><span></h2>
<p>The script will:</p>
<ul>
<li>give you a help guide</li>
</ul>
<p><code>bash launch_universc.sh -h</code></p>
<ul>
<li><p>convert R1 files so that barcodes and UMIs are where they’re
expected to be for 10x (this can take some time for larger
files)</p></li>
<li><p>runs Cell Ranger with the same parameters as for 10x and treats
samples exactly the same</p></li>
<li><p>the barcode whitelists are changed and some checks on barcodes
disabled (requires a writeable install of Cell Ranger in your user
directory)</p></li>
<li><p>it can run Cell Ranger in parallel in SGE mode on the server if
you use <code>--jobmode "sge"</code> and set up an
<code>sge.template</code> file</p></li>
<li><p>it can also restore the original Cell Ranger settings for running
10x samples</p></li>
</ul>
<p><code>bash launch_universc.sh --setup --technology "10x"</code></p>
<h3 id="valid-barcodes">Valid barcodes</h3>
<p>Please note that this script alters the barcode whitelist. Known
ICELL8 barcodes are supported but this is not possible with Nadia or
DropSeq chemistry so 100% valid barcodes will be returned.</p>
<h3 id="manual">Manual <span id="Help"><span></h3>
<h4 id="locally-install-manual">Locally install manual</h4>
<p>You can display a manual from the locally installed UniverSC
directory with:</p>
<pre><code> man man/launch_universc.sh </code></pre>
<p>Note that the working directory must be <code>universc</code> or the
full path to the man directory must be given.</p>
<h4 id="installing-the-manual-with-root-priviliges">Installing the
manual with root priviliges:</h4>
<h5 id="automated-configuration-1">Automated Configuration</h5>
<p>We provide an automated script to install the manual.</p>
<pre><code>sudo make manual</code></pre>
<p>You can remove the manual with:</p>
<pre><code>sudo make manual-clean</code></pre>
<h5 id="configuration">Configuration</h5>
<p>The manual can be installed as follows on Mac and Linux.</p>
<pre><code># add manual directory to PATH if not already found
## check config for Linux
if [[ -f /etc/manpath.config ]]
then CONFIG="/etc/manpath.config"
fi
## check config for Mac
if [[ -f /etc/manpaths ]]
then CONFIG="/etc/manpaths"
fi
if [[ ! -z $CONFIG ]]
then MANDIR=`tail -n 1 ${CONFIG}`
else if [[ ! -z $MANPATH ]]
then
SHELL_RC=`echo ~/.${0}rc`
echo "export MANPATH=/usr/local/man" >> $SHELL_RC
MANDIR=`echo ${MANPATH} | cut -d: -f1`
fi
fi
sudo mkdir -p ${MANDIR}/man1
cp man/launch_universc.sh man/launch_universc.sh.1
sudo mv man/launch_universc.sh.1 ${MANDIR}/man1/launch_universc.sh.1
gzip ${MANDIR}/man1/launch_universc.sh.1</code></pre>
<p>Alternatively the man can be installed with:</p>
<pre><code>cp man/launch_universc.sh man/launch_universc.sh.1
sudo install -g 0 -o 0 -m 0644 man/launch_universc.sh.1 ${MANDIR}/man1</code></pre>
<p>The manual can then be called from any directory as follows:</p>
<pre><code>man launch_universc.sh</code></pre>
<h4 id="help-menu">Help menu</h4>
<p>You can access the following help menu with
<code>launch_universc.sh --help</code> in the terminal.</p>
<pre><code>Usage:
bash launch_universc.sh --testrun -t THECHNOLOGY
bash launch_universc.sh -t TECHNOLOGY --setup
bash launch_universc.sh -R1 FILE1 -R2 FILE2 -t TECHNOLOGY -i ID -r REFERENCE [--option OPT]
bash launch_universc.sh -R1 READ1_LANE1 READ1_LANE2 -R2 READ2_LANE1 READ2_LANE2 -t TECHNOLOGY -i ID -r REFERENCE [--option OPT]
bash launch_universc.sh -f SAMPLE_LANE -t TECHNOLOGY -i ID -r REFERENCE [--option OPT]
bash launch_universc.sh -f SAMPLE_LANE1 SAMPLE_LANE2 -t TECHNOLOGY -i ID -r REFERENCE [--option OPT]
bash launch_universc.sh -v
bash launch_universc.sh -h
Convert sequencing data (FASTQ) from Nadia or ICELL8 platforms for compatibility with 10x Genomics and run cellranger count
Mandatory arguments to long options are mandatory for short options too.
--testrun Initiates a test trun with the test dataset
-R1, --read1 FILE Read 1 FASTQ file to pass to Cell Ranger (cell barcodes and umi)
-R2, --read2 FILE Read 2 FASTQ file to pass to Cell Ranger
-I1, --index1 FILE Index (I1) FASTQ file to pass to Cell Ranger (OPTIONAL)
-I2, --index2 FILE Index (I2) FASTQ file to pass to Cell Ranger (OPTIONAL and EXPERIMENTAL)
-f, --file NAME Path and the name of FASTQ files to pass to Cell Ranger (prefix before R1 or R2)
e.g. /path/to/files/Example_S1_L001
-i, --id ID A unique run id, used to name output folder
-d, --description TEXT Sample description to embed in output files.
-r, --reference DIR Path of directory containing 10x-compatible reference.
-t, --technology PLATFORM Name of technology used to generate data.
Supported technologies:
10x Genomics (version 2 or 3 automatically detected): 10x, chromium