You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: ChangeLog.md
+10
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,16 @@
1
1
# NEAT has a new home
2
2
NEAT is now a part of the NCSA github and active development will continue here. Please direct issues, comments, and requests to the NCSA issue tracker. Submit pull requests here insead of the old repo.
3
3
4
+
# NEAT v4.2
5
+
- After several bug fixes that constituted release 4.1 and some minor releases, we are ready to release an overhauled vesion of NEAT 4.0.
6
+
- Removed GC bias - it had little to no effect and made implementation nearly impossible
7
+
- Removed fasta creation - we had tweaked this a bit but never got any feedback. It may come back if requested.
8
+
- Improvements/fixes/full implementations of:
9
+
- heterozygosity
10
+
- read creation (now with more reads!)
11
+
- bam alignment/creation
12
+
- bed tool incorporation
13
+
4
14
-Updated "master" branch to "main." - please update your repo accordingly
5
15
# NEAT v4.0
6
16
- Rewritten the models. Models generated on old versions of NEAT will have to be redone, due to the restructuring of the codebase. These new models should be smaller and more efficient. We have replicated the previous default models in the new style. There is no straightforward way to convert between these, unfortuantely.
Copy file name to clipboardExpand all lines: README.md
+16-43
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,13 @@
1
1
# The NEAT Project v4.0
2
-
Welcome to the NEAT project, the NExt-generation sequencing Analysis Toolkit, version 4.0. This is our first (beta) release of the newest version of NEAT. There is still lots of work to be done. See the [ChangeLog](ChangeLog.md) for notes.
2
+
Welcome to the NEAT project, the NExt-generation sequencing Analysis Toolkit, version 4.2. This release of NEAT includes several fixes and a little bit of restructuring. There is still lots of work to be done. See the [ChangeLog](ChangeLog.md) for notes. We have discarded the fasta file writing for now and removed that code. We may add that in as a feature in the future, if users call for it. We also removed GC bias for now. It severely complicated implementation, and had very few noticeable effects. After discussing with some people at the Illinois Institute for Genomic Biology, it sounded like GC bias may be a bit of a non-factor with improved chemistries. These will be reintroduced if needed/called for.
3
3
4
4
We are also working on redeveloping NEAT in Rust, a memory and thread safe language that will lend itself well to the way NEAT works, check that out here: https://github.com/ncsa/rusty-neat
5
5
6
6
Stay tuned over the coming weeks for exciting updates to NEAT, and learn how to [contribute](CONTRIBUTING.md) yourself. If you'd like to use some of our code, no problem! Just review the [license](LICENSE.md), first.
7
7
8
8
NEAT's read-simulator is a fine-grained read simulator. It simulates real-looking data using models learned from specific datasets. There are several supporting utilities for generating models used for simulation and for comparing the outputs of alignment and variant calling to the golden BAM and golden VCF produced by NEAT.
9
9
10
-
This is release v4.0 of the software. While it has been tested, it does represent a shift in the software with the introduction of a configuration file. For a stable release using the old command line interface, please see: [NEAT 3.0](https://github.com/ncsa/NEAT/releases/tag/3.3) (or check out older tagged releases)
10
+
This is release v4.2 of the software. While it has been tested, it does represent a shift in the software with the introduction of a configuration file. For a stable release using the old command line interface, please see: [NEAT 3.0](https://github.com/ncsa/NEAT/releases/tag/3.3) (or check out older tagged releases)
11
11
12
12
To cite this work, please use:
13
13
@@ -31,7 +31,6 @@ Table of Contents
31
31
*[Large single end reads](#large-single-end-reads)
## Requirements (the most up-to-date requirements are found in the environment.yml file)
42
41
42
+
* Some version of Anaconda to set up the environment
43
43
* Python == 3.10.*
44
-
* poetry
44
+
* poetry == 1.3.*
45
45
* biopython == 1.79
46
46
* pkginfo
47
47
* matplotlib
@@ -71,13 +71,20 @@ the NEAT repo, after creating the conda environment:
71
71
> poetry install
72
72
```
73
73
74
+
Notes: If any packages are struggling to resolve, check the channels and try to manually pip install the package to see if that helps (but note that NEAT is not tested on the pip versions.)
75
+
74
76
Test your install by running:
75
77
```
76
78
> neat --help
77
79
```
78
80
81
+
You can also try running it using the python command directly:
82
+
```
83
+
> python -m neat --help
84
+
```
85
+
79
86
## Usage
80
-
NEAT's core functionality is invoked using the read-simulator command. Here's the simplest invocation of read-simulator using default parameters. This command produces a single ended fastq file with reads of length 101, ploidy 2, coverage 10X, using the default sequencing substitution, GC% bias, and mutation rate models.
87
+
NEAT's core functionality is invoked using the read-simulator command. Here's the simplest invocation of read-simulator using default parameters. This command produces a single ended fastq file with reads of length 151, ploidy 2, coverage 10X, using the default sequencing substitution, and mutation rate models.
81
88
82
89
Contents of neat_config.yml
83
90
```
@@ -110,7 +117,6 @@ The default is given:
110
117
111
118
produce_bam: False
112
119
produce_vcf: False
113
-
produce_fasta: False
114
120
produce_fastq: True
115
121
116
122
error_model: full path to an error model generated by NEAT. Leave empty to use default model
@@ -119,11 +125,7 @@ mutation_model: full path to a mutation model generated by NEAT. Leave empty to
119
125
model (default model based on human data sequenced by Illumina)
120
126
fragment_model: full path to fragment length model generate by NEAT. Leave empty to use default model
121
127
(default model based on human data sequenced by Illumina)
122
-
gc_model: Full path to model for correlating GC concentration and coverage, produced by NEAT.
123
-
(default model is based on human data, sequenced by Illumina)
124
128
125
-
partition_mode: by chromosome ("chrom"), or subdivide the chromosomes ("subdivision").
126
-
Note: this feature is not yet fully implemented
127
129
threads: The number of threads for NEAT to use.
128
130
Note: this feature is not yet fully implemented
129
131
avg_seq_error: average sequencing error rate for the sequencing machine. Use to increase or
@@ -134,20 +136,14 @@ include_vcf: full path to list of variants in vcf format to include in the simul
134
136
appear in the input VCF into the final VCF, and the corresponding fastq and bam files, if requested.
135
137
target_bed: full path to list of regions in bed format to target.
136
138
All areas outside these regions will have coverage of 0.
137
-
off_target_scalar: manually set the off-target-scalar when using a target bed (if you want to have some percentage of
138
-
reads from outside the targeted regions. Default is 0. (i.e., setting this to 0.02 would mean off-target areas will
139
-
have a coverage of ~2% of the total coverage). This is an experimental feature.
140
139
discard_bed: full path to a list of regions to discard, in BED format.
141
140
mutation_rate: Desired rate of mutation for the dataset. Float between 0.0 and 0.3
142
141
(default is determined by the mutation model)
143
142
mutation_bed: full path to a list of regions with a column describing the mutation rate of that region,
144
143
as a float with values between 0 and 0.3. The mutation rate must be in the third column as, e.g., mut_rate=0.00.
145
-
no_coverage_bias: Set to true to produce a dataset free of coverage bias
146
144
rng_seed: Manually enter a seed for the random number generator. Used for repeating runs. Must be an integer.
147
145
min_mutations: Set the minimum number of mutations that NEAT should add, per contig. Default is 0. We recommend setting
148
-
this to at least one for small chromosomes, so NEAT will produce at least one mutation per contig.
149
-
fasta_per_ploid: Produce one fasta per ploid. Default behavior is to produce
150
-
a single fasta showing all variants. |
146
+
this to at least one for small chromosomes, so NEAT will produce at least one mutation per contig. |
151
147
152
148
The command line options for NEAT are as follows:
153
149
@@ -156,10 +152,9 @@ Universal options can be applied to any subfunction. The commands should come be
| --log-dir LOG_DIR | Sets the log directory to custom path (default is current working directory |
160
-
| --log-name LOG_NAME | Custom name for log file (default is timestamped) |
155
+
| --log-name LOG_NAME | Custom name for log file, can be a full path (default is current working directory with a name starting with a timestamp)|
161
156
| --log-level VALUE | VALUE must be one of [DEBUG, INFO, WARN, WARNING, ERROR] - sets level of log to display |
162
-
| --log-detal VALUE | VALUE must be one of [LOW, MEDIUM, HIGH] - how much info to write for each log record |
157
+
| --log-detail VALUE | VALUE must be one of [LOW, MEDIUM, HIGH] - how much info to write for each log record |
163
158
| --silent-mode | Writes logs, but suppresses stdout messages |
164
159
165
160
read-simulator command line options
@@ -184,9 +179,8 @@ Features:
184
179
- Can simulate targeted sequencing via BED input specifying regions to sample from
185
180
- Can accurately simulate large, single-end reads with high indel error rates (PacBio-like) given a model
186
181
- Specify simple fragment length model with mean and standard deviation or an empirically learned fragment distribution using utilities/computeFraglen.py
187
-
- Simulates quality scores using either the default model or empirically learned quality scores using utilities/fastq_to_qscoreModel.py
182
+
- Simulates quality scores using either the default model or empirically learned quality scores using `neat gen_mut_model`
188
183
- Introduces sequencing substitution errors using either the default model or empirically learned from utilities/
189
-
- Accounts for GC% coverage bias using model learned from utilities/computeGC.py
190
184
- Output a VCF file with the 'golden' set of true positive variants. These can be compared to bioinformatics workflow output (includes coverage and allele balance information)
191
185
- Output a BAM file with the 'golden' set of aligned reads. These indicate where each read originated and how it should be aligned with the reference
192
186
- Create paired tumour/normal datasets using characteristics learned from real tumour data
@@ -288,27 +282,6 @@ neat read-simulator \
288
282
# Utilities
289
283
Several scripts are distributed with gen_reads that are used to generate the models used for simulation.
290
284
291
-
## neat compute_gc_bias
292
-
293
-
Computes GC% coverage bias distribution from sample (bedrolls genomecov) data.
294
-
Takes .genomecov files produced by BEDtools genomeCov (with -d option).
295
-
(Not yet implemented in NEAT 4.0)
296
-
297
-
```
298
-
bedtools genomecov
299
-
-d \
300
-
-ibam normal.bam \
301
-
-g reference.fa
302
-
```
303
-
304
-
```
305
-
neat compute_gc_bias \
306
-
-r reference.fa \
307
-
-i genomecovfile \
308
-
-w [sliding window length] \
309
-
-o /path/to/prefix
310
-
```
311
-
312
285
## neat model-fraglen
313
286
314
287
Computes empirical fragment length distribution from sample data.
0 commit comments