Version 2.7.1

A minor patch set to the v2.7.0, introduces the following changes to the Motifs search functionality:

• Allow to hide "close" icon for the track, if it's a last one open in the browser
• Disable Next or Prev buttons if there are no more motifs found in a forward or backward direction
• Allow to disable a Download file option
• Fix the inconsistency in the motifs search results tracks order (they could appear above the reference track)

Prebuilt assets

• Server: catgenome-2.7.1.3360.jar
• CLI: ngb-cli-2.7.1.3360.tar.gz
• Docs: ngb-docs-2.7.1.3360.tar.gz

Version 2.7.0

This version brings a number of features:

• Motifs search
• BLAST search
• Genes panel
• Homologs search
• Variants highlight
• Heatmap visualisation
• Strain lineage
• Multisample VCF support

See the Release 2.7.0 notes for the details

Prebuilt assets

• Server: catgenome-2.7.0.3322.jar
• CLI: ngb-cli-2.7.0.3322.tar.gz
• Docs: ngb-docs-2.7.0.3322.tar.gz

Version 2.6.1

Version 2.6.1

• Support for the Splice Junctions depth filtering

Prebuilt assets

• Server: catgenome-2.6.1.2239.jar
• CLI: ngb-cli-2.6.1.2239.tar.gz
• Docs: ngb-docs-2.6.1.2239.tar.gz

Version 2.6.0

This version brings a number of features:

• Support for the S3-hosted NGS files
• Authentication via SAML and JWT
• RBAC and permissions management
• Improvement in the BigWig and epigenetics visualizations

See the Release 2.6.0 notes for the details

Prebuilt assets

• Server: catgenome-2.6.0.2109.jar
• CLI: ngb-cli-2.6.0.2109.tar.gz
• Docs: ngb-docs-2.6.0.2109.tar.gz

Version 2.5.1

Variants of translation and strand for reference track

Previously reference track could only show forward strand.

Now several options are added:

• View reverse strand
• View AA translation for forward/reverse strands

These options are available via "General" menu of a reference track

Vertical line through all tracks

Show central line option was added for all formats. It draws a vertical line that illustrates the centre of the visualized region for each track. Previously it was only available for BAM.

Short URLs generation and handling

Previously URLs, generated for sharing NGB views, were quite long (e.g. 1000+ letters)
In 2.5.1 URL shortener is embedded into NGB

This feature allows to:

• Generate a named URL (if alias is specified)
• Generate randomly named URL (using hash-function value)

These URLs are much shorter and easier to share

So now it looks like this:

Bed Graph files support

NGB now supports Bed Graph files (.bdg file extension)
Registration and visualization of these files are the same as for WIG files

Allele frequency thresholds settings

BAM coverage track can show colored bars with mismatches distribution at a specific locus

These bars used to get colored only if a certain threshold is exceeded
Previously this value was hardcoded. Now it can be set by changing Allele frequency threshold in "Settings" -> "Alignments" -> "Coverage Options"

Other

Drop down menu sorted

Variations Table fields selector menu is now sorted in alphabetical order

Docker changes

1. NGB core docker is moved to springboot jar instead of war with a tomcat
2. /ngs is set a default folder for Open from NGB server feature. So calling docker run -v ${host_folder}:/ngs will allow to load files from a host machine without a registration
3. NGB demo docker is added, it uses core image and initializes it with demo data and a set of references
4. buildDockerDemo task it added to gradle

Bug fixes

Fix stuck loading indicator for bigWig/bedGraph tracks

Error occurred when user is loading .bw file and selecting chromosome for which data in .bw is not represented.

Now loading indicator disappears as expected.

Fix "Open from URL" to open more supported file types

Previously "Open from URL" menu only allowed to open VCF and BAM files as seen below.

Now it supports formats listed below:

• BAM
• BED
• GFF/GTF
• VCF

Version 2.5

Structural variations visualization

SV visualizer transcript selection

Previously gene-exon view was calculated be merging all transcripts into one. But this algortithm led to a number of issues.

Canonical transcript is not calculated anymore. Instead, the "best" transcript is selected to build a visualization.

By default the following logic is used to select a transcript to visualize (in order of priority):

• Transcript with domains' info and longest CDS sequence
• If no domains are available - transcript with longest CDS sequence
• If no CDS is available - transcript with longest exonic sequence is used (e.g. tRNA)
• Otherwise - random selection

A user can change automatic transcript for both interacting genes. This is made by expanding a list of transcripts for a specific gene (click arrow-up next to gene name) and select a radio-button next to desired entry

SV visualizer domain-exon fix

A bug, which caused invalid mapping of domains to exons for genes located on the reverse strand, is fixed.

Reverse strand genes are handled correctly now.

Genome annotations

Genome menu GUI

Genome menu is added to the header of the Browser tab

This menu displays currently loaded genome and allows to show/hide linked annotation files (genes, protein domains, etc.)

Note: Genes file, which was previously available from a Datasets tree, now is avaialable from a Genome menu

Managing annotations with NGB CLI

ngb-cli now contains add_ann and remove_ann commands that allow linking annotation files to a specific genome (reference), thus making them visible via Genome menu

"Organize tracks" menu

Now it is possible to manage tracks layout via Organize tracks menu added to the Browser tab header

This menu shows a list of currently opened tracks and provides the following features:

• Rearrange display order

  • UP and DOWN buttons move selected tracks by one position in a selected direction
  • TO TOP and TO BOTTOM move selected tracks to the beginning or to the end of the list
  • SORT BY NAME and SORT BY TYPE will arrange tracks according to their names or files types in ascending order
  • Drag and Drop operation using a mouse is also available for the selected list to arrange custom sort order
  • Multiple selection is available via Ctrl or Shift keys
    • Ctrl + LMB selects tracks one by one
    • Shift + LMB selects a range of tracks

• Close selected tracks using CLOSE button

• Apply changes using SAVE button

• Discard changes using CANCEL button

Fit to screen

A "fit to screen" button is added to the Browser tab header

Clicking this button fits all the data tracks to the vertical space of one screen.
No matter how many tracks are opened, each of them will get an even part of the vertical space.

Note: each track type has a minimal heght setting; if there is too little vertical space limitation on min-height, so in some cases - verticall scrolling will still occur

Pretty names for files

In some cases NGS files' names are too long (e.g. sample-name.processing-phase.index-seq.bam). To make it more comfortable for user, "pretty" names are added to the tracks definition.

Now it is possible to specify a "pretty" name for a track at registration time.

These names may be not unique, and can be used to provide a more user-friendly layout of a Datasets tree.

To specify "pretty" name, --pretty (-pt) option is added to reg_file command of ngb-cli:

$ ngb reg_file GRCh38 sample-name.processing-phase.index-seq.bam --pretty sample-name.bam

"Not registered" files list

When opening files without registration via Open from... menu, they will be listed in a Datasets panel under Not registered group.

Not registered group contains tracks that were opened without registration, thus allowing to quickly access recent files.

State of this group is persisted until a web-browser page is refreshed (e.g. F5 is pressed)

Other

Usability improvements

Highlight on hover

• Features on tracks are now highlighted when hovering with a mouse

• This can be turned off via NGB global settings

Navigating to a BAM region with a "probable coverage"

• Clicking region in a BAM track with a "Probable coverage" will zoom in to that region coordinates

View "probable coverage"

Clcik and zoom in to alignments

Synchronous group operations for BAMs

If several BAMs contain different setting for "General -> Show alignments" - ALT+W hotkey sets all BAMs to NOT showing alignments

CLI changes

• Addressing files by relative path

  • If CLI located on the same filesystem as NGB server relative paths can be used to register files
  • $ ngb reg_file GRCh38 ~/my.bam?~/my.bam.bai

• Automatic index search

  • If a folder with a file also contains index (i.e. bai, tbi, idx) with the same name, CLI will find this index automatically. This works for fa, bam, vcf, bed and gff/gtf files

• CLI version info

  • $ ngb version command is added to ngb-cli, which outputs current NGB version

$ ngb version
2.5.0

Performance improvements

Alignments streaming

BAM alignments loading is performed in a streaming manner now.

• Previously NGB server has read all requested data from a BAM file, converted to a shortened format and transmitted back to a client

• Now NGB server process requested data by chunks

This approach provides:

• Processing datasets of any size without running into OutOfMemory issues
• Increasing overall BAM viewing performance

Tracks resizing

Previously when tracks height was resized - track data was rendered for each mouse-mouve event. For large datasets (e.g. regions with thousands of variations) this lead to a poor client perforance

Now when tracks are rendered only on mouse-button-up event, thus allowinf to render any amount of data without performance downgrade

Compatibility improvements

NGB Client now tries to detect whether a web-browser is supported, if not - an alert is shown

Tick "Do not show this again" checkbox to silent such notifications

Version 2.4.2

• Optimizations of a dataset tree, when loading lots of nodes (e.g. 50.000 datasets). According to #5
• Fix bug, that caused invalid gene information visualization when a gene id was not found in NCBI database

Version 2.4.1

Fixed NGB-CLI bug that caused CLI to crash when feature file contained empty lines between header and data rows

Version 2.4

Alignments (BAM/CRAM) track improvements

BAM track now supports three visualization modes defined by zoom level:

• Alignments presence - shows presence/absence of alignments. This mode is available at the lowest zoom levels, e.g. the whole chromosome.

  Dashed ranges contain at least one read:

• Coverage mode - only shows coverage histogram, mismatches are not calculated. This mode starts being used when zooming in to maximum coverage range setting.
  Default is 500KBp, maximum is 10MBp:

• Full mode - shows all information for alignments track, including reads, coverage, splice-junctions and all other features (this was the only mode in previous versions).
  Zoom level for this mode is defined by maximum alignments range setting.
  Default is 100KBp, maximum is 150KBp:

An option is introduced to hide reads' alignments on a BAM track (coverage-only option)

• Reads can be hidden using a BAM track menu (General -> Show alignments)...

• ...or by a hotkey (default is ALT+W). This will apply to ALL the loaded BAM tracks

If there are no reads in a current region - an information message is shown (previously an empty track was shown):

BAM rendering is improved, antialiasing effect for features edges is fixed:

WIG and Alignments coverage scale configuration

WIG tracks and auto-calculated coverage "subtracks" at the alignments tracks now support scale configuration:

• Set manual scale (from-to)
• Set auto scale (default)
• Set log scale

Configuration is done from a track menu item Scale:

Clicking Manual option will show a popup with parameters
They can be applied to a single track, only WIG tracks, only BAM tracks or all tracks:

Variations table changes

Variations filtering now works inside the VARIANTS panel (FILTER panel is removed)

• To enable variations filtering - use a table menu (Show filters option):

• Select a variation type, chromosome or gene from a dropdown list (multiple selection and autocomplete are supported):

• Input filtering values for integer, float and string variations properties (From and To values are used to set a range of values):

• Tick/Untick a checkbox for flag properties:

• Clear a filter for a specific column by clicking a Clear column filter in a column menu:

• Clear all the filters by clicking a Clear all filters button:

Clicking on Variants by type bar filters a variations table by a selected type:

Variations table now supports paging. This allows loading huge VCF files without performance degradation

• Page size is 100 variations
• If a selected VCF file contains more than 100 items, table will show a paging control
• Pages can be switched by clicking a page number or by scrolling a table down (when a limit of 100 items is reached - more variations are be loaded automatically):

Improvements for opening files without registration

Path of the last opened folder is stored now. When Open from NGB server dialog is opened next time - previously used folder will be shown

Open from NGB server dialog now shows a count of selected items and allows clearing the selection:

Index files are selected automatically

• Index files (bai, tbi) are not shown in a Open from NGB server dialog anymore. If a data file and a corresponding index are found in the same directory - only a data file is shown. When a file is selected, its index is used automatically
• If no index is found for a file - its name is shown in gray color (disabled):

More user-friendly styles are applied to the breadcrumbs control:

A user mow can click breadcrumbs and paste a full path to the data directory:

Dataset tree performance improvements

Datasets tree now uses improved rendering approach that allows visualizing up to a million of nodes:

NGB Command Line Interface improvements

NGB-CLI now supports sorting of feature-files (VCF, BED, GTF, GFF, GFF3)

• ngb sort command can be used prior to registering files, if they are unsorted
• Refer to NGB CLI Command Reference for details

NGB CLI now supports recursive deletion of datasets

• -f (--force) option for dd (del_dataset) should be specified to delete a dataset and all it's children
• If a deletion is not forced - an exception will be thrown, if a dataset contains children

NGB CLI now supports registration of BAM/CRAM files without coordinate sorted flag in a header. If a bai index is specified - it is assumed that a file is sorted by coordinate

Other improvements

Now it is possible to configure default tracks visualiztion option from a server configuration file. Refer to Installation guide for details

NGB-logo menu now shows a link to GitHub - issues page, to report any problems or feature requests:

NGB now shows Loading... message when tracks are being loaded. This is useful for embedding, as it does not show placeholder page anymore

Version 2.3

Collapsed mode for track header

Now it's possible to view tracks in a "collapsed" mode. This mode hides a track header to provide more space for data (reduces each track height by 21px)

Collapsed mode:

In collapsed mode track headers are shown as floating semi-transparent bars in the top right corner of the track

Clicking a "hamburger" menu icon opens a track menu (shown on the track header in "non-collapsed" mode):

To enable/disable "collapsed" mode use "Settings -> General -> Tracks" menu:

More vertical space for data

The following changes provide even more vertical space for data:

• Global ruler height reduced by 36px
• Panels tabs' height reduced by 9px
• Variant table row height reduced by 13px

Open files without registration

Now it is possible to open files (BAM, VCF, GFF/GTF, BED) without registration.

"Open file" menu item added to the left toolbar providing two options (both of them require tbi index for the feature files and bai index for alignments being loaded):

• "From NGB Server":

• "From URL":

Build process and installation changes

• Build process is moved from build.sh to gradle, thus eliminating dependency on bash environment. See README for a list of commands
• install.sh script removed in favour of a standalone springboot jar

URL format changes

Several changes to NGB URL format (aimed to simplify embedding NGB):

• Specifying file names instead of IDs for a &tracks parameter
• Specifying all dataset tracks by using only p option of a &tracks parameter
• Filter datasets by genome (&filterByGenome=hg19)
• Display track headers in a collapsed mode by default (&collapsedTrackHeaders)
• Select first chromosome for a reference genome, in case a name is not known ({first-chromosome} should be set as a chromosome name)

Previously track ("b") and parent dataset ("b") had to be integer identifiers:

tracks=[{"b":7,"p":1}]

Now these options should preferably be specified as names (given at registration):

tracks=[{"b":"sv_sample_1.vcf","p":"SV_sample1"}]

For more details see Embedding guide

NGB Command Line Interface changes

• Several changes to NGB CLI (see NGB CLI Command Reference for details):
  • url command added to NGB CLI. It generates an URL that allows launching NGB with a set of opened files and at a specific genome position
  • Reference FASTA files now are registered without converting to internal "nib" format thus significantly decreasing the registration time
  • -ngc option added to reg_ref command, telling NGB not to calculate GC-content values for a reference track

NGB standalone desktop version preview

A preview version of a desktop version is added. This version runs locally on

• Windows
• MacOS
• Linux

Desktop version includes both NGB Java server and JS client packaged into a single platform-specific executable, using Electron framework

4 genomes are available out of the box to start using desktop version immediately:

• GRCh38
• GRCh37
• D.melanogaster
• Mus musculus

Sources are provided in a /desktop folder of the repository.

Details on building desktop version are provided in a desktop README

Other changes

• NGB supports Apple Safari now
• "Close all tracks" button added to the "Browser" panel:

• "Create session" button moved to the left toolbar: