Processing network - change refresh to not alter network topology

[justinshaffer]

Hello,

It would be helpful if the topology of the processing network was maintained following refreshing. For large networks, it can be confusing when going through results, that the network topology is often reshuffled following refreshing. One option is to maintain the same topology during refresh. A second option would be to allow the user the option to 'turn off' refreshing, particularly when no processing is occurring and the user is merely going through what was ran or outputs.

Thanks in advance,

Justin

[antgonza]

Agree this is super annoying so I spent sometime trying to figure it out. Turns out that this is an issue with the library (https://github.com/visjs/vis-network/releases) we use, see links below. In this process, I realized that we haven't updated this library in a really long time; thus, created #2969.

almende/vis#1927
almende/vis#215
almende/vis#2017
almende/vis#787
https://ww3.arb.ca.gov/ei/tools/lib/vis/docs/dataset.html
https://visjs.github.io/vis-network/docs/network/layout.html

[antgonza]

This has been fixed in the last deploy and confirmed (in qiita-rc) by @justinshaffer and @tkosciol.

[antgonza]

Reopening as in the main site the randomization of branches is still happening.

[antgonza]

I have been trying to solve this and found this issue: visjs/vis-network#926; looks like we need to find another library for this.

[antgonza]

Some "good" options to explore:

1. https://github.com/antvis/G6/
2. https://github.com/jsplumb/jsplumb/
3. https://github.com/cytoscape/cytoscape.js

[tkosciol]

Some "good" options to explore:

1. https://github.com/antvis/G6/
2. https://github.com/jsplumb/jsplumb/
3. https://github.com/cytoscape/cytoscape.js

all look really good! I wonder if there is an option from those 3 that would require least adaptation of the codebase?

[antgonza]

I'm investigating - checked those 3 and they are hard to adapt, currently checking https://observablehq.com/@d3/gallery

[antgonza]

OK, did an "in-depth" review of this and some good reads are:

• Papers worth reading about visualizations and plotting:
  a. Open source libraries and frameworks for biological data visualisation: A guide for developers. Good summary, for this issue point 5 is the most relevant
  b. stack overflow discussion a bit old but super relevant
  c. JavaScript: Discover the Graph-Visualization Ecosystem, recent but still list projects that are old and unmaintained

Took the best candidates and created a branch modifying the workflow page as it's an easy test. The summary of these tests is:

• D3: excellent for multiple plots, extremely flexible - which is great but means that we need to code all options - which will require more dev time. A good thing about this is that we could potentially change from networks to other graphs pretty easily. These links are good resources to start and helped me put together the example. The main and complex issue is how the visualization receives the data; which will mean changes also in the backend.
  https://www.youtube.com/watch?v=gda35eYXBJc
  https://github.com/d3/d3-hierarchy
  https://www.d3-graph-gallery.com/graph/dendrogram_basic.html
  https://observablehq.com/@d3/programmatic-zoom
  http://using-d3js.com/06_01_hierarchal.html
  https://javascript.plainenglish.io/learning-d3-how-to-build-a-tree-chart-w-line-by-line-code-explanations-958e04153dba
  https://bl.ocks.org/d3noob/e7e37cfe0e8763cb0915dee33cc2a24b
  http://using-d3js.com/06_02_trees.html

• jsplumb: seems like a nice option, however there is a paid and a community version - the community version seems not so well documented so it seems hard to fully integrate with Qiita AKA not worth following up.
  https://docs.jsplumbtoolkit.com/community/current/index.html
  https://docs.jsplumbtoolkit.com/toolkit/current/articles/layout-hierarchical.html

• (sigma.js)[https://github.com/jacomyal/sigma.js]: Easy adapt and implement in Qiita; however, it's not easy to create directed/hierarchy graphs (see issues below) AKA not worth following up.
  Hierarchy Graphs jacomyal/sigma.js#859
  Hierarchical graphs support Yomguithereal/gexf#7

• springy: seem pretty dead, no updates since 03/05/2018. AKA not worth following up.

• Cytoscape.js: this seems like the best candidate - note that is only for networks AKA we will not be able to use other visualizations. I have the test kind of working and I'll issue a PR adding the main functionality so we can test.

[justinshaffer]

Cool - thanks for the update! It looks like Cytoscape may also help with the issue when an artifact requires two others in the network as input? Justin

…

On Thu, Apr 22, 2021 at 8:44 AM Antonio Gonzalez ***@***.***> wrote: OK, did an "in-depth" review of this and some good reads are: - Papers worth reading about visualizations and plotting: a. Open source libraries and frameworks for biological data visualisation: A guide for developers <https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/full/10.1002/pmic.201400377>. Good summary, for this issue point 5 is the most relevant b. stack overflow discussion <https://stackoverflow.com/a/2366237> a bit old but super relevant c. JavaScript: Discover the Graph-Visualization Ecosystem <https://medium.com/analytics-vidhya/javascript-discover-the-graph-visualization-ecosystem-e637936cfeda>, recent but still list projects that are old and unmaintained Took the best candidates and created a branch <https://github.com/antgonza/qiita/tree/new-network-tests> modifying the workflow page <https://qiita.ucsd.edu/workflows/> as it's an easy test. The summary of these tests is: - D3: excellent for multiple plots, extremely flexible - which is great but means that we need to code all options - which will require more dev time. A good thing about this is that we could potentially change from networks to other graphs pretty easily. These links are good resources to start and helped me put together the example. The main and complex issue is how the visualization receives the data <http://using-d3js.com/06_01_hierarchal.html>; which will mean changes also in the backend. https://www.youtube.com/watch?v=gda35eYXBJc https://github.com/d3/d3-hierarchy https://www.d3-graph-gallery.com/graph/dendrogram_basic.html ***@***.***/programmatic-zoom http://using-d3js.com/06_01_hierarchal.html https://javascript.plainenglish.io/learning-d3-how-to-build-a-tree-chart-w-line-by-line-code-explanations-958e04153dba https://bl.ocks.org/d3noob/e7e37cfe0e8763cb0915dee33cc2a24b http://using-d3js.com/06_02_trees.html - jsplumb <https://jsplumbtoolkit.com/features>: seems like a nice option, however there is a paid and a community version - the community version seems not so well documented so it seems hard to fully integrate with Qiita AKA not worth following up. https://docs.jsplumbtoolkit.com/community/current/index.html https://docs.jsplumbtoolkit.com/toolkit/current/articles/layout-hierarchical.html - (sigma.js)[https://github.com/jacomyal/sigma.js]: Easy adapt and implement in Qiita; however, it's not easy to create directed/hierarchy graphs (see issues below) AKA not worth following up. jacomyal/sigma.js#859 <jacomyal/sigma.js#859> Yomguithereal/gexf#7 <Yomguithereal/gexf#7> - springy <https://github.com/dhotson/springy/pulls>: seem pretty dead, no updates since 03/05/2018. AKA not worth following up. - Cytoscape.js <https://js.cytoscape.org/>: this seems like the best candidate - note that is only for networks AKA we will not be able to use other visualizations. I have the test kind of working and I'll issue a PR adding the main functionality so we can test. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2968 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSDCGGE2325ONNNO2WYHN3TKA77RANCNFSM4KFVA5HA> .

-- Justin Shaffer, PhD IRACDA Postdoctoral Fellow Rob Knight Group Department of Pediatrics, School of Medicine University of California, San Diego justinshafferbio.wordpress.com

[antgonza]

Fixed on 2021.05.

[roininja]

Hi @antgonza,
Apologies if I could not understand (through above comments) but wanted to know if issue visjs/vis-network#926 is resolved since I can't get the nodes sorting in correct order even if I use current version 9.1.9.

[antgonza]

Hi @roininja; our solution was to replace visjs with cytoscape.js. Best,

[roininja]

Hi @roininja; our solution was to replace Vis JS with cytoscape.js. Best,

Thanks @antgonza. Just to clarify, did you mean to say Cytoscape's logic (sorting node order in hierarchical Layout) is included in Vis JS OR if someone wants to use that functionality, then, s/he should switch over to Cytoscape instead of Vis JS?

Many Thanks.

[antgonza]

I meant the latter; switch to cytoscape.js. However, note that we did this around 3 years ago because vis.js didn't have that functionality + it allowed us to extend some other features; however, things might have changed now.