Package Graph #102

hbcarlos · 2020-11-09T08:35:14Z

A graph to visualize packages #83
The dependencies are not correct we need to get the dependencies of the packages to show them.

I'm reusing the code from jupyterlab-plugin-graph

jtpio · 2020-11-09T11:40:27Z

The dependencies are not correct we need to get the dependencies of the packages to show them.

Maybe there are some parts that can be ported from the previous backend: https://github.com/mamba-org/mamba-navigator/blob/master/server/list_and_query.py

jtpio · 2020-11-09T11:44:09Z

packages/common/src/components/PkgGraph.tsx

+      });
+      if (ns.size >= 50) {
+        return {
+          name: 'circle',


Looking at the previous implementation from: https://github.com/mamba-org/mamba-navigator/blob/84a3d5a700b30017112d68b98fdd7e63adc9f182/src/components/Network.vue#L70

It looks like it was using the vis.Network, which is likely to be the one from https://visjs.org:

Maybe it's a just a matter of using an equivalent as a cytoscape plugin to have a similar layout?

Yes, this one looks nicer.
I'll look for a similar layout and modify the backend to get the dependencies.

Maybe something like this one: https://cytoscape.org/cytoscape.js-cose-bilkent/

it could also be good to have a lighter library than cytoscape.

Possible alternative:

https://danielcaldas.github.io/react-d3-graph/docs/index.html

vis.js has been splitted; the network part is there https://github.com/visjs/vis-network (react wrapper: https://github.com/crubier/react-graph-vis)

Thanks both for the info.
I changed the library and added an argument to the PackageHandler API, so now we can request package dependencies.

Would be great to access this graph when clicking on a package in jupyter_conda. Maybe having a new column for dependencies or a link in the package description to open the graph in a dialog.

Let me know what do you think.

Looks great!

Maybe having a new column for dependencies or a link in the package description to open the graph in a dialog.

Yes maybe that's the simplest and most visible way for now?

hbcarlos · 2020-11-17T12:07:38Z

I improved the graph and add it to the package manager.
For now, we can open the dependencies graph for a package by clicking on his version (we should change that later)

fcollonval

Thanks @hbcarlos It looks nice.

I left comments (quite a lot sorry).

I have also general question about the rendering. What happen in dark mode?
Could you add arrow to indicate the direction of the dependency?
Could you fade the node not directly dependent on the selected node?

I switch to react-window instead of react-virtualized; the main change is where a cell renderer must be defined. Could you rebase your code please? Let me know if you need help.

fcollonval · 2020-11-17T17:07:56Z

jupyter_conda/envmanager.py

+            {"package": List[dependencies]}
+        """
+        resp = {}
+        ans = await self._execute(self.manager, "repoquery", "depends", "--json", pkg)


As conda does not support this, the manager should be tested. And in this particular case a default value like {name: null} to signify it can't be done is a good candidate.

fcollonval · 2020-11-17T17:14:10Z

jupyter_conda/envmanager.py

+                if type(dep) is dict :
+                    deps = dep.get('depends', None)
+                    if deps :
+                        resp[dep['name']] = deps


I would returned an empty list rather than skipping the entry.

It seems strange to have:

pkg_depends("a") -> {"a": ["sub_a1", "sub_a2"]}

if a has two dependencies. But

pkg_depends("b") -> {}

If b has no dependency.

fcollonval · 2020-11-17T17:18:44Z

jupyter_conda/handlers.py

            query (str): optional string query
        """
+        pkg = self.get_query_argument("package", "")


I would rather use a query arguments like dependencies equals to 0 or 1 to decide if we need to search or to get the dependencies (with default value to false).
See for example PackagesEnvironmentHandler.post

fcollonval · 2020-11-17T17:19:33Z

packages/common/package.json

@@ -50,7 +50,11 @@
    "@lumino/coreutils": "^1.3.0",
    "@lumino/signaling": "^1.2.0",
    "@lumino/widgets": "^1.6.0",
+    "@types/react-d3-graph": "^2.3.4",


This is a devDependencies not a dependencies

fcollonval · 2020-11-17T17:20:09Z

packages/common/package.json

    "jupyterlab_toastify": "^4.1.2",
+    "react": "^16.4.1",


Suggested change

"react": "^16.4.1",

Unneeded as set as peerDependencies

fcollonval · 2020-11-17T17:43:08Z

packages/common/src/services.ts

+
+    let idx: number;
+    if (cancellable) {
+      idx = this._cancellableStack.push(cancel) - 1;


Follow-up on cancellation changes:

Suggested change

idx = this._cancellableStack.push(cancel) - 1;

idx = this._cancellableStack.push({type: 'getDependencies', cancel}) - 1;

fcollonval · 2020-11-17T17:45:33Z

packages/common/src/components/CondaPkgList.tsx

+        }
+        href="#"
+        onClick={(evt): void => {
+          this.props.onPkgGraph(rowData);


You need to stop the propagation of the event otherwise it will trigger an update or a remove action on the package.

Suggested change

this.props.onPkgGraph(rowData);

evt.stopPropagation();

this.props.onPkgGraph(rowData);

fcollonval · 2020-11-17T17:46:05Z

packages/common/src/components/CondaPkgList.tsx

@@ -143,6 +147,23 @@ export class CondaPkgList extends React.Component<IPkgListProps> {
      return <span>{rowData.name}</span>;
    };

+    const versionRender = ({ rowData }: ICellRender): JSX.Element => (


What happen for not installed packages?

Nothing. I think we should open this graph from another place, not from the version of the package but I'm not sure where to add it.

What do you think of this mock-up?

The idea is to add an icon in the toolbar above the packages list (it will be absent if mamba is not available - or disabled with a title advertising mamba 😉 ).
On click on the button, this would open the dependency panel (not sure if on the right of the package list is a good place or if it needs to be a separated panel to allow positioning and resize by the user).

Then selecting a package will result in the display of its dependencies graph. If not package selected, a help message such Select a package to display its dependency graph.

This means also to change the current behavior as onClick on a row changes the status of the package. This should then be restricted to the status icon probably.

Sounds good for me!
For Mamba, I would add a title in the toolbar when Mamba is not installed saying something like: "Install mamba to have all the functionalities" or maybe and alert icon to have a smaller icon in the toolbar.

fcollonval · 2020-11-17T17:54:20Z

packages/common/src/components/PkgGraph.tsx

+    if (this._package) this._updatePackages();
+  }
+
+  public showGraph(pkg: string): void {


fcollonval · 2020-11-17T17:56:10Z

packages/common/src/components/PkgGraph.tsx

+
+import { Conda } from '../tokens';
+
+export class PkgGraph extends ReactWidget {


Could you refractor this into a thin ReactWidget and a React component?

The React component would have:

props

pkgManager

package

state

data: {nodes, links} | null

Call _updatePackages in componentDidMount and componentDidUpdate

_loading is unneeded and could be replaced with _data === null

The idea is to have a component that will be easily extendable to display the dependency graph of an environment.

hbcarlos · 2020-11-18T11:27:47Z

I applied all the changes except for:

Should we handle directly here the fact that mamba will be able to render the graph but not conda?
What happen for not installed packages?

Take a look and let me know if there is something wrong.

hbcarlos · 2020-11-18T12:26:12Z

I'll look at the test tonight or tomorrow.

coveralls · 2020-11-19T08:48:28Z

Pull Request Test Coverage Report for Build 388750202

5 of 21 (23.81%) changed or added relevant lines in 2 files are covered.
130 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-1.7%) to 67.983%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
mamba_gator/handlers.py	4	5	80.0%
mamba_gator/envmanager.py	1	16	6.25%

Files with Coverage Reduction	New Missed Lines	%
src/services.ts	130	46.48%

Totals
Change from base Build 364212499:	-1.7%
Covered Lines:	745
Relevant Lines:	1002

💛 - Coveralls

fcollonval · 2020-11-20T13:20:44Z

👈 Launch a binder notebook on the branch hbcarlos/gator/master

fcollonval · 2020-11-22T14:17:43Z

Great work @hbcarlos

I have two requests regarding the style:

Is it possible to color the node of the selected package differently to highlight it?
Could you set the font size for the node to --jp-ui-font-size0? The current 8px is quite small.

hbcarlos · 2020-11-22T21:44:47Z

Sure, it's done. I'll do a new commit.

Regarding:

The idea is to add an icon in the toolbar above the packages list (it will be absent if mamba is not available - or disabled with a title advertising mamba 😉 ).
On click on the button, this would open the dependency panel (not sure if on the right of the package list is a good place or if it needs to be a separated panel to allow positioning and resize by the user).

Do you want to do it in this PR or open a new one?

fcollonval

Thanks for the reactivity, I added some final suggestions to finish simplifying the REST API and to add some documentation.

I also change a bit the default configuration because the font size of highlighted elements is not synchronized by default with the standard font size.

If you agree on those suggestions. Then we are good to go and pursuit as you suggested in another PR (I created #110 to keep track).

mamba_gator/handlers.py

mamba_gator/rest_api.yml

packages/common/src/services.ts

packages/common/src/tokens.ts

packages/common/src/components/PkgGraph.tsx

hbcarlos · 2020-11-23T13:51:54Z

Thanks, @fcollonval.
I completely agree with those suggestions. You can commit them and merge the PR.

packages/common/src/components/PkgGraph.tsx

fcollonval

Thanks for pushing this through @hbcarlos

Package Graph

170aafc

hbcarlos marked this pull request as draft November 9, 2020 08:35

jtpio reviewed Nov 9, 2020

View reviewed changes

hbcarlos added 2 commits November 16, 2020 13:49

Vis-network and new query argument in PackagesHandler GET route

f3e8446

Added graph to jupyter_conda

a559c89

hbcarlos marked this pull request as ready for review November 17, 2020 12:08

fcollonval requested changes Nov 17, 2020

View reviewed changes

hbcarlos added 4 commits November 18, 2020 09:35

Changes

86a1d28

merge

97ded1c

Cancel task

c99b81a

delete jupyter_conda bundles

7915432

Check test

bdfca10

hbcarlos added 2 commits November 22, 2020 22:45

highlight and font size

e25fe8c

eslint

516fc11

fcollonval mentioned this pull request Nov 23, 2020

Improve package dependency graph integration #110

Open

fcollonval requested changes Nov 23, 2020

View reviewed changes

jtpio reviewed Nov 23, 2020

View reviewed changes

packages/common/src/components/PkgGraph.tsx Show resolved Hide resolved

fcollonval mentioned this pull request Nov 28, 2020

Use React hooks for components instead of class based. #113

Open

fcollonval added 3 commits November 28, 2020 14:14

Apply suggestions from code review

2d497fb

Apply some more suggestions from code review

3e786e0

Fixing usage when mamba is missing

b0c10dd

Fix eslint error

906810a

fcollonval self-requested a review November 28, 2020 15:35

fcollonval approved these changes Nov 28, 2020

View reviewed changes

fcollonval merged commit 4b8378f into mamba-org:master Nov 28, 2020

fcollonval mentioned this pull request Sep 20, 2021

UX/UI proposals for Gator #157

Open

fcollonval mentioned this pull request Nov 18, 2023

Add graph visualization #83

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Package Graph #102

Package Graph #102

hbcarlos commented Nov 9, 2020

jtpio commented Nov 9, 2020

jtpio Nov 9, 2020

hbcarlos Nov 9, 2020

fcollonval Nov 9, 2020

hbcarlos Nov 16, 2020

jtpio Nov 16, 2020

hbcarlos commented Nov 17, 2020

fcollonval left a comment •

edited

Loading

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

hbcarlos Nov 17, 2020

fcollonval Nov 18, 2020

hbcarlos Nov 18, 2020

fcollonval Nov 17, 2020

fcollonval Nov 17, 2020

hbcarlos commented Nov 18, 2020

hbcarlos commented Nov 18, 2020

coveralls commented Nov 19, 2020 •

edited

Loading

fcollonval commented Nov 20, 2020 •

edited

Loading

fcollonval commented Nov 22, 2020

hbcarlos commented Nov 22, 2020

fcollonval left a comment

hbcarlos commented Nov 23, 2020

fcollonval left a comment

	idx = this._cancellableStack.push(cancel) - 1;
	idx = this._cancellableStack.push({type: 'getDependencies', cancel}) - 1;

	this.props.onPkgGraph(rowData);
	evt.stopPropagation();
	this.props.onPkgGraph(rowData);


		import { Conda } from '../tokens';

		export class PkgGraph extends ReactWidget {

Package Graph #102

Package Graph #102

Conversation

hbcarlos commented Nov 9, 2020

jtpio commented Nov 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hbcarlos commented Nov 17, 2020

fcollonval left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hbcarlos commented Nov 18, 2020

hbcarlos commented Nov 18, 2020

coveralls commented Nov 19, 2020 • edited Loading

Pull Request Test Coverage Report for Build 388750202

💛 - Coveralls

fcollonval commented Nov 20, 2020 • edited Loading

fcollonval commented Nov 22, 2020

hbcarlos commented Nov 22, 2020

fcollonval left a comment

Choose a reason for hiding this comment

hbcarlos commented Nov 23, 2020

fcollonval left a comment

Choose a reason for hiding this comment

fcollonval left a comment •

edited

Loading

coveralls commented Nov 19, 2020 •

edited

Loading

fcollonval commented Nov 20, 2020 •

edited

Loading