-
Notifications
You must be signed in to change notification settings - Fork 998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose input step order and rework calculation of node position in the workflow editor #13641
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
simonbray
commented
Mar 30, 2022
simonbray
force-pushed
the
wf-step-index
branch
from
March 31, 2022 08:07
34ff53c
to
b0e7657
Compare
simonbray
force-pushed
the
wf-step-index
branch
from
March 31, 2022 08:53
b0e7657
to
ca3ab40
Compare
…o previous position
mvdbeek
reviewed
Apr 22, 2022
simonbray
force-pushed
the
wf-step-index
branch
2 times, most recently
from
April 22, 2022 09:41
4fab832
to
6c4ffc5
Compare
simonbray
force-pushed
the
wf-step-index
branch
from
April 22, 2022 10:08
6c4ffc5
to
cce709d
Compare
simonbray
changed the title
Expose input step order in the workflow editor
Expose input step order and rework calculation of node position in the workflow editor
Apr 22, 2022
mvdbeek
reviewed
Apr 22, 2022
mvdbeek
approved these changes
Apr 22, 2022
This comment was marked as resolved.
This comment was marked as resolved.
Brilliant, that is so cool and much needed! |
nsoranzo
added a commit
to nsoranzo/bioblend
that referenced
this pull request
May 9, 2022
…ded input step The test was broken by galaxyproject/galaxy#13641 .
hexylena
added
highlight/power-user
Included at bottom of user-facing release notes (please use either this or highlight, but not both)
and removed
highlight
Included in user-facing release notes at the top
labels
Jul 11, 2022
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 3, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
4 tasks
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 3, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 4, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 4, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 5, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 5, 2022
We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up.
HadleyKing
added a commit
to biocompute-objects/galaxy
that referenced
this pull request
Nov 9, 2022
* fix and slight changes Heading component * fix heading hierarchy Toolbox fix heading hierarchy admin Home add Heading icons to admin Home * fix history import heading * toolBox and historyImport heading adjustments * rename sizes to fit bootstrap undo default heading size changes * admin heading adjustments tool recommendation heading adjustments * various heading level adjustments new hide-element utility class dataset details css refactor to avoid top-gap * various heading level changes * refactor heading levels unify heading sizes * remove headings demo * add headings documentation * fix unit tests * adjust separator heading gap * remove tool card section * remove margin for inline headings * remove toolcard heading css override * Add test for ignored comments * Fix skip XML comments when linting * Add type annotations to ``config_manage.py`` And some code cleanups. * using filter is:deleted to display deleted workflows * creating simple Vue test for restoring deleted WF * passing disabled prop for Tags * adding conditional rendering based on deleted attribute, implementing stubbed out onRestore method in both WFList (just refreshes) and WFDropdown (actually makes the call to restore). onRestore in Dropdown needs to use defined undelete endpoint * removing confirmation message for restoration * connecting to undelete endpoint * fixing test after connecting to endpoint * update the search help box with new filtering option * Test upgrade to bootstrap-vue 2.23. This uses vue-compat to (hopefully?) achieve compatibility with vue3. * Add test for ignored comments * Fix skip XML comments when linting * Fix invocation filtering Fixes galaxyproject#14851 * Rewrite `CondaContext._guess_conda_properties()` to use `conda info` * Require also a recent pyOpenSSL package when installing conda Fix galaxyproject#14848 . * Add `--strict-channel-priority` to conda create/install commands Fix galaxyproject#12790 . This is added only if using conda >=4.7.5 , which includes this bug fix: conda/conda#8819 * Drop confirmation on delete since it's easily revertible now. * Remove jest testing of workflow deletion confirmation (since we don't confirm anymore) * Fix html in helpHtml (missing tags mainly) * Update lib/galaxy/webapps/galaxy/buildapp.py * Drop accept alert from workflow deletion selenium test -- there is no confirmation now * wrap FAI icons in spans, so the tooltip triggers properly * Add breaking tests for container_resolvers index and show * lint * update lib/galaxy/webapps/galaxy/api/container_resolution.py and lib/galaxy/tool_util/deps/views.py * format and change name of class and file * add test for /api/container_resolvers/resolve * update lib/galaxy/tool_util/deps/views.py * Drop broken output.parent logic ⚠ I don't know what this was used for, but none of the dataset-like models have a children list (was probably a relationship at one point). We have no coverage at this point: https://app.codecov.io/github/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/actions/__init__.py and if you add type hints to `out_data` this breaks because there's no children attribute on DatasetInstance. * Use dnspython to check specifically for an "MX" DNS record when validating the domain name of the email address for a new user. Fix galaxyproject#14829 . * Try to fall back to the A record if MX fails * add section indicator to conditionals * Release tool-util package 22.1.3 * Start work on galaxy-tool-util 22.1.4 * ToolSearch: add a front-end Whoosh parser, add ontology search * Fix response encoding when getting tool citations via API Fix the first author surname in: https://usegalaxy.eu/api/tools/lotus2/citations without regressing on galaxyproject#9369 . When decoding the response content, it is better to default to utf-8 than trust the encoding guessed from the content itself. * Add missing ``unittest_utils`` package to galaxy-tool-util Fix: ``` ModuleNotFoundError: No module named 'galaxy.tool_util.unittest_utils' ``` in conda-forge/galaxy-tool-util-feedstock#4 . * Release tool-util package 22.1.4 * Start work on galaxy-tool-util 22.1.5 * Enable attempt restricted values for input parameter connected to subworkflow input * Inject modules & connections, then build runtime state for all steps That means steps can reason about connections that otherwise wouldn't have modules attached when building runtime inputs. * Add jest test for tool search utilities.js * ToolSearch: Conform to style changes in headings * ToolSearch: ToolBox header style changes * reintroduce decoding for PUT /dataset/set_edit calls * Move model/mapping utils into packaged code for reuse by the tool shed. * Always inject all modules, then compute runtime state * Explicitly set test data location for converter tests * Simplify head wrapper - same functionality, no pipe * Fix loading subworkflows from editor interface This probably broke in galaxyproject#13984. There were 3 issues here: Subworkflows are referenced by Workflow.id, not StoredWorkflow.id, so we'd build the wrong url. The other issue is that if we push the new url into the router nothing happens as we re-use the component. Instead we need to listen to route changes and reload the editor config. The third problem is that the main editor component loads the workflow data only in the created hook. I think this needs further refactoring to not use the controller endpoint; everything is available through exisiting APIs and can be cached (toolbox, workflows, tags), but this fixes subworkflow editing for now. * Use biocontainers * Simplify lint and test command now that planemo supports multiple test artefact paths * Adds href to collection download link so users have the ability to copy it. * Bump wrapper version and declare updates to it WF-safe * Install latest miniconda and conda * Fix mistakenly overwriting ``self._conda_version`` with conda-build version. Introduced in commit 0ed6664 . * Retry conda `install` and `create` commands without strict channel priority Restore the possibility to solve environments with packages present in multiple channels but whose requested version is not available in the highest priority one. The issue applies only when using conda >=4.7.5. Fix the following error for the `test/integration/test_resolvers.py::TestCondaResolutionIntegration::test_dependency_install` integration test: ``` ____________ TestCondaResolutionIntegration.test_dependency_install ____________ self = <integration.test_resolvers.TestCondaResolutionIntegration object at 0x7f7009682950> def test_dependency_install(self): """ Test installation of GNUPLOT dependency. """ data = GNUPLOT create_response = self._post("dependency_resolvers/dependency", data=data, admin=True) self._assert_status_code_is(create_response, 200) response = create_response.json() > self._assert_dependency_type(response) test/integration/test_resolvers.py:46: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <integration.test_resolvers.TestCondaResolutionIntegration object at 0x7f7009682950> response = {'cacheable': False, 'dependency_resolver': {'auto_init': True, 'auto_install': False, 'can_uninstall_dependencies': T...nda,defaults', ...}, 'dependency_type': 'conda', 'environment_path': '/tmp/tmp5f_37kj2/conda/envs/__gnuplot@_uv_', ...} type = 'conda', exact = True def _assert_dependency_type(self, response, type="conda", exact=True): if "dependency_type" not in response: raise Exception(f"Response [{response}] did not contain key 'dependency_type'") dependency_type = response["dependency_type"] assert dependency_type == type, f"Dependency type [{dependency_type}] not the expected value [{type}]" if "exact" not in response: raise Exception(f"Response [{response}] did not contain key 'exact'") > assert response["exact"] is exact E assert False is True test/integration/test_resolvers.py:150: AssertionError ``` due to the fact that this test tries to install version 4.6 of gnuplot, while in the highest priority conda-forge channel the oldest version available is 5.0.5. When trying to create the environment, conda fails to resolve it, so Galaxy falls back to create an environment with an unversioned gnuplot, which fails the exact version test. Introduced in commit bb000b2 . * Force recomputation of conda properties after conda `install` conda version and conda-build availability may change after an `install` in the root environment. * Add type annotation to ``conda_util.py`` Also: - Small refactorings * Install and use libmamba solver * Drop converter variable Co-authored-by: Nicola Soranzo <[email protected]> * Drop more unused stuff * Don't do biocontainers for now * Fix mistakenly overwriting ``self._conda_version`` with conda-build version. Introduced in commit 0ed6664 . * Retry conda `install` and `create` commands without strict channel priority Restore the possibility to solve environments with packages present in multiple channels but whose requested version is not available in the highest priority one. The issue applies only when using conda >=4.7.5. Fix the following error for the `test/integration/test_resolvers.py::TestCondaResolutionIntegration::test_dependency_install` integration test: ``` ____________ TestCondaResolutionIntegration.test_dependency_install ____________ self = <integration.test_resolvers.TestCondaResolutionIntegration object at 0x7f7009682950> def test_dependency_install(self): """ Test installation of GNUPLOT dependency. """ data = GNUPLOT create_response = self._post("dependency_resolvers/dependency", data=data, admin=True) self._assert_status_code_is(create_response, 200) response = create_response.json() > self._assert_dependency_type(response) test/integration/test_resolvers.py:46: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <integration.test_resolvers.TestCondaResolutionIntegration object at 0x7f7009682950> response = {'cacheable': False, 'dependency_resolver': {'auto_init': True, 'auto_install': False, 'can_uninstall_dependencies': T...nda,defaults', ...}, 'dependency_type': 'conda', 'environment_path': '/tmp/tmp5f_37kj2/conda/envs/__gnuplot@_uv_', ...} type = 'conda', exact = True def _assert_dependency_type(self, response, type="conda", exact=True): if "dependency_type" not in response: raise Exception(f"Response [{response}] did not contain key 'dependency_type'") dependency_type = response["dependency_type"] assert dependency_type == type, f"Dependency type [{dependency_type}] not the expected value [{type}]" if "exact" not in response: raise Exception(f"Response [{response}] did not contain key 'exact'") > assert response["exact"] is exact E assert False is True test/integration/test_resolvers.py:150: AssertionError ``` due to the fact that this test tries to install version 4.6 of gnuplot, while in the highest priority conda-forge channel the oldest version available is 5.0.5. When trying to create the environment, conda fails to resolve it, so Galaxy falls back to create an environment with an unversioned gnuplot, which fails the exact version test. Introduced in commit bb000b2 . * Force recomputation of conda properties after conda `install` conda version and conda-build availability may change after an `install` in the root environment. * catch two possible specific exceptions before throwing 500 * Add ludwig report datatype * Drop nose It's deprecated and unmaintained, and is broken under Python 3.10 . * Use ``galaxy_driver_class`` to create tool shed driver Use ``autouse`` fixture to configure test driver * Use local import of sqlalchemy sessions If we import too early we just get the inital None. Slightly more robust but still bad. * Drop scripts/function_tests.py * Update cryptography to 38.0.3 * remove the whole trycatch block exceptions should be handled by fastapi * Progress toward fixing extended metadata with Pulsar * refactor FormElement * move style to component * fix tests * deprecate attribute props * fix showField * fix test * add required and optional hints * add test cases for required and optional form elements * make required mark inline in workflow tool forms * remove unused styles * remove unused css class .icon * fix isEmpty for edge cases * add password type to isEmpty string check * fix title display * Consistently serialize tags in Visualization * Restore rating stars and display for published items * Add email to sharable serializer * Add hashed email for gravatar to serializer * Restore gravater in vueified published items * Fix header class * Always call strip() on data_column column values Not just when they start with `c`. Should fix running workflows that were created by manually writing columns in the text area field and then hitting enter. * Drop explicit test-data argument Co-authored-by: Nicola Soranzo <[email protected]> * Remove ability to delete individual files from tool shed. Working toward a simplified, Planemo-used API that is just uploading archives. * Update interactivetool_ml_jupyter_notebook.xml * Fix server errors when querying for invalid repos or repos without valid tools * Use _run_jobs instead of _run_workflows * Use dnspython to check specifically for an "MX" DNS record when validating the domain name of the email address for a new user. Fix galaxyproject#14829 . * Try to fall back to the A record if MX fails * Fix copy_sample_file usage * Fix displaying named tags * Upgrade vue-router to latest 3x series for composable * Add missing `dnspython` dependency in 22.05 * Mark non-slow unit tests with more relevant ``@skip_if_github_down`` decorator * Skip S3 unit tests conditionally on s3fs module presence * Replace ``GALAXY_TEST_INCLUDE_SLOW`` env variable with ``external_dependency_management`` pytest marker Fix mulled tests missing tests in `test/unit/tool_util/test_conda_resolution.py` . Also: - Add `coverage` tox factor to generate coverage report. * Move md5 utility to util * Fix the ``singularity_container_test()`` function Fix ``test_singularity_container_test`` unit test, which failed with: ``` test/unit/tool_util/mulled/test_mulled_update_singularity_containers.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ lib/galaxy/tool_util/deps/mulled/mulled_update_singularity_containers.py:62: in singularity_container_test check_output(exec_command.extend(["bash", "-c", test_command]), stderr=subprocess.STDOUT) /usr/lib/python3.8/subprocess.py:415: in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, /usr/lib/python3.8/subprocess.py:493: in run with Popen(*popenargs, **kwargs) as process: /usr/lib/python3.8/subprocess.py:858: in __init__ self._execute_child(args, executable, preexec_fn, close_fds, > args = list(args) E TypeError: 'NoneType' object is not iterable /usr/lib/python3.8/subprocess.py:1569: TypeError ``` Broken in commit 1dc1e5f . Also: - Use `tmp_path` pytest fixture. - Add type annotations. - Code refactorings. - Install singularity in the mulled GitHub workflow so that the 2 singularity unit tests are not skipped any more. * Fix conda-build version not compatible with the new miniconda Python version introduced in commit 06832df . Also: - Add unit test. - Use `tmp_path` pytest fixture. - Test code refactoring. * fix button hidden from anonymous users * Update migration instructions * Remove rating from published items * Strip item rating helper from controller endpoints * Remove item rating from legacy mako * Change custos <url> in oidc_backends_config * Fix download urls in embedded workflow component * Add workflow display component test * Fix broken subworkflow reodering on workflow import We started reordering workflow steps on import in galaxyproject#13641. If subworkflow steps are reordered before the input connections are set the wrong subworkflow steps are referenced when we build the workflow connection. This broke the IWC tests for gromacs. We now do not reorder subworkflow steps until the parent workflow is being reordered, which is after all connections are set up. * Revert "Use _run_jobs instead of _run_workflows" This reverts commit a01c8c5. _run_workflow has the right type annotations, but 21.09 didn't have this. * Adjust for disabled history edit toggle * Replace ``unittest.TestCase`` in API, framework, performance, Selenium and ToolShed tests In API tests, replace no more working `uses_test_history` decorator (because the `history_id` method parameter is interpreted as a fixture) with a real class-level pytest fixture. As a consequence, replace use of `self.history_id` in API tests because it interferes with the class-level `history_id` fixture. Also: - Some type annotations * Remove unused ``require_new`` and ``GALAXY_TEST_HISTORY_ID`` * Remove outdated check scripts * Add GitHub workflow to check for regression in test class names * Rename class which is not derived from ``TestCase`` to make `.ci/check_test_class_names.sh` happy. * Fix select2 value selection in selenium * Fix minor typo in text case * Fix merge forward of galaxyproject#14918 The new test needs to use the `history_id` fixture and pass it to `self._run_workflow()` . * Add ``require_new_history`` marker to ``pytest.ini`` * Stabilize test_edit_subworkflow selenium test The click positions don't seem to be lined up. Will probably go away with the workflow editor update I'm working on, in the meantime xfail (which turns into xpass when passing https://docs.pytest.org/en/7.1.x/how-to/skipping.html) will do the trick. * Remove incorrect references to datasets instead of dataset instances in run_request.py. See Marius' notes on https://github.com/galaxyproject/galaxy/pull/14938/files. * Fix deprecation of ``ProcessPool.schedule()`` since pebble 5.0 Fix: ``` DeprecationWarning: schedule is deprecated; use submit instead ``` Also: - Add some type annotations * Add types to galaxy/util/zipstream.py. Brought in from Marius' default file work. * More type annotations * Order steps relation of workflow_invocation by order_index * small change to job runner docs to state more explicitly what `check_watched_item()` should return * Fix indentation * Fix all B027 errors reported by new flake8-bugbear * Small refactoring * Fix pytest warning Fix: ``` PytestReturnNotNoneWarning: Expected None, but packages/tool_util/tests/tool_util/test_verify.py::test_file_list returned [TestFile(value=b'A\nB\nC', path='/tmp/tmpi72pz4ur.txt'), TestFile(value=...], which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`? ``` * Cleanup test case a bit. Co-authored-by: Laila Los <[email protected]> Co-authored-by: davelopez <[email protected]> Co-authored-by: Marius van den Beek <[email protected]> Co-authored-by: Nicola Soranzo <[email protected]> Co-authored-by: Assunta DeSanto <[email protected]> Co-authored-by: Martin Cech <[email protected]> Co-authored-by: Dannon Baker <[email protected]> Co-authored-by: Aysam Guerler <[email protected]> Co-authored-by: cat-bro <[email protected]> Co-authored-by: John Chilton <[email protected]> Co-authored-by: Ahmed Awan <[email protected]> Co-authored-by: Wolfgang Maier <[email protected]> Co-authored-by: Nicola Soranzo <[email protected]> Co-authored-by: John Davis <[email protected]> Co-authored-by: qiagu <[email protected]> Co-authored-by: Nate Coraor <[email protected]> Co-authored-by: Anup Kumar <[email protected]> Co-authored-by: Alireza Heidari <[email protected]> Co-authored-by: Simon Bray <[email protected]>
mvdbeek
added a commit
to mvdbeek/galaxy
that referenced
this pull request
Nov 14, 2022
It had always potentially had side effects (I guess it was in the name too ...), but with galaxyproject#13641 the workflow run form would receive reordered inputs if any of the steps were going to be reordered according to the new sorting riles, while the invocation endpoint would use the original, correct input order. That would lead to mixed up inputs.
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/UI-UX
area/workflows
highlight/power-user
Included at bottom of user-facing release notes (please use either this or highlight, but not both)
kind/feature
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements #13393 and some related issues:
top
and minimumleft
coordinates from the positions of each node when saving, so no node can have negative coordinates. It has the pleasant side effect that dragging the canvas around has no effect on the node positions (unlike the current behaviour).update_step_position
refactor action was therefore updated so that the coordinates provided are used to shift the current position, rather than setting a new absolute position. Hopefully this is ok, I guess that action is not being used too widely atm.getBoundingClientRect
(https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect) were being saved, although onlyleft
andtop
should be needed. This has been changed so that for new workflows onlyleft
andtop
are saved.How to test the changes?
(Select all options that apply)
License