Add a python package with doit workflows and logic for building and testing jupyter products #90

tonyfast · 2022-05-24T05:40:12Z

This pull request adds some tooling and structure for provisioning, building, and testing jupyter products. the features it adds are:

a module to capture development tasks when working jupyter products
the module provides both interactive and command line objects for interacting with the tasks
logic for building testing environments, jupyter products, and running tests.
ci workflow implementations that export accessibility results in artifacts.
this pull requests works with gitpod pull requests with a minor intervention.

the things it lacks:

a formal use of the task creation for a single command. currently it requires 4-5 but can be condensed.
it lacks documentation and ci is the reference implementation.
the ability to build tests from a pull requests which i think myself and @trallard can get done during our technical sync on monday.

@gabalafou i hope having the branch over here helps with editing and review.

here goes nothing. lets see if the ci goes 🟢 here 🤞

Fixes: Quansight-Labs/jupyter-a11y-mgmt#102
Fixes: Quansight-Labs/jupyter-a11y-mgmt#91

trallard

I left some comments - many of which are questions for my own understanding and to better scope what next steps are needed.

Overall:

@tonyfast this was a big piece of work 🤝 well done on getting all these moving pieces together
we could merge this and treat as our first step towards a maintainer tool
the only changes I would "request" is renaming some methods and classes to give more meaningful names as I know I am going to struggle later on

.github/workflows/test-a11y.yml

jupyter_accessibility/jupyter-axe/package.json

jupyter_accessibility/dodo.py

trallard · 2022-05-31T20:17:41Z

jupyter_accessibility/project.py

+        yield from (self.path / where).rglob("*/package.json")
+
+    # cloning requires the env
+    # everything else requires cloning


I am somehow confused by this statement but still need to go through some other bits

the next step in this work is to set up the conditions for completed tasks in do it. we'll need to build out the DAG.

if someone does not have git then they will need a conda environment ahead of time. these are notes for me to explicitly express the logic for running the tasks.

jupyter_accessibility/project.py

trallard · 2022-05-31T20:33:12Z

jupyter_accessibility/project.py

+            name="yarn",
+            actions=[do(f"{self.conda} yarn install", cwd=target)],
+        )
+        if CI or GITPOD:


I am not sure how playwright caching works? maybe @gabalafou does - but for now this will be our starting point

yea.... this is going to take a meeting of the minds. there should be a way to inspect the state of their cache. might take a little pair programming.

Check out caching browsers in the Playwright docs

jupyter_accessibility/project.py

jupyter_accessibility/dodo.py

trallard · 2022-05-31T20:43:09Z

As for the action on PR - we did not get to this on Monday but I have now a good mental picture of what is needed + some ideas on how we can expose the reports. Will add these to the issues to plan for the next iteration

🎉 thanks for seeing this through @tonyfast - this will be a great foundation to build from

Co-authored-by: Tania Allard <[email protected]>

tonyfast · 2022-06-08T17:29:00Z

i think i addressed as much as i could here. hope y'all have a time for another review.

gabalafou · 2022-06-14T11:03:30Z

Summary of changes my previous commit:

Updated the axe-core js test file, a11y.test.ts, with the latest updates from my version (testing/jupyterlab/tests/a11y.test.ts).
Moved files around under jupyter_accessibility/jupyter-axe to better match structure of testing/jupyterlab
Renamed js test file from a11y.test.ts to axe-violations.test.ts because I felt that a11y.test.ts was too generic.
Replace expect-axe-playwright with my fork so that it attaches axe-core results.json as well as accessibility tree to Playwright test runs.

Things I want to do in a future commit:

Move notebooks/ folder up one more level to match folder structure under testing/
Rename jupyter-axe to jupyterlab
Add docs

Maybe in this PR or a future PR, I also want to remove all of the duplicated code under testing/. It would be nice if it were as easy as just deleting the testing/ folder, but that folder also contains all of the Gitpod config. I haven't had time to think of what to do with the Gitpod work.

I don't want the testing code to live in two places, so I guess that in order to both have a single source of truth for testing code and to keep the Gitpod work, I'll need to follow this up with code that ports the Gitpod config to work with the code in this branch, then I'll be able to delete everything under testing/

gabalafou · 2022-06-14T11:14:14Z

jupyter_accessibility/jupyter-axe/package.json

+  },
+  "dependencies": {
+    "@jupyterlab/galata": "4.0.2",
+    "expect-axe-playwright": "gabalafou/expect-axe-playwright#f8af64b"


This has to be fixed.

I forked expect-axe-playwright because its API wasn't flexible enough for what we want to do.

I see two possible paths forward:

Keep dependency. Upstream my changes to expect-axe-playwright (will probably end up taking on a new form)

Discard dependency. Copy-paste whatever code is useful from expect-axe-playwright into our own source code in this repo and thereby get rid of the dependency altogether.

I'm not sure which approach is best. I lean slightly towards the second option simply because these additional libraries like expect-axe-playwright are very slim. I'm not sure they bring enough to the table to outweigh the downsides to being dependent on them. The only dependency where I don't want to reinvent the wheel is axe-reporter-html, which provides a nice HTML template for summarizing the axe-core violations.

so the changes you made in your fork are to export the output files?

other options:

could we include our patched version of axe fixture in this library itself, and ship it as part of the a package?

is this something we'd really want to target in galata?

I was reviewing PRs and noticed this is hanging. Is there anything I can do to help move the discussion forward?

isabela-pf · 2022-06-30T17:56:42Z

Woohoo! I'm merging this since it has approval and feedback has been addressed. Thank y'all for your work on this.

This reverts commit 0556994, reversing changes made to ed2254d.

tonyfast added 30 commits April 13, 2022 22:31

execute playwright with doit

ff9dc44

rewrite/reshape playwright tests

b117b5b

await violations

0181cd3

update doit tasks

76efa2f

working python classes and code

6db49d5

comment out test_setup:browser for meantime

991d92f

add comments and documentation notes

dbbaf42

add test workflow to star titerating on

4ffe70f

Merge remote-tracking branch 'upstream/master' into doit-pw

1da81a4

ci trigger

1d93756

install python package

447ec17

actions

77eac91

install playwright

5384400

different default dir

c5925b7

add playwright install if ci

c90503e

plural extras

298de10

explicitly install playwright browser on ci

073c2b4

wrong order of tasks

9db4a3a

ci logic should be right now

71b7eb9

shrug

cfd13b3

add explicit channel conda

31b44f3

you gotta have your own playwright

aaf6c03

ci logic back in project

19cf53e

come on playwright

e89ee36

add playwright in a test_setup

38bbc2f

ci condition

7455ba7

echo echo

bf62f36

rm comparison in the test because we really want the data

e65b63b

where are we!

8b52506

export artifacts

8b236b4

trallard added the type: enhancement 💅🏼 New feature or request label May 27, 2022

trallard self-requested a review May 27, 2022 17:03

trallard reviewed May 31, 2022

View reviewed changes

tonyfast and others added 10 commits June 5, 2022 17:51

Update .github/workflows/test-a11y.yml

cb72816

Co-authored-by: Tania Allard <[email protected]>

add notebook for testing, rename function/classes per review

ad51b5f

rm superfluous condition

887e685

rm redundant expression

1f38630

change node package version

4459709

add Field used in other modules

2afd222

explicitly name our depfile so its doesnt overlap with doit conventions.

c8e295b

no body is going to use the extension

cf3e20c

separate ci tasks to be more literate

a0f690d

add documentation

0655932

tonyfast requested review from gabalafou and trallard June 8, 2022 17:29

isabela-pf mentioned this pull request Jun 9, 2022

Team update - Jun 08, 2022 Quansight-Labs/jupyter-a11y-mgmt#136

Closed

trallard added the PR: needs review 👀 This PR is completed and someone needs to have a look label Jun 10, 2022

gabalafou force-pushed the doit-pw branch from 2c8fac1 to 4febf0b Compare June 14, 2022 10:38

Port js updates from testing/jupyterlab

8f82d27

gabalafou force-pushed the doit-pw branch from 4febf0b to 8f82d27 Compare June 14, 2022 10:58

gabalafou reviewed Jun 14, 2022

View reviewed changes

trallard approved these changes Jun 30, 2022

View reviewed changes

isabela-pf merged commit 0556994 into jupyter:main Jun 30, 2022

gabalafou mentioned this pull request Jul 21, 2022

Move Playwright-Axe testing code from jupyter/accessibility Quansight-Labs/jupyter-a11y-testing#1

Merged

gabalafou added a commit to gabalafou/accessibility that referenced this pull request Jul 21, 2022

Revert "Merge pull request jupyter#90 from Quansight-Labs/doit-pw"

962597e

This reverts commit 0556994, reversing changes made to ed2254d.

gabalafou mentioned this pull request Jul 21, 2022

Remove Playwright-Axe testing code #104

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a python package with doit workflows and logic for building and testing jupyter products #90

Add a python package with doit workflows and logic for building and testing jupyter products #90

tonyfast commented May 24, 2022 •

edited by trallard

Loading

trallard left a comment

trallard May 31, 2022

tonyfast Jun 6, 2022

trallard May 31, 2022

tonyfast Jun 6, 2022

gabalafou Jun 11, 2022

trallard commented May 31, 2022

tonyfast commented Jun 8, 2022

gabalafou commented Jun 14, 2022 •

edited

Loading

gabalafou Jun 14, 2022 •

edited

Loading

tonyfast Jun 15, 2022

isabela-pf Jun 28, 2022

isabela-pf commented Jun 30, 2022

Add a python package with doit workflows and logic for building and testing jupyter products #90

Add a python package with doit workflows and logic for building and testing jupyter products #90

Conversation

tonyfast commented May 24, 2022 • edited by trallard Loading

trallard left a comment

Choose a reason for hiding this comment

trallard May 31, 2022

Choose a reason for hiding this comment

tonyfast Jun 6, 2022

Choose a reason for hiding this comment

trallard May 31, 2022

Choose a reason for hiding this comment

tonyfast Jun 6, 2022

Choose a reason for hiding this comment

gabalafou Jun 11, 2022

Choose a reason for hiding this comment

trallard commented May 31, 2022

tonyfast commented Jun 8, 2022

gabalafou commented Jun 14, 2022 • edited Loading

gabalafou Jun 14, 2022 • edited Loading

Choose a reason for hiding this comment

tonyfast Jun 15, 2022

Choose a reason for hiding this comment

isabela-pf Jun 28, 2022

Choose a reason for hiding this comment

isabela-pf commented Jun 30, 2022

tonyfast commented May 24, 2022 •

edited by trallard

Loading

gabalafou commented Jun 14, 2022 •

edited

Loading

gabalafou Jun 14, 2022 •

edited

Loading