Ported code to Gymnasium API

[ppasupat]

Description

Fixes #10. Ported the class MiniWoBEnvironment to the gymnasium.Env interface. The environments are registered and a test is added to test the API.

Note: This modification changes the signatures of methods in MiniWoBEnvironment. Old code using miniwob-plusplus will no longer work.

Summary of the changes:

• Moved the miniwob package from python/ to the root directory.
• Changed the method signatures in MiniWoBEnvironment to match gymnasium.Env.
  • The original code allows multiple instances to be run at once, in which case actions, observations, etc. become lists. This conflicts with the Gymnasium interface. The behavior has been changed to fit the Gymnasium interface. The old behavior can still be triggered using the num_instances argument.
  • Added MiniWoBActionSpace and MiniWoBStateSpace. Currently, they are not extending existing spaces, and only check if the action/state are instances of MiniWoBAction/MiniWoBState.
  • Added a test (test_api.py) to test the code port.
• Registered all the environments (registration.py)
• Infer the default base_url based on the file path. This removes the need to specify an environment variable.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[pseudo-rnd-thoughts]

Im very uncertain of the MiniWoBActionSpace and MiniWoBObsSpace.
Most training libraries require the use of Box, etc from Gym / Gymnasium.
Is there no equivalent spaces in Gymnasium already?

Could we add CI test soon?

[pseudo-rnd-thoughts]

Add type hints here, it is unnecessary to have the type hints in the docstring

[pseudo-rnd-thoughts]

Add type hints

[pseudo-rnd-thoughts]

Add type hints

[ppasupat]

Thank you for the code review.

Im very uncertain of the MiniWoBActionSpace and MiniWoBObsSpace.
Most training libraries require the use of Box, etc from Gym / Gymnasium.
Is there no equivalent spaces in Gymnasium already?

The challenges are that the action space (1) has a dynamic size (e.g., can click on any element in the web page), and (2) : can be customized (e.g., one could add "drag" or "double click" actions).

I can coerce the spaces to fit the predefined spaces, but it would require big code changes and reduce flexibility somewhat. Can I do this in a different pull request?

Could we add CI test soon?

I can do this in a separate pull request, since it does not depend on the changes in this pull request.

[pseudo-rnd-thoughts]

I think getting the spaces right should be a priority and I would prefer to avoid using this custom space.
Could you look through all of the current spaces, https://gymnasium.farama.org/api/spaces/#fundamental-spaces, Im happy to get on a discord call to discuss what to do if you want

[ppasupat]

I see. Below is what I came up with for the action and observation spaces. What do you think? I would be happy to discuss further on Discord.

Action space

• Current: MiniWoBAction, which can be one of the following subclasses:
  • MiniWoBCoordClick(left, top)
  • MiniWoBElementClick(element) (element is DOMElement or ref integer)
  • MiniWoBType(text) [text is string or list[string]]
  • MiniWoBFocusAndType(element, text) [just a combination of the two above]
• Proposal: Dict (or Tuple) containing:
  • action_type: Discrete(n=NUM_ACTIONS)
    • 0 = None, 1 = CoordClick, 2 = ElementClick, 3 = Type, 4 = FocusAndType
  • clicked_coords: Box([0, 0], [SCREEN_WIDTH, SCREEN_HEIGHT], dtype=float)
    • x and y coordinates bounded by the environment size
    • Only used when action_type = 1
  • clicked_element: Box(-infinity, infinity, dtype=int)
    • integer indicating the element ID from the observation
    • Only used when action_type = 2 or 4
  • typed_text: Text(max_length=TYPING_MAX_LENGTH, charset=TYPING_CHARSET)
    • Only used when action_type = 3 or 4

Observation space

• Current: MiniWoBState, which has the following attributes:
  • phrase (Phrase object) = utterance (str) + its tokenization
  • fields (Fields object) = fields extracted from the utterance (regex-based)
  • dom = root HTML element (DOMElement)
    • DOMElement has attributes like ref, left, top, tag, etc. These will be turned into keys in the Dict space.
  • dom_elements = flattened list of elements (DOMElement)
  • screenshot (PIL Image or None)
• Proposal: Dict (or Tuple)
  • instruction: Text(max_length=INSTRUCTION_MAX_LENGTH, charset=INSTRUCTION_CHARSET)
    • max_length will be long enough to cover all environments (e.g., 512)
    • Tokenization and field extraction will no longer be done automatically.
  • dom_elements: Sequence(Dict), where the Dict keys include:
    • ref: Box(-infinity, infinity, dtype=int)
      • non-zero integer ID
      • ref for normal elements start from 1, while ref for text pseudo-elements counts down from -1
    • parent: Box(0, infinity, dtype=int)
      • ref ID of the parent (0 = no parent, for root element)
    • geom: Box([0, 0, 0, 0], [SCREEN_WIDTH, SCREEN_HEIGHT, SCREEN_WIDTH, SCREEN_HEIGHT])
      • = left, top, width, height
    • tag: Discrete(n=NUM_TAGS)
      • or should this be Text?
    • text: Text
      • Only available for leaf nodes
    • value: Text
      • Only available for <input> nodes
    • id: Text
      • Only available if the node has an "id"
    • classes: Sequence[Text]
      • or should this be Text?
      • Only available if the node has "class"es
    • bg_color: Box([0, 0, 0, 0], [255, 255, 255, 255], int)
    • fg_color: Box([0, 0, 0, 0], [255, 255, 255, 255], int)
    • flags: MultiBinary(n=4)
      • = focused, tampered, targeted, is_leaf
  • screenshot: Box([[0 ... 0] ... [0 ... 0]], [[255 ... 255] ... [255 ... 255]], (SCREEN_WIDTH, SCREEN_HEIGHT), int)

The original DOM dict (as returned by Javascript) will be put in the info dict

[pseudo-rnd-thoughts]

I think that the changes mostly look good. Add the type hinting would be great.

The final thing to solve is the action and observation spaces. Could you check that the proposed spaces are actually instances of the current action / observation spaces. i.e., generate valid actions / observations for the environment and check all fit within the spaces.
Additionally, it would be nice to know what the flattened spaces look like as this is how most users would want to use the spaces.

[pseudo-rnd-thoughts]

When you have addressed the review comments, could you react to the comment in some way for me to know that you have solved it

[ppasupat]

Ah, sorry, it's not done yet. I still need to add type annotations and verify the flattened spaces. I will mark the review comments as resolved when they're done.

[ppasupat]

@pseudo-rnd-thoughts I have finished converting the action and observation spaces into Gymnasium build-in spaces. With this observation space, the seed determinism test also passes now.

I also tried flattening the observation space: the top-level Dict couldn't be flattened since dom_elements is a variable-length Sequence, but each entry of dom_elements and other top-level values were flattened. The test is in miniwob.tests.test_api.

For modularity, can I address the type annotation in a separate pull request? I made a new GitHub Issue here: #14.

[pseudo-rnd-thoughts]

Thanks for making the changes to observation and action spaces.
This PR looks good to me