feat: integrate new genvm by kp2pml30 · Pull Request #599 · genlayerlabs/genlayer-studio

kp2pml30 · 2024-11-12T16:04:57Z

Fixes <no issue>

What

migrated all samples to actual genvm
substituted backend genvm module with actual one
changed constructor inputs a bit to reflect actual actual calldata and make it's handling predictable (removed typeof)

Why

to be closer to testnet

Testing done

integration tests passed

Decisions made

I changed how method parameters are handled in the UI to support the new calldata format the right way
Moved all llms to webrequests module to reuse your code!
returns from read methods are hacky because currently I haven't implemented calldata parsing in the frontend. For that reason it returns it in a string format, which is partially json compatible. Tests also use that fact

Checks

I have tested this code
I have reviewed my own PR
I have created an issue for this PR
I have set a descriptive PR title compliant with conventional commits

Reviewing tips

@denishacquin please review frontend changes, I tested them by hand but still

Even thought I reread this PR myself, changes are massive and there may be something I missed, feel free to ask any questions and suggest changes

WEBREQUESTSELENIUMPORT = '5001' GENVM_BIN = "/genvm/bin" was added to .env.example and so it should appear in your .env as well

User facing release notes

Users must know that new genvm looks stylish and cool 😎

codecov · 2024-11-12T16:05:59Z

Codecov Report

Attention: Patch coverage is 22.61905% with 260 lines in your changes missing coverage. Please review.

Project coverage is 19.06%. Comparing base (af6baf7) to head (c374d0c).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...ontend/src/components/Simulator/ContractParams.vue	0.00%	117 Missing and 1 partial ⚠️
...nd/src/components/Simulator/ContractMethodItem.vue	0.00%	40 Missing ⚠️
...rontend/src/components/Simulator/ContractParams.ts	0.00%	33 Missing and 1 partial ⚠️
...src/components/Simulator/ConstructorParameters.vue	0.00%	20 Missing ⚠️
frontend/src/hooks/useContractQueries.ts	0.00%	16 Missing ⚠️
frontend/src/components/global/Btn.vue	0.00%	9 Missing ⚠️
...tend/src/components/global/fields/AnyFieldValue.ts	10.00%	9 Missing ⚠️
...d/src/components/Simulator/ContractReadMethods.vue	0.00%	6 Missing ⚠️
.../src/components/Simulator/ContractWriteMethods.vue	0.00%	6 Missing ⚠️
frontend/src/hooks/useInputMap.ts	77.77%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #599      +/-   ##
==========================================
+ Coverage   18.61%   19.06%   +0.45%     
==========================================
  Files         123      127       +4     
  Lines        9618     9759     +141     
  Branches      299      300       +1     
==========================================
+ Hits         1790     1861      +71     
- Misses       7744     7813      +69     
- Partials       84       85       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles


 if __name__ == "__main__":
-    app.run(debug=True, port=os.environ.get("SELENIUMPORT"), host="0.0.0.0")
+    app.run(debug=True, port=os.environ.get("WEBREQUESTPORT"), host="0.0.0.0")


denishacquin

Awesome refactoring of the redundant logic between constructors and methods! Love it. Added a few preliminary questions and comments, have a look when you can and later I'll try to set up the branch locally like you described to do some testing locally. @kp2pml30

kp2pml30 · 2024-11-14T14:49:10Z

I also totally don't understand what pre-commit wants from me, I haven't even changed that file and pre-commit run --all-files passes locally

AgustinRamiroDiaz · 2024-11-14T19:07:19Z

I also totally don't understand what pre-commit wants from me, I haven't even changed that file and pre-commit run --all-files passes locally

probably it was left broken from another person (it's not a requirement to merge)

AgustinRamiroDiaz · 2024-11-14T19:19:22Z

@kirilaa I'm getting this error on the console when opening a contract

EDITED: the right person to tag would be @kp2pml30

endpoint_name:"gen_getContractSchemaForCode"
error:""
traceback:"Traceback (most recent call last):
File "/app/backend/protocol_rpc/endpoint_generator.py", line 46, in endpoint
result = await result
^^^^^^^^^^^^
File "/app/backend/protocol_rpc/endpoints.py", line 347, in get_contract_schema_for_code
return json.loads(await node.get_contract_schema(contract_code))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/node/base.py", line 163, in get_contract_schema
return await genvm.get_contract_schema(code.encode("utf-8"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/backend/node/genvm/base.py", line 167, in get_contract_schema
raise Exception(f"execution failed {res}")
Exception: execution failed ExecutionResult(result=ExecutionFail: Traceback (most recent call last):
File "/app/backend/node/genvm/base.py", line 306, in _run_genvm_host
get_genvm_path(),
^^^^^^^^^^^^^^^^
File "/app/backend/node/genvm/config.py", line 36, in get_genvm_path
_found_at = _find_exe("genvm")
^^^^^^^^^^^^^^^^^^
File "/app/backend/node/genvm/config.py", line 27, in _find_exe
raise Exception(f"Can't find executable {name}, searched at {checked}")
Exception: Can't find executable genvm, searched at [PosixPath('/usr/local/bin/genvm'), PosixPath('/usr/local/sbin/genvm'), PosixPath('/usr/local/bin/genvm'), PosixPath('/usr/sbin/genvm'), PosixPath('/usr/bin/genvm'), PosixPath('/sbin/genvm'), PosixPath('/bin/genvm')]
, eq_outputs={}, pending_transactions=[], stdout='', stderr='')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/backend/protocol_rpc/message_handler/base.py", line 110, in wrapper
result = await result
^^^^^^^^^^^^
File "/app/backend/protocol_rpc/endpoint_generator.py", line 50, in endpoint
raise JSONRPCError(code=-32000, message=str(e))
flask_jsonrpc.exceptions.JSONRPCError
"

AgustinRamiroDiaz · 2024-11-14T19:24:35Z

@kirilaa probably related to my prior question: do we need to install the GenVM? How can we package it so that users can use the studio (simulator)? It'd be great if it was on a container, but it could also be a script to be added to the genlayer-init cli

EDITED: the right person to tag would be @kp2pml30

AgustinRamiroDiaz

This is a partial review, I'll continue tomorrow

AgustinRamiroDiaz · 2024-11-14T19:27:59Z

are we repurposing the webrequest module to be like the node interfaces?

Well, it is really repurposed for reuse of all llms (genvm natively supports less providers right now), while original selenium python wrapper was removed (and replaced by pure selenium with rest API)

It'd be great if we had some renaming or documentation to know that. Maybe a short description of the architecture

kp2pml30 · 2024-11-14T20:03:41Z

@AgustinRamiroDiaz please see updated slack message

Also note, that WEBREQUESTSELENIUMPORT = '5001' GENVM_BIN = "/genvm/bin" was added to .env.example and so it should appear in your .env as well

added it to the pr description as well

kirilaa · 2024-11-15T10:08:01Z

@kirilaa I'm getting this error on the console when opening a contract

I guess you wanted to tag @kp2pml30 :)

AgustinRamiroDiaz · 2024-11-15T11:51:06Z

@kp2pml30 can sentence-transformers==3.1.1 be removed now from the backend/protocol_rpc/requirements.txt ?

AgustinRamiroDiaz · 2024-11-15T12:47:04Z

Whenever I move between contracts, the method gen_getContractSchemaForCode gets called twice

kp2pml30 · 2024-11-15T14:36:58Z

Whenever I move between contracts, the method gen_getContractSchemaForCode gets called twice

this thing is likely to blame (anonymous from the second trace):

watch(
    () => contract.value?.content,
    () => {
      queryClient.invalidateQueries({
        queryKey: ['schema', contract.value?.id],
      });
    },
  );

kp2pml30 · 2024-11-18T15:50:39Z

        decodedData: {
          functionName: method,
-          args,
+          ...args,


NOTE: this caused a type error, transaction type was adjusted as well

cristiam86 · 2024-11-19T12:06:06Z

@@ -1,20 +1,25 @@
-from backend.node.genvm.icontract import IContract
+# { "Depends": "py-genlayer:test" }


@kp2pml30 is this just a comment or is this something that the GenVM uses?

Something genvm uses

Ok so we need to add this line on some contracts. Do we have some documentation for general Intelligent Contracts implementation?

No we don't have it in any public repos. Where should I put it?

There was a repo for "Marvin" documentation: https://github.com/yeagerai/marvin but right now I think that just a Google Doc with the basic stuff would be great to explain that to other devs that are using the simulator today :)

Putting docs in private repos is sort of useless...

of course, but they aren't if we release them at some point.
On the other hand, at least the internal team should be able to understand how to write intelligent contracts fully.

For example this line # { "Depends": "py-genlayer:test" } how can I know when and how to use it?

…hat rose exception

kp2pml30 · 2024-11-20T09:54:16Z

 def get_transaction_by_hash(
    transactions_processor: TransactionsProcessor, transaction_hash: str
-) -> dict:
+) -> dict | None:


This one caused typing exception

jsonrpc-1 | ERROR | jsonrpc error jsonrpc-1 | Traceback (most recent call last): jsonrpc-1 | File "/usr/local/lib/python3.12/site-packages/flask_jsonrpc/site.py", line 186, in dispatch jsonrpc-1 | resp_view = current_app.ensure_sync(view_func)(*params) jsonrpc-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ jsonrpc-1 | File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 254, in __call__ jsonrpc-1 | return call_result.result() jsonrpc-1 | ^^^^^^^^^^^^^^^^^^^^ jsonrpc-1 | File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result jsonrpc-1 | return self.__get_result() jsonrpc-1 | ^^^^^^^^^^^^^^^^^^^ jsonrpc-1 | File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result jsonrpc-1 | raise self._exception jsonrpc-1 | File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 331, in main_wrap jsonrpc-1 | result = await self.awaitable(*args, **kwargs) jsonrpc-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ jsonrpc-1 | File "/usr/local/lib/python3.12/site-packages/typeguard/__init__.py", line 1055, in async_wrapper jsonrpc-1 | check_return_type(retval, memo) jsonrpc-1 | File "/usr/local/lib/python3.12/site-packages/typeguard/__init__.py", line 840, in check_return_type jsonrpc-1 | raise TypeError(*exc.args) from None jsonrpc-1 | TypeError: type of the return value must be dict; got NoneType instead

AgustinRamiroDiaz

Changes look good in general. I don't understand 100% of the code, but I think it'll make more sense for me in the future, once the behavior of the studio matches the expected behavior

Please create GitHub issues for all FIXME and TODOs

AgustinRamiroDiaz · 2024-11-20T12:41:44Z

@kp2pml30 please don't merge if e2e tests don't pass

AgustinRamiroDiaz · 2024-11-20T12:42:45Z

Please fix this error (when deploying an IC) prior to merge

jsonrpc-1             | Error running consensus unhandled errors in a TaskGroup (1 sub-exception)
jsonrpc-1             |   + Exception Group Traceback (most recent call last):
jsonrpc-1             |   |   File "/app/backend/consensus/base.py", line 109, in _run_consensus
jsonrpc-1             |   |     async with asyncio.TaskGroup() as tg:
jsonrpc-1             |   |                ^^^^^^^^^^^^^^^^^^^
jsonrpc-1             |   |   File "/usr/local/lib/python3.12/asyncio/taskgroups.py", line 145, in __aexit__
jsonrpc-1             |   |     raise me from None
jsonrpc-1             |   | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
jsonrpc-1             |   +-+---------------- 1 ----------------
jsonrpc-1             |     | Traceback (most recent call last):
jsonrpc-1             |     |   File "/app/backend/consensus/base.py", line 135, in exec_transaction_with_session_handling
jsonrpc-1             |     |     await self.exec_transaction(
jsonrpc-1             |     |   File "/app/backend/consensus/base.py", line 269, in exec_transaction
jsonrpc-1             |     |     validation_results = await asyncio.gather(*validation_tasks)
jsonrpc-1             |     |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
jsonrpc-1             |     |   File "/app/backend/node/base.py", line 106, in exec_transaction
jsonrpc-1             |     |     receipt = await self.deploy_contract(
jsonrpc-1             |     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
jsonrpc-1             |     |   File "/app/backend/node/base.py", line 165, in deploy_contract
jsonrpc-1             |     |     return await self._run_genvm(
jsonrpc-1             |     |            ^^^^^^^^^^^^^^^^^^^^^^
jsonrpc-1             |     |   File "/app/backend/node/base.py", line 272, in _run_genvm
jsonrpc-1             |     |     return self._set_vote(
jsonrpc-1             |     |            ^^^^^^^^^^^^^^^
jsonrpc-1             |     |   File "/app/backend/node/base.py", line 140, in _set_vote
jsonrpc-1             |     |     leader_receipt.execution_result == receipt.leader_receipt
jsonrpc-1             |     |                                        ^^^^^^^^^^^^^^^^^^^^^^
jsonrpc-1             |     | AttributeError: 'Receipt' object has no attribute 'leader_receipt'
jsonrpc-1             |     +------------------------------------
jsonrpc-1             |

…rollup txs

…ulator into integrate-genvm

github-actions · 2024-11-25T08:03:35Z

🎉 This PR is included in version 0.22.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Integrate new genvm

096b1e8

github-advanced-security AI found potential problems Nov 12, 2024

View reviewed changes

hotfix for failed CI tests

1fe8c32

kp2pml30 force-pushed the integrate-genvm branch from f503f68 to 1fe8c32 Compare November 12, 2024 16:39

denishacquin reviewed Nov 13, 2024

View reviewed changes

Comment thread frontend/src/hooks/useInputMap.ts

denishacquin reviewed Nov 13, 2024

View reviewed changes

Comment thread frontend/src/components/Simulator/GenericParams.vue

denishacquin reviewed Nov 13, 2024

View reviewed changes

Comment thread frontend/src/components/Simulator/GenericParams.ts

denishacquin reviewed Nov 13, 2024

View reviewed changes

rename constructor parameters

70969d4

kp2pml30 mentioned this pull request Nov 13, 2024

Port some examples from old GenVM genlayerlabs/genvm#55

Closed

AgustinRamiroDiaz self-requested a review November 13, 2024 14:44

add genvm fetching with github token

d377c99

kp2pml30 force-pushed the integrate-genvm branch from ecec85a to d377c99 Compare November 14, 2024 08:57

fix aarch64 issues and other minor bugs

35068a8

AgustinRamiroDiaz reviewed Nov 14, 2024

View reviewed changes

Comment thread backend/node/genvm/base.py

AgustinRamiroDiaz reviewed Nov 14, 2024

View reviewed changes

Comment thread backend/protocol_rpc/endpoints.py

AgustinRamiroDiaz reviewed Nov 14, 2024

View reviewed changes

Comment thread docker-compose.yml

AgustinRamiroDiaz reviewed Nov 14, 2024

View reviewed changes

Comment thread docker/Dockerfile.webrequest Outdated

minor updates addressing feedback

9539ce0

kp2pml30 commented Nov 15, 2024

View reviewed changes

Comment thread backend/consensus/base.py

Comment thread docker-compose.yml

add docs, cleanup code

b25162c

AgustinRamiroDiaz reviewed Nov 15, 2024

View reviewed changes

Comment thread backend/protocol_rpc/requirements.txt

fix for aarch64, update genvm to v0.0.6

9920b93

kp2pml30 commented Nov 18, 2024

View reviewed changes

fix a couple of bugs

ae568c7

cristiam86 reviewed Nov 19, 2024

View reviewed changes

kp2pml30 mentioned this pull request Nov 20, 2024

feat: handle serialization for objects with __dict__ magic method #619

Closed

4 tasks

add hello world example, fix consensus, fix typing bug in endpoints t…

3c64c71

…hat rose exception

kp2pml30 commented Nov 20, 2024

View reviewed changes

AgustinRamiroDiaz approved these changes Nov 20, 2024

View reviewed changes

kp2pml30 and others added 2 commits November 20, 2024 17:00

minor review related improvements

7cb2595

Merge branch 'main' into integrate-genvm

3eecf62

cristiam86 approved these changes Nov 20, 2024

View reviewed changes

Den and others added 8 commits November 20, 2024 17:20

increase reliability of detection of contract deployments on tx updates

f67eaed

add call button loading state

0679a3e

fix error in tests inroduced by Den's commit

ecd5cfb

fix routes in e2e tests

795ceb4

fix contract address mapping in deployment tx/unit test

8669c7b

fix integration tests - add migration to drop not null constraint on …

b81bbce

…rollup txs

fix precommit formatting

0d0257b

Merge commit 'refs/pull/624/head' of github.com:yeagerai/genlayer-sim…

c374d0c

…ulator into integrate-genvm

kp2pml30 added the run-tests label Nov 21, 2024

cristiam86 approved these changes Nov 22, 2024

View reviewed changes

kp2pml30 merged commit 6f07d9e into genlayerlabs:main Nov 22, 2024

github-actions Bot added the released label Nov 25, 2024

		@@ -1,20 +1,25 @@
		from backend.node.genvm.icontract import IContract
		# { "Depends": "py-genlayer:test" }

Conversation

kp2pml30 commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Testing done

Decisions made

Checks

Reviewing tips

User facing release notes

Uh oh!

codecov Bot commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Check failure

Uh oh!

Uh oh!

Uh oh!

denishacquin left a comment

Choose a reason for hiding this comment

Uh oh!

kp2pml30 commented Nov 14, 2024

Uh oh!

AgustinRamiroDiaz commented Nov 14, 2024

Uh oh!

AgustinRamiroDiaz commented Nov 14, 2024 • edited by cristiam86 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AgustinRamiroDiaz commented Nov 14, 2024 • edited by cristiam86 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AgustinRamiroDiaz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kp2pml30 commented Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kirilaa commented Nov 15, 2024

Uh oh!

Uh oh!

Uh oh!

AgustinRamiroDiaz commented Nov 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AgustinRamiroDiaz commented Nov 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kp2pml30 commented Nov 15, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

kp2pml30 commented Nov 12, 2024 •

edited

Loading

codecov Bot commented Nov 12, 2024 •

edited

Loading

AgustinRamiroDiaz commented Nov 14, 2024 •

edited by cristiam86

Loading

AgustinRamiroDiaz commented Nov 14, 2024 •

edited by cristiam86

Loading

kp2pml30 commented Nov 14, 2024 •

edited

Loading

AgustinRamiroDiaz commented Nov 15, 2024 •

edited

Loading

AgustinRamiroDiaz commented Nov 15, 2024 •

edited

Loading