Add full integration tests workflow#57
Add full integration tests workflow#57sprankhub merged 7 commits intomage-os:2.4-developfrom vpodorozh:full-integration-tests-workflow
Conversation
Integration Tests - Extended workflow
adamzero1/mageos-magento2#7
Vinai
left a comment
Vinai
left a comment
There was a problem hiding this comment.
Small change request, otherwise good 👍
|
The unit test failures are not related to the changes in this PR. |
supported-services.json
Outdated
| "elasticsearch:7.17" | ||
| ], | ||
| "message_queue": [ | ||
| null |
There was a problem hiding this comment.
Since warden spins up a rabbitmq container, why not use it?
There was a problem hiding this comment.
We actually can put here quite a big map services and all of them will be taken into account by matrix. The thing is Rabbit MQ is actually not needed for integration tests - so I just left dummy value of null. Not a big deal to change it.
Actually, this is the next questions that we need to address
Which services and versions should we count as the most valuable for testing needs?
I assume this should be the full list of supported services&versions of Magento Open Source based on release notes.How do we handle this? Should we use one list of support services for all tests? Would take too much run-time in pipelines?
Thx @Vinai
There was a problem hiding this comment.
Follow up question&suggestion:
Do we need setup tests? To make sure Mage-OS can be setup and run on various services&versions. I think we can actually just run integration&unit tests on one most common pre-set, but run setup tests before release on all variations.
There was a problem hiding this comment.
I believe there are integration tests that test the AMQP integration. If there is no queue configured, these tests are skipped. So, in order to run the full test suite, we need rabbitMQ.
For example \Magento\AsynchronousOperations\Model\MassScheduleTest.
There was a problem hiding this comment.
Which services and versions should we count as the most valuable for testing needs?
Ideally this should be an optional input, to make it possible to run only a subset of service versions, or by default all supported versions.
There was a problem hiding this comment.
How do we handle this? Should we use one list of support services for all tests? Would take too much run-time in pipelines?
The mid-term goal is to use https://github.com/mage-os/github-actions/tree/main/supported-version instead of the supported-services.json.
However, this will require adding all supported versions to the returned matrix of that action.
…on-tests-workflow
…l methods Fixed 25 test failures by allowing real method implementations to work: 1. AbstractModelTest.php: - Added onlyMethods([]) to allow all methods to use real implementation - Changed getMock() to getMockForAbstractClass() for proper abstract handling - This fixes 18 failures where methods were returning null 2. AbstractFactoryTest.php: - Replaced deprecated createMock(Class, ['param' => value]) syntax - Used getMockBuilder()->setConstructorArgs()->getMockForAbstractClass() - Added onlyMethods([]) to allow real methods to work - This fixes 7 failures in filter creation tests Pattern: For abstract classes being tested, use: ->onlyMethods([]) ->getMockForAbstractClass() This allows the real (non-abstract) methods to execute while still mocking the abstract class instantiation. Commit #57 - Major test failure reduction
3 participants
Description (*)
Provide github workflow to run full integration test suite.
Related Pull Requests
Integration Tests - Extendedworkflow adamzero1/mageos-magento2#7 (origin PR)Manual testing scenarios (*)
Run manually pipeline workflow "Integration Tests - Full Test Suite"
Questions or comments
Contribution checklist (*)