feature: test detection from runtime/go test output

[utam0k]

Did you check docs and existing issues?

• I have read the documentation.
• I have searched the existing issues.

Is your feature request related to a problem? Please describe.

Support for the table test is excellent. I'd love this feature. However, because it uses AST parser, it is not available if you are writing test cases outside of a file. We often encounter such cases when developing large OSS such as Kubernetes.
e.g., https://github.com/kubernetes/kubernetes/blob/b4f902f0371485505ff4eda39975e67bfa9b0727/test/integration/scheduler/queueing/queueinghint/queue_test.go#L32-L43

Describe the solution you'd like to see.

For example, can't the ouput tree be reflected based on the results of a test run? Like:

      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_PV_is_created (6.85s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_PV_is_updated (6.46s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_StorageClass's_AllowedTopologies_is_updated (7.07s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_CSINode_is_created (7.22s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_CSINode's_MigratedPluginsAnnotation_is_updated (7.25s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_CSIDriver's_StorageCapacity_gets_disabled (7.77s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_CSIStorageCapacity_is_created (8.09s)
      --- PASS: TestCoreResourceEnqueueWithQueueingHints/Pod_rejected_with_node_by_the_VolumeBinding_plugin_is_requeued_when_the_CSIStorageCapacity_is_increased (7.60s)
      --- FAIL: TestCoreResourceEnqueueWithQueueingHints/VolumeRestrictions/VolumeBinding/Pod_rejected_with_PVC_by_the_CSI_plugin_is_requeued_when_the_pod_having_related_PVC_is_deleted (38.61s)

If possible, I would like it to be reflected permanently, but I think that would be difficult.

Describe alternatives you've considered.

N/A

Additional context

No response

[fredrikaverpil]

Hi @utam0k !
Nice to hear you find Neotest and this adapter working for you in that quite ginormous project! 💡😊

I think your inquiry could relate to something I've been thinking about for some time; detecting tests generated by runtime and populate the Neotest positions tree with these tests (and their outcome).

If you run the tests you linked with vscode's go extension, I believe they do something similar; detecting tests via AST but then also based on the go test output. You can see how test become populated in the tree as they are detected from the test output.

We'd have to keep a separate positions tree on disk or in memory which would be based on e.g. tests from the test output (which were not found in AST) and merge this into the Neotest positions tree. However, there are numerous challenges with this which relates to how Neotest works. For example, if you save your buffer, Neotest will re-scan the AST and rebuild the positions tree, effectively removing the "runtime-detected" tests. So here we'd have to re-merge them in or something...

I wonder if it would be able to maintain a custom positions tree outside the tree based on AST. Then we wouldn't have to merge them together. 🤔

@rcarriga do you have any thoughts about how to potentially pull something like this off?

[utam0k]

I know that vscode's go extension has a similar feature. However, I'd like to use vim... 😆