Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entering text in webview doesn't work #922

Closed
jeremiahlukus opened this issue Feb 13, 2023 · 32 comments
Closed

Entering text in webview doesn't work #922

jeremiahlukus opened this issue Feb 13, 2023 · 32 comments
Labels
platform: ios iOS is affected

Comments

@jeremiahlukus
Copy link

jeremiahlukus commented Feb 13, 2023

Screen Shot 2023-02-13 at 11 24 06 AM

Describe the bug

Using https://joyful-noise-staging.joyful-noise.link/users/sign_in as a form
I have tried the way mentioned in the docs

      //await $.native.enterTextByIndex('test@hey.com', index: 0);
      await $.native.enterText(
        Selector(text: 'user_email'), // "you@example.com", "Email", etc..
        text: 'bartek@awesome.com',
      );

Nothing shows in form.
To Reproduce

Steps to reproduce the behavior:

1.Use https://joyful-noise-staging.joyful-noise.link/users/sign_in as base url
2. try to enter text in form and submit
3. App hangs with no error

Expected behavior

A clear and concise description of what you expected to happen.

Device information

  • Device: iPhone14

  • Does it also both on physical and virtual devices? [didn't check]

Additional information

flutter --version
Flutter 3.7.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 9944297138 (5 days ago) • 2023-02-08 15:46:04 -0800
Engine • revision 248290d6d5
Tools • Dart 2.19.2 • DevTools 2.20.1

patrol: ^0.10.12

patrol_cli v0.9.4

Additional context

@bartekpacia
Copy link
Contributor

Hi 👋

Did you specify the bundle ID of your app in pubspec.yaml? (see docs for it)

@jeremiahlukus
Copy link
Author

Yes i did in the pubspec.yml

patrol:
  app_name: Joyful Noise
  android:
    package_name: com.jparrack.joyful_noise
  ios:
    bundle_id: com.jparrack.joyful-noise

@jeremiahlukus
Copy link
Author

Ah i see you are corrrect. it was building com.jparrack.joyful-noise.dev so changing to that worked.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 14, 2023

@bartekpacia Sorry to keep bugging you it seems it only entered the text that one time. I am unable to enter the text again i have no idea why.

	    t =     1.89s         Wait for com.jparrack.joyful-noise.dev to idle
	2023-02-13 18:43:28.105243-0500 RunnerUITests-Runner[61203:11632578] PatrolServer: INFO: submitted 0 dart test results
	2023-02-13 18:43:28.105414-0500 RunnerUITests-Runner[61203:11632578] PatrolServer: INFO: Got 0 dart test results
	    t =     3.12s Added attachment named 'Launch Screen'
	    t =     3.12s Tear Down
	Test Case '-[RunnerUITestsLaunchTests testLaunch]' passed (3.324 seconds).
	Test Case '-[RunnerUITestsLaunchTests testLaunch]' started.
	    t =     0.00s Setting device orientation to Landscape Right
	    t =     0.03s     Wait for com.jparrack.joyful-noise.dev to idle
	    t =     0.05s Setting appearance mode to Light
	2023-02-13 18:43:29.238648-0500 RunnerUITests-Runner[61203:11632552] PatrolServer: INFO: entering text "wferem1@gmail.com" by index 0...
	    t =     0.51s     Get all elements bound by index for: Elements matching predicate 'elementType == 49 OR elementType == 50'
	    t =     2.12s     Get number of matches for: Elements matching predicate 'elementType == 49 OR elementType == 50'
	    t =     2.12s     Get number of matches for: <XCUIElementQuery: 0x60000058d950> (retry 1)
	    t =     2.12s     Get number of matches for: <XCUIElementQuery: 0x60000058d950> (retry 2)
	2023-02-13 18:43:30.854358-0500 RunnerUITests-Runner[61203:11632552] PatrolServer: INFO: found 0 text fields

when i do flutter logs i see

┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

(null) did not include status of requested cert. sounds like my error but why is my cert null?

My code is

void main() {
  patrolTest(
    'counter state is the same after going to home and switching apps',
    ($) async {
      main_dart.main();

      await $.pumpAndSettle();
      await $(#signInButtonKey).tap();
      await $.pumpAndSettle();

      logger.e("-------------------------------"); // gets to this line
      await $.native.enterTextByIndex('test@gmail.com', index: 0); //fails here
      logger.e('++++++++++++++++++++++++++++++++++++++++++++');
      await $.native.enterTextByIndex('NyanCar', index: 1);
      await $.pump(Duration(seconds: 2));
      await $.native.enterTextByIndex('heyheyhey', index: 1);
      logger.e('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@');

    },
    nativeAutomation: true,
  );
}

Ive tried adding

await $.native.enterText(
  Selector(text: 'Email'), 
  text: 'bartek@awesome.com',
);

But it doesnt find the field and errors there.

The form is
https://joyful-noise-staging.joyful-noise.link/users/sign_in again.

I feel like this is something to do with using flavors.

Once i get past the webview I can send you a tip for the help if you send me a link.

@bartekpacia
Copy link
Contributor

Thanks for all the details. I'll try to reproduce and fix the issue tomorrow.

I can send you a tip for the help if you send me a link.

I work full-time at @leancodepl on this library, and I'm compensated well :) Thank you, though - this is very kind of you.

@bartekpacia bartekpacia reopened this Feb 14, 2023
@bartekpacia bartekpacia added the platform: ios iOS is affected label Feb 14, 2023
@jeremiahlukus
Copy link
Author

Great let me know if you need more info.

That is good to hear always good to get paid for working ha

@jeremiahlukus
Copy link
Author

Ok i have the bundle identifier RunnerUITests
Screen Shot 2023-02-14 at 9 21 11 AM

I created a new debug config so i dont need to pass in the flavor
Screen Shot 2023-02-14 at 9 23 06 AM

I saw the target runner is pointing to the correct bundle identifier
Screen Shot 2023-02-14 at 9 22 25 AM

in my pubspec

  ios:
    bundle_id: com.jparrack.joyful-noise.RunnerUITests

in my test code

logger.e("-------------------------------");
await $.native.enterTextByIndex('test@gmail.com', index: 0, appId: 'com.jparrack.joyful-noise.RunnerUITests');
logger.e('++++++++++++++++++++++++++++++++++++++++++++');

In flutter logs

flutter: \^[[38;5;196m├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

In patrol logs

	2023-02-14 09:26:16.620323-0500 RunnerUITests-Runner[44748:12712462] PatrolServer: INFO: entering text "test@gmail.com" by index 0...
	    t =     0.49s     Get all elements bound by index for: Elements matching predicate 'elementType == 49 OR elementType == 50'
	    t =     2.40s     Get number of matches for: Elements matching predicate 'elementType == 49 OR elementType == 50'
	    t =     2.41s     Get number of matches for: <XCUIElementQuery: 0x60000074f070> (retry 1)
	    t =     2.41s     Get number of matches for: <XCUIElementQuery: 0x60000074f070> (retry 2)
	2023-02-14 09:26:18.541099-0500 RunnerUITests-Runner[44748:12712462] PatrolServer: INFO: found 0 text fields
	    t =     2.41s     Wait for com.jparrack.joyful-noise.RunnerUITests to idle

The app makes it to the webview and the text field is shown in the screen.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 14, 2023

after changing my entitlements to

<dict>
	<key>aps-environment</key>
	<string>development</string>
	<key>keychain-access-groups</key>
	<array>
		<string>$(AppIdentifierPrefix)com.jparrack.joyful-noise.RunnerUITests</string>
	</array>
</dict>
flutter: Patrol (native): enterTextByIndex() started
@ClxSimulated, Fix, 1, ll, 37.7858340, -122.4064170, acc, 5.00, course, -1.0, time, 698079021.1, deltaDistance, 0.0, deltaDistanceAccuracy, 1.9, odometer, 0.0, timestampGps, 698079021.1
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[65079]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[65079]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[65079]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

Now it looks like it's at least trying to do it.

@jeremiahlukus
Copy link
Author

more logs

flutter: PatrolBinding: Extension stream has no listeners, so host features won't work
flutter: PatrolBinding: registered service extension ext.flutter.patrol
flutter: Patrol (native): Android app name: Joyful Noise
flutter: Patrol (native): iOS app name: Joyful Noise
flutter: Patrol (native): Android package name: com.jparrack.joyful_noise.RunnerUITests
flutter: Patrol (native): iOS bundle identifier: com.jparrack.joyful-noise.RunnerUITests
flutter: 00:00 +0: counter state is the same after going to home and switching apps
@ClxSimulated, Fix, 1, ll, 37.7858340, -122.4064170, acc, 5.00, course, -1.0, time, 698079922.7, deltaDistance, 0.0, deltaDistanceAccuracy, 1.9, odometer, 0.0, timestampGps, 698079922.7
flutter: Patrol (native): configure() started
flutter: Patrol (native): configure() succeeded
Handling kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 (0x6000012734d0)
Got user testing notification with payload {
    controllerClass = SBIconController;
    event = ViewDidDisappear;
}
Handling kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 (0x6000012734d0)
Calling handler for AX notification kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 with payload {
    controllerClass = SBIconController;
    event = ViewDidDisappear;
}
Got user testing notification with payload {
    controllerClass = SBRootFolderController;
    event = ViewDidDisappear;
}
Calling handler for AX notification kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 with payload {
    controllerClass = SBRootFolderController;
    event = ViewDidDisappear;
}
Leaf has invalid basic constraints
Leaf has invalid basic constraints
flutter: 00:00 +0: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 903 pos 14: '_pendingExceptionDetails != null': A test overrode FlutterError.onError but either failed to return it to its original state, or had unexpected additional errors that it could not handle. Typically, this is caused by using expect() before restoring FlutterError.onError.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 903:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 681:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError
  dart:async/future_impl.dart 800:13            Future._pr<…>
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 1914 pos 12: '_pendingFrame == null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61           _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5            _AssertionError._throwNew
  package:flutter_test/src/binding.dart 1914:12     LiveTestWidgetsFlutterBinding.postTest
  dart:async/future.dart 302:31                     new Future.sync
  package:test_api/src/backend/invoker.dart 236:18  Invoker.runTearDowns.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 257:15  Invoker._waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1398:13                      _rootRun
  dart:async/zone.dart 1300:19                      _CustomZone.run
  dart:async/zone.dart 1803:10                      _runZoned
  dart:async/zone.dart 1746:10                      runZoned
  package:test_api/src/backend/invoker.dart 254:5   Invoker._waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 235:9   Invoker.runTearDowns.<fn>
  dart:async/zone.dart 1398:13                      _rootRun
  dart:async/zone.dart 130<…>
flutter: 00:00 +0 -1: (tearDownAll)
flutter: 00:00 +0 -1: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 681:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError
  dart:async/future_impl.dart 800:13            Future._pr<…>
flutter: 00:00 +0 -1: (tearDownAll)
flutter: Sending Dart test results to the native side
flutter: Warning: integration_test plugin was not detected.
flutter:
flutter: If you're running the tests with `flutter drive`, please make sure your tests
flutter: are in the `integration_test/` directory of your package and use
flutter: `flutter test $path_to_test` to run it instead.
flutter:
flutter: If you're running the tests with Android instrumentation or XCTest, this means
flutter: that you are not capturing test results properly! See the following link for
flutter: how to set up the integration_test plugin:
flutter:
flutter: https://flutter.dev/docs/testing/integration-tests#testing-on-firebase-test-lab
flutter:
flutter: 00:00 +1 -1: Some tests failed.
flutter: 00:00 +1 -1: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/zone.dart 1210:7                   _CustomZone.runGuarded
  dart:async/zone.dart 1248:23                  _CustomZone.bindCallbackGuarded.<fn>
  dart:async/zone.dart 1398:13                  _rootRun
  dart:async/zone.dart 1300:19                  _CustomZone.run
  dart:async/zone.dart 1208:7                   _CustomZone.runGuarded
  dart:async/zone.dart 1248:23                  _CustomZone.bindCallbackGuarded.<fn>
  dart:async/schedule_microtask.dart 40:21      _microtaskLoop

@bartekpacia
Copy link
Contributor

I haven't yet found time to investigate it further (sorry), but...

Screenshot 2023-02-14 at 5 07 28 PM

It should look like this in the docs:

Screenshot 2023-02-14 at 5 26 24 PM

I doubt it's the cause of the problem, though.

@jeremiahlukus
Copy link
Author

Nope i played with that as well both are now Debug and still same issue.

Ill update if i have anything else, i havent made any progress yet.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 15, 2023

Ok @bartekpacia there was an issue with flutter onError not being restored

Future<void> restoreFlutterError(Future<void> Function() call) async {
  final originalOnError = FlutterError.onError!;
  await call();
  final overriddenOnError = FlutterError.onError!;

  // restore FlutterError.onError
  FlutterError.onError = (FlutterErrorDetails details) {
    if (overriddenOnError != originalOnError) overriddenOnError(details);
    originalOnError(details);
  };
}

void main() {
..
      await restoreFlutterError(() async {
        main_dart.main();
        await $.pumpAndSettle();
      });
      ..
}

This outputs a nice patrol error now

flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following assertion was thrown running a test:
flutter: A FocusManager was used after being disposed.
Got system UI orientation (<AXUIElementRef 0x600003f67540> {pid=53935} {uid=[ID:1 hash:0x0]}[kAXApplicationOrientationAttribute]) 4
flutter: Once you have called dispose() on a FocusManager, it can no longer be used.
Preparing screenshot for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
Taking screenshot for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      ChangeNotifier.debugAssertNotDisposed.<anonymous closure> (package:flutter/src/foundation/change_notifier.dart:157:9)
flutter: #1      ChangeNotifier.debugAssertNotDisposed (package:flutter/src/foundation/change_notifier.dart:164:6)
flutter: #2      ChangeNotifier.notifyListeners (package:flutter/src/foundation/change_notifier.dart:360:27)
flutter: #3      FocusManager._applyFocusChange (package:flutter/src/widgets/focus_manager.dart:1808:7)
flutter: (elided 10 frames from dart:async)
flutter:
flutter: The test description was:
flutter:   counter state is the same after going to home and switching apps
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════
flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following PatrolActionException was thrown running a test:
flutter: Patrol action failed: GrpcError: enterTextByIndex() failed with code NOT_FOUND (text field at index
flutter: 0 doesn't exist)
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      NativeAutomator._wrapRequest (package:patrol/src/native/native_automator.dart:200:7)
flutter: <asynchronous suspension>
flutter: #1      NativeAutomator.enterTextByIndex (package:patrol/src/native/native_automator.dart:529:5)
flutter: <asynchronous suspension>
flutter: #2      main.<anonymous closure> (file:///Users/jeremiah.parrack/freelance/guitar_tabs/guitar_tabs/integration_test/app_test.dart:37:7)
flutter: <asynchronous suspension>
flutter: #3      patrolTest.<anonymous closure> (package:patrol/src/common.dart:82:7)
flutter: <asynchronous suspension>
flutter: #4      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:171:15)
flutter: <asynchronous suspension>
flutter: #5      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:935:5)
flutter: <asynchronous suspension>
flutter:
flutter: The test description was:
Converting image for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
flutter:   counter state is the same after going to home and switching apps
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 717:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError

In a later test you see the same output as before

┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[42840]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[42840]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert

So for some reason this error only triggers on the first test, after that it bypasses this error and gives a less readable error

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 15, 2023

The webview widget is

import 'package:webview_flutter/webview_flutter.dart';

  Widget build(BuildContext context) {
    return Scaffold(
      body: SafeArea(
        child: WebView(
          javascriptMode: JavascriptMode.unrestricted,
          initialUrl: widget.authorizationUrl.toString(),
          onWebViewCreated: (controller) {
            controller.clearCache();
            CookieManager().clearCookies();
          },
          navigationDelegate: (navReq) async {
            if (navReq.url.startsWith(WebAppAuthenticator.redirectUrl().toString())) {
              widget.onAuthorizationCodeRedirectAttempt(
                Uri.parse(navReq.url),
              );
              return NavigationDecision.prevent;
            }
            return NavigationDecision.navigate;
          },
        ),
      ),
    );
  }

So it looks like it's just erroring on the input when the keyboard show up?

@bartekpacia
Copy link
Contributor

I investigated this issue a bit and I'm pretty sure that it comes down to the website being Not Accessibility Friendly.

After going to the webview and taking a native view hierarchy dump, I saw:

<node NAF="true" index="1" text="" resource-id="user_password" class="android.widget.EditText" package="pl.leancode.patrol.example" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="true" selected="false" bounds="[115,1228][966,1340]" />

and

<node NAF="true" index="1" text="" resource-id="user_email" class="android.widget.EditText" package="pl.leancode.patrol.example" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="true" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[115,989][966,1102]" />

What's important is NAF="true" – this means Not Accessibility Friendly. This is Android-only, but I think it's possible iOS has the same restriction.

Apparently, UIAutomator cannot perform actions on NAF views, even when I'd expect that it should be able to perform them. For example I tried:

await $.native.enterText(
  Selector(resourceId: 'user_email'),
  text: 'charlie@root.me',
);
await $.native.enterText(
  Selector(resourceId: 'user_password'),
  text: 'ny4ncat',
);
await $.native.tap(Selector(text: 'Log in'));

and it doesn't work (output from flutter logs):

I/flutter (28599): ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
I/flutter (28599): The following PatrolActionException was thrown running a test:
I/flutter (28599): Patrol action failed: GrpcError: enterText() failed with code NOT_FOUND (selector
I/flutter (28599): UiSelector[RESOURCE_ID=user_email] found nothing)

In an act of despair, I also tried listing:

final views = await $.native.getNativeViews(Selector(className: 'android.widget.EditText'));
print("${views.length} views found")
for (final view in views) {
  print('view: $view');
}

and it finds 0 views, while I'd expect 2 to show up (login and password).

So the solution I'd suggest trying out is to make changes to the sign-in website to add "content-description" for accessibility tools so that it won't appear as Not Accessibility Friendly.

See also:

Here's the test code and webview screen code that I tried (a slightly modified example app)

@bartekpacia
Copy link
Contributor

bartekpacia commented Feb 15, 2023

I managed to get the test to pass on iOS:

record.mov

Full code is in #938 - let me know if it works if you git checkout it.

Unfortunately, the code that does it is very ugly (because of hardcoded screen refreshing (a.k.a pumping)):

import 'common.dart';

Future<void> main() async {
  testWebViewA();
}

void testWebViewA() {
  patrol('interacts with the LeanCode website in a webview', ($) async {
    await $.pumpWidgetAndSettle(ExampleApp());

    await $('Open webview screen A').scrollTo().tap();

    // this is a very ugly anti-pattern in tests
    for (var i = 0; i < 300; i++) {
      await $.pump();
    }

    await $.native.enterTextByIndex(
      'barpac02@gmail.com',
      index: 0,
    );
    await $.native.enterTextByIndex(
      'ny4ncat',
      index: 1,
    );
    await $.native.tap(Selector(text: 'Log in'));
  });
}

If I remove the pumping for 300 iterations, I'm getting:

flutter: Patrol (native): enterTextByIndex() failed
flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following PatrolActionException was thrown running a test:
flutter: Patrol action failed: GrpcError: enterTextByIndex() failed with code NOT_FOUND (text field at index
flutter: 0 in app pl.leancode.patrol.Example doesn't exist)

There are 2 solutions:

  • implement waitUntilVisible() for native views (see Make it possible to check and wait for existence of a native widget #499)

  • make $.native.enterTextByIndex() wait if no views are found immediately (just like $.native.enterText does currently)

    You cannot use $.native.enterText because, as I said in the post above, there's no way to refer to these text fields other than by their index. They don't have any kind of accessibility information that'd make it possible to locate them.

What do you think?

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 16, 2023

Ok its not entering text after i remove all the pumpAndSettle's

      await restoreFlutterError(() async {
        main_dart.main();
        // await $.pumpAndSettle();
      });
     // await $.pumpAndSettle();
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      await $(#signInButtonKey).tap();
      // await $.pumpAndSettle();
      // this is a very ugly anti-pattern in tests
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      await $.native.enterTextByIndex('test@gmail.com', index: 0);
      await $.native.enterTextByIndex('NyanCar', index: 1);

Now its not closing the dialog and adding a "q" at the end of the email for some reason.
This is copy paste from your branch

      await $.native.enterTextByIndex(
     'barpac02@gmail.com',
     index: 0,
   );
   await $.native.enterTextByIndex(
     'ny4ncat',
     index: 1,
   );
   await $.native.tap(Selector(text: 'Log in'));

Screen Shot 2023-02-15 at 8 53 39 PM

just double checked landscape isnt broken in the app ha

@bartekpacia
Copy link
Contributor

Now its not closing the dialog

What dialog?

and adding a "q" at the end of the email for some reason.

Wow, this is sooo weird. Doesn't occur on my machine.

its not entering text after i remove all the pumpAndSettle's

Yeah. I think adding the timeout to enterTextByIndex() is the best way to solve this.

@bartekpacia
Copy link
Contributor

and adding a "q" at the end of the email for some reason.

I think it's because you're in landscape mode.

After Patrol enters the email into the first text field, it taps on the second text field to enter the password. But, the keyboard is obscuring the second text field, so Patrol taps on the keyboard, specifically on the q character :)

That's just my guess.

Try portrait mode and let me know if the issue persists.

@bartekpacia bartekpacia changed the title Entering text in webview IOS Entering text in webview doesn't work Feb 16, 2023
@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 16, 2023

Is there a way to force run in portrait?
My app supports landscape so removing it from deployment info just to run tests isn't something i can do.

Screen Shot 2023-02-16 at 8 12 31 AM

Something like

exec gcloud firebase test ios run \
  --test "build/ios_integ/Build/Products/ios_tests.zip" \
  --device model=iphone8,version=15.7,locale=en_US,orientation=portrait

For the patrol cli?

@jeremiahlukus
Copy link
Author

after changing to only portrait it started working.

@bartekpacia
Copy link
Contributor

Portrait/landscape mode is a setting belonging to the iOS Simulator app, you should change it there.

Maybe you have your Simulator set to run by default in landscape mode? I don't know, really.

@jeremiahlukus
Copy link
Author

Screen Shot 2023-02-16 at 8 39 43 AM

In the file RunnerUILaunchTests it contains


+ (BOOL)runsForEachTargetApplicationUIConfiguration {
    return YES;
}

which looks like its doing it.

Back to the webview now its entering text however the login button isnt being pressed

   nans         Checking existence of `"Log in" Any`
	2023-02-16 08:37:02.148338-0500 RunnerUITests-Runner[37964:15537629] PatrolServer: INFO: found view with text "Log in", will tap on it
	    t =      nans Find the "Log in" Any
	    t =      nans Tap "Log in" Other[0.00, 0.00]
	    t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
	    t =      nans     Find the "Log in" Other
	    t =      nans     Check for interrupting elements affecting "Log in" Other
	    t =      nans     Synthesize event
	    t =      nans         Find the "Log in" Other
	    t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
	2023-02-16 08:37:02.815215-0500 RunnerUITests-Runner[37964:15537629] PatrolServer: INFO: done tapping on view with text "Log in"

I think the keyboard is covering up the button is there a way to manually close the keyboard?
Even in the video you sent you should have seen an error in the webview after clicking on submit.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 16, 2023

adding await $.native.tap(Selector(text: 'Done')); did close the keyboard but didnt click on login.

Now the output is

	2023-02-16 08:48:47.115358-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: waiting for existence of view with text "Log in"
	    t =      nans Waiting 10.0s for "Log in" Any to exist
	    t =      nans     Checking `Expect predicate `exists == 1` for object "Log in" Any`
	    t =      nans         Checking existence of `"Log in" Any`
	2023-02-16 08:48:48.215951-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: found view with text "Log in", will tap on it
	    t =      nans Find the "Log in" Any
	    t =      nans Tap "Log in" Other
	    t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
	    t =      nans     Find the "Log in" Other
	    t =      nans     Check for interrupting elements affecting "Log in" Other
	    t =      nans     Synthesize event
	    t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
	2023-02-16 08:48:48.671168-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: done tapping on view with text "Log in"
	2023-02-16 08:48:48.671422-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: result: ()
	2023-02-16 08:48:48.806536-0500 RunnerUITests-Runner[42202:15560051] PatrolServer: INFO: submitted 0 dart test results
	2023-02-16 08:48:48.806671-0500 RunnerUITests-Runner[42202:15560051] PatrolServer: INFO: Got 0 dart test results
	Test Suite 'All tests' started at 2023-02-16 08:48:49.683

@bartekpacia
Copy link
Contributor

In the file RunnerUILaunchTests it contains

Delete this file – it's not needed.

but didnt click on login.

That's because you've got 2 "Log in" texts - the first one is the heading, and the second one is on the button.

This should be possible to do with:

await $.native.tap(
      Selector(
        text: 'Log in',
        instance: 1,
      ),
    );

Unfortunately it isn't, but I see this is very important. Going to fix it as part of #663.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 16, 2023

Ok deleting that file was a good idea, now its running the tests once instead of multiple time with different orientations and dark mode/light mode.

Great thanks for working on that already.

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 16, 2023

I changed the text to "Sign in" to be unique however i still have issues clicking on the button

     await $.native.tap(
        Selector(
          text: 'Sign in',
        ),
      );


And still unable to click on it.

Screen Shot 2023-02-16 at 9 50 51 AM

@jeremiahlukus
Copy link
Author

     // works
      await $.native.enterTextByIndex(
        'test@hey.com',
        index: 0,
      );
      
         // works
      await $.native.enterTextByIndex(
        'testing',
        index: 1,
      );
      logger.e('---------------------------------');
         // works
      await $.native.tap(Selector(text: 'Done'));
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
         // broken
      await $.native.tap(Selector(text: 'Sign in'));

      logger.e('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@');
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
       // broken
      await $.native.tap(Selector(text: 'Sign in'));
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      ```
      
      

@bartekpacia
Copy link
Contributor

bartekpacia commented Feb 17, 2023

import 'common.dart';

Future<void> main() async {
  testWebViewD();
}

void testWebViewD() {
  patrol('interacts with the login form website in a webview', ($) async {
    await $.pumpWidgetAndSettle(ExampleApp());

    await $('Open webview (login form)').scrollTo().tap();

    await $.native.enterTextByIndex('test@hey.com', index: 0);
    await $.native.enterTextByIndex('some pass', index: 1);
    await $.native.tap(Selector(text: 'Sign in'));
  });
}

This reliably works on my iPhone 14 simulator (I'm using master branch of patrol):

Screen.Recording.2023-02-17.at.11.31.29.AM.mov

@jeremiahlukus
Copy link
Author

jeremiahlukus commented Feb 17, 2023

Ok finally got it flutter integration tests really deosnt like it when you overwrite onError.

If you do you need to wrap your app in

Future<void> restoreFlutterError(Future<void> Function() call) async {
  final originalOnError = FlutterError.onError!;
  await call();
  final overriddenOnError = FlutterError.onError!;

  // restore FlutterError.onError
  FlutterError.onError = (FlutterErrorDetails details) {
    if (overriddenOnError != originalOnError) overriddenOnError(details);
    originalOnError(details);
  };
}

void main() {
  patrolTest(
    'counter state is the same after going to home and switching apps',
    ($) async {
      await restoreFlutterError(() async {
        main_dart.main();
      });
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      // if you make a call that might trigger an error 
      await restoreFlutterError(() async {
        await $.native.tap(Selector(text: 'Sign in'));
      });

I thought wrapping the app with restoreFlutterError would wrap all errors however anytime there is an error the app will break if you dont wrap that call in restoreFlutterError()

thanks for all your help closing this now.

@bartekpacia
Copy link
Contributor

Thanks a lot, we'll be playing with FlutterError.onError in #951 so that's definitely useful info :) thanks!

@jeremiahlukus
Copy link
Author

Doing this is better

flutter/flutter#34499

final FlutterExceptionHandler? originalOnError = FlutterError.onError;
     main_dart.main();
           for (var i = 0; i < 300; i++) {
        await $.pump();
      }
FlutterError.onError = originalOnError;
...
expect()

The other way will load the app but will error out on the expect().
@bartekpacia

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
platform: ios iOS is affected
Projects
None yet
Development

No branches or pull requests

2 participants