Skip to content

Add UI Tests using Espresso Testing Framework #1900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions app/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ android {
multiDexEnabled = true

resourceConfigurations += listOf("ar", "bg", "bn", "bn-rIN", "bs", "cs", "da", "de", "el-rGR", "en", "eo", "es", "es-rAR", "fi", "fr", "he-rIL", "hi", "hr", "hu", "in-rID", "is", "it", "ja", "ko", "lt", "lv", "nb-rNO", "nl", "oc", "pl", "pt-rPT", "ro-rRO", "ru", "sk", "sl", "sv", "tr", "uk", "vi", "zh-rCN", "zh-rTW")
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking this might also need the following change as documented on https://developer.android.com/training/testing/instrumented-tests/androidx-test-libraries/runner to ensure that running the "LoyaltyCardCreationTest" twice in a row won't cause failures but I haven't been able to test this yet as the LoyaltyCardCreationTest currently fails for me no matter what 😅

Suggested change
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArguments clearPackageData: 'true'

}

buildTypes {
Expand Down Expand Up @@ -94,6 +95,10 @@ dependencies {
implementation("androidx.preference:preference:1.2.1")
implementation("com.google.android.material:material:1.12.0")
implementation("com.github.yalantis:ucrop:2.2.9")
implementation("androidx.tracing:tracing:1.2.0")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this dependency necessary? Given only new tests are being added, it seems weird to me to add a new runtime dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are probably right this dependency might not be necessary. While trying to integrate fastlane screengrab (unsuccessfully) within these tests I had to add some more dependencies and probably did not remove them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that the tests fail if you remove this, it seems to be a dependency of Espresso.

However, given it is an Espresso test dependency and not a runtime dependency, it seems to me this should be an androidTestImplementation instead of implementation so we don't ship it with the actual APKs users will install as it's not needed there.

So that's what I tried... and it didn't work. And then I researched more. And... it seems to be an AGP bug. Sigh: android/android-test#1755 (comment)

I decided to ask yet again for a way to track this issue as the previous request for that got no answer. It's definitely an ugly workaround, but it seems the only option currently: android/android-test#1755 (comment)

androidTestImplementation("androidx.test.ext:junit:1.2.0-beta01")
androidTestImplementation("androidx.test:rules:1.6.0-beta01")
androidTestImplementation("androidx.test.espresso:espresso-contrib:3.6.0-beta01")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing some duplicates here. All three of these seem to also be listed under the "Espresso" header. So I think those could be removed here?

coreLibraryDesugaring("com.android.tools:desugar_jdk_libs:2.0.4")

// Splash Screen
Expand All @@ -113,6 +118,14 @@ dependencies {
testImplementation("androidx.test:core:1.5.0")
testImplementation("junit:junit:4.13.2")
testImplementation("org.robolectric:robolectric:4.12.2")

// Espresso
androidTestImplementation("androidx.test.espresso:espresso-core:3.6.0-beta01")
androidTestImplementation("androidx.test:core:1.5.0")
androidTestImplementation("androidx.test:rules:1.6.0-beta01")
androidTestImplementation("androidx.test.ext:junit:1.2.0-beta01")
androidTestImplementation("androidx.test:runner:1.5.2")
androidTestImplementation("androidx.test.espresso:espresso-contrib:3.6.0-beta01")
}

tasks.withType<SpotBugsTask>().configureEach {
Expand Down
158 changes: 158 additions & 0 deletions app/src/androidTest/java/protect/card_locker/LanguageChangeUITest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package protect.card_locker;


import static androidx.test.espresso.Espresso.onData;
import static androidx.test.espresso.Espresso.onView;
import static androidx.test.espresso.action.ViewActions.click;
import static androidx.test.espresso.assertion.ViewAssertions.matches;
import static androidx.test.espresso.contrib.RecyclerViewActions.actionOnItemAtPosition;
import static androidx.test.espresso.matcher.ViewMatchers.isDisplayed;
import static androidx.test.espresso.matcher.ViewMatchers.withClassName;
import static androidx.test.espresso.matcher.ViewMatchers.withContentDescription;
import static androidx.test.espresso.matcher.ViewMatchers.withId;
import static androidx.test.espresso.matcher.ViewMatchers.withParent;
import static androidx.test.espresso.matcher.ViewMatchers.withText;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.anything;
import static org.hamcrest.Matchers.is;

import android.view.View;
import android.view.ViewGroup;
import android.view.ViewParent;

import androidx.test.espresso.DataInteraction;
import androidx.test.espresso.ViewInteraction;
import androidx.test.ext.junit.rules.ActivityScenarioRule;
import androidx.test.ext.junit.runners.AndroidJUnit4;
import androidx.test.filters.LargeTest;

import org.hamcrest.Description;
import org.hamcrest.Matcher;
import org.hamcrest.TypeSafeMatcher;
import org.hamcrest.core.IsInstanceOf;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;

@LargeTest
@RunWith(AndroidJUnit4.class)
public class LanguageChangeUITest {

@Rule
public ActivityScenarioRule<MainActivity> mActivityScenarioRule =
new ActivityScenarioRule<>(MainActivity.class);

@Test
public void languageChangeUITest() {
ViewInteraction overflowMenuButton = onView(
allOf(withContentDescription(R.string.action_more_options),
isDisplayed()));
overflowMenuButton.perform(click());

ViewInteraction materialTextView = onView(
allOf(withId(androidx.recyclerview.R.id.title), withText("Settings"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to cause a test failure if the device isn't set to English. For the rest of the test it makes sense to check explicit strings as you are changing the language, but the default Catima setting is to use the system language which may not always be English. So it seems this should just use the settings string in strings.xml

childAtPosition(
childAtPosition(
withId(androidx.appcompat.R.id.content),
0),
0),
isDisplayed()));
materialTextView.perform(click());

ViewInteraction recyclerView = onView(
allOf(withId(com.jaredrummler.android.colorpicker.R.id.recycler_view),
childAtPosition(
withId(android.R.id.list_container),
0)));
recyclerView.perform(actionOnItemAtPosition(4, click()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized this text will break whenever we add a new language. That seems unnecessarily fragile. Can we instead make the test pick the tested language by string name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed adding new languages would make the tests fail. I will make the necessary changes to pick the tested language by string name.


DataInteraction appCompatCheckedTextView = onData(anything())
.inAdapterView(allOf(withId(androidx.appcompat.R.id.select_dialog_listview),
childAtPosition(
withId(androidx.appcompat.R.id.contentPanel),
0)))
.atPosition(8);
appCompatCheckedTextView.perform(click());

ViewInteraction textView = onView(
allOf(withText("Einstellungen"),
withParent(allOf(withId(R.id.toolbar),
withParent(IsInstanceOf.<View>instanceOf(android.widget.LinearLayout.class)))),
isDisplayed()));
textView.check(matches(withText("Einstellungen")));

ViewInteraction recyclerView2 = onView(
allOf(withId(com.jaredrummler.android.colorpicker.R.id.recycler_view),
childAtPosition(
withId(android.R.id.list_container),
0)));
recyclerView2.perform(actionOnItemAtPosition(4, click()));

DataInteraction appCompatCheckedTextView2 = onData(anything())
.inAdapterView(allOf(withId(androidx.appcompat.R.id.select_dialog_listview),
childAtPosition(
withId(androidx.appcompat.R.id.contentPanel),
0)))
.atPosition(9);
appCompatCheckedTextView2.perform(click());

ViewInteraction textView2 = onView(
allOf(withText("Ρυθμίσεις"),
withParent(allOf(withId(R.id.toolbar),
withParent(IsInstanceOf.<View>instanceOf(android.widget.LinearLayout.class)))),
isDisplayed()));
textView2.check(matches(withText("Ρυθμίσεις")));

ViewInteraction recyclerView3 = onView(
allOf(withId(com.jaredrummler.android.colorpicker.R.id.recycler_view),
childAtPosition(
withId(android.R.id.list_container),
0)));
recyclerView3.perform(actionOnItemAtPosition(4, click()));

DataInteraction appCompatCheckedTextView3 = onData(anything())
.inAdapterView(allOf(withId(androidx.appcompat.R.id.select_dialog_listview),
childAtPosition(
withId(androidx.appcompat.R.id.contentPanel),
0)))
.atPosition(0);
appCompatCheckedTextView3.perform(click());

ViewInteraction appCompatImageButton = onView(
allOf(withContentDescription("Navigate up"),
childAtPosition(
allOf(withId(R.id.toolbar),
childAtPosition(
withClassName(is("com.google.android.material.appbar.AppBarLayout")),
0)),
1),
isDisplayed()));
appCompatImageButton.perform(click());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test fails for me here when testing on an Android 14 emulator:

androidx.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (view.getContentDescription() is "Navigate up" and Child at position 1 in parent (view.getId() is <2131296920> and Child at position 0 in parent view.getClass().getName() matches: is "com.google.android.material.appbar.AppBarLayout") and (view has effective visibility <VISIBLE> and view.getGlobalVisibleRect() to return non-empty rectangle))

I've attached the full view-hierarchy:
view-hierarchy-1.txt

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems weird to me because I am also running the tests on an Android 14 emulator, but I will examine this further.


ViewInteraction textView3 = onView(
allOf(withId(R.id.welcome_text), withText("Welcome to Catima"),
withParent(allOf(withId(R.id.helpSection),
withParent(withId(R.id.include)))),
isDisplayed()));
textView3.check(matches(withText("Welcome to Catima")));
}

private static Matcher<View> childAtPosition(
final Matcher<View> parentMatcher, final int position) {

return new TypeSafeMatcher<View>() {
@Override
public void describeTo(Description description) {
description.appendText("Child at position " + position + " in parent ");
parentMatcher.describeTo(description);
}

@Override
public boolean matchesSafely(View view) {
ViewParent parent = view.getParent();
return parent instanceof ViewGroup && parentMatcher.matches(parent)
&& view.equals(((ViewGroup) parent).getChildAt(position));
}
};
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
package protect.card_locker;

import static androidx.test.espresso.Espresso.onData;
import static androidx.test.espresso.Espresso.onView;
import static androidx.test.espresso.action.ViewActions.click;
import static androidx.test.espresso.action.ViewActions.closeSoftKeyboard;
import static androidx.test.espresso.action.ViewActions.replaceText;
import static androidx.test.espresso.action.ViewActions.scrollTo;
import static androidx.test.espresso.assertion.ViewAssertions.matches;
import static androidx.test.espresso.matcher.ViewMatchers.isDisplayed;
import static androidx.test.espresso.matcher.ViewMatchers.withClassName;
import static androidx.test.espresso.matcher.ViewMatchers.withContentDescription;
import static androidx.test.espresso.matcher.ViewMatchers.withId;
import static androidx.test.espresso.matcher.ViewMatchers.withParent;
import static androidx.test.espresso.matcher.ViewMatchers.withText;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.anything;
import static org.hamcrest.Matchers.is;

import android.view.View;
import android.view.ViewGroup;
import android.view.ViewParent;

import androidx.test.espresso.DataInteraction;
import androidx.test.espresso.ViewInteraction;
import androidx.test.ext.junit.rules.ActivityScenarioRule;
import androidx.test.ext.junit.runners.AndroidJUnit4;
import androidx.test.filters.LargeTest;
import androidx.test.rule.GrantPermissionRule;

import org.hamcrest.Description;
import org.hamcrest.Matcher;
import org.hamcrest.TypeSafeMatcher;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;


@LargeTest
@RunWith(AndroidJUnit4.class)
public class LoyaltyCardCreationTest {
@Rule
public ActivityScenarioRule<MainActivity> mActivityScenarioRule =
new ActivityScenarioRule<>(MainActivity.class);

@Rule
public GrantPermissionRule mGrantPermissionRule =
GrantPermissionRule.grant(
"android.permission.CAMERA",
"android.permission.WRITE_EXTERNAL_STORAGE",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catima dosen't request write permission, this seems to cause a crash when testing locally:

java.lang.SecurityException: Error granting runtime permission
at android.app.UiAutomation.grantRuntimePermissionAsUser(UiAutomation.java:1433)
at android.app.UiAutomation.grantRuntimePermission(UiAutomation.java:1399)
at androidx.test.runner.permission.UiAutomationPermissionGranter.requestPermissions(UiAutomationPermissionGranter.java:44)
at androidx.test.rule.GrantPermissionRule$RequestPermissionStatement.evaluate(GrantPermissionRule.java:149)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:162)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:68)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:463)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2402)
Caused by: java.lang.SecurityException: Package me.hackerchick.catima.debug has not requested permission android.permission.WRITE_EXTERNAL_STORAGE
at android.os.Parcel.createExceptionOrNull(Parcel.java:3057)
at android.os.Parcel.createException(Parcel.java:3041)
at android.os.Parcel.readException(Parcel.java:3024)
at android.os.Parcel.readException(Parcel.java:2966)
at android.app.IUiAutomationConnection$Stub$Proxy.grantRuntimePermission(IUiAutomationConnection.java:677)
at android.app.UiAutomation.grantRuntimePermissionAsUser(UiAutomation.java:1430)
... 31 more

"android.permission.READ_EXTERNAL_STORAGE");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not sure about READ_EXTERNAL_STORAGE, as that is also only needed up to maxSdkVersion 23. So Android 7 and up will never ask this. This line seems to fail on newer Android verisons too.

Even then, this test would never touch any code that requires this permission, even on Android 5 or 6, so it seems granting this permission should never be necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as mentioned above I added them while trying to integrate the Screengrab to create an automated screenshot workflow. Sorry for the inconvenience, I will remove everything unnecessary in the upcoming commits 😅


@Test
public void loyaltyCardCreationTest() {
ViewInteraction textView = onView(
allOf(withId(R.id.add_card_instruction), withText("Click the + plus button to add a card, or import from the ⋮ menu."),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how valuable it is to test the exact strings. It seems pretty safe her to just refer to the noGiftCards string directly. I don't think we need to test if Android will take the correct text string from the resources, I would hope the Android team tests their own core code like context.getString functions :)

I also changed my emulator's device language to Dutch and it seems to crash here now, so that seems to also support this makes the test overly fragile and causes failures under normal conditions.

withParent(allOf(withId(R.id.helpSection),
withParent(withId(R.id.include)))),
isDisplayed()));
textView.check(matches(withText("Click the + plus button to add a card, or import from the ⋮ menu.")));

ViewInteraction floatingActionButton = onView(
allOf(withId(R.id.fabAdd), withContentDescription("Add"),
childAtPosition(
childAtPosition(
withId(android.R.id.content),
0),
2),
isDisplayed()));
floatingActionButton.perform(click());

ViewInteraction extendedFloatingActionButton = onView(
allOf(withId(R.id.fabOtherOptions), withText("More options"),
childAtPosition(
allOf(withId(R.id.zxing_barcode_scanner),
childAtPosition(
withClassName(is("android.widget.RelativeLayout")),
1)),
0),
isDisplayed()));
extendedFloatingActionButton.perform(click());

DataInteraction materialTextView = onData(anything())
.inAdapterView(allOf(withId(androidx.appcompat.R.id.select_dialog_listview),
childAtPosition(
withId(androidx.appcompat.R.id.contentPanel),
0)))
.atPosition(0);
materialTextView.perform(click());

ViewInteraction editText = onView(
allOf(childAtPosition(
childAtPosition(
withId(androidx.appcompat.R.id.custom),
0),
1),
isDisplayed()));
editText.perform(replaceText("123456789"), closeSoftKeyboard());

ViewInteraction materialButton = onView(
allOf(withId(android.R.id.button1), withText("OK"),
childAtPosition(
childAtPosition(
withId(androidx.appcompat.R.id.buttonPanel),
0),
3)));
materialButton.perform(scrollTo(), click());

ViewInteraction editText2 = onView(
allOf(withId(R.id.cardIdView), withText("123456789"),
withParent(withParent(withId(R.id.cardIdField))),
isDisplayed()));
editText2.check(matches(withText("123456789")));

ViewInteraction textInputEditText = onView(
allOf(withId(R.id.storeNameEdit),
childAtPosition(
childAtPosition(
withId(R.id.storeNameField),
0),
0),
isDisplayed()));
textInputEditText.perform(replaceText("CatimaUITestCard"), closeSoftKeyboard());

ViewInteraction tabView = onView(
allOf(withContentDescription("Options"),
childAtPosition(
childAtPosition(
withId(R.id.tabs),
0),
1),
isDisplayed()));
tabView.perform(click());

ViewInteraction textInputEditText2 = onView(
allOf(withId(R.id.balanceField), withText("0"),
childAtPosition(
childAtPosition(
withId(R.id.balanceView),
0),
0),
isDisplayed()));
textInputEditText2.perform(replaceText("10000"));

ViewInteraction textInputEditText3 = onView(
allOf(withId(R.id.balanceField), withText("10000"),
childAtPosition(
childAtPosition(
withId(R.id.balanceView),
0),
0),
isDisplayed()));
textInputEditText3.perform(closeSoftKeyboard());

ViewInteraction floatingActionButton2 = onView(
allOf(withId(R.id.fabSave), withContentDescription("Save"),
childAtPosition(
childAtPosition(
withId(android.R.id.content),
0),
2),
isDisplayed()));
floatingActionButton2.perform(click());

ViewInteraction textView2 = onView(
allOf(withId(R.id.balance), withText("10000 points"),
withParent(withParent(withId(R.id.row))),
isDisplayed()));
textView2.check(matches(withText("10000 points")));
}

private static Matcher<View> childAtPosition(
final Matcher<View> parentMatcher, final int position) {

return new TypeSafeMatcher<View>() {
@Override
public void describeTo(Description description) {
description.appendText("Child at position " + position + " in parent ");
parentMatcher.describeTo(description);
}

@Override
public boolean matchesSafely(View view) {
ViewParent parent = view.getParent();
return parent instanceof ViewGroup && parentMatcher.matches(parent)
&& view.equals(((ViewGroup) parent).getChildAt(position));
}
};
}
}