-
Notifications
You must be signed in to change notification settings - Fork 21
SDK Behavior Settings #Tests #355
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Parent:
Staging
arifBurakDemiray
wants to merge
64
commits into
staging
Choose a base branch
from
sc_tests
base: staging
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 62 commits
Commits
Show all changes
64 commits
Select commit
Hold shift + click to select a range
b68a7db
feat: sync SDKs
arifBurakDemiray ca9b42c
feat: helper method to set sc before sdk init
arifBurakDemiray d1102f4
feat: add post support to mock test server
arifBurakDemiray b92325b
feat: some util functions to test
arifBurakDemiray 8fcf844
feat: base test for sbs
arifBurakDemiray 1721ce4
feat: initial SBS tests notes
arifBurakDemiray de26a22
feat: SBS tests order validation
arifBurakDemiray 8b19c1a
feat: move sbs related utils to related utils
arifBurakDemiray d74eab8
feat: get server config
arifBurakDemiray 43ad93e
feat: use name rather than id
arifBurakDemiray b2f4262
feat: apply custom handler delay beforehand
arifBurakDemiray 82dd61b
feat: backoff to the base test
arifBurakDemiray 310f73d
fix: revert delay with custom handler
arifBurakDemiray 996cba4
feat: request counter to sbs utils
arifBurakDemiray 17b8589
feat: sbs order test
arifBurakDemiray e2006ee
feat: introduce order tests
arifBurakDemiray bd07ca5
feat: DP_P order test
arifBurakDemiray 978b04c
doc: add line comment to the internal event validation function
arifBurakDemiray 758232b
feat: dp p fs test
arifBurakDemiray 55e79f5
feat: dp s fs test
arifBurakDemiray 2b000fe
feat: dp s p fs test
arifBurakDemiray 1f36955
feat: base SBS test ios
arifBurakDemiray 230401f
feat: dp s p fs iOS
arifBurakDemiray a5fddc7
feat: test comments
arifBurakDemiray 94a3c75
feat: feature test sets plan
arifBurakDemiray 34dc0d3
feat: remove backoff because duplicate
arifBurakDemiray 3a1b3ae
feat: move order tests to folder
arifBurakDemiray 67df4a0
fix: update imports
arifBurakDemiray eca7608
feat: init time status table to order tests
arifBurakDemiray cf3f26f
feat: DP FS redo
arifBurakDemiray 92ef0e8
feat: DP P redo
arifBurakDemiray 67639db
feat: DP P FS redo
arifBurakDemiray 1229dbc
feat: merge couple of order tests
arifBurakDemiray 407c808
fix: iOS temp id wont apply S SBS
arifBurakDemiray b3f8f8f
feat: new test and possible notes test mapping
arifBurakDemiray 9d3a696
fix: ts on 201E sbs
arifBurakDemiray ef36574
fix: test setup migration version
arifBurakDemiray 33352e9
feat: update orde test notes
arifBurakDemiray 4e0c2db
refactor: order tests shorten server part
arifBurakDemiray a701dfd
refactor: base test
arifBurakDemiray 7d5b492
feat: 200A feature test
arifBurakDemiray 96a46fc
feat: feature test A don
arifBurakDemiray 15c4152
feat: sbs200b tracking test
arifBurakDemiray 9919fa8
feat: 200B feature test
arifBurakDemiray 0b6d115
fix: ioS disable networking hc
arifBurakDemiray ee3702d
feat: move dort to the SBS 200 A
arifBurakDemiray 5a17440
feat: new function and move rq check in A casue ios crash
arifBurakDemiray dbfcb6e
feat: fix consent order in SBS
arifBurakDemiray 0c8f75c
feat: if checks to completion handlers widgets
arifBurakDemiray e046fd5
feat: act consent like in android sbs
arifBurakDemiray 0e26ed9
feat: 200C sbs feature tests
arifBurakDemiray fb776b0
refactor: 200C startup
arifBurakDemiray 4840576
feat: basic 200D
arifBurakDemiray dd82cab
feat: nps widget report
arifBurakDemiray 9343698
feat: nps widget report
arifBurakDemiray 0b44c9f
fix: stablize SBS200D
arifBurakDemiray 26c14dd
feat: sbs200d test eqs
arifBurakDemiray a0b2771
feat: sbs200d finalized
arifBurakDemiray bb52c1a
feat: validation tests SBS
arifBurakDemiray 00b2260
fix: remove unused import
arifBurakDemiray bbacfb4
fix: rename validation tests
arifBurakDemiray e2b15ff
fix: add unkown key
arifBurakDemiray eeaa600
Update ios/Classes/CountlyiOS/CountlyServerConfig.m
arifBurakDemiray 321c7f5
Update android/src/main/java/ly/count/dart/countly_flutter/CountlyFlu…
arifBurakDemiray File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import 'dart:io'; | ||
|
|
||
| import 'package:countly_flutter/countly_flutter.dart'; | ||
| import 'package:flutter_test/flutter_test.dart'; | ||
| import 'package:integration_test/integration_test.dart'; | ||
|
|
||
| import '../utils.dart'; | ||
| import 'sbs_utils.dart'; | ||
|
|
||
| ///This test calls all features possible | ||
| ///It is base test, tries to show how features working without SBS and defaults | ||
| void main() { | ||
| IntegrationTestWidgetsFlutterBinding.ensureInitialized(); | ||
| testWidgets('SBS_000_base', (WidgetTester tester) async { | ||
| List<Map<String, List<String>>> requestArray = <Map<String, List<String>>>[]; | ||
| createServerWithConfig(requestArray, {}); | ||
| // Initialize the SDK | ||
| CountlyConfig config = CountlyConfig('http://0.0.0.0:8080', APP_KEY).enableManualSessionHandling().setLoggingEnabled(true); | ||
| await Countly.initWithConfig(config); | ||
|
|
||
| await callAllFeatures(); | ||
|
|
||
| List<String> RQ = await getRequestQueue(); | ||
| List<String> EQ = await getEventQueue(); | ||
| expect(RQ.length, 0); | ||
| expect(EQ.length, 0); | ||
| validateRequestCounts({'events': 3, 'location': 1, 'crash': 2, 'begin_session': 1, 'consent': 0, 'end_session': 1, 'session_duration': 2, 'apm': 2, 'user_details': Platform.isIOS ? 2 : 1}, requestArray); | ||
| validateInternalEventCounts({'orientation': 1, 'view': 6, 'nps': 1}, requestArray); | ||
| validateImmediateCounts({'hc': 1, 'sc': 1, 'feedback': 1, 'queue': 2, 'ab': 1, 'ab_opt_out': 1, 'rc': 1}, requestArray); | ||
| }); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| import 'dart:convert'; | ||
| import 'dart:io'; | ||
|
|
||
| import 'package:countly_flutter/countly_flutter.dart'; | ||
| import 'package:flutter_test/flutter_test.dart'; | ||
| import 'package:integration_test/integration_test.dart'; | ||
|
|
||
| import '../utils.dart'; | ||
| import 'sbs_utils.dart'; | ||
|
|
||
| void main() { | ||
| IntegrationTestWidgetsFlutterBinding.ensureInitialized(); | ||
| testWidgets('SBS_200A_test', (WidgetTester tester) async { | ||
| List<Map<String, List<String>>> requestArray = <Map<String, List<String>>>[]; | ||
| createServerWithConfig(requestArray, { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'lkl': 5, 'lvs': 5, 'lsv': 5, 'lbc': 5, 'ltlpt': 5, 'ltl': 5, 'rcz': false, 'ecz': true, 'czi': 16, 'bom': false, 'dort': 1} | ||
| }); | ||
|
|
||
| // Initialize the SDK | ||
| CountlyConfig config = CountlyConfig('http://0.0.0.0:8080', APP_KEY).enableManualSessionHandling().setLoggingEnabled(true); | ||
|
|
||
| await Countly.initWithConfig(config); | ||
| await Future.delayed(const Duration(seconds: 2)); | ||
|
|
||
| storeRequest({'first': 'true', 'device_id': 'device_id_200C', 'app_key': APP_KEY, 'timestamp': DateTime.now().subtract(const Duration(minutes: 65)).millisecondsSinceEpoch.toString()}); | ||
| storeRequest({'second': 'true', 'device_id': 'device_id_200C', 'app_key': APP_KEY, 'timestamp': DateTime.now().subtract(const Duration(minutes: 45)).millisecondsSinceEpoch.toString()}); | ||
|
|
||
| List<Map<String, List<String>>> RQ = await getRequestQueueParsed(); | ||
| validateRequestCounts({'first': 1, 'second': 1}, RQ); // validate that requests are stored correctly | ||
|
|
||
| await callAllFeatures(disableEnterContent: true); | ||
| RQ = await getRequestQueueParsed(); | ||
| expect(RQ.length, 0); | ||
|
|
||
| validateRequestCounts({'first': 0, 'second': 1, 'events': Platform.isAndroid ? 3 : 2, 'location': 1, 'crash': 2, 'begin_session': 1, 'end_session': 1, 'session_duration': 2, 'apm': 2, 'user_details': Platform.isIOS ? 2 : 1, 'consent': 0}, requestArray); | ||
| // validate that first request is deleted from the queue because of dort: 1 | ||
| validateInternalEventCounts({'orientation': 1, 'view': Platform.isAndroid ? 6 : 5, 'nps': 1}, requestArray); // 6 android | ||
| // enter content zone is not called, but a content zone request is sent it is because server config is set cz to true | ||
| validateImmediateCounts({'hc': 1, 'sc': 1, 'feedback': 1, 'queue': 2, 'ab': 1, 'ab_opt_out': 1, 'rc': 1}, requestArray); | ||
|
|
||
| for (var queryParams in requestArray) { | ||
| if (queryParams.containsKey('method') || queryParams.containsKey('hc') || queryParams.containsKey('second')) { | ||
| continue; // skip immediate requests | ||
| } | ||
| testCommonRequestParams(queryParams); // checks general params | ||
| if (queryParams.containsKey('apm')) { | ||
| Map<String, dynamic> apm = json.decode(queryParams['apm']![0]); | ||
| expect(apm['name'].toString().length <= 5, isTrue); | ||
| } else if (queryParams.containsKey('crash')) { | ||
| Map<String, dynamic> crash = json.decode(queryParams['crash']![0]); | ||
| Map<String, dynamic> crashDetails = crash['_custom']; | ||
| expect(crashDetails.length <= 5, isTrue); | ||
| List<String> logs = (crash['_logs'] as String).split('\n').where((line) => line.trim().isNotEmpty).toList(); | ||
| expect(logs.length <= 5, isTrue); | ||
| for (var log in logs) { | ||
| expect(log.length <= 5, isTrue); | ||
| } | ||
| // iOS crash limits are not applied to the stack trace | ||
| if (Platform.isAndroid) { | ||
| List<String> stackTraces = crash['_error'].split('\n'); | ||
| for (var stackTrace in stackTraces) { | ||
| expect(stackTrace.length <= 5, isTrue); | ||
| } | ||
| } | ||
|
|
||
| for (var key in crashDetails.keys) { | ||
| expect(key.length <= 5, isTrue); | ||
| expect(crashDetails[key].toString().length <= 5, isTrue); | ||
| } | ||
| } else if (queryParams.containsKey('events')) { | ||
| var eventRaw = json.decode(queryParams['events']![0]); | ||
| for (var event in eventRaw) { | ||
| validateInternalLimitsForEvents(event, 5, 5, 5); | ||
| } | ||
| } else if (queryParams.containsKey('user_details')) { | ||
| Map<String, dynamic> userDetails = json.decode(queryParams['user_details']![0]); | ||
| if (userDetails['custom'] != null && userDetails['custom'].length <= 2) { | ||
| // operators are not truncated with segmentation values limit | ||
| expect((userDetails['custom'].values.where((v) => v is! Map).length ?? 0) <= 5, isTrue); | ||
| expect(userDetails['custom']['speci'], 'somet'); | ||
| expect(userDetails['custom']['not_s'], 'somet'); | ||
| } | ||
|
|
||
| // in iOS user data requests are formed in a different request | ||
| if (userDetails['custom'].length > 2) { | ||
| checkUnchangingUserData(userDetails, 5, 5); | ||
| } | ||
|
|
||
| if (Platform.isAndroid || (Platform.isIOS && userDetails['custom'] == null)) { | ||
| checkUnchangingUserPropeties(userDetails, 5); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| await Countly.instance.content.refreshContentZone(); // this will not affect because refresh disabled | ||
| validateImmediateCounts({'hc': 1, 'sc': 1, 'feedback': 1, 'queue': 2, 'ab': 1, 'ab_opt_out': 1, 'rc': 1}, requestArray); | ||
|
|
||
| await Countly.instance.content.exitContentZone(); | ||
| requestArray.clear(); | ||
|
|
||
| sbsServerDelay = 11; | ||
|
|
||
| await Countly.instance.sessions.beginSession(); | ||
| await Countly.instance.sessions.endSession(); // this will not be backed off because backoff disabled | ||
| await Future.delayed(const Duration(seconds: 10)); // wait for sdk to process and get the result from server | ||
|
|
||
| await Countly.instance.attemptToSendStoredRequests(); // this will take affect and trigger sending the requests | ||
| await Future.delayed(const Duration(seconds: 2)); | ||
|
|
||
| validateRequestCounts({'begin_session': 1, 'end_session': 1}, requestArray); | ||
| expect(await getServerConfig(), { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'lkl': 5, 'lvs': 5, 'lsv': 5, 'lbc': 5, 'ltlpt': 5, 'ltl': 5, 'rcz': false, 'ecz': true, 'czi': 16, 'bom': false, 'dort': 1} | ||
| }); | ||
| }); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| import 'package:countly_flutter/countly_flutter.dart'; | ||
| import 'package:flutter_test/flutter_test.dart'; | ||
| import 'package:integration_test/integration_test.dart'; | ||
|
|
||
| import '../utils.dart'; | ||
| import 'sbs_utils.dart'; | ||
|
|
||
| /// Currently it is not possible to test SCUI, we only test its value validations | ||
| void main() { | ||
| IntegrationTestWidgetsFlutterBinding.ensureInitialized(); | ||
| testWidgets('SBS_200B_test', (WidgetTester tester) async { | ||
| List<Map<String, List<String>>> requestArray = <Map<String, List<String>>>[]; | ||
| createServerWithConfig(requestArray, { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'tracking': false, 'scui': 1} | ||
| }); | ||
|
|
||
| // Initialize the SDK | ||
| CountlyConfig config = CountlyConfig('http://0.0.0.0:8080', APP_KEY).enableManualSessionHandling().setLoggingEnabled(true); | ||
|
|
||
| await Countly.initWithConfig(config); | ||
| await Future.delayed(const Duration(seconds: 2)); | ||
|
|
||
| await callAllFeatures(disableSend: true); | ||
| List<String> RQ = await getRequestQueue(); | ||
| List<String> EQ = await getEventQueue(); | ||
| expect(RQ.length, 0); | ||
| expect(EQ.length, 0); | ||
|
|
||
| await Countly.instance.attemptToSendStoredRequests(); | ||
| // check queues are empty and all requests are sent | ||
| await Future.delayed(const Duration(seconds: 10)); | ||
|
|
||
| validateRequestCounts({'events': 0, 'location': 0, 'crash': 0, 'begin_session': 0, 'end_session': 0, 'session_duration': 0, 'apm': 0, 'user_details': 0, 'consent': 0}, requestArray); | ||
| validateInternalEventCounts({}, requestArray); // 6 android | ||
| // enter content zone is not called, but a content zone request is sent it is because server config is set cz to true | ||
| validateImmediateCounts({'hc': 1, 'sc': 1, 'feedback': 1, 'queue': 2, 'rc': 1}, requestArray); // ab and ab_opt_out are not called because they are not immediate methods | ||
|
|
||
| expect(await getServerConfig(), { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'tracking': false, 'scui': 1} | ||
| }); | ||
|
|
||
| await Future.delayed(const Duration(seconds: 60)); | ||
| // wait one minute and ensure no sc requests sent | ||
| validateImmediateCounts({'hc': 1, 'sc': 1, 'feedback': 1, 'queue': 4, 'rc': 1}, requestArray); | ||
| }); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| import 'dart:convert'; | ||
| import 'dart:io'; | ||
|
|
||
| import 'package:countly_flutter/countly_flutter.dart'; | ||
| import 'package:flutter_test/flutter_test.dart'; | ||
| import 'package:integration_test/integration_test.dart'; | ||
|
|
||
| import '../utils.dart'; | ||
| import 'sbs_utils.dart'; | ||
|
|
||
| /// use auto sessions for showing session update | ||
| void main() { | ||
| IntegrationTestWidgetsFlutterBinding.ensureInitialized(); | ||
| testWidgets('SBS_200C_test', (WidgetTester tester) async { | ||
| List<Map<String, List<String>>> requestArray = <Map<String, List<String>>>[]; | ||
| createServerWithConfig(requestArray, { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'networking': false, 'cr': true, 'rqs': 5, 'sui': 10} | ||
| }); | ||
|
|
||
| setServerConfig({ | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'networking': false, 'cr': true, 'rqs': 5, 'sui': 10} | ||
| }); | ||
|
|
||
| // Initialize the SDK | ||
| CountlyConfig config = CountlyConfig('http://0.0.0.0:8080', APP_KEY).setLoggingEnabled(true).setDeviceId('device_id_200C'); | ||
|
|
||
| await Countly.initWithConfig(config); | ||
| await Future.delayed(const Duration(seconds: 2)); | ||
|
|
||
| await callAllFeatures(disableConsentCall: true); | ||
|
|
||
| // Validate that networking is disabled and no requests are sent | ||
| expect(requestArray.length, 1); // only SC request should be sent | ||
| validateImmediateCounts({'sc': 1}, requestArray); | ||
| requestArray.clear(); // clear requestArray to validate the next requests | ||
|
|
||
| // Validate that consent is required and not given and all called features are not created a request | ||
| List<Map<String, List<String>>> rq = await getRequestQueueParsed(); | ||
| validateRequestCounts({'consent': 1, 'location': 1}, rq); | ||
| Map<String, dynamic> expectedConsent = {'push': false, 'views': false, 'attribution': false, 'content': false, 'users': false, 'feedback': false, 'apm': false, 'location': false, 'remote-config': false, 'sessions': false, 'crashes': false, 'events': false}; | ||
|
|
||
| if (Platform.isAndroid) { | ||
| expectedConsent['scrolls'] = false; // Android has scrolls, content, star-rating, clicks consents extra | ||
| expectedConsent['content'] = false; | ||
| expectedConsent['star-rating'] = false; | ||
| expectedConsent['clicks'] = false; | ||
| } | ||
|
|
||
| expect(jsonDecode(rq[0]['consent']![0]), expectedConsent); | ||
| expect(rq[1]['location']![0], ''); | ||
| expect(rq.length, 2); | ||
|
|
||
| // Validate that session update occurs in every 10 seconds | ||
| await Countly.giveConsent(['sessions']); | ||
| // after giving this | ||
| // one consent, one begin session and two duration requests should be sent | ||
| // however this adds up to 6 request | ||
| // because our RQ limit is 5 the first consent request where all false is dropped | ||
|
|
||
| await Future.delayed(const Duration(seconds: 25)); | ||
| rq = await getRequestQueueParsed(); | ||
| expect(rq.length, 5); // 5 request at max could be | ||
| expect(requestArray.length, 0); // none request sent after sc request | ||
|
|
||
| validateRequestCounts({'begin_session': 1, 'session_duration': 2, 'consent': 1, 'location': 2}, rq); // one location is in begin_session | ||
| expect(rq[0]['location']![0], ''); // first request is location request from previous validations, it was consent request before but now location | ||
|
|
||
| expect(rq[1]['begin_session']![0], '1'); // second request is begin session request from auto sessions | ||
| expect(rq[1]['location']![0], ''); // show location is disabled because no consent given with tied to session request | ||
|
|
||
| expectedConsent['sessions'] = true; // now sessions consent is true | ||
| expect(jsonDecode(rq[2]['consent']![0]), expectedConsent); // second request is consent request | ||
|
|
||
| expect(rq[3]['session_duration']![0], '5'); // fourth request is session duration request | ||
| expect(rq[4]['session_duration']![0], '10'); // fifth request is session duration request and it is 10 | ||
|
|
||
| expect(await getServerConfig(), { | ||
| 'v': 1, | ||
| 't': 1750748806695, | ||
| 'c': {'networking': false, 'cr': true, 'rqs': 5, 'sui': 10} | ||
| }); | ||
| }); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.