Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Screen Broadcast Stops Unexpectedly After short time when app is in background #510

Open
creativecbr opened this issue Nov 15, 2024 · 3 comments
Open

Screen Broadcast Stops Unexpectedly After short time when app is in background #510

creativecbr opened this issue Nov 15, 2024 · 3 comments
Assignees
@hiroshihorie
Labels
bug Something isn't working

Comments

@creativecbr
Copy link
Contributor

creativecbr commented Nov 15, 2024 •
edited
Loading

UPDATED

Describe the bug

The screen broadcasting session on iOS/iPadOS automatically stops after a short period (typically between 30 seconds and 5 minutes) when the main app is moved to the background. This issue is consistently reproducible in production environments. However, when running the app with an attached debugger, the broadcasting session does not stop because the app does not enter the suspended state.

It appears that the broadcast extension functions correctly and attempts to send frames while the main app is in the background. However, the system automatically transitions the main app to a suspended state, causing the broadcast extension to stop working.

In the production build, the system automatically terminates the broadcasting session. System logs indicate that the app transitions to the suspended state, at which point the broadcasting stops:

default	11:35:41.397185+0100	runningboardd	Calculated state for app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>: running-suspended (role: None) (endowments: (null))
default	11:35:41.397223+0100	runningboardd	[app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:7642] Set jetsam priority to 30 [0] flag[1]
default	11:35:41.397285+0100	runningboardd	[app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:7642] Suspending task.
default	11:35:41.397494+0100	mDNSResponder	[R26066] DNSServiceCreateConnection STOP PID[7642](RemoteAccess)
default	11:35:41.397629+0100	runningboardd	[app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:7642] Shutdown sockets (SVC)
default	11:35:41.397665+0100	runningboardd	[app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:7642] Set darwin role to: None
default	11:35:41.397810+0100	runningboardd	[app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:7642] check if suspended process is holding locks
default	11:35:41.409815+0100	CommCenter	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.411019+0100	SpringBoard	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.411155+0100	backboardd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.411391+0100	SpringBoard	[app<io.app.demo.RemoteAccess>:7642] Setting process task state to: Suspended
default	11:35:41.411484+0100	SpringBoard	Application process state changed for io.app.demo.RemoteAccess: <SBApplicationProcessState: 0x307971460; pid: 7642; taskState: Suspended; visibility: Background>
default	11:35:41.412708+0100	SpringBoard	[io.app.demo.RemoteAccess] io.app.demo.RemoteAccess application state changed to <RBSProcessState| task:running-suspended debug:none>
default	11:35:41.412997+0100	cameracaptured	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.413139+0100	cameracaptured	<<<< FigApplicationStateMonitor >>>> fasm_updateHandler: application <private> updated to state 3 from state 4
default	11:35:41.413239+0100	symptomsd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.413566+0100	audiomxd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.413768+0100	replayd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.413840+0100	audiomxd	-MX_RunningBoardServices- mx_runningBoardServices_setApplicationStateToCache: Process '7642' state is updated to 'Background Suspended'
default	11:35:41.414843+0100	audiomxd	-CMSessionMgr- CMSessionMgrHandleApplicationStateChange:16822 Client io.app.demo.RemoteAccess with session sid:0x6dbe4, RemoteAccess(7642), 'prim' [0x69140ce00] with pid '7642' is now Background Suspended. Background entitlement: YES ActiveLongFormVideoSession: NO IsLongFormVideoApp NO
default	11:35:41.414927+0100	**audiomxd	-CMSessionMgr- CMSessionMgrHandleApplicationStateChange: Sending stop command to io.app.demo.RemoteAccess with pid '7642' because client is background suspended and there is no AirPlay video session for it**
default	11:35:41.415257+0100	audiomxd	          SSServerImp.cpp:1362  pid 7642(RemoteAccess)
default	11:35:41.415419+0100	replayd	-[RPApplicationStateMonitor applicationStateDidChange:] userInfo {
    BKSApplicationStateAppIsFrontmost = 0;
    BKSApplicationStateExtensionKey = 0;
    SBApplicationStateDisplayIDKey = "io.app.demo.RemoteAccess";
    SBApplicationStateKey = 2;
    SBApplicationStateProcessIDKey = 7642;
    SBMostElevatedStateForProcessID = 2;
} previousState {
    3252 = 8;
}
default	11:35:41.415606+0100	replayd	-[RPApplicationStateMonitor notifyInAppSessionShouldPauseOrResume:] processIdentifer 7642 observer nonnil 0
default	11:35:41.415829+0100	useractivityd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.416505+0100	callservicesd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.417187+0100	wifid	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.417649+0100	gamepolicyd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.417835+0100	UserEventAgent	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.418487+0100	WirelessRadioManagerd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.420958+0100	runningboardd	Acquiring assertion targeting [xpcservice<com.apple.WebKit.Networking([app<com.google.chrome.ios((null))>:7507])>{vt hash: 114736818}[uuid:F79A1119-BA72-4C56-B689-1A1A5CFC4286]:7550] from originator [xpcservice<com.apple.WebKit.Networking([app<com.google.chrome.ios((null))>:7507])>{vt hash: 114736818}[uuid:F79A1119-BA72-4C56-B689-1A1A5CFC4286]:7550] with description <RBSAssertionDescriptor| "Network Process is holding locked files" ID:35-7550-288746 target:7550 attributes:[
	<RBSDomainAttribute| domain:"com.apple.common" name:"FinishTaskCanSleep" sourceEnvironment:"(null)">
	]>
default	11:35:41.421325+0100	runningboardd	Assertion 35-7550-288746 (target:[xpcservice<com.apple.WebKit.Networking([app<com.google.chrome.ios((null))>:7507])>{vt hash: 114736818}[uuid:F79A1119-BA72-4C56-B689-1A1A5CFC4286]:7550]) will be created as inactive as start-time-defining assertions exist
default	11:35:41.431185+0100	dasd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.438930+0100	PerfPowerServicesExtended	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.440684+0100	kernel	ANE0: ProgramLoad: program was already loaded programHandle: 0x226112b4b00 programId: 6
default	11:35:41.441071+0100	kernel	ANE0: processTargetToHostIOCommand: DEBUG: Got the program request complete notification - programId: 6 processId: 6 uuid: 0x<private> procedureid: 0 programEvent: 1
default	11:35:41.446921+0100	RemoteAccessExtension	 [INFO] -[RPBroadcastSampleHandler _processPayloadWithAudioSample:type:]:180 Broadcast extension received audio payload from replayd
default	11:35:41.447038+0100	replayd	 [INFO] -[RPSystemBroadcastSession notifyExtensionOfAudioSampleBuffer:withType:]_block_invoke_2:621 Sending 2 audio payload to broadcast extension...
default	11:35:41.447133+0100	replayd	 [INFO] -[RPSystemBroadcastSession notifyExtensionOfAudioSampleBuffer:withType:]_block_invoke_2:621 Sending 2 audio payload to broadcast extension...
default	11:35:41.447200+0100	replayd	 [INFO] -[RPSystemBroadcastSession notifyExtensionOfAudioSampleBuffer:withType:]_block_invoke_2:621 Sending 2 audio payload to broadcast extension...
default	11:35:41.447303+0100	replayd	RPClientProxy:recordingTimerDidUpdate:
default	11:35:41.447417+0100	RemoteAccessExtension	 [INFO] -[RPBroadcastSampleHandler _processPayloadWithAudioSample:type:]:180 Broadcast extension received audio payload from replayd
default	11:35:41.447459+0100	SpringBoard	RPDaemonProxy: recordingTimerDidUpdate:
default	11:35:41.447531+0100	SpringBoard	 [INFO] -[RPScreenRecorder recordingTimerDidUpdate:]:1568 time[<private>]
default	11:35:41.447599+0100	RemoteAccessExtension	 [INFO] -[RPBroadcastSampleHandler _processPayloadWithAudioSample:type:]:180 Broadcast extension received audio payload from replayd
default	11:35:41.447688+0100	replayd	 [INFO] -[RPSystemBroadcastSession notifyExtensionOfAudioSampleBuffer:withType:]_block_invoke_2:621 Sending 2 audio payload to broadcast extension...
default	11:35:41.447738+0100	SpringBoard	 [INFO] -[RPControlCenterMenuModuleViewController updateRPCCModuleMenuItems]:503 0x876bf6a00 client=0x3012ae800
default	11:35:41.447782+0100	RemoteAccessExtension	 [INFO] -[RPBroadcastSampleHandler _processPayloadWithAudioSample:type:]:180 Broadcast extension received audio payload from replayd
default	11:35:41.447804+0100	SpringBoard	Setting 21 menu items for RPControlCenterMenuModuleViewController: <private>
default	11:35:41.448577+0100	CommCenter	QMI: Svc=0xe4(AWD) Ind MsgId=0x1012 Bin=[<private>]
default	11:35:41.448600+0100	CommCenter	QMI: Svc=0xe4(AWD) Ind MsgId=0x1012 Bin=[<private>]
default	11:35:41.448621+0100	SpringBoard	 [INFO] -[RPControlCenterClient imageForBundleID:extensionInfo:]:653 using cached image for extension
default	11:35:41.448712+0100	CommCenter	#I Received trigger 0xc762d
default	11:35:41.450670+0100	locationd	Received state update for 7642 (app<io.app.demo.RemoteAccess(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>, running-suspended-NotVisible
default	11:35:41.450771+0100	locationd	{"msg":"invoking applicationStateChange handler", "StateChangeData":"{\n    BKSApplicationStateAppIsFrontmost = 0;\n    BKSApplicationStateExtensionKey = 0;\n    SBApplicationStateDisplayIDKey = \"io.app.demo.RemoteAccess\";\n    SBApplicationStateKey = 2;\n    SBApplicationStateProcessIDKey = 7642;\n    SBMostElevatedStateForProcessID = 2;\n}"}
default	11:35:41.450874+0100	locationd	{"msg":"Post Application State Change Notification", "notification":"BackgroundTaskSuspended", "pid":7642, "bundleId":"io.app.demo.RemoteAccess"}
default	11:35:41.451488+0100	locationd	{"msg":"#Warning #ClientResolution the passed keyPath is not registered. Resolving to #nullCKP", "InputCKP":"iio.app.demo.RemoteAccess:"}
default	11:35:41.451555+0100	locationd	os_transaction created: (0x16c179c20) GetIdentifyingInformation
default	11:35:41.452038+0100	mDNSResponder	[R26089] getaddrinfo start -- flags: 0xC000D000, ifindex: 0, protocols: 0, hostname: <mask.hash: '6VY6gMVtwoCNAkNgMaTvrQ=='>, options: 0x4 {in-app-browser}, client pid: 7550 (com.apple.WebKi), delegator pid: 7507 (Chrome)
default	11:35:41.452199+0100	mDNSResponder	[R26089->Q10431] Question for <mask.hash: 'VzQ7ArJvJRIqbsKhDzDC+g=='> (HTTPS) assigned DNS service -- id: 271, type: ODoH, source: nw, scope: uuid, interface: /0, servers: {}, domains: {}, attributes: {a-ok, aaaa-ok, fail-fast, allows-failover}, interface properties: {ipv4}, resolver config: {provider name: BBPLZHAI, provider path: BBbIfmrY}, parent: 4
default	11:35:41.452441+0100	locationd	os_transaction releasing: (0x16c179c20) GetIdentifyingInformation
default	11:35:41.452481+0100	locationd	{"msg":"#CLIUA AppMonitor notification", "notification":"BackgroundTaskSuspended", "pid":7642, "bundleId":"io.app.demo.RemoteAccess", "ClientKey":"iio.app.demo.RemoteAccess:"}
default	11:35:41.452516+0100	locationd	{"msg":"skip erasing #CLIUA for RunningBoard Process State. Does not exists", "bundleId":"io.app.demo.RemoteAccess"}
default	11:35:41.452543+0100	mDNSResponder	[R26089->Q36595] Question assigned DNS service 271

The issue persists across versions 2.0.4, 2.0.15, and 2.0.17. Previously, the broadcasting feature worked correctly without this problem what is weird. Moreover, when the session is running while the main app is in the foreground, it can continue working all day without interruption.

SDK Version

  • LiveKit SDK versions tested: 2.0.4, 2.0.15, 2.0.17

iOS Version

  • Occurs on iOS 15.4, 18.1.

Xcode Version

  • Tested on Xcode 16.1 (16A242d)

Steps to Reproduce

0a. Launch app without debbuger connected.
0b. When running with an attached debugger, observe the error logs in Console.app.

  1. Connect to the LiveKit room.
  2. Start a screen broadcast session using the iOS Broadcast Extension with the LiveKit SDK.
  3. Send app to background by e.g. going to Safari app
  4. Observe the behavior for 30 seconds - 5 minutes.
    5a. The broadcast stops automatically.
    5b. Logs in Console.app occurs, broadcast status seams as continued but it is not.

Expected Behavior

The screen broadcast session should continue without interruption, as it did in previous SDK and iOS versions, without stopping after short time.

Screenshots

Project settings:

Screenshot 2024-11-19 at 13 33 48

Logs

logs.txt

Example project with reproduction

RemoteAccess_for_Livekit.zip

Please advise if there are any known workarounds or fixes for this issue.
@creativecbr creativecbr added the bug Something isn't working label Nov 15, 2024
@creativecbr
Copy link
Contributor Author

creativecbr commented Nov 19, 2024 •
edited
Loading

I found that setting background processing in background modes eliminates error:

Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=2 "Assertion's invalidation time is in the past" UserInfo={IgnoreOnReconnect=true, NSLocalizedFailureReason=Assertion's invalidation time is in the past}>

The first problem I encountered was before I updated the description. The current description is now accurate.

@creativecbr
Copy link
Contributor Author

Unfortunately, despite of added background modes aforementioned error still occurs:

default	11:54:39.493475+0100	runningboardd	Assertion 35-9504-354426 (target:[app<io.myapp.myappbundle(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:9504]) will be created as active as no start-time-defining assertions exist
error	11:54:39.493501+0100	MyApp	Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=2 "Assertion's invalidation time is in the past" UserInfo={IgnoreOnReconnect=true, NSLocalizedFailureReason=Assertion's invalidation time is in the past}>
default	11:54:39.493592+0100	runningboardd	Acquiring assertion targeting [app<io.myapp.myappbundle(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:9504] from originator [app<io.myapp.myappbundle(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:9504] with description <RBSAssertionDescriptor| "CMPhoto pool cleanup timer" ID:35-9504-354427 target:9504 attributes:[
	<RBSDomainAttribute| domain:"com.apple.common" name:"FinishTaskCanSleep" sourceEnvironment:"(null)">
	]>
default	11:54:39.493618+0100	runningboardd	Assertion 35-9504-354427 (target:[app<io.myapp.myappbundle(7E2299AE-BA30-4489-8C7F-E5EEEDBE3CC4)>:9504]) will be created as active as no start-time-defining assertions exist
error	11:54:39.493639+0100	MyApp	Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=2 "Assertion's invalidation time is in the past" UserInfo={IgnoreOnReconnect=true, NSLocalizedFailureReason=Assertion's invalidation time is in the past}>
default	11:54:39.493654+0100	kernel	AppleJPEGDriver: + virtual IOReturn AppleJPEGDriver::newUserClient(task_t, void *, UInt32, OSDictionary *, IOUserClient **) [<private>] :  task 0x<private>, securityID <private>, type 0, properties 0x0, handler <private>

@davidliu
Copy link
Contributor

iOS background mode isn't automatic, you'll need to ensure you have something to keep it active in the background.

For voip background mode, you should use CallKit to ensure the app stays active throughout the call session.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hiroshihorie hiroshihorie

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@davidliu @hiroshihorie @creativecbr