EXC_BAD_ACCESS in the TextDecoding class

[BrandonWeng]

Ran into this while transcribing, its fairly rare but have seen in 2-3 times in ~30 hours?

Thread 31 Queue : com.apple.root.default-qos (concurrent)
#0	0x000000019b97508c in Swift.IndexingIterator.next() -> Swift.Optional<τ_0_0.Element> ()
#1	0x00000001064ecb54 in closure #1 in closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at ...SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:289
#2	0x0000000106509568 in partial apply for closure #1 in closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) ()
#3	0x00000001a1928234 in partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@unowned Swift.Int) -> () to @escaping @callee_guaranteed (@unowned Swift.Int) -> () ()
#4	0x00000001a1928260 in reabstraction thunk helper from @escaping @callee_guaranteed (@unowned Swift.Int) -> () to @callee_unowned @convention(block) (@unowned Swift.Int) -> () ()
#5	0x000000010448a864 in _dispatch_client_callout2 ()
#6	0x00000001044a6264 in _dispatch_apply_invoke3 ()
#7	0x000000010448a824 in _dispatch_client_callout ()
#8	0x000000010448c768 in _dispatch_once_callout ()
#9	0x00000001044a5034 in _dispatch_apply_invoke ()
#10	0x000000010448a824 in _dispatch_client_callout ()
#11	0x00000001044a2db8 in _dispatch_root_queue_drain ()
#12	0x00000001044a377c in _dispatch_worker_thread2 ()
#13	0x00000001045250c4 in _pthread_wqthread ()

Thread 33 Queue : com.apple.root.default-qos (concurrent)
#0	0x000000019b97508c in Swift.IndexingIterator.next() -> Swift.Optional<τ_0_0.Element> ()
#1	0x00000001064ecb54 in closure #1 in closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at /.../SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:289
#2	0x0000000106509568 in partial apply for closure #1 in closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) ()
#3	0x00000001a1928234 in partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@unowned Swift.Int) -> () to @escaping @callee_guaranteed (@unowned Swift.Int) -> () ()
#4	0x00000001a1928260 in reabstraction thunk helper from @escaping @callee_guaranteed (@unowned Swift.Int) -> () to @callee_unowned @convention(block) (@unowned Swift.Int) -> () ()
#5	0x000000010448a864 in _dispatch_client_callout2 ()
#6	0x00000001044a6264 in _dispatch_apply_invoke3 ()
#7	0x000000010448a824 in _dispatch_client_callout ()
#8	0x000000010448c768 in _dispatch_once_callout ()
#9	0x00000001044a6d34 in _dispatch_apply_invoke_and_wait ()
#10	0x00000001044a56e4 in _dispatch_apply_with_attr_f ()
#11	0x00000001044a5a6c in dispatch_apply ()
#12	0x00000001a19282ec in _swift_dispatch_apply_current ()
#13	0x00000001a19281b8 in static __C.OS_dispatch_queue.concurrentPerform(iterations: Swift.Int, execute: (Swift.Int) -> ()) -> () ()
#14	0x00000001064ec904 in closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at /.../SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:287
#15	0x00000001065094f0 in partial apply for closure #1 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) ()
#16	0x00000001945fe8c0 in ___lldb_unnamed_symbol31858 ()
#17	0x00000001945ffbe8 in ___lldb_unnamed_symbol31883 ()
#18	0x00000001945fe96c in ___lldb_unnamed_symbol31859 ()
#19	0x00000001948fd030 in CoreML::MultiArrayBuffer::getBytesWithHandler ()
#20	0x00000001948fd21c in -[MLMultiArray(ScopedBufferAccess) getBytesWithHandler:] ()
#21	0x00000001945fe6c0 in __C.MLMultiArray.withUnsafeBytes<τ_0_0>((Swift.UnsafeRawBufferPointer) throws -> τ_0_0) throws -> τ_0_0 ()
#22	0x00000001064ec6f8 in closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at /.../SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:285
#23	0x0000000106509474 in partial apply for closure #1 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) ()
#24	0x00000001945fee7c in ___lldb_unnamed_symbol31860 ()
#25	0x00000001945ffc58 in ___lldb_unnamed_symbol31887 ()
#26	0x00000001945fefc8 in ___lldb_unnamed_symbol31862 ()
#27	0x00000001948fce94 in __63-[MLMultiArray(ScopedBufferAccess) getMutableBytesWithHandler:]_block_invoke ()
#28	0x00000001948fd030 in CoreML::MultiArrayBuffer::getBytesWithHandler ()
#29	0x00000001948fcdd4 in -[MLMultiArray(ScopedBufferAccess) getMutableBytesWithHandler:] ()
#30	0x00000001945feb80 in __C.MLMultiArray.withUnsafeMutableBytes<τ_0_0>((Swift.UnsafeMutableRawBufferPointer, Swift.Array<Swift.Int>) throws -> τ_0_0) throws -> τ_0_0 ()
#31	0x00000001064ec514 in closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at /.../SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:284
#32	0x0000000106509404 in partial apply for closure #1 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) ()
#33	0x00000001945fe8c0 in ___lldb_unnamed_symbol31858 ()
#34	0x00000001945ffbe8 in ___lldb_unnamed_symbol31883 ()
#35	0x00000001945fe96c in ___lldb_unnamed_symbol31859 ()
#36	0x00000001948fd030 in CoreML::MultiArrayBuffer::getBytesWithHandler ()
#37	0x00000001948fd21c in -[MLMultiArray(ScopedBufferAccess) getBytesWithHandler:] ()
#38	0x00000001945fe6c0 in __C.MLMultiArray.withUnsafeBytes<τ_0_0>((Swift.UnsafeRawBufferPointer) throws -> τ_0_0) throws -> τ_0_0 ()
#39	0x00000001064ec2f8 in closure #4 in static TextDecoding.updateKVCache(keyTensor:keySlice:valueTensor:valueSlice:insertAtIndex:) at /.../SourcePackages/checkouts/WhisperKit/Sources/WhisperKit/Core/TextDecoder.swift:283

[ZachNagengast]

Thanks for the report, need to rework this section because we're nesting a lot of closures and sending to different threads inside of them. The DispatchQueue.concurrentPerform is faster in testing, but sliceShape can be deallocated which would lead to this. Needs a weak reference for a simple fix, but looking for a more robust solution if you have any suggestions.

[BrandonWeng]

Thanks for the quick response. Honestly no suggestion, we're struggling with memory issues ourselves. It only happened a handful of times, not too urgent

[BrandonWeng]

Would making a copy of sliceLength make sense?

let sliceLength = sliceShape[3] 
...
DispatchQueue.concurrentPerform(iterations: tensorShape[1]) { j in
    for k in 0..<sliceLength { 
        // ...
    }
}