iOS: Some phones fail to init AudioUnit when using shared context

[camgaertner]

Hi, first of all thanks for the work on this library. It seems like there may be an issue on some iOS devices.

Background

My use case is VOIP, where I use two ma_devices, with two threads, for input and output. After reading this comment, I began sharing an ma_context between the devices in order to play/record at the same time. This works well on my iPhone 13, but on an iPhone SE and iPhone 15, the input fails to initialize. When not sharing the context, there are no errors and the devices "start" but don't work, probably for the reasons in the above comment.

it results in two context's being created which is why your session category is getting overwritten

Behavior

My output device is initialized first and starts fine. Then, my input device tries to initialize. When it hits line 34287 in ma_device_init_internal__coreaudio, it fails with error.
This line:

status = ((ma_AudioUnitSetProperty_proc)pContext->coreaudio.AudioUnitSetProperty)(pData->audioUnit, kAudioUnitProperty_StreamFormat, formatScope, formatElement, &bestFormat, sizeof(bestFormat));

returns -10868 which corresponds to kAudioUnitErr_FormatNotSupported. I was able to print some variables at the time of failure:

(AudioStreamBasicDescription) bestFormat = {
  mSampleRate = 48000
  mFormatID = 1819304813
  mFormatFlags = 41
  mBytesPerPacket = 4
  mFramesPerPacket = 1
  mBytesPerFrame = 4
  mChannelsPerFrame = 0
  mBitsPerChannel = 32
  mReserved = 0
}
formatScope = 2
formatElement = 1

My Code

Context

std::shared_ptr<ma_context> shared_context::init(uint32_t category_options)
{
  std::unique_lock<std::mutex> lock(m_mutex);
  
  if (m_ma_context)
      return m_ma_context;
  
  m_ma_context = std::make_shared<ma_context>();
  
  ma_context_config contextConfig = ma_context_config_init();
  contextConfig.coreaudio.sessionCategory = ma_ios_session_category_play_and_record;
  contextConfig.coreaudio.sessionCategoryOptions = category_options;
  const auto result = ma_context_init(NULL, 0, &contextConfig, m_ma_context.get());
  
  if (result != MA_SUCCESS)
  {
      log_util::error("Error initializing ma_context");
      return nullptr;
  }
  
  ma_log_register_callback(ma_context_get_log(m_ma_context.get()), ma_log_callback_init(on_log, NULL));
          
  return m_ma_context;
}

Output

uint32_t category_options = output->has_speaker() ? ma_ios_session_category_option_default_to_speaker : ma_ios_session_category_option_allow_bluetooth | ma_ios_session_category_option_allow_bluetooth_a2dp;

m_ma_context = shared_context->init(category_options);

if (!m_ma_context)
    return;
      
// Device
ma_device_config deviceConfig = ma_device_config_init(ma_device_type_playback);
deviceConfig.playback.format = ma_format_s16;
deviceConfig.playback.channels = 1;
deviceConfig.sampleRate = 16000;
deviceConfig.periodSizeInFrames = 320;
deviceConfig.dataCallback = &output::DataCallback;
deviceConfig.pUserData = this;
      
m_ma_device.reset(new ma_device());
ma_result result;

// Initialize
result = ma_device_init(m_ma_context.get(), &deviceConfig, m_ma_device.get());
if (result != MA_SUCCESS)
{
  log_util::error("Error initializing output device");
  
  return;
}

// Start
result = ma_device_start(m_ma_device.get());
if (result != MA_SUCCESS)
{
  log_util::error("Error starting output device");
  ma_device_uninit(m_ma_device.get());
  
  return;
}

Input

uint32_t category_options = output->has_speaker() ? ma_ios_session_category_option_default_to_speaker : ma_ios_session_category_option_allow_bluetooth | ma_ios_session_category_option_allow_bluetooth_a2dp;

m_ma_context = shared_context->init(category_options);

if (!m_ma_context)
    return;

log_util::error("Using category options: %d", category_options);

// Device
ma_device_config deviceConfig = ma_device_config_init(ma_device_type_capture);
deviceConfig.capture.format   = ma_format_s16;
deviceConfig.capture.channels = 1;
deviceConfig.sampleRate       = 16000;
deviceConfig.periodSizeInFrames = 320;
deviceConfig.dataCallback     = &input::DataCallback;
deviceConfig.pUserData = this;

ma_result result;
m_ma_device.reset(new ma_device());

// Initialize
result = ma_device_init(m_ma_context.get(), &deviceConfig, m_ma_device.get());
if (result != MA_SUCCESS)
{
    log_util::error("Error initializing input device");
    log_util::error("Result was: %d", result);
    
    return;
}

// Start
result = ma_device_start(m_ma_device.get());
if (result != MA_SUCCESS)
{
    log_util::error("Error starting input device");
    ma_device_uninit(m_ma_device.get());
    
    return;
}

What I've tried

• Removing my format preference, i.e not specifying sampleRate or periodSizeInFrames, but the input still doesn't start properly. It may be failing in a different place, as I didn't step through this scenario
• Waiting for the output to start before initializing the input, because these happen at the same time in my program, but this didn't help.
• Not sharing a context - no issue initializing, but can't play and record at the same time
• Double checking mic permissions - definitely have it
• Debugging in the simulator - but I can't get this to work for an iPhone SE simulator or iPhone 15 simulator - I think this may be broken in general, or there is some aspect of simulator setup that I'm missing

Thanks for any guidance.

[mackron]

The idea of having one ma_context to many ma_device is certainly something that's supposed to work. It's strange that it works on some iPhone's but not others. I'm not an expert on Apple platforms (I depend on advice from the community for a lot of this stuff) so I'm not sure how much use I can be, but one thing I was questioning reading your code - is your m_ma_device object referring to two separate ma_device objects? You're not reusing the one ma_device for both output and input are you? If so, don't do that.

The other suspicious thing is mChannelsPerFrame = 0 in the format description.

If you were to do only playback, or only capture (as in only do one or the other at once), does that work for both sides?

[camgaertner]

Thanks for the reply.

Yes, I'm using separate ma_devices, I should have specified all of those code sections are in separate classes.

If you were to do only playback, or only capture (as in only do one or the other at once), does that work for both sides?

I will give this a try when I get access to one of these iOS devices that reproduce the problem again. I have a feeling this would work though.

The other suspicious thing is mChannelsPerFrame = 0 in the format description.

This should be 1, right? Could coreaudio be rejecting the format based on this? I will look into this more and see what I can find.

[camgaertner]

This turned out to be a problem in my code, and was affected by thread races, so it's not a bug in miniaudio. Thanks for entertaining this though.

The details if it helps someone else:
Before starting the input/output devices, I was playing a short sound. The sound was played from QMediaPlayer, because we were using Qt's audio engine before moving over to miniaudio. On most platforms this is fine, but on iOS, everything has to be cognizant of the AVAudioSession sharedInstance, wrt category, mode, frame rate, etc. After debugging more, I believe that Qt's media player was setting the category to AVAudioSessionCategoryPlayback sometime after miniaudio set it to AVAudioSessionCategoryPlayAndRecord. When miniaudio tried to set the format, core audio wasn't accepting it, probably because we were trying to do input when the sharedInstance was set for output/playback only. The solution was to remove these sounds for now. Later, I may use ma_engine_play_sound. Thanks again, and sorry for the bogus report.