[Sponsorship] Need Help with New Features

[RainyTunes]

Describe the bug

Hi shogo4405,

Thank you for your insistent work with HaishnKit!

We're using HaishinKit a lot in our product and it works great for most of what we need. But we're stuck on a few features and really need your help. We're also happy to sponsor these improvements, anywhere from $50 to $500 or more if needed.

Here are the challenges we're facing:

1. Streaming to Multiple Platforms (Like OBS): We want to send streams to multiple platforms at the same time and manage these streams by adding or removing sources. We saw someone asked about this in 2021 (Issue #852). Has anything changed since then that could help us do this?

2. Streaming Local Videos (Like OBS): We want to stream local videos and be able to pause and play them during the stream. We tried two ways:

   • Using registerVideoEffect() and hack the video frames to overlay a video, but the video and audio don't sync well.
   • Using appendBuffer(_, sampleBuffer:), but this also messes up the video and audio sync and doesn't really work well for controlling video playback.

3. Customizing Camera Positions in the Stream: HaishinKit lets us use both the front and back cameras at the same time, but we can't adjust where these camera views show up in the stream. We’d like to be able to move and resize these views.

If it's too hard to add these features directly into HaishinKit, maybe you could guide us on how to add them ourselves? Any advice on how complicated this might be and what kind of work it would involve would be really helpful.

To Reproduce

Please see the description below.

Expected behavior

Please see the description below.

Version

1.8.1

Smartphone info.

No response

Additional context

No response

Screenshots

No response

Relevant log output

No response

[shogo4405]

1, RTMPStream has a method called addObserver (https://shogo4405.github.io/HaishinKit.swift/Classes/IOStream.html#/s:10HaishinKit8IOStreamC11addObserveryyAA0cE0_pF). You can use this to subscribe to CMSampleBuffer and then use RTMPStream's append(_ buffer: CMSampleBuffer) to link to other streams. This might fulfill your requirements, so please try it out.

2 and 3, they should be addressed by #1449. This has already been merged into the main branch and is scheduled for release as version 1.9.0. Please try it and provide feedback.

[shogo4405]

This is not a bug but rather a question about using HaishinKit, so I have moved it to the Discussion section. For anything other than clear bugs, please use the Discussion section.

[shogo4405]

@RainyTunes
Regarding continuous sponsorship, I am currently seeking sponsors, so I would greatly appreciate your consideration. Please visit here.

On the other hand, the aforementioned features 2 and 3 are new implementations, so I am seeking appropriate feedback. Please review them when you have time.

[RainyTunes]

@shogo4405

Thank you very much for your kindly advice. We are going to work on the features with your suggestions but we currently have new high-priorities things to deal with this month, so I will provide my feedback for feature 2 and 3 next month, as soon as possible.

My employer was willing to sponsor for your sincere help, but it takes a few time to apply for the financial application. I will turn back when the process is done.

[RainyTunes]

We have provided a $50 sponsorship, and while we are currently operating with limited financial resources, we truly appreciate your continued support and dedication to the project. @shogo4405

[RainyTunes]

I have tried version 1.9.0, and in most of cases, it works realy well and meets my expectation.

However, for AssetScreenObject, it only ingests the video part not the audio part. As a result, the video resource can be streamed but with no sound.

This feature is very urgent for us, so I would like to ask that if there are any recent plans to support this, or alternatively, could you guide us on how to implement it ourselves? Any advice on the complexity and the kind of work involved would be really helpful. @shogo4405

[shogo4405]

1. Unfortunately, I do not have plans for this. I'm currently working on support for Swift 6 and Swift Concurrency, which is expected to take until September.
2. While I'm not sure if it meets your requirements, there is an audio mixing feature available.

   stream.isMultiTrackAudioMixingEnabled = true
   stream.append(yourVideoAudioDataSampleBuffer, track: 1)
   

   This allows you to combine the microphone audio and video audio for RTMP transmission. However, since this use case has not been tested, customization may be necessary.

[RainyTunes]

#1512
#1513

Thank you very much for your response.
I met two issues in 1.9.1, I put them in the issue page as feedback.
I am still working on solving this problem, so any further suggestions would be really helpful.