Write past end of buffer error when sending byte[] using SendNamedMessage

[Robotnik08]

Description

This bug is about Custom Messages.
When trying to send a byte[] Using SendNamedMessage you get a "Write past end of buffer error"

Reproduce Steps

1. Write a script that sends a simple byte[] to a client.
2. The FastBufferArray will write perfectly fine, but The SendNamedMessage will get an "Write past end of buffer error";

Actual Outcome

SendNamedMessage with a byte[] will give an error

Expected Outcome

The written FastBufferArray will be send without issue

My code:

public void SendChunkMessage(byte[] messageContent, ulong clientID)
    {
        int writeSize = FastBufferWriter.GetWriteSize(messageContent);
        FastBufferWriter writer = new FastBufferWriter(writeSize, Allocator.Temp);
        CustomMessagingManager customMessagingManager = NetworkManager.CustomMessagingManager;
        using (writer)
        {
            writer.TryBeginWrite(writeSize);
            writer.WriteBytes(messageContent, writeSize, 0);
            Debug.Log("Written to FastBufferWriter");
            if (IsServer)
            {
                customMessagingManager.SendNamedMessage("Chunk", clientID, writer);
            }
        }
        Debug.Log("Send to client");
    }

My error:

Environment

• OS: Windows 10 pro
• Unity Version: 2022.2.0b16
• Netcode Version: 1.1.0

[NoelStephensUnity]

Hello @Robotnik08 !
Apologies for the delayed response on your issue.
I was able to get something I put together on my end work by making the below adjustments:

FastBufferWriter writer = new FastBufferWriter(writeSize, Allocator.Temp, writeSize * 2);
CustomMessagingManager customMessagingManager = NetworkManager.CustomMessagingManager;
using (writer)
{
    writer.TryBeginWrite(writeSize);
    writer.WriteBytes(messageContent, writeSize, 0);
    Debug.Log("Written to FastBufferWriter");
    if (IsServer)
    {
        customMessagingManager.SendNamedMessage("Chunk", clientID, writer, NetworkDelivery.ReliableFragmentedSequenced);
    }
}

I added the maximum buffer size to be 2x the size you planned on writing. Will double check why this made it work on my end, but I think it includes header information and such. Either case, it still will only allocate the amount specified by the first parameter and the third parameter just allows it to grow beyond your payload size (at least I believe that is what the issue is...or it is a bug and the default -1 doesn't automatically adjust)

Also, I wasn't sure if you planned on sending something greater than the non-fragmented packet size (i.e. 1300 bytes) so I made it deliver fragmented packets (which are always reliable sequenced) so it could handle large payloads.

With the above modifications, let me know if your issue is resolved?

[Robotnik08]

I already fixed this issue a few days ago,
the issue ended up being the NetworkDelivery.ReliableFragmentedSequenced part
Even though the max packet size was set to like 2 million, it somehow kept not working, until I added NetworkDelivery.ReliableFragmentedSequenced!
I don't know if that is a bug or not, I'm just happy its fixed.

[NoelStephensUnity]

Ah...good to hear you fixed it on your end.
If the data you are sending is > 1300 bytes then you need to specify that it is ReliableFragmentedSequenced, otherwise in the lower messaging system it will only allocate enough for a single packet.

Of course, there should be a better message to help guide you towards using ReliableFragmentedSequenced than just an overflow exception.

@ShadauxCat Just keeping you in the loop that it might be useful to check the number of bytes being sent and if it exceeds 1300 bytes and the delivery method is not ReliableFragmentedSequenced possibly still throw an exception but include additional information that will help point users to change the delivery method to ReliableFragmentedSequenced.

[Robotnik08]

Yea the error and documentation should be more clear about that in my opinion.

[TheCaveOfWonders]

(at least I believe that is what the issue is...or it is a bug and the default -1 doesn't automatically adjust)

@NoelStephensUnity I believe the default -1 is bugged and doesn't automatically adjust the size.

[ShadauxCat]

(at least I believe that is what the issue is...or it is a bug and the default -1 doesn't automatically adjust)

@NoelStephensUnity I believe the default -1 is bugged and doesn't automatically adjust the size.

It does adjust the size (this is pretty thoroughly tested both by unit tests and by usage throughout the SDK) but only up to a set maximum. If you're sending with Unreliable delivery, the max is 1300 bytes, because unreliable delivery, at the transport and even kernel level, has to fit into a single UDP datagram. FastBufferWriter won't expand beyond that size because the delivery method won't allow it - if it did, what would happen is that instead of the error you're seeing, you'd see a more confusing error later on, or possibly your message would be silently lost as the network wouldn't be able to deliver it.

As Noel said, if you switch to ReliableFragmentedSequenced, this should allow the buffer to grow large enough to fit your data.

[TheCaveOfWonders]

(at least I believe that is what the issue is...or it is a bug and the default -1 doesn't automatically adjust)

@NoelStephensUnity I believe the default -1 is bugged and doesn't automatically adjust the size.

It does adjust the size (this is pretty thoroughly tested both by unit tests and by usage throughout the SDK) but only up to a set maximum. If you're sending with Unreliable delivery, the max is 1300 bytes, because unreliable delivery, at the transport and even kernel level, has to fit into a single UDP datagram. FastBufferWriter won't expand beyond that size because the delivery method won't allow it - if it did, what would happen is that instead of the error you're seeing, you'd see a more confusing error later on, or possibly your message would be silently lost as the network wouldn't be able to deliver it.

As Noel said, if you switch to ReliableFragmentedSequenced, this should allow the buffer to grow large enough to fit your data.

Oh sorry my comment was unrelated to this actual issue, it was specifically about the default -1 value.
Just tested it in my project when I saw Noel's message, I tried it again now just to see (in case it was bugged before cause it had never worked for me).

This works fine (the size of values is 16)

  using (FastBufferWriter writer = new FastBufferWriter(8, Allocator.Temp, 256))
                {
                    writer.WriteValueSafe(values);
                    NetworkManager.Singleton.CustomMessagingManager.SendNamedMessage(SharedConstants.Network.Tech.SyncTech, senderClientId, writer);
                }

This won't work and will throw a "Write past end of buffer" exception. (the size of values is 16)

  using (FastBufferWriter writer = new FastBufferWriter(8, Allocator.Temp))
                {
                    writer.WriteValueSafe(values);
                    NetworkManager.Singleton.CustomMessagingManager.SendNamedMessage(SharedConstants.Network.Tech.SyncTech, senderClientId, writer);
                }