Skip to content

Byte Buffers

Jump to bottom
Tony Arcieri edited this page Dec 27, 2017 · 6 revisions

NOTE: This feature was added in nio4r 2.0 thanks to a Google Summer of Code project by Upekshe Jayasekera

The NIO::ByteBuffer class represents a fixed-sized native buffer, and is modeled after the corresponding Java NIO class. The closest Ruby equivalent is a StringIO. However, unlike Ruby's String and StringIO types there are no hidden performance gotchas involving string encodings or pathological usage patterns to worry about. Instead, byte buffers provide the most efficient means of performing I/O operations on in-memory data.

Creating Byte Buffers

To create a byte buffer, construct it with a given capacity:

>> buffer = NIO::ByteBuffer.new(16384)
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=16384 @capacity=16384>

All byte buffers have the following attributes:

  • position: a cursor from which all I/O operations will take place
  • limit: size of the current data in the buffer in bytes. Defaults to same value as capacity, but is updated each time we call #flip
  • capacity: total size of the buffer in bytes

These values uphold a position <= limit <= capacity invariant.

Adding data: #<<

To add data to the buffer directly, use the #<< method:

>> buffer << "Hello, world!" 
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=13 @limit=16384 @capacity=16384>

The intended use of a byte buffer is to first read some data into it, then once we've done one or more reads to get the complete data, read the data out of it. Before we read the data out, let's add some more data:

>> buffer << " This is a byte buffer."
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=36 @limit=16384 @capacity=16384>

Switching read/write modes: #flip

Before reading data out, call the #flip method. Pay extra special attention to #flip because it's the byte buffer API's secret sauce:

>> buffer.flip
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=36 @capacity=16384>

Calling #flip changed the limit value to be the previous position cursor value, and set position to be 0. In other words, it moved the cursor from the end to the beginning, and set the limit to the cursor's previous position.

Retrieving data as a string: #get

After calling #flip, we can read data out of the buffer by using the #get method:

>> buffer.get
 => "Hello, world! This is a byte buffer."
>> buffer
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=36 @limit=36 @capacity=16384>

Calling #get returned all of the data up to the limit as a string, and also moved the position cursor to match the limit. We can also call #get with a length:

>> buffer.flip
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=36 @capacity=16384>
>> buffer.get(13)
 => "Hello, world!"

Setting the limit: #limit=

We can set the limit back to its original value using the #limit= method:

>> buffer.flip
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=36 @capacity=16384>
>> buffer.limit = 16384
 => 16384

Non-blocking I/O: #read_from and #write_to

To perform I/O operations using the buffer, use the #read_from and #write_to methods. These methods perform non-blocking I/O on the remaining space in the buffer after the position cursor:

>> buffer << "GET / HTTP/1.0\r\n\r\n"
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=18 @limit=16384 @capacity=16384>
>> buffer.flip
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=18 @capacity=16384>
>> socket = TCPSocket.new("github.com", 80)
 => #<TCPSocket:fd 11>
>> buffer.write_to(socket)
 => 18
>> buffer.clear
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=16384 @capacity=16384>
>> buffer.read_from(socket)
 => 93
>> buffer.flip
 => #<NIO::ByteBuffer:0x007fc60fa41528 @position=0 @limit=93 @capacity=16384>
>> buffer.get
 => "HTTP/1.1 301 Moved Permanently\r\nContent-length: 0\r\nLocation: https:///\r\nConnection: close\r\n\r\n"

Clearing the buffer: #clear

The #clear method, used in the above example, returns a buffer to its original state.

Putting it all together

The following shows the basic flow you should use when processing incoming network data using an NIO::ByteBuffer and how all the methods relate to each other:

$ pry -rnio
[1] pry(main)> buf = NIO::ByteBuffer.new(64)
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=0 @limit=64 @capacity=64>
[2] pry(main)> buf << "GET /" # simulated client write to process
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=5 @limit=64 @capacity=64>
[3] pry(main)> buf.flip # begin processing write by flipping
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=0 @limit=5 @capacity=64>
[4] pry(main)> buf.get(1) # read first byte
=> "G"
[5] pry(main)> buf.get(1) # keep reading until space
=> "E"
[6] pry(main)> buf.get(1) # keep reading until space
=> "T"
[7] pry(main)> buf.get(1) # keep reading until space
=> " "
[8] pry(main)> buf.compact # found space, so compact
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=1 @limit=64 @capacity=64>
[9] pry(main)> buf << "foobar HTTP/1.0" # simulate client sending more data
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=16 @limit=64 @capacity=64>
[10] pry(main)> buf.flip # begin processing by flipping
=> #<NIO::ByteBuffer:0x007f9f49055038 @position=0 @limit=16 @capacity=64>
[11] pry(main)> buf.get # ok, so what have we got?
=> "/foobar HTTP/1.0"

See Also

Clone this wiki locally