Skip to content

Commit

Permalink
hps_accel/stream: Update stream API
Browse files Browse the repository at this point in the history 
Defines a new Stream API
- remove distinction between Source and Sink
- make a StreamDescription class that can be used to document whether
  the Stream endpoint ignores the handshaking rules.

These changes were made in light of comments on
amaranth-lang/amaranth#317
  • Loading branch information
@alanvgreen
alanvgreen committed Aug 21, 2021
1 parent c5cc676 commit 3f9f042
Show file tree
Hide file tree
Showing 4 changed files with 145 additions and 187 deletions.
57 changes: 29 additions & 28 deletions proj/hps_accel/gateware/stream/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,32 +17,37 @@
Streams allows data to be transferred from one component to another.
A stream of data consists of a series of packets transferred from a Source
to a Sink. Each packet contains one Payload. A stream of packets may optionally
be organized into Frames.
A stream of data consists of a series of payloads transferred from an
upstream (aka producer or source) to a downstream (aka consumer or
sink). Each packet contains one payload.
Source and Sink operate in the same clock domain. They communicate with these
signals:
The upstream and downstream must operate in the same clock domain. They
communicate with these signals:
- payload: The data to be transferred. This may be a simple Signal or else a
Record.
- valid: Set by source to indicate that a valid payload is being presented.
- ready: Set by sink to indicate that it is ready to accept a packet.
- last: Indicates the end of a frame.
- payload: The data to be transferred. This may be a simple Signal or
else a Record. Set by upstream.
- valid: Set by upstream to indicate that a valid payload is being
presented.
- ready: Set by downstream to indicate that it is ready to accept a
packet.
A transfer takes place when both of the valid and ready signals are asserted on
the same clock cycle. This handshake allows for a simple flow control - Sources
are able to request that Sinks wait for data, and Sinks are able to request for
Sources to temporarily stop producing data.
A transfer takes place when both of the valid and ready signals are
asserted on the same clock cycle. This handshake allows for a simple
flow control - upstreams are able to request that downstreams wait for
data, and downstreams are able to request for upstreams to pause data
production.
There are cases where a Source or a Sink may not be able to wait for a
valid-ready handshake. For example, a video phy sink may require valid data be
presented at every clock cycle in order to generate a signal for a monitor. If
a Source or Sink cannot wait for a transfer, it should be make the behavior
clear in its interface documentation.
In order to ensure that there are no combinatorial cycles between
pipelines, the valid signal of a stream must not combinatorially
depend on the stream's ready signal. As a general principle, it is
also useful to avoid having ready combinatorially depend on the
stream's valid Signal.
The use of frames - and hence the "last" signal, is optional and must be agreed
between Source and Sink.
There are cases where an upstream or downstream may not be able to
wait for a valid-ready handshake. For example, a video phy sink may
require valid data be presented at every clock cycle in order to
generate a signal for a monitor. If an upstream or downstream cannot
wait for a handshake, it declares this in its Definition.
This implementation was heavily influenced by the discussion at
https://github.com/nmigen/nmigen/issues/317, and especially the existing design
Expand All @@ -51,13 +56,9 @@
Major differences from LiteX Streams:
- Source and Sink are distinct types. They share a common superclass for
implementation purposes, but this is not exposed in the API.
- A "payload_type" attribute contains the type of the payload.
- "Parameters" are omitted. The value of parameters as a general mechanism is
unclear.
- "first" is omitted. A "first" signal can be derived from the "last" signal.
- There are no "first" or "last" signals.
- The payload is a single signal or Record named "payload".
"""

from .actor import BinaryCombinatorialActor
from .stream import Sink, Source, glue_sources, glue_sinks
from .stream import Stream, StreamDefinition, connect
41 changes: 19 additions & 22 deletions proj/hps_accel/gateware/stream/actor.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,21 @@
from migen import Module
from util import SimpleElaboratable

from .stream import Sink, Source
from .stream import StreamDefinition, Stream


class BinaryActor(SimpleElaboratable):
"""A binary actor is an object that transforms stream packets.
It has one sink and one source. Each packet received at the sink
has a corresponding packet sent from the source.
It has one input stream and one output stream. Each packet received
at the input has a corresponding packet sent from the output.
Parameters
----------
input_type: The Shape or Layout for the sink
input_type: Defines input stream
output_type: The Shape or Layout for the source.
output_type: Defines output stream
Attributes
----------
Expand All @@ -43,14 +43,15 @@ class BinaryActor(SimpleElaboratable):
"""

def __init__(self, input_type, output_type):
self.input_type = input_type
self.output_type = output_type
self.sink = Sink(input_type)
self.source = Source(output_type)
self.input_type = StreamDefinition.cast(input_type, ignores_valid=True)
self.output_type = StreamDefinition.cast(
output_type, ignores_ready=True)
self.input = Stream(input_type)
self.output = Stream(output_type)

def elab(self, m: Module):
self.control(m)
self.transform(m, self.sink.payload, self.source.payload)
self.transform(m, self.input.payload, self.output.payload)

def control(self, m):
"""Adds flow control to sink and source."""
Expand All @@ -60,8 +61,12 @@ def control(self, m):
def transform(self, m, input, output):
"""Transforms input to output.
input: self.input_type, in
output: self.output_type, out
m: Module
The module for this elaboratable
input:
The input payload to be transformed
output:
The transformed value
"""
raise NotImplementedError(
"BinaryActor subclass must implement transform()")
Expand All @@ -70,23 +75,15 @@ def transform(self, m, input, output):
class BinaryCombinatorialActor(BinaryActor):
"""Base for a combinatorial binary actor.
Performs a combinatorial operation on a sink payload and routes it to a
source.
Performs a combinatorial operation on a input payload to transform
it to an output.
Parameters
----------
input_type: The Shape or Layout for the sink
output_type: The Shape or Layout for the source.
Attributes:
sink: Sink(input_type), in
Sink for incoming data
source: Source(source_type), out
Source for outgoing data
"""

def __init__(self, input_type, output_type):
Expand Down
198 changes: 79 additions & 119 deletions proj/hps_accel/gateware/stream/stream.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,146 +14,106 @@
# limitations under the License.

from nmigen import Shape
from nmigen.hdl.rec import Record, DIR_FANIN, DIR_FANOUT
from nmigen.hdl.rec import Layout, Record

__all__ = ['StreamDefinition', 'Stream', 'connect']

class _Endpoint:
"""Abstract base class for Sinks and Sources."""

def __init__(self, payload_type, name, src_loc_at):
self.payload_type = payload_type
self._record = Record([
("valid", Shape(), DIR_FANOUT),
("ready", Shape(), DIR_FANIN),
("last", Shape(), DIR_FANOUT),
("payload", payload_type, DIR_FANOUT),
], src_loc_at=2+src_loc_at, name=name)
self.valid = self._record.valid
self.ready = self._record.ready
self.last = self._record.last
self.payload = self._record.payload
class StreamDefinition:
"""Defines a stream and the guarantees that it makes.
def is_transferring(self):
"""Returns an expression that is true when a transfer takes place."""
return (self.valid & self.ready)
ignores_valid:
Stream may use payload without valid being set.

class Source(_Endpoint):
"""A stream source.
Parameters
----------
payload_type: Shape(N) or Layout
The payload transferred from this Source.
name: str
Base for signal names.
Attributes:
-----------
payload_type: Shape(N) or Layout
valid: Signal(1), out
ready: Signal(1), in
last: Signal(1), out
payload: Signal(N) or Record, out
ignores_ready:
Stream may present a new payload while valid is asserted
and before ready is asserted.
"""

def __init__(self, payload_type, name=None, src_loc_at=0):
super().__init__(payload_type, name, src_loc_at)

def connect(self, sink):
"""Returns a list of statements that connects this source to a sink.
Parameters:
sink: This Sink to which to connect.
"""
assert isinstance(sink, Sink)
return self._record.connect(sink._record)


class Sink(_Endpoint):
"""A stream sink
@staticmethod
def cast(obj, src_loc_at=0, ignores_ready=False, ignores_valid=False):
if isinstance(obj, StreamDefinition):
return StreamDefinition(
paylad_type=obj.payload_type,
ignores_ready=obj.ignores_ready,
ignores_valid=obj.ignores_valid)
else:
return StreamDefinition(
payload_type=obj, src_loc_at=1 + src_loc_at)

def __init__(self, *, payload_type,
ignores_ready=False, ignores_valid=False, src_loc_at=0):
self.ignores_ready = ignores_ready
self.ignores_valid = ignores_valid
self.payload_type = payload_type
self.layout = Layout([
("valid", Shape()),
("ready", Shape()),
("payload", payload_type),
],
src_loc_at=1 + src_loc_at
)


class Stream:
"""Interface to a stream
Parameters
----------
payload: Signal(N) or Record
The payload transferred to this Sink.
name: str
Base for signal names.
Attributes:
-----------
payload_type: Shape(N) or Layout
valid: Signal(1), in
ready: Signal(1), out
last: Signal(1), in
payload: Signal(N) or Record, in
definition: StreamDefintion
Specifies the payload type and other parameters of this type.
"""

def __init__(self, payload_type, name=None, src_loc_at=0):
super().__init__(payload_type, name, src_loc_at)


def glue_sources(source_in: Source, source_out: Source):
"""Combinatorially glues two sources together.
source_in is combinatorially glued to source_out. This is useful when
exposing a submodule's Source as part of the interface of the current
module.
The two sources must have identical payload types.
def __init__(self, definition=None, *, payload_type=None, name=None, src_loc_at=0):
if definition is None:
self.definition = StreamDefinition(
payload_type=payload_type,
src_loc_at=src_loc_at + 1)
else:
self.definition = StreamDefinition.cast(definition)
self._record = Record(
self.definition.layout,
name=name,
src_loc_at=1 + src_loc_at)
self.valid = self._record.valid
self.ready = self._record.ready
self.payload = self._record.payload

Parameters:
source_in:
The source that forms part of the submodule's interface.
source_out:
The source that forms part of the current module's interface.
@staticmethod
def like(other):
return Stream(other.definition)

Result:
A sequence of statements that connects the two sources.
"""
# Checking to catch simple mistakes
assert isinstance(source_in, Source)
assert isinstance(source_out, Source)
assert source_in.payload_type == source_out.payload_type
def is_transferring(self):
"""Is a transfer taking place this cycle?
return [
source_in.ready.eq(source_out.ready),
source_out.valid.eq(source_in.valid),
source_out.last.eq(source_in.last),
source_out.payload.eq(source_in.payload),
]
True iff valid and ready are both asserted.
"""
return self.valid & self.ready


def glue_sinks(sink_in: Sink, sink_out: Sink):
"""Combinatorially glues two sinks together.
def connect(from_stream, to_stream):
"""Convenience function for connecting an upstream to a downstream.
sink_in is combinatorially glued to sink_out. This is useful when
exposing a submodule's Sink as part of the interface of the current
module.
Examples:
The two sinks must have identical payload types.
m.d.comb += connect(one_components_output, another_components_input)
m.d.comb += connect(my_input, child_input)
Parameters:
sink_in:
The sink that forms part of the current module's interface.
sink_out:
The sink that forms part of the submodule's interface.
Arguments
---------
from_stream:
The upstream side of the stream. Presents payload and valid.
to_stream:
The downstream side of the stream. Presents ready.
Result:
A sequence of statements that connects the two sinks.
Result
------
A list of statements.
"""
# Checking to catch simple mistakes
assert isinstance(sink_in, Sink)
assert isinstance(sink_out, Sink)
assert sink_in.payload_type == sink_out.payload_type

return [
sink_in.ready.eq(sink_out.ready),
sink_out.valid.eq(sink_in.valid),
sink_out.last.eq(sink_in.last),
sink_out.payload.eq(sink_in.payload),
to_stream.valid.eq(from_stream.valid),
to_stream.payload.eq(from_stream.payload),
from_stream.ready.eq(to_stream.ready),
]
Loading

0 comments on commit 3f9f042

Please sign in to comment.