**NOT TO BE REVIEWED UNTIL:**

• Fix availability and naming of properties and state structs for devices #276 is merged.
• Documentation has been updated.
• A related examples PR has been created.
• Some tests have been added?

 
This PR:

• Implements support within the Softswitch and Orchestrator for Tinsel's Hardware Idle feature and supports applications that have an OnHardwareIdle handler. resolves Softswitch: Implement Hardware Idle #242:
  • Builds "dummy" binaries for unused cores that do nothing other than start and sit at a call to tinselIdle().
  • Modifies Composer to generate default 0-initialised ThreadContext structures for all threads on used cores rather than just used threads.
  • Modifies the Softswitch to sit at a call to tinselIdle() instead of tinselWaitUntil(TINSEL_CAN_RECV) in the control loop if the device type has a Hardware Idle handler.
  • Modifies the Softswitch to sit at a call to tinselIdle() if it hosts no devices.
  • Modifies the deployment logic as required.
• Configurably uses the Hardware Idle feature as part of the barrier release. The default behaviour is now to use the Hardware Idle feature. fixes Mothership/Softswitch barrier logic breaks "simple" GALS applications #285.
• progresses Mothership freezes with some large problem sizes, deadlocking the system #247
• fixes RTS_FLAG_{device_type}_{name} not emitted #308
• fixes Conditional send through doSend is not supported #309
• Is related to Mothership-facing documentation for hardware idle compatibility orchestrator-documentation#23
• Is related to Add simple tests for Hardware Idle and doSend Orchestrator_examples#15
• Requires Support boot in hostlink without starting the cores. tinsel#111

 
A more verbose summary:
This PR implements Hardware Idle support in the Orchestrator and Softswitch. Some fairly significant changes to deployment logic have been required in order to support this feature and tweaks to tinsel have been required.

In summary, Hardware Idle requires all threads in the system, whether they have been started or not, to be sat at a call to tinselIdle() for the hardware synchronisation point to be. This is at odds with the existing paradigm in the Orchestrator where binaries are only built and loaded onto used cores and only used threads are started.

To address this, we now build a common dummy binary that facilitates Hardware Idle and define a default behaviour for unused threads. The Mothership now deploys the "dummy" binaries to all cores before deploying application-specific binaries to used cores. Changes were also made to Tinsel to enable a common binary to be loaded onto all cores without starting them.

Additionally, the start-up barrier was still found to be a problem for some application types despite the fixes in #248. To address this, we make use of the Hardware Idle feature as an (optional) way to implement barrier release. This behaviour is configurable prior to compilation with two new Composer commands:

• compose /hwbarrier = * enables the Hardware Idle barrier. This is now the default behaviour.
• compoae /swbarrier = * disabled the Hardware Idle barrier and falls back to existing behaviour.

As an extra bonus, this PR also implements support for doSend in the Send Handler and fixes #309. This PR also emits the "global" RTS flags and indices (into MessageFormats.h) as required by #308.

 
Testing:
The basic behaviour of Hardware Idle has been tested with a new Hardware Idle simple test (included in POETSII/Orchestrator_examples#15).

The modified barrier release has been extensively tested with the GALS Arithmetic application (pending commit to the Examples repository) at various sizes between 25 and 250,000 devices deployed with the spread algorithm. There were zero instances of an application failing to be released from the barrier.

The basic behaviour of doSend has been tested with a new doSend simple test (included in POETSII/Orchestrator_examples#15).

The functionality of the global RTS variables has been checked with clock_tree_broad_s2.v4.xml from the 2019 benchmarks

 
Conflicts:
This changeset potentially makes #262 easier as we can discriminate between the barrier releasing naturally or being broken by an inbound packet. Care is required.

This changeset makes multi-application runs significantly harder.

This changeset makes implementing support for multiple supervisors more involved and will likely require changes to Tinsel.