Linking related processes

[duncandewhurst]

Linking related processes

This issue is under consideration for the 1.1 milestone.

It builds on previous discussions in #72, #184 and #144

The issue

An OCDS contracting process is defined as:

“All the planning, tendering information, awards, contracts and contract implementation information related to a single initiation process.”

This has been interpreted to mean we have a single planning and tender section, and multiple award and contract sections

A single planning process often leads to many contracting processes.

Some processes involve multiple tender stages, including:

• PQQ processes;
• Frameworks;
• Processes where a tender fails, and is re-issued;

What we are proposing

Introducing a relatedProcess field under release which can be used to refer back to a prior contracting process, or to refer to related subcontracts. relatedProcess should contain the OCID of the prior process, a URI of a record package, and a relationship type selecting from an open codelist including:

• framework
• planning
• parentContract (for subcontracts Parent contract - planning data relates to multiple contracting processes. #72 and Subcontracting Information #144)
• qualification (?)
• unsuccessfulTender
• renewalProcess (for handling issue in Linking contracts nearing expiry with the process for procuring their replacements #184)

Draft modelling

"relatedProcess": {
    "title":"Framework agreement for XYZ",
    "ocid":"ocds-123456-23673",
    "relationship":"framework",
    "uri":"http://example.open-contracting.org/records/ocds-123456-123562"
    }

Engagement

Please indicate support or opposition for this proposal using the +1 / -1 buttons or a comment. If opposing the proposal, please give clear justifications, and where possible, make an alternative proposals.

[timgdavies]

A few things we may need to consider here:

(1) Should relatedProcess be an array? A contract could both refer back to the framework it is from, and onwards to the process relating to it's renewal.

(2) Would it be better for us to use explicit properties for each type of relationship?

[LindseyAM]

the strongest use case for identifying related processes that i have heard (today in Ukraine) is that there is a need to be able to show related procedures that have failed. So, for a tender that fails due to no bidders or is canceled before signature due to some irregulatory, there is a desire to be able to know when it has been relaunched in a new procedure. an alternative solution (in Ukraine) may be to just be able to filter to show recent procedures for same procuring entity & item code or something like that...

[timgdavies]

This was discussed at the upgrade session in Bogota yesterday, again with the example of identifying failed tenders.

One of the issues raised by @juanpane was that it might be necessary for cross-references to also point to the particular stage of a relatedProcess, not only to the ocid of the process itself.

We should do some more work on this issue to develop some worked examples for different scenarios, and to explore in more depth the user stories for both applications and individual users accessing data which includes relatedProcess information.

[duncandewhurst]

The World Bank Procurement Regulations provide a couple of useful definitions here for processes with a qualification stage:

Term	Definition
Prequalification	The shortlisting process which can be used prior to inviting request for bids in the procurement of Goods, Works or Non-consulting Services.
Initial Selection (IS)	The shortlisting process used prior to inviting request for proposals in the procurement of Goods, Works or Non-consulting Services.

[duncandewhurst]

The UNCITRAL Model Law on Public Procurement (2011) also draws a distinction between pre-qualification and pre-selection:

Term	Definition
Pre-qualification	Procedure to identify, prior to solicitation, suppliers or contractors that are qualified
Pre-selection	Procedure to identify, prior to solicitation, a limited number of suppliers or contractors that best meet the qualification criteria for the procurement concerned.

[andrewlorien]

NSW eTendering (Australia) uses "related items" as the only means of linking the stages of the tendering process.
An "Annual Procurement Plan" may be linked to many "Planned Procurements".
A Tender process may link to a Planned Procurement, or to another Tender process. The link is bi-directional and the type of relationship is not specified, but it is used to link new tenders to failed tenders (as described above), and to link tender processes which select a list of qualified suppliers to a process which selects a particular supplier from that list (example). From the user guide:
"Enter the related RFT ID – any RFT apart from the one currently being created.
Please Note: This field is normally used to link this RFT to another one for reference purposes. It could be used in the case of multi-stage sourcing projects to link the RFT IDs of the two or more phases of the project."
A Contract may link to a Tender (and so a Tender may have any number of Contracts), or a Standing Offer Notice but I think that is covered by OCDS 1.0

[AlCollier]

Bit concerned that we're losing the original use case from #184 here - to link an existing contract with the tender process for its replacement.

[timgdavies]

From work on mapping against EU forms, three new field suggestions have come up captured in the merge_patch below.

The first of these could be added to the lots extension #381. The second two might be proposed inclusions in relatedProcess core - or could be added as extensions on this.

{
  "definitions": {
    "relatedProcess": {
      "description": "A link to a related contracting process in OCDS and the type of relationship.",
      "type": "object",
      "title": "Related Process",
      "properties": {
        "relatedLots": {
          "title": "Related lots",
          "description": "For subcontracting relationships: an array containing the lot identifiers of the lots within the current contracting process that may be subcontracted in the related process. Required by the EU",
          "type": [
            "array",
            "null"
          ],
          "items": {
            "type": "string"
          }
        },
        "description": {
          "title": "description",
          "description": "A description of the related process. In the case of subcontracting relationships, this should include a description of the part of the contract to be subcontracted. Required by the EU",
          "type": "string"
        },
        "value": {
          "title": "value",
          "description": "Estimated or actual value of the related process. Required by the EU for estimates of the value of related subcontracting processes.",
          "$ref": "#/definitions/Value"
        }
      }
    }
  }
}

[duncandewhurst]

In the draft V1.1 standard related process is added at both release and contract level (see documentation for details)

[duncandewhurst]

The draft documentation for related processes states the following uses for the building block:

(1) At the release level - used to point backwards to prior processes, such as planning, PQQ or framework establishment.

(2) At the contract level - used to point onwards to sub-contracts, renewal or replacement processes that relate solely to the particular contract the property appears in.

In the UK Contracts Finder implementation of OCDS, related process could be used both to link backwards from subsequent tenders to the OCDS record for an early engagement notice (e.g. early planning/market engagement) and to link forwards from the early engagement notice to the OCDS records for subsequent tenders.

As such I propose updating the above documentation to read:

(1) At the release level - used to point backwards to prior processes, such as planning, PQQ or framework establishment or to point forwards to subsequent processes resulting from a planning or PQQ process.

(2) At the contract level - used to point onwards to sub-contracts, renewal or replacement processes that relate solely to the particular contract the property appears in.

This would also require the addition of a code to the related process codelist to represent this relationship:

code	title	description
tender	Tender process	This contracting process resulted in the related tender process

[LindseyAM]

In order to point forward, someone would have to go back and edit the prior notice to point to the related process?

[duncandewhurst]

My thinking was that in a noticing system which is producing this information, if the user publishing a tender notice needs to select the prior planning notice in order to establish the backwards link, then the system would be able to automatically update the prior planning notice with the forwards link to the tender notice.

[ekoner]

@AlCollier Are your concerns addressed here? http://standard.open-contracting.org/1.1-dev/en/schema/reference/#relatedprocess - Please let us know what you think.

[timgdavies]

Feedback from Jachym Hercher during peer review process

"This proposal dilutes the intuitive and very valuable meaning of a ""contracting process"" as the steps taken from planning to contract implementation. If this was implemented as proposed, I would consider it a serious detriment to the structure of the standard. Please find comments below in more detail, structured along the code list of http://standard.open-contracting.org/1.1-dev/en/schema/codelists/#related-process :

1. This proposal proposes to model procurement processes (in the 1.0 sense) containing ""preQualification"", ""preSelection"", and ""frameworks"" as multiple procurement processes (in the 1.1. sense). This is not correct for many reasons. a) If we accept this split, it means that the first procurement process will never have a contract nor implementation stage, while the second process will never have a planning stage. This shows that we've just arbitrarily split a whole contracting process into two. b) Furthermore, many (most?) procurement processes would become two procurement processes - e.g. all EU procedures except the open non-framework ones. c) Some procurement procedures (e.g. competitive dialogue) would have to be modelled as many, e.g. 20, procurement processes. d) We are not consistent in dealing with frameworks - based on ""some details"" (whether competition is reopened or not), some would be modelled as two processes, some as more. All of this would absolutely dilute the meaning of ""procurement process"". A different way to model this needs to be found (e.g. subsections in the tender section; multiple tender sections per one procurement process).

Introducing this change would also make it essentially impossible to use the same identifier approach for the EU forms and the OCDS. Currently, a ""procurement procedure"" according to EU law corresponds quite nicely with the OCDS concept of procurement process. However, in EU law, framework agreements and two-stages procedures are still clearly just part of one procedure, so if these start receiving multiple OCDS procurement process IDs, this match would be broken.

2. Planning - no opinion. It might be useful, but I'm not not sure whether additional subsections in the planning section, or multiple planning sections, wouldn't be better instead.

3. I think using RelatedProcess for Subcontracting, renewal, replacement, unsuccessful procedure is a good idea. Including this information both in the ""child"" release and (I assume through an amendment) ""parent"" release is a bit cumbersome, but I guess there can be an advantage of having this information in every release - so I have no opinion on this. However, if it's implemented in both notices, an appropriate code (""relatedProcess"") should be added to the amendment codelist proposed above. "

[duncandewhurst]

@JachymHercher thanks for the detailed feedback

Changing tender from an object to an array would be a backward incompatible change, which we're not able to make in V1.1, however in the draft PPP schema we've taken the approach of introducing an optional preQualification section which is effectively a duplicate of the tender section, so that more detailed information can be provided about 'two-stage' processes.

Would this approach work for the EU procedures?

[timgdavies]

A major revisions comment was raised noting that the use of relatedProcess for 'preQualitification', 'preSelection' and 'frameworks', breaks the idea of an OCID applying to a single contracting process, and risks undermining a key vaue of OCDS - as well as introducing difficulty for mapping between a OCDS process, and a European procurement processes.

The point is well made. The introduction of these relatedProcesses had been an attempt to avoid a backwards incompatible change of OCDS by converting 'tender' from an object to an array. This change, (tender from object to array, allowing a single contracting process to have multiple tender stages), is something that has been informally considered for a 2.0 version.

Based on the feedback here, our draft proposal is to::

• Remove the 'preQualification', 'preSelection' and 'framework' codes from the relatedProcess codelist;
• Update documentation to remove the reference to relatedProcess being used for mini-competitions or pre-qualification processes;
• Work up a proposal for version 2.0 to support 'tenders' (array) rather than 'tender', and developing an extension to support this in the interim - and, subject to feedback, recommending this as the correct approach for frameworks or multi-stage competitions;

Additionally:

A question was raised about the inclusion of 'title' within the RelatedProcess block. Whilst, in well integrated systems, this would be redundant (as a user should be able to look up the referenced OCID and get a process title), our view is that this field is useful in cases when either the related process does not have a stable OCID (due to the way technical systems are implemented), or to help less technical users to gain an easier intuitive understanding of the related process, particularly when using tools (e.g. spreadsheets) that are not able to easily resolve the OCID and lookup the title.

A question was raised as to whether relatedProcess can be used to cover cases when a tender is declared deserted, and a new tender started. The documentation will be updated to reflect that it can be used to cover this situation.

[JachymHercher]

@timgdavies The proposal sounds good. (@duncandewhurst that could work for two-stage, I'm not sure if it would have worked for more-stage such as competitive dialogue or perhaps innovative partnership.)

One questions about

A question was raised as to whether relatedProcess can be used to cover cases when a tender is declared deserted, and a new tender started. The documentation will be updated to reflect that it can be used to cover this situation.

• I'm not sure what "deserted" means, but is declaring it in relatedProcess in line with releaseTag tenderCancellation? (This might be also linked to my comment in Amendment handling #357.)

[duncandewhurst]

Flagging that my proposal above for the inclusion of the following code in the relatedProcess codelist didn't make it into 1.1:

code	title	description
tender	Tender process	This contracting process resulted in the related tender process

This code is required to model one-to-many relationships between early engagement and tender notices in the UK.

Adding this issue to the version 2.0 milestone for the addition of this code.

[timgdavies]

@duncandewhurst could you open a new issue with your proposal, so that we can cleanly capture that change needed for next version. I would suggest leaving off the milestone - as it might be that we have a 1.2 before 2.0.

[jpmckinney]

Remove the 'preQualification', 'preSelection' and 'framework' codes from the relatedProcess codelist

'framework' was not removed, FYI.

Update: See #577 for discussion about why it was retained.