-
Notifications
You must be signed in to change notification settings - Fork 7
ResourcePoolAPI
A Resource Pool is a Kubernetes cluster that pulls for full workflows called Agendas. A Resource Pool may have one or more queues it pulls from filtered by Insights.
-
The puller runs as a Kubernetes pod that pulls for work by calling Agenda Service getAgenda
-
Creates On-Demand Kubernetes Pod for Agenda Execution
-
Executes all operations in Agenda by spinning up resources dynamically
-
Metrics / Alerting
-
Progress reporting
-
Parallel processing
-
Node Graph / DAG model processing
-
Executor for a single Agenda Operation
-
Metrics / Alerting
-
API owner
-
Progress reporting
-
Kubernetes
-
Kubernetes Annotations
-
Docker
-
Bananas
-
Graphite
Path: < stagename > /resourcepool
byId
byCid
byLinkId
Field | Type | Description |
---|---|---|
contentType | String | This will be application/json for the immediate term, but we might also support application/xml later |
Field | Purpose | Type | Notes |
---|---|---|---|
id | Id of the original requestString | GET, PUT | |
customerId | Id of the Customer from the customer table entry | String | POST / PUT |
cid | Correlation identifier to combine all data objects created within a request submission | String | |
title | An easy way to reference | String | |
updatedTime | Generated upon PUT | Timestamp | |
addedTime | Generated upon POST | Timestamp | |
insightIds | Array of insight identifiers, generated from insight association | String[] |
{
"id": null,
"cid": null,
"customerId": "http://my.customer.url/data/Account/3333333",
"updatedTime": 1576615017354,
"addedTime": null,
"title": "POPPool-dev",
"insightIds": \[
\]
}
- Submission
- Scheduling
-
Execution
the ResourcePool
Agenda
the workflow
Agenda Template
the workflow definition
Customer
Insight
the scheduling queue definition
Operation Progress
the state of the running Agenda operations
Progress
the state of the running Agendas
ResourcePool
the processing resources
Agenda Service
the workflow submission
Progress Service
rolled up agenda progress summary
ResourcePool Service
getting work and updating progress
AgendaReclaimer
restarting stuck Agendas
AgendaRetry
retrying failed Agendas
DataObjectReaper
reaping expired data objects
PodReaper
reaping stuck Kubernetes pods
DevKubernetesSetup
RunLocalExecution
- RunWithMiniKube