[Discussion] Action Versioning

[style95]

This is to discuss the new feature, "action versioning".

It originates from the need for invokers to fetch a specific version of the action.
After a user invokes an action and before an invoker receives the activation message if the user updates the action, the version of action is updated, but the invoker still tries to fetch action with old revision. At this point, Boxed Error happens.

[2019-07-19T08:58:14.747Z] [ERROR] [#tid_ec27fe8f4334dc324422f95769477660] [CouchDbRestStore] [GET] 'whisk_local_whisks' internal error, doc: 'id: guest/noop, rev: 1-5f1211b4597236ee3dfe504d5db6ab59', failure: 'Boxed Error' [marker:database_getDocument_error:434:40]

Generally, it does not happen under a normal situation as activation passing is faster than action update, but it can happen in some cases.

CouchDB has the functionality to fetch data with a specific revision. But this feature is not being used by OpenWhisk.
https://docs.couchdb.org/en/stable/api/document/common.html#obtaining-a-specific-revision

One limitation is, old revision of data can be removed by the compaction.
apache/couchdb-fauxton#1069
https://stackoverflow.com/questions/9730299/are-old-data-accessible-in-couchdb

Also, other ArtifactStores such as CosmosDBArtifactStore may not have such a versioning functionality.
So it would be great for OpenWhisk itself to have the versioning feature.

AWS Lambda also supports this kind of versioning feature at the platform level.

I think the versioning feature can enable many deployment scenarios such as blue-green, canary deployment and so on. Such deployment methods are already supported in K8S with Istio.

Similarly, we can imagine the scenario that we assign two different versions of an action to one execution URL, and control the routing ratio between them. This would be useful for users to test their new version of the action with the real traffic.

Currently, OpenWhisk has a version field, but mostly it defers the versioning to users.
Above mentioned is just one use case. I believe users can benefit from it in many directions.

So it would be great to list up requirements and use cases of this feature.

[style95]

My initial thought is as follow:

1. An action should be uniquely identified with versions.

With the fact that many databases do not support such versioning functionality, we may need to redefine the id for documents.

ex) "_id": "guest/hello/0.0.1",

2. Users should be able to specify versions at least when reading/invoking an action.

A version header or URL parameter can be options to specify versions.
I think it is a minimum requirement to support reading and invoking with a specific version.
Regarding deleting/updating, it can be controversial.

3. A way to handle spatial quota.

Currently, OW does not limit the storage quota. Users can create as many actions as they want.
As we introduce the versioning feature, storage usage can increase drastically.
It implies the need to manage/limit spatial quota.
A simple way can be to limit the number of versions for an action to recent 5 ~ 10 versions.

We may define a more complicated method to handle it.

4. Treat an action with different versions as different.

This is related to the reuse of action containers.
We need to treat them just like different actions as underlying action codes can be different.
So only if activations with the same action version will reuse action containers.

Also, if we would support some kinds of routing features as I mentioned above, we may need a way to correlate them with a single URL, and control the scheduling ratio between them as well.

Sharing your thinking about this feature, requirements, and any use cases would be appreciated.

[style95]

Let me share some excerpts from the discussion threads on Slack with @rabbah and @chetanmeh.

chetanm 1:03 PM
Internally model does has support for version (as a SemVer) but its not currently used much apart form helping in distinguishing changed entities

rr 1:06 PM
There’s a history of discussions around versioning in the issues on GitHub. iirc it comes down to does the system enforce semver or leave it to the user. To mimic aws lambda versions could be done a user level for the most part, and the api would allow an alias for “latest” (by user level I mean you make it part of the action name)

chetanm Yesterday at 1:12 PM
This scheme also leaves gc of old action to user. Worst cases it can bloat up storage a bit … so for not that big issue

rr 1 day ago
Indeed. At user level means you have to clean up at user level also.

chetanm 1 day ago
and currently there is no quota on user storage. So for now user has no incentive to clean up!

rr 1 day ago
yep that’s true

style95 1 day ago
I think we can define some default limits such as 10 recent versions though there ist still no quota in a namespace. (edited)

style95 1 day ago
Users can benefit from SCMs to manage the versioning of their actions, but I think it would be much useful if the platform itself supports the versioning.

[bdoyle0182]

Sorry to bump, just curious if any progress has been made on this feature this year @style95

[style95]

@bdoyle0182 No problem.
There is no more discussion until now.
And we(Naver) are preparing for contribution of this feature and would start it soon.

I believe @jiangpengcheng would drive the contribution.

[bdoyle0182]

Cool couple thoughts / questions I wanted to ask.

1. What will backwards compatibility look like if we need to update docIds? Will no appended version to the docid just act as default version?
2. I think for quota it would be good to have an auto delete for oldest semver if the new version being deployed hits the threshold. I think this can be on the api level through a param rather than global config wide if certain namespace's want to leverage the feature. Pretty much an optimization to force add the action so definitely doesn't need to be there for mvp.

[style95]

1. Regarding the backward compatibility, I believe there can be two options.
   First, we can have a switch to turn on/off the feature.
   Second, each component can take care of it. So if it finds any document with old docids, it can consider it as the latest one.
   Since we are still at the design phase internally, I didn't look into the issue deeply yet, but many parts of OpenWhisk were driven with the above two way and I think we can take the same approach here.

2. That sounds great to me. We would consider the feature.