bulk promotion rule information does not include expiry

[sjmudd]

Discussed with Shlomi some time ago.

api/bulk-promotion-rules returns a list of hosts known to orchestrator and their promotion rules. This can be used by systems to validate that the values are correct and to push out changes if this is needed.

Orchestrator has a cache of how long to keep the rules after which they are expired.
An application calling api/bulk-promotion-rules can not see the "orchestrator expiry date" to be applied to any of the servers so it won't be aware of when the rule needs to be updated.

If this script/program runs periodically there is a period during which the rules in orchestrator may expire and consequently failures during those times may trigger the use of an unintended master or intermediate master.

Suggested change is to export the expiry timestamp that orchestrator would use so the script can figure out when the current values need "updating" even if they are correct.

A naive implementation might update these values each time but that seems like an unnecessary action if the number of values (servers) being updated is large as may be the case of a large installation.

I have a patch which I'll provide as a PR shortly.

Note: it is possible to push information into a meta table on each mysql server managed by orchestrator. However, that requires that scripting on that server is always aware of the correct state of the server and will update it as appropriate. With a large number of servers being managed this way there's a chance that this mechanism may not be 100% safe on all servers so it may be fragile. Clearly this depends on the setup used, but it was also one of the reasons that the bulk mechanism exists as a more central bit of code can handle this for all servers and be aware of the complete environment.

[sjmudd]

For reference the patch to be provided will return something of the form:

[
    {
        "Hostname": "host-000001.example.com",
        "Port": 3306,
        "PromotionRule": "must_not",
        "LastSuggestedString": "2019-03-19 09:48:53",
        "PromotionRuleExpiry": "2019-03-19 10:48:53"
    },
    {
        "Hostname": "host-000002.example.com",
        "Port": 3306,
        "PromotionRule": "prefer",
        "LastSuggestedString": "2019-03-19 10:03:50",
        "PromotionRuleExpiry": "2019-03-19 11:03:50"
    },
...
]