From e04c2b10c0bb1b6ac96e0a028ce51849054c10e7 Mon Sep 17 00:00:00 2001 From: Vitalii Zabaznov Date: Fri, 15 Feb 2019 15:07:09 -0600 Subject: [PATCH 1/5] MC-13864: Consumer always read config from memory --- .../asynchronous-operations/poison-pill.md | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 design-documents/asynchronous-operations/poison-pill.md diff --git a/design-documents/asynchronous-operations/poison-pill.md b/design-documents/asynchronous-operations/poison-pill.md new file mode 100644 index 000000000..b8c97ef8a --- /dev/null +++ b/design-documents/asynchronous-operations/poison-pill.md @@ -0,0 +1,20 @@ +## Glossary + +poison pill +: message in Data Storage that signals that consumer should die + +### Overview + +Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count we need to introduce the way to reinit them. + +### Design + +Before process new message consumer should ask Data Storage if there are poison pill for him. If yes, consumer dies and next cron:run will start new instance of consumer. + +#### Acceptance Criteria Fulfillment + +1. If consumer takes message to process after poison pill was placed, consumer must be reinited + +#### Extension Points and Scenarios + +We will put poison pill into Data Storage based on common Magento extension points like events and plugins From 69e1a53cabf5bdd2b1475fd5cbe78857646a0d9c Mon Sep 17 00:00:00 2001 From: Zabaznov Vitalii Date: Tue, 19 Feb 2019 13:12:47 -0600 Subject: [PATCH 2/5] Update poison-pill.md --- design-documents/asynchronous-operations/poison-pill.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design-documents/asynchronous-operations/poison-pill.md b/design-documents/asynchronous-operations/poison-pill.md index b8c97ef8a..0e73861bf 100644 --- a/design-documents/asynchronous-operations/poison-pill.md +++ b/design-documents/asynchronous-operations/poison-pill.md @@ -5,7 +5,7 @@ poison pill ### Overview -Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count we need to introduce the way to reinit them. +Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it leads to wrong im-memory config/objects states and we need to introduce the way to reinit them. ### Design From 1e7ee95608e434fd5478323fbc3bd4c490834bd9 Mon Sep 17 00:00:00 2001 From: Zabaznov Vitalii Date: Tue, 19 Feb 2019 15:06:06 -0600 Subject: [PATCH 3/5] Update poison-pill.md --- design-documents/asynchronous-operations/poison-pill.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design-documents/asynchronous-operations/poison-pill.md b/design-documents/asynchronous-operations/poison-pill.md index 0e73861bf..d08897f2b 100644 --- a/design-documents/asynchronous-operations/poison-pill.md +++ b/design-documents/asynchronous-operations/poison-pill.md @@ -17,4 +17,4 @@ Before process new message consumer should ask Data Storage if there are poison #### Extension Points and Scenarios -We will put poison pill into Data Storage based on common Magento extension points like events and plugins +We will put poison pill into Data Storage based on common Magento extension points From b05689ab6310f9e5d9e03de8221138421f4d6480 Mon Sep 17 00:00:00 2001 From: Zabaznov Vitalii Date: Wed, 20 Feb 2019 09:44:00 -0600 Subject: [PATCH 4/5] Update poison-pill.md --- design-documents/asynchronous-operations/poison-pill.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design-documents/asynchronous-operations/poison-pill.md b/design-documents/asynchronous-operations/poison-pill.md index d08897f2b..84e94295a 100644 --- a/design-documents/asynchronous-operations/poison-pill.md +++ b/design-documents/asynchronous-operations/poison-pill.md @@ -5,7 +5,7 @@ poison pill ### Overview -Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it leads to wrong im-memory config/objects states and we need to introduce the way to reinit them. +Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it leads to outdated im-memory config/objects states and we need to introduce the way to reinit them. ### Design From 31062a6f4a058526a5df973001964b103648b842 Mon Sep 17 00:00:00 2001 From: Vitalii Zabaznov Date: Fri, 22 Feb 2019 15:48:23 -0600 Subject: [PATCH 5/5] MC-13864: Consumer always read config from memory --- .../asynchronous-operations/poison-pill.md | 26 +++++++++++++++---- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/design-documents/asynchronous-operations/poison-pill.md b/design-documents/asynchronous-operations/poison-pill.md index 84e94295a..2ce356f40 100644 --- a/design-documents/asynchronous-operations/poison-pill.md +++ b/design-documents/asynchronous-operations/poison-pill.md @@ -1,20 +1,36 @@ ## Glossary poison pill -: message in Data Storage that signals that consumer should die +: message that added after list of events in Data Storage that signals that consumer should die. ### Overview -Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it leads to outdated im-memory config/objects states and we need to introduce the way to reinit them. +Right now our consumer depends on config/objects in-memory state. After application was bootstrapped it initialize a lot of object that are always kept in memory after first load - config, websites, etc.., since our consumers are long-living and their live time limited only by message count it could happens that our consumer keep outdated im-memory config/objects states. Considering that, we need to introduce the way to reinit them. ### Design -Before process new message consumer should ask Data Storage if there are poison pill for him. If yes, consumer dies and next cron:run will start new instance of consumer. +Basic implementation of how the pill will be added should be implemented inside Magento Framework. We will share the api that let each module call whenever it needed. + +Api Interface - *PoisonPillInterface* will be inside Magento Framework, implementation in MessageQueue module. + +PoisonPillInterface +```php +/** + * Put poison pill inside DB. + * + * @throws \Exception + * @return int + */ +public function put(); +``` + +Before process new message, consumer will compare version of PoisonPill in-memory state with latest in DB. If version is different consumer dies and next cron:run will start new instance of consumer. #### Acceptance Criteria Fulfillment -1. If consumer takes message to process after poison pill was placed, consumer must be reinited +1. If consumer takes message to process after poison pill was placed, consumer must be reinited. #### Extension Points and Scenarios -We will put poison pill into Data Storage based on common Magento extension points +We will put poison pill into Data Storage based on common Magento extension points. +