Add coverage for generic state parameter using Azure Storage (after #1897) by jdom · Pull Request #1915 · dotnet/orleans

jdom · 2016-07-08T00:31:34Z

While I was doing this minor cleanup, I encountered that this is still failing on the call to ClearState.
@Maarten88 do you know if this is still related to the grain type name or is this a new bug?

Both tests (with long and short name) are now failing, these are the details:

Test Name:  UnitTests.General.GenericGrainsInAzureStorageTests.Generic_OnAzureTableStorage_LongNamedGrain_EchoValue
Test FullName:  UnitTests.General.GenericGrainsInAzureStorageTests.Generic_OnAzureTableStorage_LongNamedGrain_EchoValue
Test Source:    D:\projects\OrleansVso\test\Tester\GenericGrainsInAzureStorageTests.cs : line 28
Test Outcome:   Failed
Test Duration:  0:00:00.286

Result StackTrace:  
Server stack trace: 
   at Orleans.Core.GrainStateStorageBridge.<ClearStateAsync>d__7.MoveNext() in D:\projects\OrleansVso\src\Orleans\Core\GrainStateStorageBridge.cs:line 147
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at UnitTests.Grains.SimpleGenericGrainUsingAzureTableStorage`1.<ClearState>d__1.MoveNext() in D:\projects\OrleansVso\test\TestGrains\GenericGrains.cs:line 69
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Orleans.PublicOrleansTaskExtensions.<BoxAwait>d__7.MoveNext() in D:\projects\OrleansVso\src\Orleans\Async\TaskExtensions.cs:line 128
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Orleans.Runtime.InsideRuntimeClient.<Invoke>d__57.MoveNext() in D:\projects\OrleansVso\src\OrleansRuntime\Core\InsideRuntimeClient.cs:line 395

Exception rethrown at [0]: 
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at UnitTests.General.GenericGrainsInAzureStorageTests.<Generic_OnAzureTableStorage_LongNamedGrain_EchoValue>d__1.MoveNext() in D:\projects\OrleansVso\test\Tester\GenericGrainsInAzureStorageTests.cs:line 32
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
----- Inner Stack Trace -----
   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result)
   at Microsoft.WindowsAzure.Storage.Table.CloudTable.EndExecute(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Orleans.AzureUtils.AzureTableDataManager`1.<UpdateTableEntryAsync>d__21.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Storage\AzureTableDataManager.cs:line 305
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Orleans.Storage.AzureTableStorage.GrainStateTableDataManager.<Write>d__9.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Providers\Storage\AzureTableStorage.cs:line 500
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Orleans.Storage.AzureTableStorage.<ClearStateAsync>d__33.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Providers\Storage\AzureTableStorage.cs:line 221
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Orleans.Core.GrainStateStorageBridge.<ClearStateAsync>d__7.MoveNext() in D:\projects\OrleansVso\src\Orleans\Core\GrainStateStorageBridge.cs:line 133
----- Inner Stack Trace -----
   at Microsoft.WindowsAzure.Storage.Table.Protocol.TableOperationHttpResponseParsers.TableOperationPreProcess(TableResult result, TableOperation operation, HttpWebResponse resp, Exception ex)
   at Microsoft.WindowsAzure.Storage.Table.TableOperation.<>c__DisplayClass13.<ReplaceImpl>b__12(RESTCommand`1 cmd, HttpWebResponse resp, Exception ex, OperationContext ctx)
   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndGetResponse[T](IAsyncResult getResponseResult)
Result Message: 
Orleans.Runtime.OrleansException : Error from storage provider during ClearState for grain Type=UnitTests.Grains.SimpleGenericGrainUsingAzureTableStorage`1[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] Pk=*grn/F4F91C41/4226a67b4367d6461eedf64f82301bad0303c6e2f4f91c41-0xC9D474F8 Id=GrainReference:*grn/F4F91C41/82301bad<[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]> Error=

Exc level 0: Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (400) Bad Request.
   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result)
   at Microsoft.WindowsAzure.Storage.Table.CloudTable.EndExecute(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Orleans.AzureUtils.AzureTableDataManager`1.<UpdateTableEntryAsync>d__21.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Storage\AzureTableDataManager.cs:line 305
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Orleans.Storage.AzureTableStorage.GrainStateTableDataManager.<Write>d__9.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Providers\Storage\AzureTableStorage.cs:line 500
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Orleans.Storage.AzureTableStorage.<ClearStateAsync>d__33.MoveNext() in D:\projects\OrleansVso\src\OrleansAzureUtils\Providers\Storage\AzureTableStorage.cs:line 221
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Orleans.Core.GrainStateStorageBridge.<ClearStateAsync>d__7.MoveNext() in D:\projects\OrleansVso\src\Orleans\Core\GrainStateStorageBridge.cs:line 133
Exc level 1: System.Net.WebException: The remote server returned an error: (400) Bad Request.
   at Microsoft.WindowsAzure.Storage.Table.Protocol.TableOperationHttpResponseParsers.TableOperationPreProcess(TableResult result, TableOperation operation, HttpWebResponse resp, Exception ex)
   at Microsoft.WindowsAzure.Storage.Table.TableOperation.<>c__DisplayClass13.<ReplaceImpl>b__12(RESTCommand`1 cmd, HttpWebResponse resp, Exception ex, OperationContext ctx)
   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndGetResponse[T](IAsyncResult getResponseResult)
---- Microsoft.WindowsAzure.Storage.StorageException : The remote server returned an error: (400) Bad Request.
-------- System.Net.WebException : The remote server returned an error: (400) Bad Request.

Maarten88 · 2016-07-08T10:55:11Z

There is still a problem with TableStorage on generic grains. But I'd expect not only ClearState to fail, but ReadState and WriteState as well. The problem is that the grainType contains the generic parameters as fully qualified types (including assembly info) instead of just the base type names.

The Table Storage Provider then does grainReference.ToKeyString(), which creates a string that, in the case of a generic grain, adds the complete generictype in the key string, creating a partitionkey that is too long; TableStorage does not accept it.

Simplest solution would be to change the ToKeyString() function to return a more compact string for generic grain. A better solution would be to fix the whole internal handling of generic grain type names to not use type.FullName anymore, but type.ToString() or TypeUtils.GetParseableName()

Maarten88 · 2016-07-08T15:54:28Z

I am debugging this; I'm completely at loss why ClearState behaves differently from ReadState and WriteState. It seems the culprit is somewhere in the rowkey value: Azure Table Storage does not like the value in it (which is the grainType), but only in Replace and Delete table operations. Insert and Read seem to work fine with the same values!

Sanitizing the row key with AzureStorageUtils.SanitizeTableProperty() does not help. Putting the base type in the rowKey also does not work. But simply making it empty does help! Maybe there's a limitation on the combined length of partitionKey and rowKey? I can't find good documentation on this; only speculation in forums.

Simplest solution would be to do away with the rowkey completely as it's not helping anyway, but for backwards compatibility it's probably wiser to keep the grainType in there for non-generic grains and make it empty for generic grains (breaking generic state partition key will happen anyway now, so we can break the rowkey too)

veikkoeeva · 2016-07-08T16:37:16Z

@Maarten88, @jdom Just in case, I note here that relational storage provider needs a enough data in grainType (or then glance it from the grain reference) to either use straight away or allow to get enough consistent data out of it so as to create the type key consistently (the ClearState won't work otherwise, which seem to be different compared to ATS here as currently there aren't problems).

Maarten88 · 2016-11-15T18:46:40Z

I found some recent documentation on differences between Azure Storage and Storage Emulator that might help here: https://docs.microsoft.com/en-gb/azure/storage/storage-use-emulator

specifically:

The storage emulator supports partition key and row key property values of less than 512 bytes each. Additionally, the total size of the account name, table name, and key property names together cannot exceed 900 bytes.

By doing this, I encountered that this is still failing on the call to ClearState

jdom · 2017-03-22T02:30:39Z

Rebased this (since there were many conflicts) and added a way to skip the tests if they are being run from the emulator, given the differences in combined property lengths supported in comparison with the production service.
These tests are all passing now (or skipped when real azure storage is not configured)

ReubenBond · 2017-03-22T02:43:00Z

This ought to have been fixed by #2715. I'll take a look and merge

jdom · 2017-03-22T02:47:01Z

Could be, not sure, I did not run it again before rebase, but after the rebase I was still seeing the same stack trace on ClearState that failed the test, when running the test using the storage emulator. So I assume this was just a test issue.

ReubenBond · 2017-03-22T04:49:53Z

Ok, so it seems that blob names can be at most 261 characters when using the Azure Storage Emulator. If I truncate the name to 260 chars, then I get a 404 instead of a 400.

jdom added the Status: work-in-progress label Jul 8, 2016

dnfclas added the cla-already-signed label Jul 8, 2016

Maarten88 mentioned this pull request Jul 8, 2016

[WIP] Fix storage for generic grains in a way that is backward-compatible for normal grains #1920

Closed

sergeybykov modified the milestone: 1.4.0 Nov 3, 2016

sergeybykov modified the milestones: 1.5.0, 1.4.0 Jan 14, 2017

jdom added 3 commits March 21, 2017 19:05

Cleanup for PR dotnet#1897 (generic state parameter bug).

8d4eae2

By doing this, I encountered that this is still failing on the call to ClearState

Add tests for azure blob storage too

e58b93a

Skip tests if being run in the emulator

1582af0

jdom force-pushed the generic-arguments-bug-cleanup branch from 49bbb08 to 1582af0 Compare March 22, 2017 02:23

Rename test grain to align with its usage

081c85a

jdom changed the title ~~WIP: Cleanup for PR #1897 (generic state parameter bug)~~ Cleanup for PR #1897 (generic state parameter bug) Mar 22, 2017

jdom removed the Status: work-in-progress label Mar 22, 2017

jdom requested a review from ReubenBond March 22, 2017 02:37

jdom assigned ReubenBond Mar 22, 2017

jdom removed the request for review from ReubenBond March 22, 2017 02:38

ReubenBond approved these changes Mar 22, 2017

View reviewed changes

jdom changed the title ~~Cleanup for PR #1897 (generic state parameter bug)~~ Add coverage for generic state parameter using Azure Storage (after #1897) Mar 22, 2017

ReubenBond merged commit 7a29756 into dotnet:master Mar 22, 2017

jdom deleted the generic-arguments-bug-cleanup branch March 22, 2017 17:40

github-actions Bot locked and limited conversation to collaborators Dec 12, 2023

Navigation Menu

Repository navigation

Add coverage for generic state parameter using Azure Storage (after #1897) - #1915

Add coverage for generic state parameter using Azure Storage (after #1897)#1915
ReubenBond merged 4 commits into
dotnet:masterfrom
jdom:generic-arguments-bug-cleanup

jdom commented Jul 8, 2016

Uh oh!

Maarten88 commented Jul 8, 2016

Uh oh!

Maarten88 commented Jul 8, 2016 •

edited

Loading

Uh oh!

veikkoeeva commented Jul 8, 2016 •

edited

Loading

Uh oh!

Maarten88 commented Nov 15, 2016

Uh oh!

jdom commented Mar 22, 2017 •

edited

Loading

Uh oh!

ReubenBond commented Mar 22, 2017

Uh oh!

jdom commented Mar 22, 2017 •

edited

Loading

Uh oh!

ReubenBond commented Mar 22, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

jdom commented Jul 8, 2016

Uh oh!

Maarten88 commented Jul 8, 2016

Uh oh!

Maarten88 commented Jul 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

veikkoeeva commented Jul 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Maarten88 commented Nov 15, 2016

Uh oh!

jdom commented Mar 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ReubenBond commented Mar 22, 2017

Uh oh!

jdom commented Mar 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ReubenBond commented Mar 22, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Maarten88 commented Jul 8, 2016 •

edited

Loading

veikkoeeva commented Jul 8, 2016 •

edited

Loading

jdom commented Mar 22, 2017 •

edited

Loading

jdom commented Mar 22, 2017 •

edited

Loading