Fix Additional Metadata for App Submission packages

In v1.6.0, we added [additional metadata](391954b) to the generated JSON file for application submission packages. Unfortunately, we operated under the incorrect assumption that there could only ever be a single Min OS version specified for a package. In fact, a single package can target multiple device families, and each device family can target a different minimum OS version. The only requirement around Min OS Versions for packages is that if there is an appxbundle, all appx files within that bundle must target the same minimum os version for a given device family. Given this change in our understanding of how things work, we're modifying the additional metadata that we were generating. `targetDeviceFamiliesEx` will now contain the same version that `targetDeviceFamilies` contains (both the device family name and the minimum os version), but they will be split into named properties. Additionally, the `minOsVersion` property has been removed since the data it has is less relevant when it's not paired with the specific deviceFamily that it's related to. Finally, we are now adding a new `sbschema` property (which is simply an integer) which we'll be increasing from now on whenever we make changes to non-standard properties in our output JSON in the event that StoreBroker (or other applications leveraging the StoreBroker payload) are depending on those values and need to know how to behave if they're not there. USAGE.md has been updated to now track (and explain) the [schema version changes](https://github.com/Microsoft/StoreBroker/blob/master/Documentation/USAGE.md#schema-versions) (for botn App submissions and for IAP submissions).
microsoft · Oct 24, 2017 · c8b643c · c8b643c
1 parent 7378e93
commit c8b643c
Show file tree

Hide file tree

Showing 3 changed files with 139 additions and 23 deletions.
diff --git a/Documentation/USAGE.md b/Documentation/USAGE.md
@@ -24,6 +24,7 @@
     *   [IAP Overview](#iap-overview)
     *   [Creating Your IAP Payload](#creating-your-iap-payload)
     *   [IAP Commands](#iap-commands)
+*   [Schema Versions](#schema-versions)
 *   [Using INT vs PROD](#using-int-vs-prod)
 *   [Telemetry](#telemetry)
 *   [FAQ](#faq)
@@ -879,6 +880,104 @@ To monitor an IAP submission:
 Follow the steps in [monitoring a submission](#monitoring-a-submission), and be sure to include
 **`-IapId`** in the function parameters _instead of_ **`-AppId`**.
 
+## Schema Versions
+
+StoreBroker's packaging commands (`New-SubmissionPackage` and `New-InAppProductSubmissionPackage`)
+add properties to the generated .json file that are not part of the official Submission API JSON
+schema.  These additional properties are added in order to enable additional StoreBroker scenarios
+(like validating the `AppId` or `IapId` being used with a submission package, and determining which
+packages are redundant when using the `-UpdatePackages` switch).
+
+It is our intention to never change how these custom properties work once we've published out that
+version of StoreBroker, however it is possible that bugs or additional circumstances may simply make
+it necessary.
+
+To that end, starting with version `1.11.1` of StoreBroker, we will always add a `sbSchema` property
+to the generated json file, with a simple **integer value** that will be incremented whenever we make any
+change to those additional properties.  We'll document what those differences are below so that you'll
+be able to make corresponding changes to your own code if you depend on these properties.
+
+### App Submission Packages
+
+#### Version 1
+
+We never used the value `1` for a published schema.  Any App JSON package file that doesn't have an
+`sbSchema` property is effectively version 1.
+
+**Properties Added**
+ * `appId` - Stores the `appId` of the In-App Product that the submission package is for.
+ * `applicationPackages.minOSVersion` - An array of all Min Versions referenced in the app packages.
+ * `applicationPackages.targetFamiliesEx` - An array of all device families that the package targets.
+ * `applicationPackages.innerPackages.[architecture].minOSVersion` - An array of all Min OS Versions
+    referenced in the specific architecture app package.
+ * `applicationPackages.innerPackages.[architecture].targetFamiliesEx` - An array of all device
+    families targeted in this specific architecture app package.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+#### Version 2
+
+**Properties Added**
+ * `sbSchema` - Tracks the version number of the schema to identify what additional StoreBroker properties
+   should be expected within.
+
+**Properties Removed**
+ * `applicationPackages.minOSVersion` - This information can now be found in better context in
+   `applicationPackages.targetFamiliesEx.minOSVersion`
+ * `applicationPackages.innerPackages.[architecture].minOSVersion` - This information can now
+    be found in better context in `applicationPackages.innerPackages.[architecture].targetFamiliesEx.minOSVersion`
+
+**Properties Changed**
+ * `applicationPackages.targetDeviceFamiliesEx` - This used to be an array of strings of each
+   device family. This is now an array of dictionaries where each dictionary is a
+   `name`/`minOSVersion` pair, since a single appx can have different Min OS Versions depending
+   on which platform is being looked at.
+ * `applicationPackages.targetDeviceFamilies` - There was a bug here that if there was more than
+   one Min OS Version within a package, the string would look like this:
+   `Windows.Desktop min version System.Object[]`.  This is now fixed.
+ * `applicationPackages.innerPackages.[architecture].targetDeviceFamiliesEx` - This used to be
+   an array of strings of each device family. This is now an array of dictionaries where each dictionary
+   is a `name`/`minOSVersion` pair, since a single appx can have different Min OS Versions depending on
+   which platform is being looked at.
+ * `applicationPackages.innerPackages.[architecture].targetDeviceFamilies` - There was a bug here that
+   if there was more than one Min OS Version within a package, the string would look like this:
+   `Windows.Desktop min version System.Object[]`.  This is now fixed.
+
+### In-App Product (IAP) Submission Packages
+
+#### 1
+
+We never used the value `1` for a published schema.  Any IAP JSON package file that doesn't have an
+`sbSchema` property is effectively version 1.
+
+**Properties Added**
+ * `iapId` - Stores the `iapId` of the In-App Product that the submission package is for.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+#### 2
+
+This schema is equivalent to v1, but now with the addition of `sbSchema` tracking the schema version.
+
+**Properties Added**
+ * `sbSchema` - Tracks the version number of the schema to identify what additional StoreBroker properties
+   should be expected within.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+
 ## Using INT vs PROD
 
 > This option is only available for Microsoft employees using the official Microsoft

diff --git a/StoreBroker/PackageTool.ps1 b/StoreBroker/PackageTool.ps1
@@ -33,10 +33,20 @@ $script:applicationMetadataProperties = @(
     "capabilities",
     "targetDeviceFamilies",
     "targetDeviceFamiliesEx",
-    "minOSVersion",
     "innerPackages"
 )
 
+# The API formats version numbers as "[device family] min version [min version]"
+$script:minVersionFormatString = "{0} min version {1}"
+
+# The current version of the StoreBroker schema that PackageTool is authoring for app and IAP submissions.
+# The StoreBroker schema may include metadata that isn't a core part of the official Submission API
+# JSON schema (like the appId or iapId, package metadata, etc...). These values should be updated any time
+# we alter what additional metadata is added to the schema for that submission type.
+$script:appSchemaVersion = 2
+$script:iapSchemaVersion = 2
+$script:schemaPropertyName = 'sbSchema'
+
 function Get-StoreBrokerConfigFileContentForIapId
 {
 <#
@@ -1296,7 +1306,7 @@ function New-ApplicationMetadataTable
 
         The hashtable has keys for
          "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages"
 
     .OUTPUTS
@@ -1384,7 +1394,7 @@ function Read-AppxMetadata
         Reads various metadata properties about the input .appx file.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1437,7 +1447,6 @@ function Read-AppxMetadata
 
         $metadata.version        = $manifest.Package.Identity.Version
         $metadata.architecture   = $manifest.Package.Identity.ProcessorArchitecture
-        $metadata.minOSVersion   = $manifest.Package.Dependencies.TargetDeviceFamily.MinVersion
         $metadata.targetPlatform = Get-TargetPlatform -AppxManifestPath $appxManifest
         $metadata.name           = $manifest.Package.Identity.Name -creplace '^Microsoft\.', ''
 
@@ -1453,14 +1462,15 @@ function Read-AppxMetadata
             Sort-Object -Unique
 
         $metadata.targetDeviceFamiliesEx = @()
-        $metadata.targetDeviceFamiliesEx += $manifest.Package.Dependencies.TargetDeviceFamily.Name |
-            Where-Object { $null -ne $_ } |
-            Sort-Object -Unique
+        $metadata.targetDeviceFamiliesEx += $manifest.Package.Dependencies.TargetDeviceFamily |
+            Where-Object { $null -ne $_.Name } |
+            Sort-Object -Property Name -Unique |
+            ForEach-Object { [PSCustomObject]@{ 'name' = $_.Name; 'minOSVersion' = $_.MinVersion } }
 
         $metadata.targetDeviceFamilies = @()
         foreach ($family in $metadata.targetDeviceFamiliesEx)
         {
-            $metadata.targetDeviceFamilies += ("{0} min version {1}" -f $family, $metadata.minOSVersion)
+            $metadata.targetDeviceFamilies += ($script:minVersionFormatString -f $family.Name, $family.minOSVersion)
         }
 
         # A single .appx will never have an inner package, but we will still set this property to
@@ -1497,7 +1507,7 @@ function Read-AppxUploadMetadata
         Reads various metadata properties about the input .appxupload.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1596,7 +1606,7 @@ function Read-AppxBundleMetadata
         Reads various metadata properties about the input .appxbundle.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1677,21 +1687,20 @@ function Read-AppxBundleMetadata
             $appxFilePath = (Get-ChildItem -Recurse -Path $expandedContainerPath -Include $application).FullName
             $appxMetadata = Read-AppxMetadata -AppxPath $appxFilePath -AppxInfo $AppxInfo
 
-            # minOSVersion and targetPlatform will always be the values of the last .appx processed.
-            $metadata.minOSVersion    = $appxMetadata.minOSVersion
+            # targetPlatform will always be the values of the last .appx processed.
             $metadata.targetPlatform  = $appxMetadata.targetPlatform
 
             $capabilities            += $appxMetadata.capabilities
             $targetDeviceFamilies    += $appxMetadata.targetDeviceFamilies
             $targetDeviceFamiliesEx  += $appxMetadata.targetDeviceFamiliesEx
 
             $metadata.innerPackages.$($appxMetadata.architecture) = @{
-                version              = $appxMetadata.version;
-                targetDeviceFamilies = $appxMetadata.targetDeviceFamiliesEx;
-                languages            = $appxMetadata.languages;
-                capabilities         = $appxMetadata.capabilities;
-                minOSVersion         = $appxMetadata.minOSVersion;
-                targetPlatform       = $appxMetadata.targetPlatform;
+                version                = $appxMetadata.version;
+                targetDeviceFamiliesEx = $appxMetadata.targetDeviceFamiliesEx
+                targetDeviceFamilies   = $appxMetadata.targetDeviceFamiliesEx | ForEach-Object { $script:minVersionFormatString -f $_.name, $_.minOSVersion }
+                languages              = $appxMetadata.languages;
+                capabilities           = $appxMetadata.capabilities;
+                targetPlatform         = $appxMetadata.targetPlatform;
             }
         }
 
@@ -1701,7 +1710,11 @@ function Read-AppxBundleMetadata
         # results in $m.capabilities being a String type instead of Array type.
         $metadata.capabilities           += $capabilities | Sort-Object -Unique
         $metadata.targetDeviceFamilies   += $targetDeviceFamilies | Sort-Object -Unique
-        $metadata.targetDeviceFamiliesEx += $targetDeviceFamiliesEx | Sort-Object -Unique
+
+        # https://stackoverflow.com/questions/31343752/how-can-you-select-unique-objects-based-on-two-properties-of-an-object-in-powers
+        $metadata.targetDeviceFamiliesEx += $targetDeviceFamiliesEx |
+            Group-Object -Property name, minOSVersion |
+            ForEach-Object { $_.Group | Select-Object -Property name, minOSVersion -First 1 }
     }
     finally
     {
@@ -1741,7 +1754,7 @@ function Get-FormattedFilename
         System.String. The ManifestType_AppName_Version_Architecture string.
 
     .EXAMPLE
-        Get-FormattedFilename @{ name="Maps"; version="2.13.22002.0"; architecture="x86"; targetDeviceFamiliesEx=@("Windows.Desktop") }
+        Get-FormattedFilename @{ name="Maps"; version="2.13.22002.0"; architecture="x86"; targetDeviceFamiliesEx=@(@{ name = "Windows.Desktop"); minOSVersion="1.2.3.0" } }
 
         Would return something like "Desktop_Maps_2.13.22002.0_x86.appxupload"
 #>
@@ -1776,7 +1789,7 @@ function Get-FormattedFilename
     }
 
     # Simplify 'Windows.Universal' to 'Universal'
-    $deviceFamilyCollection = $Metadata.targetDeviceFamiliesEx | ForEach-Object { $_ -replace '^Windows\.', '' }
+    $deviceFamilyCollection = $Metadata.targetDeviceFamiliesEx.Name | ForEach-Object { $_ -replace '^Windows\.', '' }
 
     $formattedBundleTags = @($Metadata.name, $version, $architectureTag)
     if ($deviceFamilyCollection.Count -gt 0)
@@ -1798,7 +1811,7 @@ function Read-ApplicationMetadata
         Reads metadata used for submission of an .appx, .appxbundle, or appxupload.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx",
         "innerPackages", and "name".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -2149,6 +2162,8 @@ function Get-SubmissionRequestBody
 
     $submissionRequestBody = Remove-DeprecatedProperties -SubmissionRequestBody $submissionRequestBody
 
+    $submissionRequestBody | Add-Member -Name $script:schemaPropertyName -Value $script:appSchemaVersion -MemberType NoteProperty
+
     return $submissionRequestBody
 }
 
@@ -2261,6 +2276,8 @@ function Get-InAppProductSubmissionRequestBody
         $submissionRequestBody.listings = Convert-InAppProductListingsMetadata @listingsResources
     }
 
+    $submissionRequestBody | Add-Member -Name $script:schemaPropertyName -Value $script:iapSchemaVersion -MemberType NoteProperty
+
     return $submissionRequestBody
 }
 

diff --git a/StoreBroker/StoreBroker.psd1 b/StoreBroker/StoreBroker.psd1
@@ -6,7 +6,7 @@
     CompanyName = 'Microsoft Corporation'
     Copyright = 'Copyright (C) Microsoft Corporation.  All rights reserved.'
 
-    ModuleVersion = '1.11.0'
+    ModuleVersion = '1.11.1'
     Description = 'Provides command-line access to the Windows Store Submission REST API.'
 
     RootModule = 'StoreIngestionApi'