Import-PSGsuiteConfig failing on macOS - 7.0.0 -- All versions

[phatmandrake]

Describe the bug

The Cmdlet completes successfully, but it only succeeds in partially importing the data. Distinctly it does set the first letter of appemail, adminemail, customerid, domain, and preference.

Setting the -Temporary parameter results in a successful import.

This issue doesn't seem to be happening on Windows 10.

Environment

• OS: macOS Catalina
• PowerShell Version: 7.0.0-rc.2
• PSGSuite Version: 2.35.1

[scrthq]

heyo @phatmandrake ! Thanks for sending this in! Just to confirm, Import-PSGSuiteConfig is failing when trying to import a config from JSON if it involves saving it back to the config file as well, otherwise using the Temporary switch to just update the $script:PSGSuite variable in the current session works fine?

If you open the config file on that Mac, do you see the same info there? Since you should have already saved it at this point, you can test by...

1. Start a new PowerShell session
2. Import-Module PSGSuite
3. Show-PSGSuiteConfig

After the above, does it show the same thing you're seeing now?

If that is the only config on that machine, you can also try clearing it out and reimporting:

Remove-Item (Show-PSGSuiteConfig).ConfigPath -Force
Import-PSGSuiteConfig $configJson

[scrthq]

also - if you look at the JSON you're importing, does it have an array for the AppEmail, AdminEmail, CustomerId, Domain, and Preference values? Or is it a single string on the JSON as expected?

[phatmandrake]

The answer to your first question is Yes.

Interestingly, the config file does not exist at the configpath.

/Users/phatmandrake/.config/powershell/AnonymousModules/PSGSuite/Configuration.psd1

It instead exists here:

/Users/phatmandrake/.config/powershell/SCRT HQ/PSGSuite/Configuration.psd1

[UPDATE:] So I deleted the file (I swear I already did this), and it now recreates it in the anticipated directory. Under SCRT HQ rather than AnonymousModules. So chased my tail on that trying to make that make sense longer than I care to admit 😅

---

Pretend I didn't say any of that.

I did some digging and here's what I'm finding.

Import-PSGsuiteConfig successfully imports the config file. Set-PSGsuiteConfig is called, and the information is then Encrypted with the private Encrypt function, and the config is successfully exported. Get-PSGsuiteConfig is called and this where things get murky:

Write-Verbose "Retrieved configuration '$choice'"
        if (!$NoImport) {
            $script:PSGSuite = $decryptedConfig
        }

$decryptedConfig outputs the same as the import file, so the failure seems to be stemming from the decrypt function.

I found this:

https://docs.microsoft.com/en-us/archive/blogs/besidethepoint/decrypt-secure-strings-in-powershell

The article suggest the following as an example:

        function Decrypt-SecureString {
            param( 
                [Parameter(ValueFromPipeline=$true,Mandatory=$true,Position=0)] 
                [System.Security.SecureString] $sstr
                )
                $marshal = [System.Runtime.InteropServices.Marshal]
                $ptr = $marshal::SecureStringToBSTR( $sstr )
                $str = $marshal::PtrToStringBSTR() $ptr )
                $marshal::ZeroFreeBSTR( $ptr )
                $str
            }

It is currently:

        function Decrypt {
            param($String)
            
            if ($String -is [System.Security.SecureString]) {
                [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
                    [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
                        $String)
                        )
            }
            else {
                $String
            }
        }

[System.Runtime.InteropServices.Marshal]::PtrToStringAuto() for some reason in PWSH 7 this method terminates after the first character. In 5.1 and 6 this behaves as exactly as expected.

https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.ptrtostringauto?view=netframework-4.8

I haven't yet found a way around this yet, other than using PtrToStringBSTR(). This might be preferred since we know that it will always be a BSTR.

I'm not sure why most examples for this specific use case use PtrToStringAuto() really.

[phatmandrake]

Every other char is treated as a NULL value...hmm 🤔

Related(?) dotnet/runtime#12343

[scrthq]

@phatmandrake - Weird and interesting findings!!! On your JSON, is the ConfigPath listed? If it is, that should be pulled from the Export-PSGSuiteConfig process as that may be breaking import.

For the Decrypt bit, you mention PowerShell 7. Is it behaving the same way on PS7 outside of Mac? I'm not able to replicate it on PS7-rc2 on Windows 10 personally:

>> [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
>>     [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
>>         (ConvertTo-SecureString 'hello' -AsPlainText -Force)
>>     )
>> )
hello

[phatmandrake]

The config path listed was listed as the anonymousmodules path shown in my first screenshot; however, by deleting the config file in SCRT HQ I was able to re-import the configuration and have it show as being in the correct SCRT HQ path. By deleting and reimporting, I could replicate this preferred behavior multiple times. I think what happens is when there is an existing config.psd1 the path changes, but even still the modules goes to the right location (SCRT HQ) ultimately when looking for a config to load even if it lists it wrong. I also think this only happens when the config fails to load properly.

I noticed when I would Import-Module -Force that the "anonymousmodules" path would be injected after Import-SpecificConfiguration is called. Haven't dug into why it's not idempotent (boy I never thought I would get to use that word lol).

As for the decrypt bit (ha). It works just fine in Windows 10. I run macOS and Windows in a VM so I bounce between environments all day. This is only a PS7 on MacOS issue that I can tell right now.

https://stackoverflow.com/questions/11022555/manipulating-c-strings-with-multiple-null-characters-in-memory

Some of the discussion leads me to believe the issue is related to UTF-16 on MacOS I don't understand why this is now an issue when it's not in previous versions.

[scrthq]

Super strange!

What version of the Configuration module do you have installed on your Mac?

Get-Module Configuration

[phatmandrake]

Version 1.3.1 -- but the issue related to that could be because of something I did while I was searching for the PtrToStringAuto() problem. I am unable to replicate the behavior where the wrong configpath is being specified now.

RC3 has the same issue where null chars are being added between each char in the secure string on macOS, which is impacting PtrToStringAuto() to function the way we'd like on macOS.

[phatmandrake]

I opened an issue here: PowerShell/PowerShell#11953

[phatmandrake]

Well @mklement0 (+1 this guy is great) confirmed the behavior for me, and reaffirmed that using PtrToStringBSTR would be most appropriate for resolving the issue in place using the existing code structure. (Along with a different approach using [NetworkCredential])

Is there any room here for adopting a new secrets model in preparation for version 7? I feel like I remember seeing something on secrets management recently 🤔

Short term it would be trivial to update the decrypt function to utilize the PtrToStringBSTR function though, but I would love to see universal encryption come to the module.

https://stackoverflow.com/questions/60404847/are-you-able-to-use-ptrtostringauto-to-decrypt-a-secure-string-in-powershell-7-o

[scrthq]

Thank you @phatmandrake + @mklement0 ❤️ I'll get that change in as soon as I can!

[scrthq]

@phatmandrake Deployed! Also just seeing your note about possibly moving to the new Secrets storage. I'm definitely interested in exploring that, but I want to ensure that being able to retrieve the configuration is reliably the same across platform. I'm keeping an eye on it though 🙂