Accessing a variable set inside a timer from a route

[ssugar]

I'd like to have a Server timer running every X seconds that will pull some data and update a hash table. I would then like to read from that hash table when a specific api route is called.

I haven't been able to figure out a way to do that. I looked through the readme and the timer example, but that is more about how to create/set a timer, and not about the logic and variables within the timer. Any guidance would be appreciated.

[Badgerati]

I have an idea on how this could be possible, but i'll need to experiment with it.

Basically: since the timers are all running in a separate runspace, the variables defined within the timers will only be within the scope of that runspace - so inaccessible to routes/loggers.

My thinking is that runspaces have a defined session state variable, and maybe it could be possible to have a "user defined" hashtable within that state where you could tell pode to set variables against it so they're accessible everywhere.

ie:

timer 'forever' 5 {
    $some_hashtable = @{}

    # will add hashtable to user-defined session state, calling it 'earth'
    state add 'earth' $some_hashtable

    # give it some data
    $some_hashtable['hello'] = 'world'
}

route 'get' '/' {
    param($session)

    # in theory this should write out the text 'world'
    $user_state['earth']['hello'] | Out-Default

    # or keep to method syntax:
    (state get 'earth').hello | Out-Default
}

I'm not too sure what's in your hashtable, but for now it could be possible to ConvertTo-Json and write it into a file; then have the routes read back the file?

[ssugar]

I figured that the variables wouldn't be accessible due to the runspace.

I thought about writing to a file and then reading back in the route, but wanted to check in with you if there was a cleaner way of doing this via a runspace state variable or something like that.

Thanks for the detailed response. Much appreciated.

[Badgerati]

Commit above adds in the state function idea I was talking about; There's an example and readme docs in there, and it allows you to share variables in routes, timers and loggers.

[ssugar]

Tested, and accessing variables I set in a timer from a route is working for me now. I have started to see some exceptions though running with this version. They happen about 10-15 minutes after starting up pode, so things work for a while, and then these exceptions occur. Note that I have some browser-side javascript that is calling api routes served by pode every 1-5 seconds.

Invoke-Command : Global scope cannot be removed.
At C:\users\ssugar\Documents\github\pode\src\Tools\WebServer.ps1:126 char:21
+ ...             Invoke-Command -ScriptBlock $route.Logic -ArgumentList $P ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (GLOBAL:String) [Invoke-Command], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : GlobalScopeCannotRemove,Microsoft.PowerShell.Commands.InvokeCommandCommand

another one

Write-ToResponse : The term 'Test-Empty' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\users\ssugar\Documents\github\pode\src\Tools\Responses.ps1:190 char:5
+     Write-ToResponse -Value $Value -ContentType 'application/json; ch ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Test-Empty:String) [Write-ToResponse], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException,Write-ToResponse

[Badgerati]

Real headache this one. I can't seem to get the global error you've put, but I can get the other error plus more.

These errors don't seem to happen in master, so seems it definitely caused by this work. It appears to happen when under load, and when you have a route and timer altering the same shared state variable - good old multi threading :/

I've put in some additional error logging, and also locking in the State function. Although the rate of errors has dropped they can still happen.

I'll keep digging.

[ssugar]

Yeah, I've had pode under similar load before without any issue, but when I try to make frequent changes to these state variables in timers and read them in routes, I run into issues pretty quick.

[Badgerati]

This should now be all right however, you'll need to update anywhere you're using state and wrap them with a lock - grouping multiple usages together. There's an example of using state and lock in the examples:

timer 'forever' 2 {
    param($session)
    $hash = $null

    lock $session.Lockable {
        if (($hash = (state get 'hash')) -eq $null) {
            $hash = (state set 'hash' @{})
            $hash['values'] = @()
        }

        $hash['values'] += (Get-Random -Minimum 0 -Maximum 10)
    }
}

route get '/get-array' {
    param($session)

    lock $session.Lockable {
        $hash = (state get 'hash')
        json $hash
    }
}

I've made it so that routes, timers and loggers each have a Lockable resource passed in their $session params for use with lock.

I've tested it with quite a bit of load:

• a timer running every second
• a route to get the current hashtable value
• a route to reset the current hashtable value

The latter two I hit 100 times per second. Before the locking, lots of errors, after it there no errors after multiple runs.

[ssugar]

Ok, not sure why, but with this latest commit, I'm getting an exception thrown on the first page load.

Listening on https://localhost:443/
The property 'Web' cannot be found on this object. Verify that the property exists and can be set.
At C:\users\ssugar\documents\github\pode\src\Tools\WebServer.ps1:61 char:13
+             $PodeSession.Web = @{}
+             ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : PropertyNotFound

ForEach-Object : Stack empty.
At C:\users\ssugar\documents\github\pode\src\Tools\Helpers.ps1:165 char:61
+ ... deSession.Runspaces | Where-Object { !$_.Stopped } | ForEach-Object {
+                                                          ~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [ForEach-Object], InvalidOperationException
    + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.ForEachObjectCommand

Let me know if you want me to send along the code for the server I'm running.

Once that error is thrown, I will continue to get the same error until I actually close my powershell window and reopen it:

C:\users\ssugar\documents\pode> .\web_page.ps1
The property 'Web' cannot be found on this object. Verify that the property exists and can be set.
At C:\users\ssugar\documents\github\pode\src\Tools\WebServer.ps1:61 char:13
+             $PodeSession.Web = @{}
+             ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : PropertyNotFound

Exception calling "Stop" with "0" argument(s): "Cannot access a disposed object.
Object name: 'System.Net.HttpListener'."
At C:\users\ssugar\documents\github\pode\src\Tools\WebServer.ps1:154 char:13
+             $listener.Stop()
+             ~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : ObjectDisposedException

[Badgerati]

That "Web" error was the one I was getting most frequently until I put locking in place. If it's all right I might need some of the code you're using for the server that reproduces the issue?

[ssugar]

I've cleaned things up a bit and removed everything non-essential. In the attached zip is my server script, a basicindex.pode view file, and the javascript that basicindex.pode file calls that runs the api calls.
pode-issue-40.zip. Hope this helps.

[Badgerati]

Excellent, thank you.

I've added in a bunch of logging, and managed to narrow it down to precisely when it happens. The moment you make a web request at the precise moment a timer is within the Enter section of the lock, and before the Exit - something there, somehow, is causing the $PodeSession to mysteriously become null.

Very odd. There are no errors, nothing. Just somehow, it's... null 😕

[Badgerati]

@ssugar Turns out it was some weird behaviour on Invoke-Command. Still haven't worked out what it was, but using & instead seems to have resolved it.

[ssugar]

Using your latest commit, the prior exceptions are gone. Seeing this exception after about 20-30 seconds though:

Exception setting "ContentLength64": "This operation cannot be performed after the response has been submitted."
At C:\users\ssugar\documents\github\pode\src\Tools\Responses.ps1:25 char:5
+     $PodeSession.Web.Response.ContentLength64 = $Value.Length
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : ExceptionWhenSetting

[Badgerati]

I've a feeling that might be caused by something else; is there a chance a route is trying to set a a response twice? That's when that error gets thrown, as the response has already had content set against it.

[Badgerati]

Or maybe not! I just got it on your example files. But... it was working yesterday o-o

walks away and bangs head on desk for a couple hours.

[Badgerati]

@ssugar Any luck with this one? I've had the scripts you gave me running fine for 2hrs and going.

[ssugar]

@Badgerati - Got it running now. Will keep you up-to-date

[ssugar]

Looks good so far. It hasn't thrown any exceptions in the last 10 minutes, which is much better than any of the prior commits.

One thing I do notice is that around 10% of the log entries are missing their response size (just shows "-")

Seems to be pretty random at this point.

[Badgerati]

@ssugar Response sizes should be back to normal now

[ssugar]

@Badgerati - Running it now. Response sizes look good again.

[ssugar]

@Badgerati - Stability is good too. Amazing.

[Badgerati]

@ssugar Still going strong?

[ssugar]

@Badgerati - Yup, been going strong for the last 12 hours. No exceptions thrown. Looks great.

Edit: over 24 hours running smoothly now

[Badgerati]

Excellent, sounds like it's all fine now; in which case I'll do some extra testing tonight and then release it

[ssugar]

@Badgerati - Sounds good to me.