Question: low load BGS

[StuartIanNaylor]

Murari, if you where focusing on low load high FPS BGS do you have any suggestions for an alg/model or is there such a thing?

I guess under IR/Snow storm/Windy weather a scene can pretty much change, but just wondering if there is a compromise solution that works most of the time minus adverse and freak conditions that is low load in terms of computation.
If you where going to pick one, what would you pick as starting point?
Especially with seen scene vs unseen as its likely for a cam to have a stream history so is unseen also important?