Fix random zoom in zoom out

[XuYicong]

Fixes PlayCover/PlayCover/issues/1483 and PlayCover/PlayCover/issues/767 and PlayCover/PlayCover/issues/1492

Update 0530

I think I finally found the real cause (many thanks to testing help of discord user @tyler): The data race in PTFakeMetaTouch

Basically a reusageMask is used to record allocation of UITouch objects to touch points. The fake touch process has two procedures, sendEventCallback runs in main thread, and touchcam runs in touchqueue which is a separate thread. For reusageMask, there is a &= operation in touchqueue and a |= operation in main thread.

Theoretically, there should be some protection method on this cross-thread data manipulation, but for some reason there are not. How could this cause the random zoom-in zoom-out issue?

So the &= operation in touchcam is used to allocate an object for the touch point, and the |= operation in sendEventCallback for releasing an object for subsequent use. These two contains firstly reading the value, then doing the calculation, then writing it back. The operation sequence could be like this:

1. Thread A read data
2. Thread B read data
3. Thread A write back data
4. Thread B write back data

So clearly in this situation, the write operation of thread A doesn't take effect. It was overriden by thread B. That is, the &= operation and |= operation could fail. Let thread A be the touchqueue thread, and the object allocation operation fails. In this case, this point may continue its lifecycle, but as long as another point needs an allocation too, the former object may be released immediately. This would cause its lifecycle ends without an ended event, and the game would always think this touch point is on screen, while we completely lost control of it. That would cause a ghost touch point always on the screen. And at this point if you try to pan or rotate, there are two touch points on the screen and it would be recognized as a pinch gesture.

What if the |= operation fails? In this case, we failed to release an object. That object would always exist in the touch array in an ended phase, which interestingly wouldn't cause any issue, except maybe several bytes of memory leak. In summary, the only issue that data race would cause is the ghost touch issue, thus a random zoom-in zoom-out.

Solution is simply making reusageMask atomic and all operations regarding it atomic.

Update 0531

Thanks to extensive testing of @RyufM, I found another concurency issue.

When the &= operation is executed in touchqueue thread, it occupies an ID for use, then replaces that object with a new object. At the same time, in the main thread, the code checks if the touch object at every ID is in an ended state and if so, mark that ID as free. The execution sequence could be like this:

1. touchqueue marks an ID as occupied
2. main thread checks the object at the same ID
3. main thread found it's in ended state and marks this ID as free
4. touchqueue replaces the touch object with a new one

In this case, even when that ID has a new object, it is still marked as free. Subsequent ID allocation requests would occupy that same ID and replace that new object, making its lifecycle end without and "ended" event, thus becoming a ghost touch point.

Solution is to use a mutex lock to lock the ID occupation operation and the object replacement operation together.

Below are the original description which may help but doesn't really matter:

---

This PR tries to fix things in three aspects:

1. Pressed keys may not release when people press option to hide/unhide cursor. (as described by @kang49)

For that, I have added release of all touch points related to mouse whenever option is pressed.

6. The game developer might assume every touch point is inside window. If a touch point ends up out of window, the game may somehow fail to end it properly and cause resources to leak over time. So I limited every swipeAction point inside the window.

The moved touch action inside swipeAction is moved before position update, so as to fix some glitch, where the initial touch position becomes the first moved touch (which is a little shifted from the began touch), and the terminal velocity becomes 0 because the end touch has the same position as the last moved touch.

7. When touch point phase is set to ended, it might fail. Previous implementation assumes that this alwasys succeed, so as to create an inconsistency between the actual phase of the touch point and the recorded point phase. When we think it's ended, it might still be moved and we lost control of it. By checking on the actual phase of a point, this is mitigated.

Let me know if your issue behaves differently or this PR doesn't fix your issue. (and most importantly, describe your issue, what triggers it, what can be done to temporarily fix it. It's better to provide screen recording of your issue.)

[huafeng-yang]

修复了 PlayCover/PlayCover/issues/1483（至少是@kang49）

我认为这个问题只在人们选择隐藏光标时。

为此，我单击按钮选择释放与鼠标相关的所有触摸点的功能。

如果您的问题表现不同或此PR无法解决您的问题，请告诉我。（重要的是，描述您的问题、触发原因、可以参考哪些措施解决它。最好提供问题的屏幕广播。）

我看到您修复了这个问题，但我怎么把它应用在自己的playcover上呢？纯电脑小白，希望您可以大致告诉我该怎么做，感激不尽

[kang49]

修复了 PlayCover/PlayCover/issues/1483（至少是@kang49）
我认为这个问题只在人们选择隐藏光标时。
为此，我单击按钮选择释放与鼠标相关的所有触摸点的功能。
如果您的问题表现不同或此PR无法解决您的问题，请告诉我。（重要的是，描述您的问题、触发原因、可以参考哪些措施解决它。最好提供问题的屏幕广播。）

我看到您修复了这个问题，但我怎么把它应用在自己的playcover上呢？纯电脑小白，希望您可以大致告诉我该怎么做，感激不尽

We need to wait for the project owner to approve this solution, review the code to ensure it doesn't break other parts of the project, and then deploy it to production. This process is quite time-consuming.

If possible, please use English when discussing on GitHub.

[kang49]

Looks good! The fix for the random zoom issue seems solid. Just a minor suggestion: add a comment in invalidateNonButtonActions to clarify its purpose. Otherwise, great job!

[ugzhu]

By restricting swipeAction inside window, does this PR happen to also resolve PlayCover/PlayCover#1492 ? These two isseus seem to be related in some way.

[XuYicong]

By restricting swipeAction inside window, does this PR happen to also resolve PlayCover/PlayCover#1492 ? These two isseus seem to be related in some way.

Yeah that issue should have been at least mitigated in some sort, but I can't say for sure because I can't reproduce the issue thus testing the fix.

[Depal1]

This PR also fixes the issue described in PlayCover/PlayCover#1022

[TheMoonThatRises]

Something in this PR messes with scrolling through UI elements.

Heres an example before:

before.mov

and heres with this PR:

after.mov

[XuYicong]

Something in this PR messes with scrolling through UI elements.

This is because the touch points are limited in the window. Whenever the point moves on the window edge, it's released and re-touched from the starting point again.

In details, if you scroll too fast (and too close to the edge), touch points would reset so fast that it's not even recognized as scrolling.

I was thinking this could fix random zooming in some sort, but it turns out not working. Since it now causes problems, I think I shall revert this change.

[TheMoonThatRises]

This issue still exists in this PR, but to a less extent. The scrolling does work, but it bugs out after scrolling for a little bit.

[Depal1]

Perhaps we could try testing this together with a polling rate limiter for scroll wheel events, as mentioned in the discussion of #142 #142 (comment)

[XuYicong]

The scrolling does work, but it bugs out after scrolling for a little bit.

I have investigated more into this issue, and found that limiting touches inside window is necessary for Wuthering Waves camera to work(otherwise PlayCover/PlayCover#1492 would happen).

For the scrolling issue, it seems that Wuthering Waves' scroll view allows touch outside of window on top/left direction, but not on bottom/right direction. If a touch moves outside of window on bottom/right, the whole scroll would freeze. Moreover, if edge-resetting causes a touch sequence having too few moved events, it also causes problems, so simply resetting once it touches the edge doesn't work. I'm thinking of adding a velocity rescaler to the scroll. If the velocity is predicted to cause less than 4 moved events before an edge-resetting, then velocity is rescaled so that there would be enough moved events to be recognized as a scroll. This would come at a cost of limiting max scrolling speed.

Moreover, not resetting on touching edge may have better UX on other games. So maybe add an option for the user to toggle this behavior? And default resetting-on-edge to on.

Update: It works. As far as my own testing, limiting min event count does fix the Wuthering Waves scroll issue.