Eyetoy Support #30

seta-san · 2020-05-03T22:15:52Z

i see there is at least some preliminary support for eyetoy in the code even if it isn't active. is there plans to make that available?

SlyCooperReloadCoded · 2020-05-10T09:13:48Z

Yes, please. No one seems to have attempted to get into EyeToy emulation, but all EyeToy titles would most likely jump directly to the Playable category if there was full EyeToy emulation. It's just a camera, it can't be that hard, right? Heck, the EyeToy can be used as a PC webcam.

Florin9doi · 2020-05-27T22:22:14Z

Notes:

The develop branch (and others?) has a issue: it reorders the endpoints and descriptors:

Patch 1: try to resolve it but SuperSpeed endpoints are disabled: usb_desc1.txt. Untested
Patch 2: avoid the issue only for EyeToy: eyetoy.txt. This patch also clears a reserved bit from bmAttributes which is not present in a hardware device.

The Windows and linux (a windows clone) drivers are configuring the camera to transfer jpeg pictures.
PS2, or at least EyeToyPlay(1), is using a different format/compression. Immediately after OV header, partially documented by the linux driver, it sends an additional constant header: 69 70 75 6D 00 00 00 00 40 01 F0 00 01 00 00 00 and then compressed(?) data.
EyeToyPlay(1)(SCES_515.13) and his OV driver are unstripped.

jackun · 2020-05-27T22:31:17Z

it reorders the endpoints and descriptors:

Is there any difference? Seemed to "work" anyway.

Florin9doi · 2020-05-27T22:46:06Z

Is there any difference? Seemed to "work" anyway.

I haven't seen any difference. It is getting stuck the same

Florin9doi · 2020-05-28T07:47:32Z

At the bytes [5:8] from header: ff ff ff 50 00 [ef bb ba bc] 00 01 00 00 00 14 00
is present an little-endian increasing counter. Between SOP (ff ff ff 50) and EOP (ff ff ff 51) it increases with ~320k, and between EOP (ff ff ff 51) and SOP (ff ff ff 50) it increases with ~10k: counter.txt

Florin9doi · 2020-05-31T07:49:30Z

I modified the plugin to send a few frames captured from a hardware camera. The frames are received and then sent to the image processing unit (IPU) to decode them: ipu_decompressYCrCb(void *, void *, sceImage *, sceImage *, sceImage *), but the decoding fails and this message is observed in console: IPU1 running when IPU1 DMA disabled! CHCR 1 QWC 126. It seems that the PCSX2 decoding is bugged too.
[69 70 75 6D](magic number) 00 00 00 00 [40 01](width) [F0 00](height) 01 00 00 00

Florin9doi · 2020-05-31T09:20:34Z

EyeToy Kinetic Combat works with pre-recorded data 😛

jackun · 2020-05-31T10:15:59Z

So the data was still MJPG, just different header or some huffman compressed raw?

Florin9doi · 2020-05-31T10:26:24Z

I have no idea yet. I saved the usb isoc packets using a linux test program. But because the decoder is already implemented in PCSX2 and not in game's driver, I think that it cannot be something very custom.
//LE: something MPEG with these tables:

seta-san · 2020-05-31T20:24:42Z

Thank you guys for working g so hard on this. I think peripheral emulation is going to be one of the last biggest compatibility boosters!

I also have an eyetoy and singstar. If doing some sort of straight passthrough would help figure out what's the plugin issue and what's a core issue I'm here to test

SlyCooperReloadCoded · 2020-06-02T17:07:07Z

Yes, thank you. I'm looking forward to this, this is most likely the only thing keeping all the EyeToy games from immediately jumping to the Playable category.

Florin9doi · 2020-06-09T09:55:00Z

System and method for USB compatible I-Frame only MPEG image compression

Florin9doi · 2020-06-09T16:07:11Z

Frames conversion works now: Florin9doi@85ffc3c
example: https://i.imgur.com/2MZJcEM.mp4

seta-san · 2020-06-09T18:53:34Z

You guys are amazing

jackun · 2020-06-09T19:57:31Z

Play 2 is still just green background and keeps spamming reg 0x71 which i don't remember what it was for

[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x01 (1)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)
[USBqemu] [eyetoy_handle_control]:360	*** WRITE reg 0x71 = 0x00 (0)

Florin9doi · 2020-06-09T20:50:19Z

I know and I made a ticket @PCSX2/pcsx2#3432 but no reply so far.
The game receives the usb packets and send them to IPU for decoding, but the IPU fails.

seta-san · 2020-06-10T00:17:30Z

The eye toy has a couple led lights on the front of it that it controls. Might it be trying to access those?

jackun · 2020-06-10T04:09:50Z

Doh, it's the led.

SlyCooperReloadCoded · 2020-06-10T19:16:33Z

In this case, it's the misLED.

seta-san · 2020-06-10T20:35:38Z

Play 2 is still just green background and keeps spamming reg 0x71 which i don't remember what it was for

so it's basically bitching at you for a brighter room

Florin9doi · 2020-06-13T15:54:23Z

I pushed a new commit to capture video from a real webcam (v4l) but is noisy with many corrupted frames:

https://github.com/Florin9doi/USBqemu-wheel/commits/eyetoy
Works only if the webcam supports native: 320x240/YUYV. No resize/format conversion.

jackun · 2020-06-13T16:02:48Z

Extend/modify the API in videodev.h as needed as if this actually starts to work then the plan is to hide platform specific stuff in VideoDevice-inheriting classes.

Eyetoy v4l driver itself support YUYV? Keeps sending JPEG :D
I guess not

ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'JPEG' (JFIF JPEG, compressed)

Florin9doi · 2020-06-13T16:20:30Z

It doesn't. The frames should be decompressed.

Florin9doi · 2020-06-17T16:42:04Z

I fixed the frames corruption and I added jpeg support for EyeToy cams:

Florin9doi · 2020-06-17T16:48:58Z

What's left :

check if the image should be mirrored ?
support for windows
speed and quality optimizations ?

seta-san · 2020-06-17T16:57:16Z

What's left :

check if the image should be mirrored ?

support for windows

speed and quality optimizations ?

I can test on hardware to see if it needs to be mirrored. It might also be nice to have a working driver for the actual eyetoy on windows 10.

Florin9doi · 2020-06-17T17:00:58Z

It might also be nice to have a working driver for the actual eyetoy on windows 10.

it works disabling testsigning, installing the old driver and enabling testsigning again

seta-san · 2020-06-17T17:03:08Z

It might also be nice to have a working driver for the actual eyetoy on windows 10.

it works disabling testsigning, installing the old driver and enabling testsigning again

You mean driver signing?

Florin9doi · 2020-06-17T17:03:56Z

yes

seta-san · 2020-06-17T17:04:55Z

yes

Tried that

SlyCooperReloadCoded · 2020-10-30T21:10:21Z

Any progress on this? Still waiting for a fix for Antigrav. I also don't see the eyetoy branch anywhere.

seta-san · 2020-10-30T21:13:39Z

Any progress on this? Still waiting for a fix for Antigrav. I also don't see the eyetoy branch anywhere.

The two active branches are master and develop. Eye toy support is there. The problem seems to be in pcsx2's ipu code around certain color spaces and no one seems terribly anxious to fix it.

SlyCooperReloadCoded · 2020-10-30T21:20:47Z

Dang, that's the only reason I wanted EyeToy support honestly. If knew anything about programming I'd look into it, but I couldn't script or program to save my life.

Florin9doi · 2020-10-30T22:05:30Z

The problem seems to be in pcsx2's ipu code around certain color spaces

You have more information ?

seta-san · 2020-10-30T22:08:40Z

The problem seems to be in pcsx2's ipu code around certain color spaces

You have more information ?

I made a patch that got eyetoy play to show a picture by activating the rgb functions. The picture is fucked up and because of that the game isn't playable.

Florin9doi · 2020-10-30T22:09:44Z

where is the patch ?

seta-san · 2020-10-30T22:10:38Z

I'll send it to you when I get home

seta-san · 2020-10-30T22:11:46Z

Basically I sent the emulator from the ypbpr function to the rgb function

seta-san · 2020-10-30T22:15:01Z

where is the patch ?

Again. This is the video of it "working"
https://www.youtube.com/watch?v=oW4YexlmuCM

seta-san · 2020-10-31T08:16:33Z

i can't find the patch anymore. it's just a straight jump from 0015DE80 to 0015EB30 in Eyetoy Play NTSC.

All i'm doing is moving the game from this function.
ipu_decompressYCrCb__FPvPvP8sceImageP8sceImageP8sceImage

to this one.
ipu_decompressRGB__FPvPvP8sceImageP8sceImageP8sceImage

Florin9doi · 2020-10-31T10:08:21Z

Ok, I made a patch as you said for PAL :

Serial = SCES-51513
[patches = C0FCD1CE]
	// replace decompressYCrCb with decompressRGB
	patch=0,EE,0015cc80,word,16620033
[/patches]

The color issue is expected considering that it uses an incorrect function to decode the frames.
My observations are that:

"IPU1 running when IPU1 DMA disabled" isn't observed while the patch is applied.
ipu_decompressRGB function is 3 times shorter than ipu_decompressYCrCb : 0x1A8 bytes vs 0x4F4 bytes
ipu_decompressRGB contains a single sceIpuSync call, but ipu_decompressYCrCb contains 6 sceIpuSync calls
GameIndex.dbf contains a few sceIpuSync patches; sceIpuSync isn't properly emulated??

seta-san · 2020-10-31T10:16:59Z

there's something there. the plugin IS sending the data. I have no idea why the emulator isn't doing anything with it.

seta-san · 2020-10-31T10:28:57Z

i also found an eyetoy game that currently works and has debug symbols. it's Boboboubo Boubobo: Shuumare! Taikan Boubobo

https://wiki.pcsx2.net/Boboboubo_Boubobo:_Shuumare!_Taikan_Boubobo

SlyCooperReloadCoded · 2020-10-31T16:19:24Z

For some reason this no longer works on the latest PCSX2 builds, specifically the 1.7.0 dev builds. It detects my webcam like the 1.5.0 builds, but just gives this weird static color image instead of using the webcam:
https://imgur.com/a/p8gGeeS

jackun · 2020-10-31T16:22:16Z

@SlyCooperReloadCoded just try opening/closing config dialog or pausing/resuming PCSX2. There's some weirdness with DShow currently.

SlyCooperReloadCoded · 2020-10-31T16:26:00Z

Also, where's the latest plugin in the develop branch? I tried downloading an artifact, but it wasn't detected as a plugin in PCSX2.

Edit: tried pausing, resuming, and messing with the config menu. Still getting that weird screen.

jackun · 2020-10-31T16:47:34Z

FFS why can't you still trigger a branch build from appveyor ui, gaaah

@SlyCooperReloadCoded try these https://github.com/jackun/USBqemu-wheel/releases/tag/0.10.0-10

SlyCooperReloadCoded · 2020-10-31T20:46:37Z

Hm, that one doesn't have EyeToy. Isn't develop the branch with EyeToy?

jackun · 2020-10-31T20:54:19Z

Uh? I have only 32bit version installed and that works fine so check that you actually selected USBqemu-wheel-0.10.0-Win32 or USBqemu-wheel-0.10.0-x64

seta-san · 2020-10-31T20:58:03Z

0.10.0 release version has eyetoy in it but so do the master and develop branches. develop has the most up to date code in it and it works on 32 bit but crashes on 64bit

SlyCooperReloadCoded · 2020-10-31T21:01:30Z

I downloaded that develop version and EyeToy wasn't an option.

seta-san · 2020-10-31T21:04:02Z

here's the latest develop builds for 32 and 64bit
USBqemu-wheel-0.10.0- OCT 31 2020.zip

SlyCooperReloadCoded · 2020-10-31T23:05:36Z

Not only did that work, but it also fixed the static color image and it now works fine.

Also, I found another game that doesn't work with it yet. The PS2 version of Harry Potter and the Prisoner of Azkaban has EyeToy minigames. Similar to AntiGrav, it doesn't detect input. However, it also doesn't detect that an EyeToy is even plugged in, probably for that reason.

SlyCooperReloadCoded · 2020-11-06T01:55:22Z

Kinda unrelated, but PCSX2 is slowly working on integrating its plugins into itself, meaning no plugin selector. They're working with the devs of another plugin that adds network and HDD support, but I hope this is getting integrated too.

Florin9doi · 2021-02-20T21:44:48Z

OV519, register 0xA0 is setting the output format : 0x33=JPEG, 0x42=MPEG. Needed if we will support different formats.

			case VendorDeviceOutRequest | 0x1: //Write register
				switch (index)
				{
+					case 0xA0:
+						if (data[0] == 0x42)
+							Console.WriteLn("EyeToy : configured for MPEG format");
+						else if (data[0] == 0x33)
+							Console.WriteLn("EyeToy : configured for JPEG format; Unimplemented");
+						else
+							Console.WriteLn("EyeToy : configured for unknown format");
+						break;
					case R518_I2C_CTL:

OV7648, register 0x12, b6 (0x40) is setting the image mirroring. Now we always send a mirrored image, but some games ask for a straight image.

@@ -376,6 +403,14 @@ namespace usb_eyetoy
                                                        {
                                                                s->i2c_regs[reg] = val;
                                                        }
+                                                       if (reg == 0x12 && (val & 0x40))
+                                                               Console.WriteLn("EyeToy : mirroring ON");
+                                                       else if (reg == 0x12 && (val & 0x40) == 0)
+                                                               Console.WriteLn("EyeToy : mirroring OFF");
                                                }
                                                else if (s->regs[R518_I2C_CTL] == 0x03 && data[0] == 0x05)
                                                {

jackun · 2021-02-20T22:07:24Z

Of course datasheet skips 0xA0 👹

mirh · 2023-09-22T19:10:28Z

I'm going to assume MPEG might have been added after/while partnering with sony?
It is not mentioned whatsoever in the annoucement or datasheet of the bridge (only JPEG is ever highlighted), conversely you *can* find it in the press release and presentation of the eyetoy.

Florin9doi · 2023-09-22T20:20:37Z

Omnivision have had webcam controllers with some kind of JPEG support previously: ov511 and ov518. I assume ov519 was developed together with Sony from the beginning (they also made PS3 and PS4 webcams together), but the MPEG support wasn't advertised because the format is customised to take advantage of the PS2's IPU hardware: it adds some special headers intended for decoder, unrelated to Mpeg. Additionally, it wouldn't make sense to reuse the same name for a different product.

I don't have any other ov519 webcam but it's something which may be tested on one of many other products with the same controller. It was used even by Microsoft for its Xbox Webcam.

Florin9doi mentioned this issue Jan 12, 2021

[BUG] Bakufuu Slash!! Kizna Arashi PCSX2/pcsx2#4125

Closed

Florin9doi mentioned this issue May 3, 2021

[BUG] Motion Buttons Don't work PCSX2/pcsx2#4239

Closed

Eyetoy Support #30

Eyetoy Support #30

Comments

seta-san commented May 3, 2020

SlyCooperReloadCoded commented May 10, 2020

Florin9doi commented May 27, 2020 • edited Loading

jackun commented May 27, 2020

Florin9doi commented May 27, 2020

Florin9doi commented May 28, 2020 • edited Loading

Florin9doi commented May 31, 2020 • edited Loading

Florin9doi commented May 31, 2020

jackun commented May 31, 2020 • edited Loading

Florin9doi commented May 31, 2020 • edited Loading

seta-san commented May 31, 2020 • edited Loading

SlyCooperReloadCoded commented Jun 2, 2020

Florin9doi commented Jun 9, 2020 • edited Loading

Florin9doi commented Jun 9, 2020

seta-san commented Jun 9, 2020

jackun commented Jun 9, 2020

Florin9doi commented Jun 9, 2020 • edited Loading

seta-san commented Jun 10, 2020

jackun commented Jun 10, 2020

SlyCooperReloadCoded commented Jun 10, 2020

seta-san commented Jun 10, 2020

Florin9doi commented Jun 13, 2020

jackun commented Jun 13, 2020 • edited Loading

Florin9doi commented Jun 13, 2020

Florin9doi commented Jun 17, 2020 • edited Loading

Florin9doi commented Jun 17, 2020 • edited Loading

seta-san commented Jun 17, 2020

Florin9doi commented Jun 17, 2020

seta-san commented Jun 17, 2020

Florin9doi commented Jun 17, 2020

seta-san commented Jun 17, 2020

SlyCooperReloadCoded commented Oct 30, 2020

seta-san commented Oct 30, 2020

SlyCooperReloadCoded commented Oct 30, 2020

Florin9doi commented Oct 30, 2020

seta-san commented Oct 30, 2020

Florin9doi commented Oct 30, 2020

seta-san commented Oct 30, 2020

seta-san commented Oct 30, 2020

seta-san commented Oct 30, 2020

seta-san commented Oct 31, 2020

Florin9doi commented Oct 31, 2020

seta-san commented Oct 31, 2020

seta-san commented Oct 31, 2020

SlyCooperReloadCoded commented Oct 31, 2020

jackun commented Oct 31, 2020 • edited Loading

SlyCooperReloadCoded commented Oct 31, 2020 • edited Loading

jackun commented Oct 31, 2020

SlyCooperReloadCoded commented Oct 31, 2020

jackun commented Oct 31, 2020

seta-san commented Oct 31, 2020

SlyCooperReloadCoded commented Oct 31, 2020

seta-san commented Oct 31, 2020

SlyCooperReloadCoded commented Oct 31, 2020

SlyCooperReloadCoded commented Nov 6, 2020

Florin9doi commented Feb 20, 2021

jackun commented Feb 20, 2021

mirh commented Sep 22, 2023

Florin9doi commented Sep 22, 2023 • edited Loading

Florin9doi commented May 27, 2020 •

edited

Loading

Florin9doi commented May 28, 2020 •

edited

Loading

Florin9doi commented May 31, 2020 •

edited

Loading

jackun commented May 31, 2020 •

edited

Loading

Florin9doi commented May 31, 2020 •

edited

Loading

seta-san commented May 31, 2020 •

edited

Loading

Florin9doi commented Jun 9, 2020 •

edited

Loading

Florin9doi commented Jun 9, 2020 •

edited

Loading

jackun commented Jun 13, 2020 •

edited

Loading

Florin9doi commented Jun 17, 2020 •

edited

Loading

Florin9doi commented Jun 17, 2020 •

edited

Loading

jackun commented Oct 31, 2020 •

edited

Loading

SlyCooperReloadCoded commented Oct 31, 2020 •

edited

Loading

Florin9doi commented Sep 22, 2023 •

edited

Loading