Serial over ESPUSB

[leopck]

Hi sorry for creating double issue (Joystick over ESPUSB) but I didn't want to mix these two projects together.

I thought of sharing these with others, just in case someone here is interested.
I've been attempting serial over espusb as how I've seen DigiSpark did it using Low-Speed CDC. So I tried replicating their method as well. I have only did the change to the usb_config.h for the moment.

usb_config.h (https://gist.github.com/leopck/0d3d1e38a11f3742f0a86bdcb3d88740)

So after multiple trial and errors I somehow got something on my 'dmesg'

[ 1331.813737] usb 2-1.2: Product: ESPUSB2
[ 1331.813741] usb 2-1.2: Manufacturer: CNLohr
[ 1331.838176] cdc_acm 2-1.2:1.0: ttyACM0: USB ACM device
[ 1331.838974] usbcore: registered new interface driver cdc_acm
[ 1331.838981] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters

Which looks like good news! Until I see more messages below:

[ 1331.845052] ------------[ cut here ]------------
[ 1331.845068] WARNING: CPU: 7 PID: 862 at /build/linux-a2WvEb/linux-4.4.0/drivers/usb/core/urb.c:449 usb_submit_urb.part.6+0x142/0x560()
[ 1331.845072] usb 2-1.2: BOGUS urb xfer, pipe 3 != type 1
[ 1331.845075] Modules linked in: cdc_acm cp210x usbserial rfcomm drbg ansi_cprng ctr ccm bbswitch(OE) bnep nvidia_uvm(POE) nvidia_modeset(POE) nvidia(POE) arc4 snd_hda_codec_hdmi snd_hda_codec_realtek ath9k snd_hda_codec_generic intel_rapl ath9k_common x86_pkg_temp_thermal uvcvideo ath9k_hw snd_hda_intel snd_usb_audio intel_powerclamp videobuf2_vmalloc coretemp videobuf2_memops snd_usbmidi_lib kvm_intel snd_hda_codec ath videobuf2_v4l2 snd_hda_core mac80211 videobuf2_core snd_hwdep btusb kvm v4l2_common snd_pcm btrtl snd_seq_midi videodev btbcm snd_seq_midi_event btintel snd_rawmidi media bluetooth cfg80211 snd_seq irqbypass crct10dif_pclmul snd_seq_device crc32_pclmul snd_timer jmb38x_ms memstick mei_me snd joydev input_leds mei soundcore shpchp lpc_ich serio_raw cryptd binfmt_misc ideapad_laptop
[ 1331.845172]  sparse_keymap mxm_wmi wmi mac_hid parport_pc ppdev lp parport autofs4 hid_generic usbhid hid i915 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops psmouse sdhci_pci ahci drm sdhci tg3 libahci ptp pps_core video fjes
[ 1331.845215] CPU: 7 PID: 862 Comm: ModemManager Tainted: P           OE   4.4.0-36-generic #55-Ubuntu
[ 1331.845220] Hardware name: LENOVO 20090                           /Base Board Product Name, BIOS 47CN28WW(V2.06) 06/14/2011
[ 1331.845224]  0000000000000286 00000000aa160979 ffff8802097fba08 ffffffff813f13b3
[ 1331.845231]  ffff8802097fba50 ffffffff81d486b8 ffff8802097fba40 ffffffff810810f2
[ 1331.845236]  ffff88010c30c0c0 0000000000000003 ffff88017a991800 0000000000000000
[ 1331.845242] Call Trace:
[ 1331.845255]  [<ffffffff813f13b3>] dump_stack+0x63/0x90
[ 1331.845263]  [<ffffffff810810f2>] warn_slowpath_common+0x82/0xc0
[ 1331.845269]  [<ffffffff8108118c>] warn_slowpath_fmt+0x5c/0x80
[ 1331.845284]  [<ffffffff81618802>] usb_submit_urb.part.6+0x142/0x560
[ 1331.845289]  [<ffffffff81618c82>] usb_submit_urb+0x62/0x70
[ 1331.845297]  [<ffffffffc073020b>] acm_submit_read_urb+0x3b/0x80 [cdc_acm]
[ 1331.845304]  [<ffffffffc07330b6>] acm_port_activate+0x186/0x1e8 [cdc_acm]
[ 1331.845310]  [<ffffffff814f3a7e>] tty_port_open+0x8e/0xe0
[ 1331.845317]  [<ffffffffc0730d03>] acm_tty_open+0x33/0x60 [cdc_acm]
[ 1331.845324]  [<ffffffff814ebfff>] tty_open+0x11f/0x6b0
[ 1331.845331]  [<ffffffff81211b8f>] chrdev_open+0xbf/0x1b0
[ 1331.845337]  [<ffffffff8120acef>] do_dentry_open+0x1ff/0x310
[ 1331.845341]  [<ffffffff81211ad0>] ? cdev_put+0x30/0x30
[ 1331.845346]  [<ffffffff8120be84>] vfs_open+0x54/0x80
[ 1331.845352]  [<ffffffff81217aeb>] ? may_open+0x5b/0xf0
[ 1331.845358]  [<ffffffff8121b667>] path_openat+0x1b7/0x1330
[ 1331.845365]  [<ffffffff8121d9d1>] do_filp_open+0x91/0x100
[ 1331.845371]  [<ffffffff8122b266>] ? __alloc_fd+0x46/0x190
[ 1331.845377]  [<ffffffff8120c258>] do_sys_open+0x138/0x2a0
[ 1331.845383]  [<ffffffff8106b544>] ? __do_page_fault+0x1b4/0x400
[ 1331.845389]  [<ffffffff8120c3de>] SyS_open+0x1e/0x20
[ 1331.845395]  [<ffffffff8182dfb2>] entry_SYSCALL_64_fastpath+0x16/0x71
[ 1331.845400] ---[ end trace 53721cd36948f595 ]---

This issue is similar to an issue from DigiSpark (digistump/DigistumpArduino#16) but it wasn't resolved at the end. Any ideas?

Also, there's another option which is to use v-USB HID Serial as shown here (http://rayshobby.net/hid-class-usb-serial-communication-for-avrs-using-v-usb/) but I haven't attempted this.

@cnlohr I was reading thru how you sent data over the espUSB and from the file 'execute_reflash.c' I saw that you used libusb_control_transfer to transfer pieces of the software over. Is it possible to reuse this for serial?

[cnlohr]

You have gotten MUCH MUCH further than I did!!!! I tried doing this and all it would tell me was "not enough bandwidth" errors over and over. Are you using a bulk endpoint? Are you somehow getting CDC to work over an interrupt endpoint? If that works, any chance at a RNDIS device working?

You can have whatever custom commands you want with the control_transfer messages... Totally up to you!

[leopck]

Oops I forgot to get back to this message... I'm sorry
I've been steadily going back and forth with this Serial over ESPUSB due to work constraint.

First, I can't get this to work on bulk endpoint. The dmesg doesn't recognize it. However, using interrupt endpoint, I could get some results but lots of kernel warning and errors. I've read from a thread forum somewhere that the newer Linux kernel doesn't support low-speed CDC anymore.

I haven't attempted the HID-serial but this seems to be a very good method to attempt if low speed CDC really doesn't work. I'm going to test this many more times before I disregard this method. Hopefully, this works :D

What's RNDIS? virtual Ethernet?
Wiki states "Remote Network Driver Interface Specification (RNDIS) is a Microsoft proprietary protocol
used mostly on top of USB"
Isn't this a proprietary protocol? Do we know how it looks like?

[cnlohr]

(1) I always used control messages instead of HID, but if by some miracle you can get it to enumerate as a serial device, I suppose we could do it. Control messages with libusb are really beneficial because of two reasons (a) they handle all framing, you basically say "send this command, with this code and this payload" or "send this command and give me the response buffer" and it goes and does it... and (b) it's fast!

(2) RNDIS: Yes, virtual ethernet. LUFA supports it. It's how I made my Minecraft server that fits in a USB port. https://www.youtube.com/watch?v=YNrFOClrzTA

(3) I still don't know if the ESP32 can do USB full speed, but hopefully around Christmas time I'm going to have an opening to give it a whirl.

[leopck]

(1) Wow, I'll definitely look thru the control message more now. Sounds promising.

(2) That's some very interesting thing you did there. It's amazing how LUFA did it, what is needed to get RNDIS to work? Isn't LUFA based on hardware USB? But I guess it's always worth a try to make the impossible possible :)

(3) I saw nosdk8266 and 346MHz ESP8266. Is it possible to run full speed USB on that? It's exciting to see you work. ESP32 full speed is really somethg to look forward to. Let me know if I could be of help in any way possible since I've also received my ESP-devkitC from Espressif.

[cnlohr]

(2) I recommend not using LUFA since it is not nearly as lightweight as their name implies :-p. LUFA is meant for hardware stacks. It might be possible to use in our situation bout would be difficult. If full-speed works, it wouldn't be difficult to get RNDIS running. But I agree serial matters most, first.

(3) I do suppose it may be "technically" "maybe" possible... But it would take re-jiggering the USB stack. It would take a fair bit of hand-coding. 346 MHz is 4.325x faster than 80 MHz. Full-speed is 8x faster than low-speed.

I can't find the thing that said how many cycles it takes to read from the GPIOs. So I don't know for sure it would be possible. I seem to recall "19" cycles being thrown around which would make it a tight pinch if at all possible :-/.

MORE IMPORTANTLY: Would this /actually/ be useful? Since wifi cannot be used at all when overclocking. And, if we used it during the bootloader phase, we'd have to come up with some way of "disconnecting" the pullup and attaching it to the other leg, to switch between full and low speed.

[leopck]

(3) I gave myself a few days to think of an idea on what to use full speed on the bootloader.... If full-speed is not attainable on the 'normal' SDK maybe, what this could do is that each time you want to use full speed, the esp8266 will reboot to bootloader and perform the USB full speed task and come back to WiFi SDK again.

Use case:

1. Cloud Printer/Cloud 3D Printer?

• User access ESP8266's website and uploads .txt file
• ESP8266 saves txt file in SD card/Flash
• Reboots to bootloader
• Connects to USB full-speed on bootloader to printer
• Sends txt file to printer.
  (Okay, I know each printer has their own driver and method of communication but this is just an example)

[cnlohr]

Man... I don't know why OSes don't permit all the endpoints at low speed :(. Fingers crossed for time around Christmas to give full-speed a go.