basic usb hub and mouse/keyboard handling

[BigFishDev22]

Adds handling for USB hub and mouse and keyboards events. Events are passed along to ps/2 controller. Tested using Anker - 4-Port USB 3.0 Hub with 5Gbps Data Transfer, Ultra-Slim Data USB C Hub. Mouse and keyboard were Amazon basics brand hardware.

Tested in windows 3.1 image, created using pc version of the tiny386.

USB HID code is modified from ESPIDF HID host example project.

[hchunhui]

Great!
It seems the code allocates 8KB+ stack, so I will do some tests to see if it uses too much memory. From my experience, the system will crash randomly due to main memory shortage.

[hchunhui]

Unfortunately the driver doesn't recognize my usb wireless mouse. I will try more usb hid devices.

Another problem is the wifi and the usb cannot work at the same time. I think it's due to memory shortage.
And the speed optimizations part breaks the sound.

So I think perhaps we need a option such as "enable_usb" in ini file, which is disabled by default. And the speed optimizations patch may be reverted. After that the PR will be ready to merge, and we will receive feedback from more users.

[BigFishDev22]

Unfortunately the driver doesn't recognize my usb wireless mouse. I will try more usb hid devices.

Another problem is the wifi and the usb cannot work at the same time. I think it's due to memory shortage. And the speed optimizations part breaks the sound.

So I think perhaps we need a option such as "enable_usb" in ini file, which is disabled by default. And the speed optimizations patch may be reverted. After that the PR will be ready to merge, and we will receive feedback from more users.

I think the USB issue would possibly be the mouse not supporting USB boot protocol.

Im thinking the stack size for usb can be reduced from the defaults.

The speed optimizations were mostly me just testing and we're meant to go to a branch. It might be possible to apply the optimizations to some source files and leave things like sound alone. But that probably just means stuff in the background breaking in unexpected ways that aren't obvious. Maybe a flag to choose between faster but less accurate emulation?

[hchunhui]

Unfortunately the driver doesn't recognize my usb wireless mouse. I will try more usb hid devices.
Another problem is the wifi and the usb cannot work at the same time. I think it's due to memory shortage. And the speed optimizations part breaks the sound.
So I think perhaps we need a option such as "enable_usb" in ini file, which is disabled by default. And the speed optimizations patch may be reverted. After that the PR will be ready to merge, and we will receive feedback from more users.

I think the USB issue would possibly be the mouse not supporting USB boot protocol.

Im thinking the stack size for usb can be reduced from the defaults.

I have tried reducing the stack size, but it doesn't work.

The speed optimizations were mostly me just testing and we're meant to go to a branch. It might be possible to apply the optimizations to some source files and leave things like sound alone. But that probably just means stuff in the background breaking in unexpected ways that aren't obvious. Maybe a flag to choose between faster but less accurate emulation?

I agree.

[BigFishDev22]

I have tried reducing the stack size, but it doesn't work.

Yeah, after some testing I was only able to get it to run with 3072 for the usb_stack task and still needed 4096 for the host driver.

So I think perhaps we need a option such as "enable_usb" in ini file, which is disabled by default.

It looks like the frambuffer, wifi, and usb use up too much DMA ram, so I agree that an option for one or the other is the best way to go. The ESP-32 P4 appears to have much more ram that is DMA capable, so it will likely work there, especially since the wifi is offloaded to another module. I've never used the P4, so im not sure how integrating the current wifi implementation will look.

It'd be interesting to do the same with an S3 to offload the wifi. A dev board built around this project could be interesting, including a usb hub, wifi and ethernet.

[RPDevJesco]

I took a look at the source code and from what I gather (without running it on my own device), the source of the problem with USB not working with WIFI is due to:

#define MINCSIZE 4096

Every time a network buffer needs to grow, it allocates at least 4KB more.
If IF_MTU is 1500 (standard Ethernet), every mbuf starts at ~1.5KB. Then grows by 4KB increments.

If you reduce MINCSIZE to 256 or even 128, USB and wifi should not cause too much DMA ram to be used anymore and allow for USB to be used with WIFI.

[hchunhui]

I took a look at the source code and from what I gather (without running it on my own device), the source of the problem with USB not working with WIFI is due to:

#define MINCSIZE 4096

It's not true. The macro belongs to the "slirp" library. The library is used in PC version, for user networking, and it's not used in esp port.

[RPDevJesco]

Once I get my own ESP32-S3, I'll pull the source code and do my own testing to see if I can't figure out a proper fix. Like I said in my original comment, it was an educated guess based off the discussion and behaviors described.

[SoftwareGuy]

Alright, so apparently when I've built and flashed this to my device, it seems that something to do with the USB is causing it to get in a reset loop, until I unplug the Simplecom Keyboard & Mouse combo (which is like a TV remote controller shaped thing) and then plug it back in after the SeaBIOS appears. I unfortunately can't get a debug log because the /dev/ttyACM0 port disappears when the ESP32 disconnects.

The hub I'm using is actually a Nintendo Switch USB-C to HDMI out, USB in and USB-C PD in hub. Not optimal, but it works since I got the screen kit that only has 1 USB port.

Pre-post EDIT: It seems if I plug in the USB devices AFTER the BIOS initializes and it starts loading Windows, then the USB is stable. I have no idea, is it because of unhandled events as soon as usb setup is called?

EDIT: I noticed that the top of the screen had some blue-like corruption lines when the desktop was loading. Sound was very choppy, like the CPU speed was very low. Eventually the mouse cursor moved a bit and then it seemed to just lock up or something.

EDIT 2: Could the lockup/crash be due to running out of memory, and one of the processes just hangs? It doesn't seem to reboot, it just locks up.

[SoftwareGuy]

This is potentially bumping an old thread but after modifying the code to use poll timeouts - as some of my USB wifi keyboard/mouse combo devices could hang the whole emulator and cause it to fail to catch back up - the USB code seems to be decently stable.

The USB devices causing a ESP32 reset is an order of operations. We set up the USB stuff when other stuff is still NULL. By doing null checks, the emulator initializes and then the USB code plugs in the keyboard and mouse logic. I understand this PR is only a prototype, but with the null checks in place, ejecting and replugging keyboard/mouse work fine.