Compile for VMware ESXi

[ryandesign]

Your hwmond works great on macOS but I run VMware ESXi 6 on my Xserves and it would be great to be able to run hwmond there instead. ESXi 6 includes libusb so all that's needed is to compile hwmond for ESXi, but of course ESXi doesn't ship with a compiler. Do you have the ability to produce a binary of hwmond for ESXi and add it to the downloads of your latest release? Or do you have any guidance for how such a binary could be compiled? I might try to figure it out myself if you don't know but I thought I'd ask first.

[castvoid]

Hi,

I actually had a go at getting the project to run on ESXi myself. The project should compile, link, and run just fine on Linux, but I couldn't get it to run on ESXi.

I don't have an ESXi setup to try this out on right now, but I'd love to hear about it if you get it working! You'll either need a similar Linux environment (CentOS?), or an appropriate cross-compiler. I found this article online that might be helpful: https://www.vm-help.com/esx40i/esx41/developing-for-vmware-esxi

Best,
Harry

[Fingerz]

I have a spare Xserve front panel and have always toyed with the idea of using a Raspberry Pi to drive the front panel to simulate activity. I would also like to try that for one of my Xserve RAIDs that was water damaged in a past life. That could be another port of interest!

[osxster]

I have compiled this successfully on OS X, but I was thinking of trying to compile it on Linux or BSD to see if coping the binary over to ESXi would work. Has anyone successfully compiled it on Linux? What flavor? I am trying to compile it on BSD right now, but I am having problems with missing libraries, namely mach_host.h, once I installed every library dependency I was missing.

/root/xserve-frontpanel-master/cpu_usage.c:5:10: fatal error: 'mach/mach_host.h' file not found

I am not an expert in this field, but it seems that this header file is part of the Mach kernel which seems to only be used on MacOS. If that is the case, can this be compiled on Linux or BSD without replacement header files which are compatible? Any information would be great, thanks!

[ryandesign]

The project should compile, link, and run just fine on Linux

It appears that this is not true of most Linux distributions, in the code's current form, because it unconditionally includes headers and calls functions that are specific to Mach microkernel operating systems like macOS/Darwin (and apparently GNU Hurd). These will need to be made conditional on that (i.e. #ifdef __MACH__) and alternative implementations will need to be created for operating systems that are based on other kernels.

Linux for example uses the Linux kernel. #8 was filed requesting Linux support, and parsing /proc/stat is apparently the way to get the information there, but as a sanity check before writing any code I looked on my ESXi system and although it has /proc is does not have /proc/stat so we need a different method. ESXi does not use the Linux kernel; it uses the proprietary VMkernel.

I found a way to get CPU usage percentages from ESXi by using the undocumented vsish command to show nodes within the power/pcpu tree. I explain how in this Stack Overflow answer. This information is not in the same form as it is in /proc/stat on Linux nor via host_processor_info on Mach. There may be more closely matching information elsewhere in the vsish node tree that I haven't found yet.

I hate to have to run an executable every time I want to get current CPU usage data. That's got to incur some overhead. That executable must be calling some function or getting the information from the kernel somehow. If only we knew how and could use that same method. Unfortunately the VMkernel is closed source so we may never know.

Because we don't have a compiler on ESXi and would be compiling for ESXi on another OS, we could not detect the OS at build time; we would have to detect it at runtime, perhaps by using uname. But we would only have to do so either when compiling for ELF format (though I'm not sure how to know that programmatically) or just when compiling on Linux (I'm not sure which of the two).

Before spending time implementing ESXi-specific methods of getting real CPU data, a simpler test would be to conditionalize all the Mach stuff and just use fake data on Linux. Then see if that executable will work on ESXi. If so, further work can be done.

[ryandesign]

we would have to detect it at runtime

or offer a compile time switch in CMakeLists.txt. That's probably simpler.

[osxster]

If you had time to take this up, that would be pretty cool. I get the point about not wanting to run an executable every time this runs and views the CPU status, but even running that less often would produce CPU stats that would be available in the front panel. I'd be happy with that as right now I have nothing! I was considering running this on an OSX VM within ESXi and pass the USB through, but that isn't really the right way to do this.

I guess a start would be trying to compile the executable in linux to just control the lights (without reading in CPU status), see if that executable will run on ESXi and see if it will control the lights. If that works, either pass the stats through an executable (I get not preferred) or figure out a better way to read the stats natively.

[ryandesign]

I was considering running this on an OSX VM within ESXi and pass the USB through, but that isn't really the right way to do this.

This works but of course only shows you CPU activity of that VM.

I guess a start would be trying to compile the executable in linux to just control the lights (without reading in CPU status), see if that executable will run on ESXi

Here's what currently happens when I run my WIP Linux version (#8) on ESXi 6.0.0U3:

libusb: 0.000000 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0202 is read-only; cannot determine active configuration descriptor
libusb: 0.000107 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0102 is read-only; cannot determine active configuration descriptor
libusb: 0.053320 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0701 is read-only; cannot determine active configuration descriptor
libusb: 0.053464 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0601 is read-only; cannot determine active configuration descriptor
libusb: 0.105622 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0501 is read-only; cannot determine active configuration descriptor
libusb: 0.158221 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0401 is read-only; cannot determine active configuration descriptor
libusb: 0.212660 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0301 is read-only; cannot determine active configuration descriptor
libusb: 0.212802 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0201 is read-only; cannot determine active configuration descriptor
libusb: 0.212915 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0101 is read-only; cannot determine active configuration descriptor
libusb: 0.215910 warning [usbfs_set_configuration] failed to update cached config descriptor, error -5
Couldn't claim interface: LIBUSB_ERROR_NOT_FOUND

So on the plus side the executable is in the right format, it finds ESXi's libusb library, and runs. But on the minus side it can only read from USB devices not write to them (something I also encountered under Linux but which I solved by adding an entry to /lib/udev/rules.d/ but a different solution will be needed for ESXi since ESXi doesn't appear to use udev) and it can't "claim" the USB device (which I also see when running under Linux and haven't solved there yet either).

[ryandesign]

The read-only device messages turned out to be irrelevant because they refer to other USB devices, not the frontpanel device we're looking for.

I now have random LEDs working in Ubuntu Server 20.04 and ESXi 6.0.0. Next step will be to implement real CPU activity data capturing for Linux and then for ESXi.

[osxster]

This is really great news! Now will this work with ESXi 6.5? That is what I am running on my xServe. Not sure under the covers what is different between 6.0 and 6.5 and if you foresee any issues there. I'm running a 2009 xServe 3,1, which the processors support up to 6.5.

[ryandesign]

I haven't upgraded to ESXi 6.5 because I read somewhere that the internal Apple SSD would no longer be supported with that version, which was relevant to me until mine died last month. But I don't foresee any reason why this program wouldn't also work on other ESXi versions. You're welcome to try it: I've uploaded the ESXi binary here temporarily (or you can compile it yourself from my linux branch).

[osxster]

I tried running the executable, getting descriptor errors trying to use libusb. I am running it as root, so it isn't a permissions problem...

./hwmond-esxi
libusb: 0.000000 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0702 is read-only; cannot determine active configuration descriptor
libusb: 0.006344 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0701 is read-only; cannot determine active configuration descriptor
libusb: 0.006454 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0601 is read-only; cannot determine active configuration descriptor
libusb: 0.006558 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0501 is read-only; cannot determine active configuration descriptor
libusb: 0.006661 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0401 is read-only; cannot determine active configuration descriptor
libusb: 0.006765 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0301 is read-only; cannot determine active configuration descriptor
libusb: 0.006868 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0201 is read-only; cannot determine active configuration descriptor
libusb: 0.006976 warning [initialize_device] access to /vmfs/devices/char/vmkdriver/usb0101 is read-only; cannot determine active configuration descriptor
libusb: 0.032163 error [usbfs_kernel_driver_active] get driver failed error -1 errno 22
Couldn't determine if a kernel driver is active: LIBUSB_ERROR_OTHER

[ryandesign]

Oh, that kernel-driver-checking code I added seemed not to be relevant after all. I've commented it out and recompiled. Please try downloading the revised version.

[osxster]

Still get the warnings, but I get some funky moving CPU lights now on ESXi 6.5! Very good so far!

[osxster]

Curious if there was any progress in this. If there are no ways to easily pull CPU data via the kernel in ESX, I would be fine actually with a version of this which accepts a pipe. It wouldn't be difficult at all to execute it every minute via cron for example and pipe in the CPU data from maybe esxtop or some other way. While the CPU panel will only update every minute (or more if coded in a shell script loop of some type), it would at least work. Any thoughts on that?

[mav2287]

Also curious to see if any progress was made on this.

[ryandesign]

Well as you know I made a number of commits in branches in my fork to work on implementing Linux support. In the course of doing this I had to do some refactoring and I fixed a few bugs. I did not want to submit a PR containing all these unrelated changes at once, nor did I want to submit many PRs at once since many of the changes are interdependent I am not sufficiently skilled at using git to be able to submit a set of interrelated PRs. So I submitted PRs for just two of the simplest non-Linux-support-related fixes in #9 and #10. I had hoped that the owner of this repository would quickly merge those PRs as obviously correct. I would then have submitted other PRs for additional self-contained small changes, discussed any concerns with the repository owner, and hopefully have ended up with Linux support merged into the repository, and I was planning to continue to investigate implementing ESXi support and then getting that merged as well. However, the repository owner's complete lack of response to the simplest of changes gave me no hope that any further changes would be regarded either. Therefore I did not pursue further work. Alternately, I could consider this repository dead and continue the work in my fork, and I may do that if I return to this project some day.

[mav2287]

I think you should consider this dead and just fork it. You will then be able to push in your changes. Based on my understanding this was a pet project repo that once it was up and running didn't really get much more attention. If you need help with anything just tag me and I will try to help with whatever you need. I have hit the point with my XServe 3,1s where ESXi is the only option other than retirement for them so I would love to have this working for ESXi 6.5.