Multiple JetKVM random restart USB PD issue

[toms-tech]

Disclaimer

• I have read and understood the disclaimer.

Application version

0.5.1

System version

0.2.5

Device model

JetKVM

Extension model

None

Remote device Hardware

No response

Remote device OS

No response

Bug description

I have 3 JetKVMs from the original batch. They seemed to have developed a random restart issue. After about 5 days of uptime, the units reboot by themselves and get stuck at Initializing.

Unplugging them does not cause the issue to go away. It seems like a USB PD issue since if I flip the USB power connector and plug it in the device fully boots. I can do a restart and it works fine.

Then after about 5 days the devices randomly reboot. These are on different USB power sources. The problem seems to have started on the 0.5.0 update as these all ran for mornths without issue. 0.5.1 exhibits the same issue. Im unable to roll back to.0.4.8 to test.

This affects all 3 units. The fact I need to flip the USB C cable and it works again points to somekind of USB PD firmware issue.

[adamshiervani]

Could you please describe your hardware setup? I haven't been able to reproduce this, and would love to know what gear you're using to see if I can trigger the issue here.

Could you also send your the diagnostics file? Settings > Troubleshooting Mode > Download Diagnostics

[toms-tech]

No problem!

In my first setup, it uses the JetKVM Y split cable, and the USB C power comes from a Unifi USP PDS Pro USB C port. The data part goes to a USB 3 Hub. I've also had the power plugged into a USB C 5 volt charger as well. Same result, a reboot and sitting at Initializing after a few days of uptime.

The second JetKVM, it's USB C to A cable plugging into a Lenovo PC Dock. I'm also had the USB plugged into a different dock. Same result, a reboot and sitting at Initializing after a few days of uptime.

The third, I've had it plugged directly into the back of a Lenovo P360, and now it's on a USB 3 Dock. Same result, a reboot and sitting at Initializing after a few days of uptime.

They have been stable in this configuration for months. I find if I unplug it, turn the cable, replug it, it starts up. If I just replug it, it will hang at Initializing. I can leave them off for 10 minutes and if I leave them plugged in the same way, it will sit at Initializing. Change the cable orientation, sure enough it boots just fine. If I switch it back to plugging it in the old way, it will now work. That's why it seems like a hardware issue to me.

I tried to create the Diagnostics download, but each time I do, each KVM crashes, and reboots. The one I was testing started in fial safe mode, and I tried to update it back to 4.8.0, it crashed, and now sits at initializing no matter what I do.

Is there any warranty or RMA?

[toms-tech]

Managed to DFU to get it back and up and running. I can confirm it's a software issue with the PD controller in the firmware, as the DFU mode works every time, no matter the 5v USB source. I'm going to try to stay on 0.4.8 for this particular unit and see if the problem continues.

[Galbar]

I just realized I have the same issue. I installed the my jetkvm, with the ATX Power Control card, about 3-4 weeks ago. I set it up, did some tests, loved it and left it there as a way to recover the computer in case something went wrong.

Well, today something went wrong and, when I went to connect to it, it wasn't available. Lucky enough today I was able to access the machine, which is not always the case and the reason why I got the kvm in the first place.

The jetkvm was stuck with the screen saying "initializing". I had to unplug both the RJ12 and the USB going to the PC so that it would power off, waited a bit and then plugged it back and it immediately booted.

I then went to check my router's logs. It was first plugged on Dec 27th and then again on Jan 1st (I was toying with the setup). After that, it worked well until Jan 16th. From then on I see the following pattern: disconnect -> ~10 minutes -> disconnect -> 12 hours -> repeat.

It is running version 0.5.1. It is not connected to the internet (I access it through a VPN). I tried downloading the diagnostics but it also crashes and reboots.

I'd like some guidance on how to make the device stable. I won't always be able to go and unplug it and now I don't know that I can rely on it for when 💩 hits the fan.

[Galbar]

I saw that there is a developer mode and ssh'd into the device. Looking at the code in the repo I found the folder /userdata and inside of it /userdata/jetkvm/crashdump. I'm attaching the two relevant files.

Notice that the dates (except for the year) mostly align because I last plugged it on Jan 1st. I had a quick look. You will see in the logs that it tries many many times to timesync (it never succeeds because it has no access to the internet) and auto_update_enabled=false.

The main thing that stands out to me is that the first log file stops on the same day that the repeated disconnecting pattern logged by the router start and that the next and last logfile has the timestamp of 47 minutes later and only has a single line. Nothing else is logged between that and today when I unplugged it and re-plugged it, which caused logs to be logged with date 20210101 again. I'm not including those because they don't seem relevant.

Another interesting thing is that the longer log file is (almost) precisely 100MB. Maybe it hit some limit? I don't know the system, just mentioning everything I see in case it helps 😄

jetkvm-20210116-125640.log.gz
jetkvm-20210116-125727.log

[toms-tech]

This was happening to all of my JetKVM's.

If you open up the SSH shell, do a "df -k" are any of the partitions full? Aka very little space left?

I managed to get all of my JetKVM's stable again, by factory resetting them using the instructions, and getting it on to 0.5.2 and 0.2.7. So far it's been good. I have noticed that JetKVM does not do a great job at cleaning up disk space. I had a few issues previously to this where userdata got full from logs and upgrades would fail. I guess in testing the units are not running for long periods so I think they missed disk clean up.

The factory wipe and reinstall resolved my issues. You might want to try that too.

[Galbar]

I just checked and my device is not full:

I didn't have to factory reset it to get it working again, just unplug it from power and plug it back. 🤔

[Galbar]

It just happened again. I wanted to connect to it and it was unavailable. Went to the server room and it said "Initializing" but nothing else happening. I had to unplug both the USB-C and RJ12 for it to turn off, I had to repeat the process twice for it to boot and show the UI on screen.

Interestingly enough, I have an automated monitor pinging the device every ~5 minutes and it said that it was healthy, meaning it was returning pings.