Server shuts down randomly

Software & Operating Systems Linux
,
1

I have a server running ubuntu server. Occasionally when I check on it, it is not powered on. I have kept it up to date and It is running the 5.8.0 kernel. I am not even sure where to start looking. It used to remain on with no issues, so I am not sure what changed.

Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.8.0-050800-generic x86_64)

System information as of Wed 09 Dec 2020 11:47:59 AM PST

System load: 0.0
Usage of /: 10.8% of 108.79GB
Memory usage: 11%
Swap usage: 0%
Temperature: 16.8 C
Processes: 318
Users logged in: 0

0 updates can be installed immediately.
0 of these updates are security updates.

OS: Ubuntu 20.04.1 LTS x86_64
Kernel: 5.8.0-050800-generic
CPU: AMD Ryzen 5 3600X (12) @ 3.800G
GPU: NVIDIA GeForce GT 710
Memory: 1752MiB / 16016MiB

2

Check the BIOS for accidentally set shut-off times. Check system health (too hot -> shutdown). Install remote management tools like Webmin, check various settings related to shutdown. Check logs (/var/log/) for unusual entries. Run rkhunter to check if rootkits have taken over your system. Look at dmesg for entries you can’t explain.
sudo dmesg | tail

2 Likes
3

Another possibility is power supply or motherboard VRM failure. If those capacitors fail anything weird can happen. Including sudden power off.

1 Like
4

Could the system also be trying to go into a “low power” mode while quiet, which is not supported by the O/S which just makes it unresponsive?

5

Dmesg would be fresh each boot, no?

So would not explain what happened last time, if the computer needed to be rest.

(from the man page) Show all kernel logs from previous boot:

     journalctl -k -b -2 -r

(the -2 chan be changed to how many times ago the system was booted. the -r is to reverse order, so last message put first)

might be useful, otherwise have a poke around the /var/log for anything usedful.

6

not sure anything here looks interesting https://paste.ubuntu.com/p/7bRDXkGcxm/

Thanks for all of the suggestions, I will take another look at it when I have more time to debug.

1 Like
7

Ah ok so it’s not shutting down, but it is suspending for some reason. when I was SSHed into it, it stopped responding. still displayed the prompt, but could not type anything in. when I hit the power button I regained the ability to type and the characters I was typing from before I hit the power button began displaying in addition to the prompt.

8

You did not say if it was headless or not. If it is not, then turn that off in power settings. you should be fine. Some systems still have issues with hibernate and suspend due to non-compliant ACPI implementations.

1 Like
9

Brownouts can cause that kind of behavior. Even with a UPS if it’s got a heavy load. Yeah they’re not supposed to allow that but I’ve had one with a battery that was failing which it haddent alerted to end up with that sort of behavior.

10

I VNCed in and changed the screen shutdown time from 5min to never and it already seems to be staying on for longer. Has not shutoff yet for a few hours. I will check it periodically to see if anything changes.

1 Like
11

Sorry, those loud mist-horns you’ve been hearing are my alarm-bells… :woozy_face:

If I read you right, you actually have a desktop environment installed on your server. That’s not really a good idea. For the reasons you’ve experienced: it interferes with server operations. As well as being a security risk, that is. Any service you run on your server is a potential attack vector for a hacker. That includes desktop stuff. Make sure to purge it all from your system.