April 2024
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
2930  

Categories

April 2024
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
2930  

Auto reboot after kernel panic

When a kernel encounters certain errors it calls the “panic” function which results from a unrecoverable error. This panic results in LKCD (Linux Kernel Crash Dump) initiating a kernel dump where kernel memory is copied out to the pre-designated dump area. The dump device is configured as primary swap by default. The kernel is not completely functional at this point, but there is enough functionality to copy memory to disk. When the system boots back up, it checks for a new crash dump. If a new crash dump is found it is copied from the dump location to the file system, “/var/log/dump” directory by default. After copying the image, the system continues to boot normally and forensics can be performed at a later date.

By default after a kernel panic, system just waits there for a restart.  This is because of the value set on “kernel.panic” parameter.

# cat /proc/sys/kernel/panic
0

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer value greater than zero to the paramter “kernel.panic“, where the value is the number of seconds to wait before a automatic reboot.  For example , if you set it to “10” , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and add following like at end of the file.

kernel.panic = 10

Linux is a robust and stable operating system kernel, but there are instances where it can panic, be it due to bad hardware or bad software. It does not happen often, but it can happen.

If you’re running a server or some other always-on system that you may not have easy access to, a kernel panic typically means an inconvenient trip to reboot a system or a phone call to inconvenience someone else. You can, however, configure Linux to automatically reboot on a kernel panic by making a small modification to /etc/sysctl.conf, a configuration file that tweaks many kernel operating parameters.

Add the following to /etc/sysctl.conf:

kernel.panic = 20

This tells the kernel that if it encounters a panic, it is to reboot the system after a 20 second delay. By default, the kernel will never reboot when it encounters a panic, but with the above setting you can force it to.

Of course, if you enable this, make sure you are using swatch or some other means of observing log files to make sure you aware of the fact when the system panics so you can take appropriate steps to correct the problem.

On local systems, it is also convenient to be able to reboot the system with a key-press in the case of a panic. Instead of having the system reboot automatically on a local system, consider using the magic SysRq keys to reboot your system if X locks up or keyboard entry is being ignored.

To enable magic SysRq support, you must again edit /etc/sysctl.conf; some Linux distributions have this enabled by default whereas others do not.

kernel.sysrq = 1

If the time comes when the SysRq keys are required, use the magic SysRq combination, which is: [ALT]+[SysRq]+[COMMAND], where the [SysRq] key is the “print screen” key and [COMMAND] is one of the following:

  • b – reboot immediately without syncing or unmounting disks
  • e – sends a SIGTERM to all running processes, except for init
  • o – shut down system
  • s – attempt to sync all mounted filesystems
  • u – attempt to remount all mounted filesystems as read-only

These keys need to be pressed together simultaneously to take effect.

Auto-rebooting is great for remote systems, and the magic SysRq combo is very useful for local systems.

 

By default after a kernel panic, Linux kernel just waits there for a system administrator to hit the restart or powercycle button.  This is because of the value set on “kernel.panic” parameter.

[root@linux23 ~]# cat /proc/sys/kernel/panic
0
[root@linux23 ~]# sysctl -a | grep kernel.panic
kernel.panic = 0
[root@linux23 ~]#

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer N greater than zero to the paramter “kernel.panic”, where “N” is the number of seconds to wait before a automatic reboot. 

For example , if you set N = 10 , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and set it.

[root@linux23 ~]# echo “10” > /proc/sys/kernel/panic
0
[root@linux23 ~]# grep kernel.panic /etc/sysctl.conf
kernel.panic = 10
[root@linux23 ~]#

2 comments to Auto reboot after kernel panic

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>