How to Automatically Restart Failed Services Using systemd

When running production servers, ensuring your critical services stay up and running is essential. Service failures can happen for various reasons — memory issues, unexpected crashes, or resource limitations. Fortunately, modern Linux distributions include systemd, which offers built-in functionality to automatically restart failed services without requiring constant manual intervention.

Server reliability is crucial for maintaining uptime and providing consistent service to users. Learning how to automatically restart failed services using systemd is one of the most effective ways to improve system stability. Whether you’re running web servers like Apache or Nginx, database services like MySQL, or custom applications, unexpected failures can significantly impact your operations unless you configure systemd to handle recovery automatically.

This guide focuses on using systemd (the modern standard service manager in most current Linux distributions) to automatically restart failed services when they crash or terminate unexpectedly. We’ll walk through the process of configuring systemd to monitor and automatically restart services like MariaDB when they fail. If you’re looking for an alternative approach, you might also be interested in using Monit for service monitoring and automatic restarts.

Which Method Should You Choose?

There are two main approaches to automatically restarting services in Linux: systemd (covered in this guide) and Monit (a separate monitoring utility). Here’s how they compare:

Choose systemd if:

• You’re running a modern Linux distribution
• You need a simple, built-in solution
• You’re primarily concerned with basic service restarts

Choose Monit if:

• You need advanced monitoring features
• You want email notifications
• You’re running an older Linux distribution
• You need to monitor resource usage or network connectivity

For details on implementing the Monit approach, see our guide on How to Automatically Restart Failed Services in Linux Using Monit.

Using systemd for Automatic Restarts

Systemd is the default init system and service manager in most modern Linux distributions (CentOS 7+, Ubuntu 16.04+, Debian 8+). It offers built-in functionality for monitoring and automatically restarting services without requiring additional software.

Finding Your Service Files

Before you can configure auto-restart, you need to locate the correct service file. The most reliable way to do this is by checking the service status:

sudo systemctl status mariadb.service

For example, with MariaDB you might see:

● mariadb.service - MariaDB 11.4.5 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: enabled)
   Drop-In: /etc/systemd/system/mariadb.service.d
            └─limits.conf, migrated-from-my.cnf-settings.conf
   Active: active (running) since Thu 2025-04-24 06:34:22 UTC; 6min ago

The important line is Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: enabled), which tells you exactly where the service file is located.

Configuring Auto-restart

Once you’ve identified your service file, you have two options:

Option 1: Create an Override

This approach is best when working with system-provided service files:

sudo systemctl edit mariadb.service

This opens an editor where you can add just the settings you want to override. Add these lines:

[Service]
Restart=always
RestartSec=3s

When you save and exit, systemd will automatically create a drop-in file in /etc/systemd/system/mariadb.service.d/ and reload the configuration. However, sometimes it doesn’t work and you need to edit the service file directly.

Option 2: Edit the Service File Directly

If you’re creating a custom service file or if you really must edit a file directly, use the path from the status output:

sudo vim /usr/lib/systemd/system/mariadb.service

Add these lines to the [Service] section:

Restart=always
RestartSec=3s

Understanding Restart Options

The Restart directive offers several options:

• always: Restart the service regardless of the exit reason (most comprehensive)
• on-failure: Restart only if the service exits with a non-zero status or crashes
• on-abnormal: Restart if the service was terminated by a signal or timed out
• on-watchdog: Restart only if the watchdog timeout is triggered
• on-abort: Restart only if the service dumps core
• on-success: Restart only if the service exits cleanly with code 0
• no: Never restart the service (default behaviour)

Complete Example: MariaDB Service

Here’s a complete example of auto-restart configuration for MariaDB. First, find the service file:

sudo systemctl status mariadb.service

This will show something like:

● mariadb.service - MariaDB 11.4.5 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: enabled)

Now create an override:

sudo systemctl edit mariadb.service

Add these lines to the editor:

[Service]
Restart=on-failure
RestartSec=5s

When you save and exit, systemd will create the override file at /etc/systemd/system/mariadb.service.d/override.conf.

Verifying Your Configuration

After making your changes, you can verify the complete service configuration with:

systemctl cat mariadb.service

This will show both the original service file and any overrides you’ve created. You should see your Restart settings in the output.

Managing Your Service

After configuring auto-restart:

1. If you used systemctl edit, the changes are automatically applied. Otherwise, reload the systemd daemon:

   sudo systemctl daemon-reload
   

2. Restart your service to apply the new configuration:

   sudo systemctl restart mariadb.service
   

3. Enable the service to start automatically on boot:

   sudo systemctl enable mariadb.service
   

Verifying Auto-restart Works

To confirm your service will restart automatically:

1. Check the service properties:

   systemctl show -p Restart,RestartSec mariadb.service
   

2. Test the restart functionality:

   sudo systemctl status mariadb.service  # Note the process ID
   sudo kill -9 [process-id]              # Force kill the process
   sudo systemctl status mariadb.service  # Verify it restarted automatically
   

Troubleshooting systemd Restarts

If you’re having issues with auto-restart:

• Service doesn’t restart: Check journal logs with journalctl -u mariadb.service for errors
• Continuous restart loops: If your service keeps failing and restarting, add these lines to limit restart attempts:

  StartLimitIntervalSec=500StartLimitBurst=5
  

• Permission issues: Ensure the service has proper permissions to restart

Conclusion

Automatic service restart capabilities are essential tools for maintaining high availability in Linux environments. Systemd offers a reliable, built-in solution that requires no additional software installation.

By implementing automatic restarts with systemd, you can significantly improve your server reliability and reduce downtime. No more manually restarting crashed services at 3 AM!

Take the time to properly configure automatic restarts for your critical services—it’s one of the simplest yet most effective ways to improve your system’s reliability.