Linux Security Audit and Hardening: Production Guide

Anil Jalela · Published Dec 10, 2020 · 25 min read

Linux hardening is a controlled reduction of attack paths, privileges, and recovery uncertainty. It is not a script that disables everything it recognizes. A useful audit establishes what the server does, which identities and network paths it needs, what evidence it records, how each change will be tested, and how access will be recovered if a control behaves differently in production.

This guide works from read-only discovery through risk-ranked remediation for current Debian, Ubuntu, Red Hat Enterprise Linux, Rocky Linux, AlmaLinux, and compatible systems. It preserves the practical checks administrators need for services, packages, remote access, users, logs, filesystems, auditing, integrity, mandatory access control, firewalls, and vulnerability management. Commands that can interrupt boot, networking, authentication, applications, or package dependencies are retained with explicit Unsafe warnings and safer change procedures.

Define the system and threat model

A hardened host must still perform its approved function. Record whether it is an Internet-facing web server, database, mail transfer agent, bastion, virtualization host, container worker, developer system, or appliance. The expected users, data sensitivity, availability objective, compliance scope, management path, and recovery method determine which controls are appropriate.

Use an audit-to-remediation workflow

1. Authorize the scope and name the system, owner, maintenance window, and recovery operator.
2. Collect a read-only baseline and protect the evidence.
3. Compare the observed state with the intended workload and a release-specific benchmark.
4. Rank findings by exploitability, exposure, privilege, data impact, and recovery cost.
5. Test one bounded control in a representative environment.
6. Back up the exact changed file or policy and record the restoration command.
7. Apply through configuration management where possible, validate syntax, then reload or reboot only when required.
8. Verify both the security outcome and the business transaction.
9. Retain exceptions with owner, reason, compensating control, and expiry date.

A scanner result is an input to this workflow. It is not authorization to edit the host. Automated remediation can change hundreds of controls simultaneously, making a failure difficult to attribute and a rollback incomplete.

Protect access and collect a baseline first

audit_id="linux-hardening-20260821"
evidence_dir="/var/tmp/${audit_id}"

sudo install -d -m 0700 -o root -g root "$evidence_dir"
sudo sh -c 'umask 077; date -u +%FT%TZ > "'$evidence_dir'/started_utc.txt"'

# Keep the current privileged session open. In another terminal, prove the
# approved administration path and test console or provider recovery access.
hostnamectl | sudo tee "$evidence_dir/hostnamectl.txt" >/dev/null
uname -a | sudo tee "$evidence_dir/uname.txt" >/dev/null
cat /etc/os-release | sudo tee "$evidence_dir/os-release.txt" >/dev/null
uptime | sudo tee "$evidence_dir/uptime.txt" >/dev/null

Unsafe: do not close the only working root or sudo session before SSH, PAM, firewall, routing, and console recovery tests pass. A syntactically valid configuration can still deny the real operator because of source address, group membership, key permissions, identity-provider failure, or a rule evaluated in an unexpected order.

Capture hardware, firmware, boot, and virtualization state

sudo dmidecode -t system -t bios 2>/dev/null
lsblk -o NAME,TYPE,FSTYPE,SIZE,MOUNTPOINTS,ROTA,RO
findmnt --real --output TARGET,SOURCE,FSTYPE,OPTIONS
systemd-detect-virt
cat /proc/cmdline

# Secure Boot status where supported.
command -v mokutil >/dev/null && mokutil --sb-state

# Identify encrypted block devices and swap without exposing keys.
lsblk -f
swapon --show --output NAME,TYPE,SIZE,USED,PRIO

For physical systems, restrict boot from removable media, protect firmware configuration, control chassis and console access, and monitor firmware changes. For cloud systems, the provider console, serial console, snapshot API, rescue image, instance metadata, and account roles are the physical-equivalent boundary. Full-disk or volume encryption protects powered-off media, while Secure Boot and measured boot address different integrity risks.

Protect the boot loader without losing recovery

# RHEL-family GRUB2 password workflow. Review vendor documentation first.
sudo grub2-setpassword

# Debian/Ubuntu can generate a PBKDF2 hash for a reviewed GRUB custom entry.
grub-mkpasswd-pbkdf2

# Inspect configuration sources. Do not edit generated grub.cfg directly.
sudo find /etc/default/grub /etc/grub.d -maxdepth 1 -type f -ls

A boot-loader password can prevent casual editing of kernel arguments, but it does not replace firmware controls, Secure Boot, storage encryption, or console access policy. Store the credential in the approved secrets system and test the documented recovery procedure.

Unsafe: editing /boot/grub*/grub.cfg directly, running an unverified generator, changing root-device arguments, or protecting every boot entry before testing can make the host unbootable. Back up the exact source files and boot metadata, keep rescue media or provider console access available, validate the generated configuration, and schedule a monitored reboot.

Inventory listeners, sockets, routes, and processes

sudo ss -lntup
sudo ss -lxnp
sudo lsof -nP -i 2>/dev/null
ip -brief address
ip route show table all
ip -6 route show table all
sudo nft list ruleset 2>/dev/null

# Map a listener to its process, executable, unit, package, and owner.
sudo readlink -f /proc/1234/exe
ps -o pid,ppid,user,group,lstart,cmd -p 1234
systemctl status example.service

ss is the current replacement for the commonly used netstat -tulpn. Inspect TCP, UDP, and Unix-domain sockets because a service can expose a privileged local control path without a network port. Determine whether each listener binds to loopback, a management address, a private interface, every IPv4 address, or every IPv6 address.

Inventory enabled services, timers, and socket activation

systemctl list-unit-files --state=enabled
systemctl list-units --type=service --state=running
systemctl list-sockets --all
systemctl list-timers --all
systemctl --failed

# Show why a unit starts and its effective definition.
systemctl cat example.service
systemctl show example.service \
  -p FragmentPath -p UnitFileState -p ActiveState -p SubState \
  -p Wants -p Requires -p Triggers -p TriggeredBy

On current systemd distributions, these commands replace routine use of chkconfig --list. Socket, path, timer, device, and dependency activation mean a service can start even when it is not simply “enabled.” Record the workload dependency and owner before disabling it.

# First prevent new automatic starts, then stop only in an approved window.
sudo systemctl disable example.service
sudo systemctl stop example.service

# Mask only when policy requires preventing all ordinary activation paths.
sudo systemctl mask example.service

# Roll back.
sudo systemctl unmask example.service
sudo systemctl enable --now example.service

Unsafe: disabling or masking an unfamiliar unit can break networking, storage, authentication, monitoring, backup, time synchronization, cluster membership, mail queues, scheduled work, or boot. mask is stronger than disable. Prove dependencies with systemctl list-dependencies, test the application, and keep an exact rollback.

Audit installed packages, repositories, and update state

# RHEL-family inventory and available security advisories.
rpm -qa --last
dnf repolist --all
dnf updateinfo list --security 2>/dev/null

# Debian/Ubuntu inventory and pending upgrades.
dpkg-query -W -f='${binary:Package}\t${Version}\n'
apt-cache policy
apt list --upgradable 2>/dev/null

# Kernel and reboot indicators.
uname -r
test -f /var/run/reboot-required && cat /var/run/reboot-required

Remove software because its function is unapproved, not because its name appears on a generic list. NFS, DNS, DHCP, SNMP, printing, mail, web, database, discovery, compiler, and debugging packages may be essential on one host and unnecessary exposure on another. Verify repository signing, release support, package origin, and update automation.

# Preview removal and dependencies before accepting anything.
sudo dnf remove --assumeno package-name
sudo apt-get -s remove package-name

# Review files owned by an installed package.
rpm -ql package-name
dpkg -L package-name

Unsafe: confirming package removal without reading the dependency transaction can remove schedulers, mail transport used by applications, libraries, authentication modules, kernels, or management agents. Capture the package list, repository state, configuration and data paths, dependent units, and reinstall command. Test the application after the change.

Patch through a release and reboot policy

Security patches are a recurring control, not a one-time update command. Define repository trust, advisory intake, testing, rollout rings, maintenance windows, kernel/live-patch policy where applicable, reboot ownership, failure rollback, and evidence that the running process loaded corrected libraries.

# RHEL-family controlled update examples.
sudo dnf check-update
sudo dnf upgrade --security

# Debian/Ubuntu controlled update examples.
sudo apt update
sudo apt upgrade

# Ubuntu package for scheduled security updates; review its policy files.
sudo apt install unattended-upgrades

# Processes still using deleted files may require restart.
sudo lsof +L1

Do not use unattended reboot behavior on a stateful or clustered service without orchestration. A patched file on disk does not guarantee that a long-running process or kernel is using it.

Harden SSH with an effective-configuration test

# Put organization changes in a managed drop-in when supported.
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitEmptyPasswords no
MaxAuthTries 4
LoginGraceTime 30
AllowGroups ssh-admins
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no

Forwarding controls depend on the workload. Bastions and deployment systems may require narrowly governed forwarding; turning it off globally without testing can break automation. Modern OpenSSH uses protocol 2, so adding Protocol 2 to current configurations is normally unnecessary.

sudo install -d -m 0700 /root/linux-hardening-backup
sudo cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config.d \
  /root/linux-hardening-backup/ 2>/dev/null || true

# Validate syntax and inspect the effective configuration.
sudo sshd -t
sudo sshd -T | sort

# Match blocks can change the result for a real connection.
sudo sshd -T -C user=admin,host=server.example,addr=192.0.2.50 | sort

sudo systemctl reload sshd 2>/dev/null || sudo systemctl reload ssh

Unsafe: disabling passwords before every administrator has a tested key, correct home-directory permissions, group authorization, and console recovery can lock out the system. Test a new connection while the original session stays open. Do not weaken host-key checking in automation to make deployment easier.

Manage SSH keys as identities

find /home /root -xdev -type f -name authorized_keys \
  -printf '%m %u:%g %p\n' 2>/dev/null

# Inspect fingerprints without publishing private material.
while IFS= read -r keyfile; do
  echo "== $keyfile"
  ssh-keygen -lf "$keyfile" 2>/dev/null || true
done < <(find /home /root -xdev -type f -name authorized_keys 2>/dev/null)

Each key needs an owner, purpose, approved source, creation date, rotation or expiry policy, and removal event. Prefer individual keys or certificate-backed SSH identities over shared administrator keys. Use from=, restrict, forced commands, or certificate principals where the automation case permits them.

Audit users, groups, shells, and UID 0

getent passwd
getent group

# Every UID 0 identity requires explicit authorization.
getent passwd | awk -F: '$3 == 0 {print $1 ":" $6 ":" $7}'

# Empty local password hashes. Requires root to read /etc/shadow.
sudo awk -F: '($2 == "") {print $1}' /etc/shadow

# Human-account threshold comes from the distribution, not a hard-coded 500.
awk '/^UID_MIN/{print}' /etc/login.defs

# Review account status, expiry and last login.
sudo passwd -S username
sudo chage -l username
last -Fai | head -n 50
lastb -Fai 2>/dev/null | head -n 50
lastlog | head -n 50

Check separated users, dormant accounts, generic identities, service accounts with interactive shells, groups granting sudo or application administration, duplicate UIDs/GIDs, home ownership, and identities returned by LDAP, SSSD, or another directory. A local-file-only audit can miss network identities.

# Locking a password does not necessarily terminate sessions, keys, tokens,
# scheduled jobs, or service credentials.
sudo passwd -l username
sudo usermod --expiredate 1 username
sudo loginctl terminate-user username

# Deliberate rollback after authorization.
sudo usermod --expiredate -1 username
sudo passwd -u username

Unsafe: locking a service identity or setting its shell to /usr/sbin/nologin can stop applications, backups, agents, file transfer, or scheduled jobs. Identify how the account authenticates and what runs under its UID before changing it.

Govern sudo instead of sharing root

sudo -l -U username
sudo visudo -c
sudo find /etc/sudoers.d -maxdepth 1 -type f -printf '%m %u:%g %p\n'

# Audit broad or shell-capable grants for manual review.
sudo grep -RInE 'NOPASSWD|ALL[[:space:]]*=[[:space:]]*\(ALL' \
  /etc/sudoers /etc/sudoers.d 2>/dev/null

Administrators should use named accounts, strong authentication, and approved sudo roles so actions are attributable. Avoid allowing unrestricted editors, package managers, interpreters, service managers, or commands with user-controlled path arguments when the role is intended to be narrow; many can provide a root shell indirectly. Validate every change with visudo.

Set password and authentication policy through supported tooling

Current systems commonly use pam_pwquality for password quality and pam_pwhistory for history. Older pam_cracklib examples and direct edits to generated system-auth files should not be copied into a current host. RHEL-family systems may manage PAM through authselect; Debian/Ubuntu use their packaged PAM configuration. Directory-managed identities may be governed elsewhere.

grep -RInE 'pam_(pwquality|pwhistory|faillock|unix)\.so' \
  /etc/pam.d /etc/security 2>/dev/null
sudo authselect current 2>/dev/null || true

# Review account aging defaults and a specific user.
grep -E '^(PASS_MAX_DAYS|PASS_MIN_DAYS|PASS_WARN_AGE)' /etc/login.defs
sudo chage -M 90 -m 1 -W 14 username

Password length, compromised-password screening, MFA, rate limits, and identity-provider controls are generally more valuable than forcing arbitrary character classes or frequent changes without evidence of compromise. Apply account aging according to the organization's identity policy, not one universal number.

Audit cron, at, and systemd scheduled work

sudo systemctl list-timers --all
sudo find /etc/cron.d /etc/cron.daily /etc/cron.hourly \
  /var/spool/cron /var/spool/cron/crontabs \
  -xdev -type f -printf '%m %u:%g %p\n' 2>/dev/null
sudo atq 2>/dev/null

# Look for writable scripts and parent directories before execution.
namei -l /usr/local/sbin/example-backup
sudo stat /etc/crontab /etc/cron.allow /etc/cron.deny 2>/dev/null

cron.allow and cron.deny behavior varies by implementation and file presence. Writing a literal ALL to /etc/cron.deny is not a portable method to disable every user. Control scheduler access according to the local manual page, and audit the commands, environment, PATH, file ownership, secrets, output handling, and service-account scope.

Collect and protect system logs

journalctl --disk-usage
journalctl --verify
journalctl -p warning..alert --since '24 hours ago'
journalctl -u sshd --since '24 hours ago' 2>/dev/null || \
  journalctl -u ssh --since '24 hours ago'

# Common distribution files may include these when rsyslog writes them.
sudo ls -l /var/log/auth.log /var/log/secure /var/log/messages \
  /var/log/syslog /var/log/cron /var/log/maillog 2>/dev/null

systemctl status systemd-journald rsyslog 2>/dev/null

Define which events are required, local retention, rate-limit behavior, time synchronization, remote transport protection, central retention, access separation, alerting, and loss detection. A local root compromise can alter local logs, so high-assurance systems forward them promptly to a restricted collector or security platform.

Unsafe: copying complete journals, authentication logs, process environments, command histories, or application logs into an unrestricted ticket can disclose usernames, internal addresses, message content, tokens, and personal data. Preserve integrity and timestamps, restrict access, and redact only the copy used for collaboration.

Configure Linux Audit rules for security events

sudo auditctl -s
sudo auditctl -l
sudo augenrules --check 2>/dev/null
sudo ausearch -ts today -m USER_LOGIN,USER_AUTH,USER_ACCT
sudo aureport --summary

Persistent audit rules normally belong in /etc/audit/rules.d/*.rules and are compiled with augenrules. Design rules from required events and actual filesystem layout. Watches are simple but can be noisy; syscall rules need architecture, filters, and performance testing.

# /etc/audit/rules.d/50-identity.rules
-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/sudoers -p wa -k privilege
-w /etc/sudoers.d/ -p wa -k privilege
-w /etc/ssh/sshd_config -p wa -k ssh_config
-w /etc/ssh/sshd_config.d/ -p wa -k ssh_config

sudo install -m 0600 /path/to/reviewed/50-identity.rules \
  /etc/audit/rules.d/50-identity.rules
sudo augenrules --check
sudo augenrules --load
sudo auditctl -l
sudo ausearch -k identity -ts recent

Unsafe: loading a large copied ruleset can exhaust audit backlog, consume disk, degrade a busy host, or create an operational denial of service. Measure event rate, backlog, loss counters, disk behavior, and searchability under representative load. Some audit service operations intentionally require the service command or reboot according to distribution policy.

Audit filesystem permissions, ACLs, and ownership

# Stay on each filesystem and store results securely.
sudo find / -xdev -type f -perm /6000 -printf '%m %u:%g %p\n'
sudo find / -xdev \( -type f -o -type d \) -perm -0002 \
  -printf '%m %u:%g %p\n'
sudo find / -xdev \( -nouser -o -nogroup \)

# Inspect every path component and any extended ACL.
namei -l /etc/ssh/sshd_config
getfacl -p /path/to/review 2>/dev/null
getcap -r / 2>/dev/null

SUID and SGID files execute with additional identity semantics and deserve review, but they are not automatically vulnerabilities. Compare them with the package baseline, business purpose, file hash, owner, permissions, capabilities, and recent changes. World-writable directories such as /tmp need the sticky bit; a generic removal of world write can break normal operation.

# Verify package ownership and integrity before changing a privileged file.
rpm -qf /usr/bin/example 2>/dev/null
rpm -V package-name 2>/dev/null
dpkg -S /usr/bin/example 2>/dev/null
dpkg -V package-name 2>/dev/null

# Record before any approved permission change.
stat /usr/bin/example
getfacl -p /usr/bin/example

Unsafe: running recursive chmod or chown from /, /usr, an application root, or a shared data tree can destroy package, ACL, service, and container semantics. Change one verified path, preserve ACLs and extended attributes, and test the owning workload.

Set umask in the correct scope

umask
grep -RInE '(^|[[:space:]])umask[[:space:]]' \
  /etc/profile /etc/bash.bashrc /etc/profile.d /etc/login.defs 2>/dev/null

# A service has its own effective creation mask.
systemctl show example.service -p UMask
systemctl cat example.service

A common interactive baseline is 0022; sensitive services may use 0027 or 0077. The correct value depends on required group collaboration and application behavior. Shell profile settings do not reliably govern systemd services, containers, cron jobs, or applications that set their own permissions. Use a systemd drop-in such as UMask=0027 for a tested service-specific policy.

Review mounts before adding nodev, nosuid, or noexec

findmnt --real --output TARGET,SOURCE,FSTYPE,OPTIONS
sudo findmnt --verify --verbose

# Resolve the unit and test a temporary option only in an approved environment.
systemd-escape --path --suffix=mount /var/tmp

nodev, nosuid, and noexec reduce particular execution paths when applied to suitable, separate filesystems. They are not complete boundaries: interpreters can read scripts from a noexec mount, and applications may legitimately need execution from temporary or data paths. Separate filesystems also create capacity and boot dependencies.

Unsafe: applying new mount options to /, /usr, /var, /tmp, application data, container storage, or /boot without workload and reboot testing can break package updates, boot generation, installers, databases, containers, and applications. Test with a temporary mount or representative host, validate /etc/fstab, keep console recovery, and schedule a reboot test.

Establish file-integrity monitoring early

# RHEL-family.
sudo dnf install aide
sudo aide --init

# Debian/Ubuntu.
sudo apt install aide
sudo aideinit

# Check using the distribution's configured database path.
sudo aide --check

AIDE and enterprise integrity tools are useful only when the baseline is created from a trusted state, stored or anchored securely, updated through approved change, and reviewed by an owner. Modern configurations use strong hashes rather than relying on MD5. Package verification complements integrity monitoring but does not cover unowned application files or prove that every difference is malicious.

Unsafe: initializing the database after compromise records the attacker's changes as trusted. Do not automatically accept every post-patch difference. Correlate package transactions, configuration management, maintenance records, hashes, signatures, ownership, and runtime evidence.

Keep SELinux or AppArmor enforcing

# SELinux systems.
getenforce 2>/dev/null
sestatus 2>/dev/null
sudo ausearch -m AVC,USER_AVC -ts recent 2>/dev/null
sudo semanage boolean -l 2>/dev/null | head

# AppArmor systems.
sudo aa-status 2>/dev/null
sudo journalctl -k --since '1 hour ago' | grep -i apparmor

SELinux and AppArmor provide mandatory access control beyond owner/group/mode permissions. If a legitimate action is denied, confirm the expected label or profile, application path, port, boolean, and policy before writing a local exception. Use permissive or complain behavior only as a bounded diagnostic step where supported, then return to enforcement.

# SELinux: inspect and restore expected labels before creating policy.
ls -lZ /path/to/application
sudo restorecon -RFv /path/to/application

# AppArmor: review logged behavior with the supported helper.
sudo aa-logprof

Unsafe: setting SELinux permanently permissive, disabling AppArmor, or generating a broad allow policy from unreviewed denial logs removes a containment layer and can authorize attacker behavior. Correct labeling and application configuration first; make the smallest reviewed policy change; test and monitor it.

Apply host firewall policy without breaking management

sudo nft list ruleset
sudo firewall-cmd --list-all-zones 2>/dev/null
sudo ufw status verbose 2>/dev/null

# Confirm listeners and the source path before editing policy.
sudo ss -lntup
ip route get 192.0.2.50

Use one authoritative host-firewall management layer. A cloud security group or network firewall does not replace the host boundary, and overlapping unmanaged iptables, nftables, firewalld, UFW, container, and orchestration rules create misleading audits. Start from the required traffic matrix: source, destination address, protocol, port, direction, owner, and expiry.

Unsafe: flushing firewall rules or setting a default drop remotely before the established management connection and return traffic are permitted can immediately lock out the host. Use an out-of-band console, timed rollback, syntax check, atomic ruleset loading, and a second-session test. Preserve container and orchestration-managed chains.

Harden kernel and network settings selectively

# /etc/sysctl.d/60-host-hardening.conf
kernel.randomize_va_space = 2
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_fifos = 2
fs.protected_regular = 2
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

sudo sysctl --system
sysctl kernel.randomize_va_space fs.protected_hardlinks \
  net.ipv4.conf.all.accept_redirects net.ipv4.conf.all.rp_filter

Reverse-path filtering can break asymmetric routing, policy routing, multi-homing, tunnels, containers, and load balancers. IPv6 should be secured with equivalent policy, not disabled simply because the team is less familiar with it. Blocking all ICMP echo does not make a host invisible and can harm monitoring and diagnosis; blocking broadcast echo is a different control. Choose settings from the actual network role and release-specific vendor guidance.

Use systemd sandboxing for exposed services

systemd-analyze security example.service
systemctl show example.service \
  -p User -p Group -p NoNewPrivileges -p PrivateTmp \
  -p ProtectSystem -p ProtectHome -p CapabilityBoundingSet
systemctl cat example.service

# sudo systemctl edit example.service
[Service]
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/example /var/log/example
CapabilityBoundingSet=

systemd-analyze security measures exposure based on unit settings; it is not a vulnerability verdict. Add one directive at a time, verify with systemd-analyze verify, restart during a window, and test startup, steady state, log rotation, updates, backup, and failure recovery. Capabilities and writable paths must match actual application behavior.

Control application execution where the risk justifies it

RHEL-family systems can use fapolicyd to allow or deny execution based on trust and policy. It is powerful on controlled servers and can disrupt custom applications, interpreters, agents, or software installed outside the RPM database. Begin with the vendor's permissive/debug workflow and observe representative operations before enforcement.

sudo dnf install fapolicyd
sudo fapolicyd --debug-deny --permissive
sudo fapolicyd-cli --check-config
sudo fapolicyd-cli --check-trustdb
sudo ausearch -ts recent -m fanotify

Do not copy a deny-execute policy from another server. Inventory package-managed software, local releases, interpreters, update tools, maintenance paths, and emergency utilities; then create the smallest reviewed trust and rule set.

Scan against a tailored benchmark

# RHEL-family examples when OpenSCAP content is installed.
oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
sudo oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
  --results /var/tmp/oscap-results.xml \
  --report /var/tmp/oscap-report.html \
  /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

# Confirm the exact profile and data stream on the installed release.

Unsafe: applying scanner-generated remediation directly to a production system can alter boot parameters, PAM, SSH, mounts, cryptography, services, and audit rules together. Review every proposed change, exception, dependency, reboot requirement, and rollback in the context of the workload.

Rank findings by attack path and consequence

Severity labels from tools are useful but incomplete. Add reachability, authentication required, exploit maturity, current controls, affected data, persistence, safety impact, operational dependency, and recovery time. Findings that indicate compromise move into incident response; routine hardening must not overwrite their evidence.

Verify each hardening change

Roll back the exact control

# Example: restore one SSH file, validate, and reload only SSH.
sudo install -m 0600 \
  /root/linux-hardening-backup/sshd_config \
  /etc/ssh/sshd_config
sudo sshd -t
sudo systemctl reload sshd 2>/dev/null || sudo systemctl reload ssh

# Re-run the same approved and denied connection tests and retain the failed file.

Unsafe: restoring whole configuration directories, flushing every firewall rule, disabling SELinux/AppArmor, removing all audit rules, or rebooting without knowing which control failed expands the outage and loses attribution. Restore the exact changed unit, validate it, activate only the owning subsystem, and repeat the acceptance tests.

Operate hardening continuously

• Reconcile assets, owners, support status, and approved workload after every material architecture change.
• Monitor new listeners, enabled units, packages, repositories, privileged identities, sudo grants, keys, SUID/SGID files, capabilities, and scheduled jobs.
• Track security advisories, patch latency, reboot debt, authentication failures, audit loss, integrity changes, MAC denials, firewall drops, and backup restoration tests.
• Review benchmark exceptions before expiry and remove controls that no longer match the system role.
• Rebuild compromised or untrustworthy hosts from a controlled image instead of attempting to “harden away” uncertain persistence.

Production completion checklist

• The system role, data, exposure, owners, administration paths, availability objectives, and exact benchmark release are documented.
• A protected baseline covers OS, kernel, boot, mounts, interfaces, routes, listeners, processes, services, sockets, timers, packages, repositories, users, groups, keys, sudo, logs, audit, firewall, and MAC state.
• Console or provider recovery and a second administrative session are tested before remote-access changes.
• Every listener, enabled service, repository, privileged identity, scheduled task, SUID/SGID file, capability, and writable execution path has an owner or finding.
• SSH effective configuration and Match behavior pass for real administrators and automation.
• Patch policy includes repository trust, testing, rollout, process restart, reboot, and failure recovery.
• Audit and centralized logging record the required event, retain it securely, and alert if events are lost.
• SELinux or AppArmor remains enforcing with narrow, reviewed policy changes.
• Firewall, sysctl, mount, systemd sandbox, and execution-policy controls match the workload and have service tests.
• Each remediation has a backup, syntax check, activation method, acceptance evidence, rollback trigger, owner, and change record.
• Findings that suggest compromise are handled as incidents without destroying evidence.

Primary references

• Red Hat Enterprise Linux 9 Security hardening
• Ubuntu Server security suggestions
• Ubuntu Server AppArmor guidance
• CIS Benchmarks overview
• SCAP Security Guide
• systemd-analyze documentation
• auditctl manual
• ss socket-inspection manual
• OpenSSH manuals
• AIDE file-integrity monitoring