Using dd Safely: Swap Files, Large Test Files, and Disk Images

Anil Jalela · Published Dec 13, 2022 · 18 min read

dd can create a test file, populate a swap file, capture a disk image, or overwrite the wrong device with equal efficiency. The safe way to use it is to identify the job first, choose the tool that matches that job, verify every path, and define how success and rollback will be proved.

The examples use /var/tmp/nw-storage-lab, documentation device names, and small sizes where possible. Commands that write a block device are retained because they are useful, but they are marked Unsafe and must never be pasted before the source and destination have been independently verified.

Choose the operation before choosing the command

A large file can be useful for upload limits, download handling, storage alerts, backup tests, LAN transfers, and development workflows. Do not assume one file is suitable for every test. A sparse zero-filled file may compress or deduplicate almost completely, while random data may stress CPU and entropy generation more than the target application does.

Read a dd command from left to right

GNU size suffixes matter. bs=1M count=1024 writes 1 GiB because GNU M represents 1,048,576 bytes. A copied command such as bs=1024 count=1024000 writes 1,048,576,000 bytes, which is 1.048576 GB but only 0.9765625 GiB. State whether a requirement uses decimal GB or binary GiB.

Prepare a safe working directory

sudo install -d -m 0750 -o root -g root /var/tmp/nw-storage-lab
cd /var/tmp/nw-storage-lab
pwd
df -hT .
df -i .
findmnt -T . -o TARGET,SOURCE,FSTYPE,OPTIONS
ls -la

Confirm available bytes and inodes, filesystem type, quotas, encryption, snapshots, thin provisioning, and the cleanup owner. A successful 10 GiB allocation on a thin or deduplicating layer does not prove that 10 GiB of physical capacity is safely available.

Create large files with the right allocation behavior

Logical sparse file with truncate

truncate -s 10G logical-10g.bin
ls -lh logical-10g.bin
du -h logical-10g.bin
stat --format='logical=%s bytes allocated=%b blocks block_unit=%B' logical-10g.bin

The file reports a 10 GiB logical length, but its holes consume few data blocks. This is useful for code paths that inspect length. It is unsuitable for proving free-space consumption, sustained writes, backup volume, or a valid swap file.

Preallocated file with fallocate

fallocate -l 10G allocated-10g.bin
ls -lh allocated-10g.bin
du -h allocated-10g.bin
stat --format='logical=%s bytes allocated=%b blocks block_unit=%B' allocated-10g.bin

fallocate reserves blocks quickly on supporting filesystems, usually without writing ten GiB of zeroes through the storage stack. Its exact behavior and swap suitability depend on the filesystem and kernel. Do not use it as a universal substitute for the portable dd swap-file method.

Fully written zero-filled file with dd

dd if=/dev/zero of=zero-10g.bin \
  bs=1M count=10240 \
  status=progress conv=excl,fsync

ls -lh zero-10g.bin
du -h zero-10g.bin
sha256sum zero-10g.bin

conv=excl refuses to replace an existing regular file, and conv=fsync includes a final data-and-metadata flush before completion. Zeroes can still be compressed, deduplicated, or optimized by lower layers, so document the storage path before interpreting the rate.

High-entropy file for transfer and compression tests

head -c 10G </dev/urandom > random-10g.bin
sync -f random-10g.bin
ls -lh random-10g.bin
du -h random-10g.bin
sha256sum random-10g.bin

Unsafe at scale: this command intentionally writes ten GiB and may consume significant CPU, storage bandwidth, quota, snapshot capacity, replication bandwidth, and backup space. Start with 100M in a development environment and increase only after confirming headroom and cleanup.

XFS utility when xfs_mkfile is installed

xfs_mkfile -v 10g xfs-written-10g.bin
xfs_mkfile -n -v 10g xfs-sparse-10g.bin
ls -lh xfs-*.bin
du -h xfs-*.bin

The normal command pads with zeroes. The -n form intentionally creates a holey file. Use the explicit filename and allocation checks rather than inferring behavior from the utility name.

Know why /dev/zero and /dev/null are different

dd if=/dev/null of=empty.bin bs=1M count=10 status=progress
stat --format='size=%s bytes' empty.bin

dd if=/dev/zero of=ten-mebibytes.bin \
  bs=1M count=10 status=progress conv=excl,fsync

The first command creates or truncates the output but copies zero bytes because the input is already at EOF. It cannot be used to populate a large file or swap file.

Prove logical size, allocation, and content separately

One command rarely answers all three questions. ls normally reports the apparent length seen by an application. du reports allocated filesystem space. stat exposes both the byte length and allocated block count. filefrag can show extents and holes on supporting Linux filesystems.

for test_file in logical-10g.bin allocated-10g.bin zero-10g.bin random-10g.bin; do
  test -e "$test_file" || continue
  printf '\nfile=%s\n' "$test_file"
  ls -lh -- "$test_file"
  du -h -- "$test_file"
  stat --format='size=%s blocks=%b block_unit=%B mode=%a owner=%U:%G' \
    -- "$test_file"
done

Keep filenames fixed inside a reviewed lab directory. Do not feed filenames from an untrusted upload into an administrative shell loop. A newline or option-like filename needs safe argument handling, and an application test should use its normal validation layer rather than a root shell.

Inspect before creating swap

free -h
swapon --show --output=NAME,TYPE,SIZE,USED,PRIO
cat /proc/swaps
grep -E '^(MemTotal|MemAvailable|SwapTotal|SwapFree):' /proc/meminfo
df -hT /
findmnt -T / -o TARGET,SOURCE,FSTYPE,OPTIONS
grep -vE '^[[:space:]]*(#|$)' /etc/fstab

Swap is not additional physical RAM and is not a backup. It can absorb cold anonymous pages and change out-of-memory behavior, but heavy swap use may turn a memory shortage into severe I/O latency. Size it from workload behavior, hibernation needs, storage endurance, incident policy, and available capacity rather than a fixed RAM multiplier.

Create a portable swap file with dd

This example creates a 2 GiB file. Confirm that /swapfile does not exist and that the root filesystem has enough real space before running it.

sudo test ! -e /swapfile
df -hT /

sudo dd if=/dev/zero of=/swapfile \
  bs=1M count=2048 \
  status=progress conv=excl,fsync

sudo chmod 0600 /swapfile
sudo chown root:root /swapfile
sudo ls -lh /swapfile
sudo du -h /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

swapon --show --output=NAME,TYPE,SIZE,USED,PRIO
free -h

Unsafe if the path is wrong: mkswap writes a swap signature, and swapon allows the kernel to overwrite the area with memory pages. Never point either command at an unidentified partition, mounted filesystem, application file, or disk image.

On hosts with util-linux 2.41 or newer, mkswap --file can create and format an appropriately populated swap file in one operation. Check the installed help before choosing that workflow:

mkswap --version
mkswap --help | grep -E -- '--file|--size'

# Current util-linux workflow when supported:
sudo mkswap --file --size 2G /swapfile-current
sudo swapon /swapfile-current
swapon --show

Do not run both creation workflows for the same path. The explicit dd sequence remains useful across a wider range of installed util-linux releases.

Persist swap and test the boot-equivalent path

Back up /etc/fstab, edit it with the system's normal configuration process, and add exactly one entry:

/swapfile  none  swap  defaults  0  0

sudo cp -a /etc/fstab /etc/fstab.before-swapfile
sudo findmnt --verify --verbose
sudo systemctl daemon-reload

# Only after proving enough available memory for swapoff:
free -h
sudo swapoff /swapfile
sudo swapon -a
swapon --show
free -h

swapon -a reads swap entries from /etc/fstab. mount -a handles mountable filesystems and is not the swap activation test. On a busy or memory-constrained host, swapoff may fail or trigger intense memory pressure while pages return to RAM.

Unsafe without headroom: do not run swapoff -a merely to test configuration. It targets every active swap area and can provoke an out-of-memory event. Disable only the intended path after checking MemAvailable, swap used, workload growth, and rollback access.

Handle Btrfs swap files with Btrfs rules

Btrfs supports swap files on current kernels, but the file must be NOCOW, preallocated, and free of holes. The filesystem must use a supported single-device and single-data-profile layout. An active swap file also constrains snapshots, balance, device changes, and scrub coverage for affected block groups.

findmnt -T /swap -o TARGET,SOURCE,FSTYPE,OPTIONS
sudo btrfs filesystem mkswapfile --size 2G /swap/swapfile
sudo swapon /swap/swapfile
swapon --show

If the installed Btrfs tools do not provide filesystem mkswapfile, follow the current Btrfs documentation for chattr +C, preallocation, permissions, mkswap, and activation. Do not use truncate, a copied sparse file, or a snapshotted file as swap.

Observe swap without manufacturing an incident

watch -n 2 'free -h; echo; swapon --show'
vmstat 1 10
sar -W 1 10
cat /proc/pressure/memory
sysctl vm.swappiness

In vmstat, sustained nonzero si and so show swap-in and swap-out activity. Combine that with memory pressure, major faults, application latency, I/O latency, queue depth, and OOM logs. Swap configured but unused is not a fault.

Current kernels accept vm.swappiness values from 0 through 200. The default is commonly 60, but the right value is workload-dependent and reflects the relative cost of swap I/O and filesystem paging. Do not copy a universal “set it to 10” rule into every server.

# Temporary, reviewed experiment:
sudo sysctl vm.swappiness=30

# Persist only after measuring the workload:
printf 'vm.swappiness = 30\n' \
  | sudo tee /etc/sysctl.d/60-nw-swappiness.conf
sudo sysctl --system

Use dd benchmarks as narrow evidence

cd /var/tmp/nw-storage-lab

dd if=/dev/zero of=write-test-2g.bin \
  bs=1M count=2048 \
  status=progress conv=excl,fdatasync

dd if=write-test-2g.bin of=/dev/null \
  bs=1M iflag=direct status=progress

This is a sequential test, not a database, VM, mail queue, or mixed application workload. Results depend on caches, thin provisioning, compression, deduplication, RAID, network storage, background activity, direct-I/O support, block size, and queue depth. iflag=direct may reject an unsupported alignment or filesystem.

Unsafe on production: writing a multi-gigabyte test file consumes capacity and I/O. Dropping all kernel caches with echo 3 > /proc/sys/vm/drop_caches is sometimes suggested before read tests, but it disturbs unrelated workloads and still does not create a controlled benchmark. Use an isolated test volume and a reviewed fio job instead.

Monitor a running copy without guessing its state

Use status=progress for a new GNU dd command. For an already running process, GNU dd prints statistics when it receives USR1 on Linux and then continues. Capture the exact child PID instead of using a broad pkill or a process-name search that might signal another administrator's job.

dd if=/dev/zero of=/dev/null \
  bs=1M count=4096 iflag=fullblock &
dd_pid=$!

kill -USR1 "$dd_pid"
wait "$dd_pid"
dd_status=$?
printf 'dd_exit=%s\n' "$dd_status"

This harmless example reads zero bytes and discards them, so it demonstrates process control rather than disk throughput. An exit status of zero indicates that dd completed its requested copy. Capture standard error and the final status in automation. A progress line without a successful exit is not completion evidence.

Ctrl-C sends an interrupt and normally stops a foreground copy after printing final statistics. Stopping a regular-file copy can leave a partial output. Stopping a block-device restore can leave an unbootable, partially overwritten destination. Never improvise a “resume” offset from the displayed byte count without aligning units and proving the destination state.

Copy a bounded region when that is the actual task

# Copy the first 64 MiB of a regular image into a new file.
dd if=source-device.img of=first-64m.img \
  bs=1M count=64 iflag=fullblock \
  status=progress conv=excl,fsync

# Copy 16 MiB beginning at input offset 128 MiB.
dd if=source-device.img of=region-128m-144m.img \
  bs=1M skip=128 count=16 iflag=fullblock \
  status=progress conv=excl,fsync

stat --format='size=%s bytes' first-64m.img region-128m-144m.img
sha256sum first-64m.img region-128m-144m.img

skip counts input blocks, while seek counts output blocks. Changing bs changes the offsets represented by both values. For forensic, boot-record, database, or recovery work, record byte offsets explicitly and use a tool designed for the format when one exists.

Unsafe: combining seek with conv=notrunc can modify a region inside an existing output while preserving the rest. That is useful in specialized image construction and destructive on an ordinary filesystem or disk. Keep it out of general-purpose automation unless a reviewed binary layout requires it.

Test LAN and application transfers separately

# Network-only test between approved systems:
iperf3 --server
iperf3 --client 192.0.2.20 --time 30 --parallel 4

# File-transfer integrity test:
sha256sum random-10g.bin
scp random-10g.bin [email protected]:/var/tmp/nw-storage-lab/
ssh [email protected] \
  'sha256sum /var/tmp/nw-storage-lab/random-10g.bin'

Run the server only on an approved test endpoint and restrict its firewall exposure. iperf3 isolates the network more effectively. The file transfer then tests SSH, encryption, disks, filesystem, and end-to-end integrity together. A zero-filled file can produce misleading results when transport or storage compression is active.

Image a healthy device to a file

Quiesce the application and unmount the source filesystem when a crash-consistent raw image is required. Database and virtual-machine workloads may need their own snapshot or backup mechanism.

lsblk -e7 -o NAME,PATH,SIZE,MODEL,SERIAL,TYPE,FSTYPE,MOUNTPOINTS
findmnt

# Documentation names only. Replace after independent verification.
sudo dd if=/dev/disk/by-id/SOURCE_DEVICE \
  of=/mnt/backup/source-device.img \
  bs=4M iflag=fullblock \
  status=progress conv=excl,fsync

sha256sum /mnt/backup/source-device.img \
  | sudo tee /mnt/backup/source-device.img.sha256
ls -lh /mnt/backup/source-device.img*

Unsafe: a mounted, changing source can produce an inconsistent image, and a mistaken if/of reversal can destroy the device. Use stable /dev/disk/by-id/ names, record model and serial, verify the backup filesystem has capacity, and require a second-person check for important systems.

Clone or restore to a block device

lsblk -e7 -o NAME,PATH,SIZE,MODEL,SERIAL,TYPE,FSTYPE,MOUNTPOINTS
sudo blockdev --getsize64 /dev/disk/by-id/SOURCE_DEVICE
sudo blockdev --getsize64 /dev/disk/by-id/DESTINATION_DEVICE

# UNSAFE: overwrites the destination from its first block onward.
sudo dd if=/dev/disk/by-id/SOURCE_DEVICE \
  of=/dev/disk/by-id/DESTINATION_DEVICE \
  bs=4M iflag=fullblock \
  status=progress conv=fsync

Unsafe and destructive: the destination loses its partition table, filesystems, boot records, and data. Confirm the destination is not mounted or used by LVM, RAID, encryption, swap, a database, a VM, or the running system. The destination must be large enough. Keep a tested backup and console access. Do not rely on changing /dev/sdX letters.

After a planned clone, reread the partition table or reboot only according to the change plan, then validate partitions, filesystem identifiers, bootability, application integrity, and expanded capacity. Duplicate filesystem UUIDs can cause ambiguous mounting when source and clone remain attached.

Use a recovery tool for failing media

sudo ddrescue --force --no-scrape \
  /dev/disk/by-id/FAILING_SOURCE \
  /mnt/recovery/device.img \
  /mnt/recovery/device.map

sudo ddrescue --force --direct --retry-passes=3 \
  /dev/disk/by-id/FAILING_SOURCE \
  /mnt/recovery/device.img \
  /mnt/recovery/device.map

The map file lets ddrescue resume and distinguish copied, failed, and pending areas. Recovery strategy depends on device health and data value; repeated reads can worsen some failures. For important data, stop and use a qualified recovery service rather than experimenting.

Place swap at the correct virtualization boundary

On a physical server, the operating system owns the swap area and its storage path. In a virtual machine, guest swap consumes virtual-disk I/O that ultimately reaches the hypervisor and backend storage. The host may also be under memory pressure. Monitor both layers before concluding that more guest swap solves the problem.

A container normally shares the host kernel and does not own a private swap device merely because a file exists inside its filesystem view. Container memory and swap behavior is governed through cgroups and the orchestration platform. Running swapon inside an application container usually requires excessive privilege and creates an unmanaged host-level side effect. Configure memory limits, swap policy, eviction behavior, and observability at the platform boundary.

Managed Kubernetes and container services can restrict or define node swap behavior differently by version and provider. Do not enable host swap underneath a cluster by copying a standalone-server procedure. Start with the current platform policy, node image, kubelet configuration, workload class, and support boundary.

Cloud virtual disks add another layer of limits. A dd result may be constrained by provisioned IOPS, throughput caps, burst credits, instance bandwidth, attachment type, encryption, snapshots, or noisy neighbors. Record the disk and instance limits beside the command. A short burst is not sustained capacity.

Protect memory contents and plan hibernation separately

Swap can contain fragments of credentials, message bodies, application data, cryptographic material, and other memory-resident information. Apply the same data-classification and disposal policy used for memory dumps. Whole-disk encryption may protect a swap file at rest when the key lifecycle and boot path are designed correctly. An unencrypted swap partition or file can expose data after shutdown or disk removal.

Ephemeral encrypted swap, where a new key is created at boot, is useful when resume across reboot is not required. Hibernation needs the system to find and decrypt the saved memory image during early boot, so it has different key-retention and resume requirements. A swap file used for hibernation also needs a correct resume device and physical offset. Merely making the file as large as RAM is not a complete hibernation configuration.

Do not enable hibernation on servers, encrypted systems, Btrfs files, or managed images without following the distribution and bootloader procedure. Test resume after kernel, initramfs, storage, encryption, and bootloader changes. A failed resume can become a boot incident or expose stale memory contents.

When retiring swap, disable it, remove the persistent configuration, verify it is inactive, and follow the storage platform's sanitization policy. Overwriting a file once does not guarantee physical erasure on SSDs, copy-on-write filesystems, snapshots, thin pools, replicated volumes, or cloud storage. Cryptographic erasure or provider-managed destruction may be the correct control.

Remove a swap file safely

1. Confirm swap usage, available memory, and the exact active path.
2. Remove or comment the matching /etc/fstab entry through the normal configuration process.
3. Reload the service manager if required.
4. Disable only that swap file and verify it disappeared.
5. Delete the exact inactive file only after validation.

swapon --show
free -h
grep -nF '/swapfile' /etc/fstab

sudo cp -a /etc/fstab /etc/fstab.before-swapfile-removal
# Edit only the validated /swapfile entry.
sudo systemctl daemon-reload
sudo swapoff /swapfile
swapon --show
sudo file /swapfile

# UNSAFE: irreversible regular-file deletion after all checks pass.
sudo rm -f -- /swapfile
df -hT /

Unsafe: deleting a file that is still active as swap may fail, create confusing hidden space use, or leave an inconsistent boot configuration. A broad wildcard or unresolved variable in the removal command is unacceptable.

Troubleshoot from evidence

Completion record

• The goal, test size, logical-versus-allocated requirement, owner, capacity limit, duration, and cleanup time are recorded.
• Every dd input and output was resolved to an exact regular file or stable device identity before execution.
• The filesystem, free bytes, inodes, quotas, snapshots, thin provisioning, compression, deduplication, and backup impact were checked.
• Swap creation used a populated non-sparse file and filesystem-specific rules.
• Permissions, mkswap, activation, /etc/fstab, swapon -a, monitoring, and rollback passed.
• Performance results state block size, durability, cache, allocation, storage path, concurrency, and what was not measured.
• Images and clones have recorded source/destination identities, consistency method, capacity proof, checksums, recovery plan, and validation.
• Temporary files and swap areas are removed only by exact path after they are inactive and their evidence is retained.

Primary references

• GNU Coreutils dd documentation
• util-linux swapon documentation
• util-linux mkswap documentation
• util-linux fallocate documentation
• XFS xfs_mkfile documentation
• Btrfs swap-file documentation
• Linux kernel virtual-memory sysctl documentation