How to Investigate Linux Memory Leaks on a Server

Suspected Linux memory leaks need a timeline, not just a screenshot showing little free RAM. Linux uses spare memory for useful cache. Applications also retain memory for reuse. Neither behavior proves a leak.

The stronger warning is continued growth under comparable workloads, without a stable plateau across repeated cycles. This guide focuses on collecting that evidence. It separates host memory, individual processes, and service limits before recommending deeper application testing.

Define the Suspected Leak Before Changing Anything

A true leak occurs when software retains allocations it no longer needs and cannot reuse or release properly. However, legitimate caches, growing queues, and inefficient workloads can produce similar symptoms.

Start by recording the affected service, business impact, and approximate start time. Note recent deployments, traffic changes, scheduled tasks, and configuration edits.

• Does memory grow continuously or only during busy periods?
• After the same workload repeats, does memory usage stabilize?
• Does a service restart reduce usage temporarily?
• Are requests failing, workers restarting, or queues growing?
• Is the problem inside a container or on the whole host?

A restart that releases memory supports a process-related explanation. It does not prove a programming defect. Restarting also clears legitimate caches and unfinished work.

If the server faces immediate failure, prioritize controlled recovery. Otherwise, preserve measurements before restarting. Avoid clearing caches, disabling swap, or changing limits just to improve a dashboard.

Separate Linux Cache From Actual Memory Pressure

Begin with a host-level snapshot:

date -Is
free -h
vmstat 5 12

These commands inspect state on typical Linux installations. Review tool availability and access rules first. The sampling command runs for roughly one minute.

In free, focus on available, not only free. Available memory estimates what applications can use without swapping. Linux can reclaim much of its file cache when applications need space.

In vmstat, inspect si and so, which show swap movement. The first report generally summarizes activity since boot; later reports reflect each sampling interval. Existing swap usage alone does not prove current pressure.

Falling available memory, sustained swap activity, and worsening response times deserve attention. Large cache totals without those symptoms may reflect normal operation.

Also check shared memory, temporary filesystems, and kernel allocations if process totals do not explain host usage. Simply adding process memory figures will not produce an accurate host total.

Track Process RSS to Investigate Linux Memory Leaks

Resident set size, or RSS, measures a process’s memory currently resident in physical RAM. It is useful for trends, but it includes shared pages.

ps -eo pid,ppid,etimes,rss,vsz,comm --sort=-rss | head -n 16

With typical procps tools, RSS and virtual size appear in KiB. Virtual size, shown as vsz, includes reserved address space. A large virtual size alone does not establish a leak.

Select a candidate process, then sample it over time. Replace 1234 with the verified process ID:

for sample in $(seq 1 60); do
  date -Is
  ps -p 1234 -o pid=,etimes=,rss=,vsz=,comm=
  sleep 60
done

This bounded loop collects about an hour of observations. Stop it with Ctrl+C if needed. Record results in an approved monitoring system or protected file.

Track process identity as well as memory. After a restart, the old ID disappears; Linux can later reuse it. A worker-based service may distribute growth across several processes, so follow the complete worker group too.

Look Beyond RSS Before Calling It a Leak

RSS can double-count shared pages when you add multiple processes together. Proportional set size, or PSS, divides shared pages among the processes using them.

For a closer view, inspect the candidate’s memory summary where available:

cat /proc/1234/smaps_rollup

Replace the process ID first. Permissions and kernel support vary. Mapping inspection adds overhead, so avoid frequent collection across every process on a busy host.

Compare RSS, PSS, and anonymous memory over time. Anonymous memory often contains application heaps and stacks, unlike file-backed mappings. Its growth narrows the investigation but still does not prove a leak.

Managed runtimes may retain heap space after garbage collection. Memory allocators can also keep freed regions for reuse or suffer fragmentation. Use runtime-specific metrics to distinguish live application objects from reserved heap capacity.

The practical question is whether comparable work causes continued retained growth. A rising graph that eventually levels off may indicate a bounded cache rather than leaked allocations.

Inspect OOM Events and Service Restarts Together

An out-of-memory event, or OOM event, means an allocation domain could not satisfy memory demand. The kernel may kill a process to recover. The victim is not necessarily the original source of growth.

journalctl -k --since "2 hours ago" --no-pager
journalctl -u example.service --since "2 hours ago" --no-pager

Replace example.service with the actual unit name. Review kernel entries for messages such as Out of memory, Killed process, and Memory cgroup out of memory.

Match timestamps against application failures and process changes. Kernel OOM handling differs from userspace memory managers, such as systemd-oomd. Their logs may explain terminations without a matching kernel OOM entry.

Consult the journalctl manual for filtering options. Missing journal entries do not rule out memory pressure, especially after log rotation or nonpersistent logging.

If automatic restarts hide the pattern, build a service restart timeline. A repeated rise followed by a sudden drop may reflect process replacement, not recovery within the same process.

Check cgroup Memory Limits, Not Just Host Capacity

Control groups, or cgroups, account for and constrain groups of processes. A container or service can hit its own memory limit while the host still has available RAM.

For a systemd service, inspect the unit’s accounting and configuration:

systemctl show example.service \
  -p MainPID -p ControlGroup -p MemoryCurrent \
  -p MemoryHigh -p MemoryMax -p NRestarts

Property availability and accounting behavior depend on the installed systemd version and configuration. The main process alone may not represent the whole service.

On cgroup v2 systems, locate the service’s directory using its reported control-group path. Relevant files include:

• Current usage: memory.current shows memory charged to the group.
• The hard limit: memory.max gives its ceiling, or max.
• Event counters: memory.events records pressure, limit encounters, and OOM activity.
• Usage breakdown: memory.stat includes anonymous and file-backed memory.

Compare event-counter changes during the incident, not just their absolute values. Check parent groups too; an ancestor may impose the effective limit. Group totals include charges beyond summed process RSS, including file cache.

Do not raise limits blindly. That can move the failure from one service to the entire host.

Correlate Memory Growth With Comparable Workloads

For Linux memory leaks, workload correlation matters more than an isolated high reading. Collect request volume, concurrent jobs, queue depth, worker count, and memory on one timeline.

Compare quiet periods with busy periods and repeated batch runs. Note deployments and scheduled imports. More workers naturally consume more memory, even when each worker remains stable.

• Growth followed by a plateau: investigate cache sizing and normal warm-up.
• Growth matching queue depth: investigate slow consumers or retained jobs.
• Repeated growth after equivalent jobs: prioritize application-level allocation analysis.
• Stable RSS but rising group usage: inspect cache, child processes, and other group charges.

If open connections rise alongside memory, examine file descriptor and socket exhaustion as a related symptom.

Observe beyond one short burst. Some applications release memory only after idle cleanup or garbage collection. Choose a window that covers the application’s normal lifecycle.

Validate the Fix Without Losing the Evidence

Once you identify a likely component, reproduce the workload in staging when practical. Application profiling can reveal retained allocations, but attachment and instrumentation may increase overhead.

Heap dumps can contain credentials, customer records, and other sensitive data. Confirm authorization, storage capacity, and handling rules before collecting them.

Change one factor at a time, then repeat comparable work. A useful fix produces a stable memory envelope without unacceptable latency or errors. Temporary worker recycling may contain risk, but it does not identify the underlying defect.

Escalate with timestamps, process identities, RSS trends, cgroup limits, OOM evidence, and workload details. Professional remote server assistance may help when outages recur or profiling threatens production stability. Tech Rescue Ops LLC can help organize that investigation without treating every high memory reading as a leak.