Processes vs Threads

Every execution path on your computer runs inside either a Process or a Thread. Understanding how operating systems manage these two units of execution—specifically their memory boundaries, isolation levels, and communication overhead—is a foundational requirement for low-level system design.

1. Learning Objectives

• Differentiate between processes (isolated memory) and threads (shared memory).
• Analyze virtual memory mapping, page tables, and Translation Lookaside Buffer (TLB) invalidation during context switches.
• Compare Inter-Process Communication (IPC) with Inter-Thread Communication (ITC) methods and performance.
• Evaluate the reliability and fault tolerance trade-offs of multi-processing versus multi-threading.
• Measure context switching latency differences between processes and threads.

2. Problem & Naive Solution

Suppose you are building a web scraping crawler. The scraper needs to download 1,000 pages and parse their HTML. The page download is I/O-bound (network latency), while parsing is CPU-bound.

The Naive Solution (Processes for everything)

A developer decides to use the OS command line tool to launch a brand new process for *each* download task, executing curl or wget in a loop:

3. Issues with the Naive Approach

• Huge Resource Overhead: Spawning a process requires allocating virtual memory space, creating new page tables, and loading executable binaries. Spawning thousands of processes will quickly exhaust system RAM.
• Slow Context Switching: The OS must switch page directories and flush cache lines (TLB cache) when changing processes, causing massive context switching delays.
• Complex Communication: Because processes have isolated address spaces, passing downloaded HTML data back to the main aggregator process requires heavy IPC mechanisms (pipes, sockets, or shared memory).

4. Memory Layout Comparison

A process contains its own text, data, and heap. Sibling threads nested inside a process share this heap but maintain private stacks and program counters.

5. Theory: OS Virtual Memory, TLB & Context Switching

Let's explore the core OS and hardware-level differences between processes and threads:

A. OS Address Spaces & Isolation

• Processes (Heavyweight): Possess their own virtual-to-physical memory mapping (Page Table). If Process A attempts to write to an address belonging to Process B, the CPU MMU (Memory Management Unit) throws a Segment Fault (SegFault), blocking cross-process corruption. This ensures high fault isolation.
• Threads (Lightweight): Share the same page table. If Thread 1 crashes with a NullPointer or memory exception, it can corrupt shared heap references, crashing all threads inside the process.

B. Context Switch Overhead: Process vs. Thread

A context switch requires changing CPU execution states:

• Thread Context Switch: The CPU swaps register states, stack pointer (SP), and the program counter (PC). The page directory reference remains unchanged. This is fast (typically < 1-2 microseconds).
• Process Context Switch: In addition to register swapping, the CPU must switch the active page directory (CR3 register in x86). This invalidates the Translation Lookaside Buffer (TLB)—the hardware cache mapping virtual addresses to physical RAM. The CPU must perform slow RAM lookups to rebuild the cache, degrading execution performance.

C. Communication Boundaries (IPC vs. ITC)

• Inter-Process Communication (IPC): Requires OS-level mechanisms because processes are isolated. Includes Pipes (standard input/output redirection), Unix Domain Sockets, Message Queues, and Shared Memory segments. IPC involves system calls and context switches, making it relatively slow.
• Inter-Thread Communication (ITC): Fast because threads share variables on the heap directly. However, they require synchronization (locks, volatiles) to prevent concurrent write corruption.

6. Syntax Explanation

To implement multiprocessing or multithreading, we use core runtime classes:

• Java: Launching processes is managed by ProcessBuilder. Threads are instantiated by passing Runnable functional structures to Thread.
• Python: Provides multiprocessing.Process for process-level isolation (allocating separate memory maps) and threading.Thread for shared heap threading.
• C++: Creating threads uses std::thread. Spawning subprocesses in native Unix systems uses fork() and execvp(), requiring manual file descriptor routing.

7. Step-by-Step Implementation

1. Determine Execution Boundaries: Identify if the task requires process-level isolation (e.g., executing third-party binaries) or thread-level speed.
2. Spawning Processes: Configure argument lists, route input/output streams, and call startup triggers.
3. Spawning Threads: Wrap tasks in thread objects and invoke execution.
4. Manage Lifecycles: Wait for subprocesses (waitFor()) or threads (join()) to complete.

8. Complete Code (Process Spawning vs. Thread Execution)

This program compares spawning a separate process (launching a system ping) and spawning a concurrent thread within the application.

9. Code Walkthrough

• Process Creation Overhead: The process example calls external ping binaries (ProcessBuilder, subprocess.run, std::system). This requires the OS kernel to verify file permissions, duplicate virtual memory pages, and establish standard file descriptors, which takes significantly longer than creating a thread.
• Shared Memory Thread Execution: The thread example targets a local lambda function or worker method. Because it runs within the caller's memory mapping, the OS does not create new page tables, resulting in much faster execution.

10. Execution Flow

1. Process Fork Initiated: The parent process requests memory allocation from the OS kernel.
2. Page Table Duplication: The OS kernel duplicates page table mappings and marks physical RAM pages as Copy-on-Write (COW).
3. Execution Swap (TLB Flush): The CPU switches the active page directory register to point to the new process's page table. The TLB is flushed.
4. Execution Complete: The child process exits, and the kernel triggers an interrupt to notify the parent thread.

11. Complexity Analysis

• Time Complexity:
  • Process Context Switch: $O(\text{Pages})$ since virtual memory mapping registers must be swapped and the TLB is invalidated.
  • Thread Context Switch: $O(1)$ registers swap.
• Space Complexity:
  • Process allocation: $O(\text{Address Space})$ requiring virtual memory structures.
  • Thread allocation: $O(1)$ fixed stack allocation (typically 1MB).

12. Best Practices

• Use Processes for Isolation: Run unsafe, third-party executables or scripts in separate processes. If the script crashes or leaks memory, the main application remains unaffected.
• Use Threads for High-Throughput I/O: Use threads for tasks like handling API requests or network connections, where speed and shared memory access are critical.

13. Common Mistakes

• Failing to Clean Up Zombie Processes: Forgetting to call waitFor() or read exit statuses on spawned processes. This prevents the OS from cleaning up the process table, leaving "zombie" entries that exhaust system resources.
• Sharing Mutable Data Without Locks: Sharing memory between threads without synchronization, causing data corruption due to race conditions.

14. Interview Discussion

15. Practice Exercises

• Easy: Write a program that spawns a thread and prints the thread ID along with the process ID. Compare the process IDs of the main and worker threads.
• Medium: Write a Python script that benchmarks the time taken to spawn 100 threads versus 100 subprocesses.
• Hard: Build a basic Inter-Process Communication system using shared memory files (mmap) to pass data between two separate Python processes.

16. Challenge Problem

Design an OS-level Task Isolation Hub. The hub takes custom Java code snippets as strings, compiles them, and runs them. To prevent user snippets from freezing the hub (e.g., via infinite loops or memory leaks), execute each snippet in an isolated subprocess. Set a 5-second timeout on the subprocess. If a snippet times out or crashes, terminate the subprocess without crashing the main application.

17. Summary & Cheat Sheet

18. Quiz

19. Next Lesson Preview

In the next lesson, we will cover the Thread Lifecycle & States. We will study the states a thread transitions through (NEW, RUNNABLE, BLOCKED, WAITING, TIMED_WAITING, TERMINATED) and how scheduler triggers control them!