# Synchronization Using `struct mutex` in Linux Kernel Modules > A **hands-on Linux Kernel Module (LKM)** demonstrating kernel mutexes, kernel threads, critical sections, thread synchronization, and proper module lifecycle management. --- # Table of Contents - Overview - Learning Objectives - Concepts Covered - Project Structure - Build Requirements - Compilation - Module Signing (Secure Boot) - Loading the Module - Viewing Kernel Logs - Unloading the Module - Execution Flow - Program Architecture - Source Code Walkthrough - Kernel Threads - Driver Private Context - Understanding Mutex - Critical Section - `mutex_lock()` - `mutex_lock_interruptible()` - Thread Competition - Expected Execution Timeline - Kernel Log Explanation - Module Cleanup - Important Kernel APIs - Why Mutex Instead of Spinlock? - Common Interview Questions - Exercises - References --- # Overview This project demonstrates one of the most fundamental synchronization primitives in the Linux kernel: - `struct mutex` The module creates **two kernel threads** that continuously compete for a shared mutex while modifying a shared variable. The program intentionally introduces contention so the kernel scheduler blocks one thread while the other owns the mutex. This is exactly how real Linux device drivers protect shared resources. Examples include - USB Drivers - PCI Drivers - Character Drivers - I2C Drivers - SPI Drivers - Network Drivers - Filesystem Drivers --- # Learning Objectives After completing this example you should understand - Linux Kernel Modules - Kernel Threads - Driver Private Data - Shared Resources - Race Conditions - Critical Sections - Sleeping Locks - Mutex Initialization - Mutex Acquisition - Mutex Release - Interruptible Waiting - Thread Scheduling - Proper Cleanup --- # Concepts Covered | Concept | Demonstrated | |----------|--------------| | Linux Kernel Module | ✅ | | `struct mutex` | ✅ | | `mutex_init()` | ✅ | | `mutex_lock()` | ✅ | | `mutex_unlock()` | ✅ | | `mutex_lock_interruptible()` | ✅ | | `mutex_destroy()` | ✅ | | Kernel Threads | ✅ | | `kthread_run()` | ✅ | | `kthread_stop()` | ✅ | | Shared Data | ✅ | | Critical Section | ✅ | | Thread Contention | ✅ | | Race Condition Prevention | ✅ | | Proper Module Cleanup | ✅ | --- # Project Structure ``` mutex_demo/ │ ├── Makefile ├── mutex_demo.c └── README.md ``` --- # Build Requirements - Linux Kernel Headers - GCC - Make - Root Privileges - Secure Boot Keys (if Secure Boot enabled) # High Level Execution Flow ``` insmod │ ▼ module_init() │ ▼ Initialize Mutex │ ▼ shared_counter=0 │ ▼ Create Kernel Thread T1 │ ▼ Create Kernel Thread T2 │ ▼ Both compete for mutex │ ▼ Enter Critical Section │ ▼ Increment Counter │ ▼ Release Mutex │ ▼ Repeat │ ▼ rmmod │ ▼ Stop Both Threads │ ▼ Destroy Mutex │ ▼ Module Exit ``` --- # Program Architecture ``` +------------------------+ | Linux Kernel Module | +-----------+------------+ | | +-----------v------------+ | Driver Private Context | +------------------------+ | shared_counter | | struct mutex mymtx | +-----------+------------+ | +---------------------+---------------------+ | | | | +---------v---------+ +-----------v----------+ | Kernel Thread T1 | | Kernel Thread T2 | | mutex_lock() | | mutex_lock_interruptible() | +---------+---------+ +-----------+----------+ | | +---------------------+---------------------+ | ▼ Shared Counter ``` --- # Driver Private Context ```c struct mydrv_priv { int shared_counter; struct mutex mymtx; }; ``` This structure stores - Driver state - Shared resources - Synchronization objects Nearly every Linux device driver maintains a private structure similar to this. --- # Shared Counter ```c drvctx.shared_counter++; ``` Both threads modify this variable. Without synchronization ``` T1 reads 10 T2 reads 10 T1 writes 11 T2 writes 11 ``` Expected ``` 12 ``` Actual ``` 11 ``` This is called a # Race Condition --- # Critical Section A **critical section** is any code accessing shared resources. ``` mutex_lock() ↓ Modify Shared Data ↓ mutex_unlock() ``` In this example ```c mutex_lock(&drvctx.mymtx); drvctx.shared_counter++; mutex_unlock(&drvctx.mymtx); ``` Only one thread may execute this block at any time. --- # Kernel Threads The module creates two kernel threads. ``` Thread 1 ↓ worker_lock() ``` ``` Thread 2 ↓ worker_interruptible() ``` Creation ```c kthread_run() ``` Stopping ```c kthread_stop() ``` Thread loop ```c while (!kthread_should_stop()) ``` This is the standard Linux kernel thread pattern. --- # Understanding Mutex Mutex means > Mutual Exclusion Only one thread may own the mutex. ``` Mutex Locked │ ▼ Thread 1 Thread 2 waits Thread 3 waits ``` Once unlocked ``` Scheduler wakes ↓ Next waiting thread ``` --- # mutex_init() ```c mutex_init(&drvctx.mymtx); ``` Initializes the mutex. Internally ``` Unlocked Owner = NULL Wait Queue = Empty ``` --- # mutex_lock() ```c mutex_lock(&drvctx.mymtx); ``` If mutex is free ``` Acquire immediately ``` If mutex is busy ``` Sleep ↓ Scheduler switches CPU ↓ Wake later ``` This is called a **sleeping lock**. --- # mutex_unlock() ```c mutex_unlock(&drvctx.mymtx); ``` Releases ownership. If another thread is waiting ``` Wake waiting thread ↓ Scheduler ↓ Acquire mutex ``` --- # mutex_lock_interruptible() ```c ret = mutex_lock_interruptible(&drvctx.mymtx); ``` Unlike ```c mutex_lock() ``` this version may return early if interrupted. Return value ``` 0 ↓ Lock acquired ``` Non-zero ``` Interrupted while waiting ``` Program ```c if (ret) { printk(...); continue; } ``` --- # Thread Competition ``` Time T1 -------------------- LOCK ------------------- UNLOCK T2 waiting LOCK UNLOCK T1 waiting LOCK UNLOCK ``` Only one thread executes inside the critical section. --- # Expected Timeline Counter ``` 0 ``` Thread 1 ``` Acquire Mutex Counter=0 Increment Counter=1 Sleep Unlock ``` Thread 2 ``` Waiting... Acquire Counter=1 Increment Counter=2 Unlock ``` Thread 1 ``` Acquire Counter=2 Increment Counter=3 ``` and so on. --- # Expected Kernel Log ``` Mutex Demo: Module Loaded Mutex initialized T1: waiting for mutex T1: acquired mutex, counter=0 T2: waiting for mutex (interruptible) T1: releasing mutex, counter=1 T2: acquired mutex, counter=1 T2: releasing mutex, counter=2 T1: acquired mutex, counter=2 T1: releasing mutex, counter=3 ... ``` Observe - No overlapping critical sections - Counter always increases correctly - Mutex serializes access --- # Module Cleanup ``` rmmod ↓ module_exit() ↓ kthread_stop(T1) ↓ kthread_stop(T2) ↓ mutex_destroy() ↓ Exit ``` --- # mutex_destroy() ```c mutex_destroy(&drvctx.mymtx); ``` Normally performs little work. Useful when ``` CONFIG_DEBUG_MUTEXES=y ``` Kernel debug builds perform additional consistency checks. --- # Important Kernel APIs | API | Purpose | |------|----------| | `module_init()` | Module entry point | | `module_exit()` | Module exit point | | `kthread_run()` | Create kernel thread | | `kthread_stop()` | Stop kernel thread | | `kthread_should_stop()` | Thread termination check | | `mutex_init()` | Initialize mutex | | `mutex_lock()` | Acquire mutex | | `mutex_unlock()` | Release mutex | | `mutex_lock_interruptible()` | Interruptible acquire | | `mutex_destroy()` | Destroy mutex | | `printk()` | Kernel logging | | `msleep()` | Sleep current thread | --- # Why Use Mutex? Advantages - Simple API - Prevents race conditions - Sleeping lock - Scheduler friendly - No busy waiting - Excellent for long critical sections Ideal for - Device Drivers - Filesystems - Networking - USB - Character Drivers --- # Mutex vs Spinlock | Feature | Mutex | Spinlock | |----------|--------|----------| | Sleeps | Yes | No | | Busy Wait | No | Yes | | Scheduler Friendly | Yes | No | | Long Critical Sections | Yes | No | | Interrupt Context | No | Yes | | Process Context | Yes | Yes | --- # Real Driver Examples Mutexes commonly protect ``` Device Registers Configuration Structures Driver State Shared Buffers Reference Counters Linked Lists Open File State Device Queues ``` --- # Key Takeaways This project demonstrates - Linux Kernel Module lifecycle - Driver private context - Shared resource protection - Mutex initialization - Mutex acquisition - Mutex release - Interruptible mutex locking - Kernel thread creation - Thread synchronization - Scheduler interaction - Critical section protection - Race condition avoidance - Safe module cleanup Although intentionally simple, the synchronization pattern used here is the same pattern employed throughout production Linux kernel drivers. --- ## Summary This Linux Kernel Module serves as a compact yet practical introduction to **sleeping synchronization** in the Linux kernel. By creating two competing kernel threads that protect a shared resource with a mutex, it demonstrates the complete lifecycle of mutex usage—from initialization and acquisition to release and cleanup—while reinforcing essential concepts such as race conditions, critical sections, scheduler interaction, and proper kernel module design. It provides a solid foundation before progressing to more advanced synchronization primitives such as spinlocks, semaphores, completions, wait queues, reader-writer locks, and Read-Copy-Update (RCU). GitHub Repository: 👉 **[linux_kernel_mutex](https://github.com/aj333git/linux_kernel_mutex)** Explore the complete source code, build files, and module implementation on GitHub.