A hands-on Linux Kernel Module (LKM) demonstrating kernel mutexes, kernel threads, critical sections, thread synchronization, and proper module lifecycle management.

---

Table of Contents

• Overview
• Learning Objectives
• Concepts Covered
• Project Structure
• Build Requirements
• Compilation
• Module Signing (Secure Boot)
• Loading the Module
• Viewing Kernel Logs
• Unloading the Module
• Execution Flow
• Program Architecture
• Source Code Walkthrough
• Kernel Threads
• Driver Private Context
• Understanding Mutex
• Critical Section
• mutex_lock()
• mutex_lock_interruptible()
• Thread Competition
• Expected Execution Timeline
• Kernel Log Explanation
• Module Cleanup
• Important Kernel APIs
• Why Mutex Instead of Spinlock?
• Common Interview Questions
• Exercises
• References

---

Overview

This project demonstrates one of the most fundamental synchronization primitives in the Linux kernel:

• struct mutex

The module creates two kernel threads that continuously compete for a shared mutex while modifying a shared variable.

The program intentionally introduces contention so the kernel scheduler blocks one thread while the other owns the mutex.

This is exactly how real Linux device drivers protect shared resources.

Examples include

• USB Drivers
• PCI Drivers
• Character Drivers
• I2C Drivers
• SPI Drivers
• Network Drivers
• Filesystem Drivers

---

Learning Objectives

After completing this example you should understand

• Linux Kernel Modules
• Kernel Threads
• Driver Private Data
• Shared Resources
• Race Conditions
• Critical Sections
• Sleeping Locks
• Mutex Initialization
• Mutex Acquisition
• Mutex Release
• Interruptible Waiting
• Thread Scheduling
• Proper Cleanup

---

Concepts Covered

Concept	Demonstrated
Linux Kernel Module	✅
struct mutex	✅
mutex_init()	✅
mutex_lock()	✅
mutex_unlock()	✅
mutex_lock_interruptible()	✅
mutex_destroy()	✅
Kernel Threads	✅
kthread_run()	✅
kthread_stop()	✅
Shared Data	✅
Critical Section	✅
Thread Contention	✅
Race Condition Prevention	✅
Proper Module Cleanup	✅

---

Project Structure

mutex_demo/
│
├── Makefile
├── mutex_demo.c
└── README.md

---

Build Requirements

• Linux Kernel Headers
• GCC
• Make
• Root Privileges
• Secure Boot Keys (if Secure Boot enabled)

High Level Execution Flow

           insmod
              │
              ▼
     module_init()
              │
              ▼
     Initialize Mutex
              │
              ▼
      shared_counter=0
              │
              ▼
    Create Kernel Thread T1
              │
              ▼
    Create Kernel Thread T2
              │
              ▼
   Both compete for mutex
              │
              ▼
 Enter Critical Section
              │
              ▼
 Increment Counter
              │
              ▼
 Release Mutex
              │
              ▼
 Repeat
              │
              ▼
         rmmod
              │
              ▼
     Stop Both Threads
              │
              ▼
     Destroy Mutex
              │
              ▼
       Module Exit

---

Program Architecture

                    +------------------------+
                    | Linux Kernel Module    |
                    +-----------+------------+
                                |
                                |
                    +-----------v------------+
                    | Driver Private Context |
                    +------------------------+
                    | shared_counter         |
                    | struct mutex mymtx     |
                    +-----------+------------+
                                |
          +---------------------+---------------------+
          |                                           |
          |                                           |
+---------v---------+                     +-----------v----------+
| Kernel Thread T1  |                     | Kernel Thread T2     |
| mutex_lock()      |                     | mutex_lock_interruptible() |
+---------+---------+                     +-----------+----------+
          |                                           |
          +---------------------+---------------------+
                                |
                                ▼
                        Shared Counter

---

Driver Private Context

struct mydrv_priv
{
    int shared_counter;
    struct mutex mymtx;
};

This structure stores

• Driver state
• Shared resources
• Synchronization objects

Nearly every Linux device driver maintains a private structure similar to this.

---

Shared Counter

drvctx.shared_counter++;

Both threads modify this variable.

Without synchronization

T1 reads 10

T2 reads 10

T1 writes 11

T2 writes 11

Expected

12

Actual

11

This is called a

Race Condition

---

Critical Section

A critical section is any code accessing shared resources.

mutex_lock()

↓

Modify Shared Data

↓

mutex_unlock()

In this example

mutex_lock(&drvctx.mymtx);

drvctx.shared_counter++;

mutex_unlock(&drvctx.mymtx);

Only one thread may execute this block at any time.

---

Kernel Threads

The module creates two kernel threads.

Thread 1

↓

worker_lock()

Thread 2

↓

worker_interruptible()

Creation

kthread_run()

Stopping

kthread_stop()

Thread loop

while (!kthread_should_stop())

This is the standard Linux kernel thread pattern.

---

Understanding Mutex

Mutex means

Mutual Exclusion

Only one thread may own the mutex.

          Mutex

      Locked
         │
         ▼

      Thread 1

Thread 2 waits

Thread 3 waits

Once unlocked

Scheduler wakes

↓

Next waiting thread

---

mutex_init()

mutex_init(&drvctx.mymtx);

Initializes the mutex.

Internally

Unlocked

Owner = NULL

Wait Queue = Empty

---

mutex_lock()

mutex_lock(&drvctx.mymtx);

If mutex is free

Acquire immediately

If mutex is busy

Sleep

↓

Scheduler switches CPU

↓

Wake later

This is called a sleeping lock.

---

mutex_unlock()

mutex_unlock(&drvctx.mymtx);

Releases ownership.

If another thread is waiting

Wake waiting thread

↓

Scheduler

↓

Acquire mutex

---

mutex_lock_interruptible()

ret = mutex_lock_interruptible(&drvctx.mymtx);

Unlike

mutex_lock()

this version may return early if interrupted.

Return value

0

↓

Lock acquired

Non-zero

Interrupted while waiting

Program

if (ret)
{
    printk(...);
    continue;
}

---

Thread Competition

Time

T1 -------------------- LOCK ------------------- UNLOCK

                    T2 waiting

                              LOCK

                              UNLOCK

T1 waiting

LOCK

UNLOCK

Only one thread executes inside the critical section.

---

Expected Timeline

Counter

0

Thread 1

Acquire Mutex

Counter=0

Increment

Counter=1

Sleep

Unlock

Thread 2

Waiting...

Acquire

Counter=1

Increment

Counter=2

Unlock

Thread 1

Acquire

Counter=2

Increment

Counter=3

and so on.

---

Expected Kernel Log

Mutex Demo: Module Loaded

Mutex initialized

T1: waiting for mutex

T1: acquired mutex, counter=0

T2: waiting for mutex (interruptible)

T1: releasing mutex, counter=1

T2: acquired mutex, counter=1

T2: releasing mutex, counter=2

T1: acquired mutex, counter=2

T1: releasing mutex, counter=3

...

Observe

• No overlapping critical sections
• Counter always increases correctly
• Mutex serializes access

---

Module Cleanup

rmmod

↓

module_exit()

↓

kthread_stop(T1)

↓

kthread_stop(T2)

↓

mutex_destroy()

↓

Exit

---

mutex_destroy()

mutex_destroy(&drvctx.mymtx);

Normally performs little work.

Useful when

CONFIG_DEBUG_MUTEXES=y

Kernel debug builds perform additional consistency checks.

---

Important Kernel APIs

API	Purpose
module_init()	Module entry point
module_exit()	Module exit point
kthread_run()	Create kernel thread
kthread_stop()	Stop kernel thread
kthread_should_stop()	Thread termination check
mutex_init()	Initialize mutex
mutex_lock()	Acquire mutex
mutex_unlock()	Release mutex
mutex_lock_interruptible()	Interruptible acquire
mutex_destroy()	Destroy mutex
printk()	Kernel logging
msleep()	Sleep current thread

---

Why Use Mutex?

Advantages

• Simple API
• Prevents race conditions
• Sleeping lock
• Scheduler friendly
• No busy waiting
• Excellent for long critical sections

Ideal for

• Device Drivers
• Filesystems
• Networking
• USB
• Character Drivers

---

Mutex vs Spinlock

Feature	Mutex	Spinlock
Sleeps	Yes	No
Busy Wait	No	Yes
Scheduler Friendly	Yes	No
Long Critical Sections	Yes	No
Interrupt Context	No	Yes
Process Context	Yes	Yes

---

Real Driver Examples

Mutexes commonly protect

Device Registers

Configuration Structures

Driver State

Shared Buffers

Reference Counters

Linked Lists

Open File State

Device Queues

---

Key Takeaways

This project demonstrates

• Linux Kernel Module lifecycle
• Driver private context
• Shared resource protection
• Mutex initialization
• Mutex acquisition
• Mutex release
• Interruptible mutex locking
• Kernel thread creation
• Thread synchronization
• Scheduler interaction
• Critical section protection
• Race condition avoidance
• Safe module cleanup

Although intentionally simple, the synchronization pattern used here is the same pattern employed throughout production Linux kernel drivers.

---

Summary

This Linux Kernel Module serves as a compact yet practical introduction to sleeping synchronization in the Linux kernel. By creating two competing kernel threads that protect a shared resource with a mutex, it demonstrates the complete lifecycle of mutex usage—from initialization and acquisition to release and cleanup—while reinforcing essential concepts such as race conditions, critical sections, scheduler interaction, and proper kernel module design. It provides a solid foundation before progressing to more advanced synchronization primitives such as spinlocks, semaphores, completions, wait queues, reader-writer locks, and Read-Copy-Update (RCU).

GitHub Repository: 👉 linux_kernel_mutex Explore the complete source code, build files, and module implementation on GitHub.