# Linux Kernel `rwlock_t` ## Introduction Concurrency is one of the fundamental challenges in Linux kernel development. Multiple kernel threads, interrupt contexts, and subsystem components can access shared kernel state concurrently. Without proper synchronization, even a seemingly simple linked-list operation can result in race conditions, corrupted state, or use-after-free bugs. This experiment demonstrates the use of `rwlock_t` to protect a shared kernel linked list. The module creates: * Two **reader kernel threads** * One **writer kernel thread** * A shared `struct list_head` based linked list * An `rwlock_t` protecting the list The goal is not simply to demonstrate an API, but to understand the **reader-writer synchronization model**, critical-section design, and module-lifecycle implications of protecting shared kernel data. * * * ## What Is `rwlock_t`? Linux provides reader-writer locking for situations where shared data is accessed frequently for reading but less frequently for modification. The basic semantics are: * Multiple readers can hold the lock simultaneously. * A writer requires exclusive ownership. * Readers cannot execute the protected section while a writer owns the lock. * Multiple writers cannot execute simultaneously. The important distinction is therefore: **Reader + Reader → permitted** **Reader + Writer → serialized** **Writer + Writer → serialized** This makes `rwlock_t` fundamentally different from a traditional mutex. * * * ## The Shared Data Structure The experiment uses a kernel linked list. Each element contains an integer value and a Linux kernel `list_head`: ```c struct my_item { int value; struct list_head list; }; ``` The list itself is declared using: ```c static LIST_HEAD(my_list); ``` The list is shared by all three kernel threads, which means every traversal or modification must respect the synchronization rules. * * * ## Why Protect the Linked List? A linked list is more than a collection of independent values. Its correctness depends on the relationships between nodes and the internal `next` and `prev` pointers maintained by the kernel list infrastructure. A writer inserting a node modifies the list structure. At the same time, a reader may be traversing that structure. Therefore, the synchronization requirement is not simply: > Protect the integer value. It is: > Protect the consistency of the shared list while it is being accessed or modified. This distinction becomes extremely important when moving from toy synchronization examples toward real kernel subsystem development. * * * ## Declaring the Reader-Writer Lock The module uses: ```c static DEFINE_RWLOCK(my_rwlock); ``` This creates the reader-writer spinlock used to protect `my_list`. Readers acquire the lock with `read_lock()` and release it with `read_unlock()`. Writers use `write_lock()` and `write_unlock()`. * * * ## Reader-Side Synchronization The reader threads traverse the list while holding the read lock. The essential operation is: ```c read_lock(&my_rwlock); list_for_each_entry(item, &my_list, list) { count++; sum += item->value; } read_unlock(&my_rwlock); ``` The critical section calculates two properties: * Number of elements * Sum of all element values Because the operation is read-only, multiple reader threads can execute this section concurrently. This is the central property being demonstrated by the experiment. * * * ## Concurrent Readers Suppose both reader threads are executing simultaneously. There is no requirement for reader 2 to wait merely because reader 1 already holds the read lock. This allows the synchronization primitive to exploit **read-side concurrency**. However, this does not mean readers execute without synchronization. They are still synchronized with respect to writers. A reader must not traverse the list while a writer is changing its structure. * * * ## Writer-Side Synchronization The writer periodically adds a new node. The shared-list modification occurs under the write lock: ```c write_lock(&my_rwlock); list_add_tail(&item->list, &my_list); write_unlock(&my_rwlock); ``` The write lock provides exclusive access. While the writer holds this lock: * Other writers cannot modify the list. * Readers cannot enter their protected read-side critical section. This creates the required serialization between list traversal and list modification. * * * ## Why Allocation Happens Before the Lock The new object is allocated using `kmalloc()` before modifying the shared list. Conceptually, the sequence is: 1. Allocate memory. 2. Initialize the object. 3. Acquire the write lock. 4. Insert the object into the shared list. 5. Release the write lock. This is a useful critical-section design principle. The lock should protect the **shared-state transition**, not unrelated preparation work. Keeping unnecessary operations outside the lock can reduce lock hold time and therefore reduce contention. * * * ## `rwlock_t` Is a Spinlock-Based Primitive One of the most important technical details in this example is that `rwlock_t` belongs to the **spinlock-based synchronization family**. Therefore, the critical section must be short. The module deliberately performs: ```c read_lock(&my_rwlock); /* short list traversal */ read_unlock(&my_rwlock); ``` The thread then sleeps outside the lock. This distinction matters because sleeping while holding a spinlock-based lock is not a valid synchronization design. * * * ## Why `msleep()` Is Outside the Critical Section The reader thread periodically sleeps between iterations. The writer also sleeps between insertions. Those delays are intentionally placed outside the lock. The design is effectively: **Acquire → operate on shared state → release → sleep** rather than: **Acquire → sleep → operate → release** The second pattern would unnecessarily extend lock ownership and block other participants. For kernel synchronization, **critical-section duration matters**. * * * ## `rwlock_t` Versus `mutex` Both `rwlock_t` and mutexes provide synchronization, but their execution models differ. A mutex provides exclusive ownership to one thread. An `rwlock_t` allows multiple readers while maintaining exclusive writer access. For example, a read-heavy workload can conceptually have: * Reader 1 reading * Reader 2 reading * Reader 3 reading at the same time. A writer must wait until the relevant readers have left their critical sections. This makes reader-writer synchronization useful for particular access patterns, but it does not mean an `rwlock_t` should automatically replace a mutex. The correct primitive depends on the execution context, contention characteristics, critical-section duration, and workload. * * * ## Kernel Threads and Shared State The module uses kernel threads through the `kthread` infrastructure. Each reader executes a loop controlled by: ```c while (!kthread_should_stop()) ``` The writer follows the same lifecycle model. This creates a simple concurrent environment in which multiple execution contexts interact with the same kernel data structure. The experiment therefore combines two important concepts: **Thread lifecycle management** and **Shared-state synchronization** Understanding both is necessary for writing safe kernel modules. * * * ## Module Teardown Is Part of Synchronization Synchronization does not end when the main operation finishes. Module unloading also requires careful ordering. The module first stops the kernel threads and only then frees the linked-list objects. The lifecycle is therefore: **Stop threads → ensure thread termination → free shared objects** This prevents a reader from continuing to access a list node after the node has been released. This is a fundamental kernel concept: > Resource lifetime and concurrency must be designed together. * * * ## Safe Linked-List Cleanup The cleanup routine uses `list_for_each_entry_safe()` because the current node is removed during traversal. The basic operation is: ```c list_for_each_entry_safe(item, tmp, &my_list, list) { list_del(&item->list); kfree(item); } ``` The `_safe` variant preserves the information required to continue traversal after removing the current element. This is different from merely traversing a list for read-only purposes. * * * ## What Happens During Execution? Initially, the module creates several list elements. The readers observe the initial list state. The writer subsequently adds new elements. As the list grows, the readers observe increasing element counts and sums. The interesting part is not the arithmetic itself. The important observation is that the readers and writer are accessing the same kernel data structure under a defined synchronization protocol. This gives a practical demonstration of how a kernel lock establishes ownership and exclusion around shared state. * * * ## Why This Is More Than a Counter Example A simple shared counter demonstrates atomicity or mutual exclusion, but it does not expose the structural problems involved in protecting a complex data structure. Here, the protected object is a linked list. The synchronization protects: * List traversal * List insertion * Structural consistency * Object visibility * Object lifetime during teardown This makes the example a useful bridge between basic locking experiments and more realistic kernel subsystem synchronization. * * * ## Important Design Lessons ### 1\. Protect Shared State Identify exactly which data is shared between execution contexts and define the synchronization boundary around it. ### 2\. Keep Critical Sections Small Lock hold time directly affects contention. ### 3\. Do Not Sleep While Holding a Spinlock-Based Lock `rwlock_t` is not a sleeping lock. ### 4\. Synchronize Data Structure Operations Protect structural modifications, not just individual fields. ### 5\. Consider Object Lifetime Stopping concurrent users before freeing shared objects is essential. ### 6\. Choose the Primitive Based on the Workload `rwlock_t`, mutexes, spinlocks, semaphores, atomics, and RCU solve different synchronization problems. * * * ## What This Experiment Teaches This small kernel module connects several Linux kernel concepts: * **Kernel threads** * **Reader-writer synchronization** * `rwlock_t` * **Kernel linked lists** * **Dynamic memory allocation** * **Critical-section design** * **Concurrent readers** * **Exclusive writers** * **Thread termination** * **Resource lifetime** * **Module cleanup** The deeper lesson is that synchronization is not simply about preventing two threads from changing the same variable. It is about maintaining the **invariants of shared kernel state across concurrent execution**. * * * ## Beyond This Example Once the basic `rwlock_t` model is understood, several more advanced experiments become interesting. Possible extensions include: * Measuring **lock contention** * Increasing the number of reader threads * Increasing writer frequency * Comparing `rwlock_t` with a mutex * Investigating lock hold time * Studying lock ordering * Exploring **lockdep** * Comparing `rwlock_t` with **RCU** * Studying read-heavy scalability * Examining synchronization under different kernel configurations These experiments move the discussion from API usage toward **kernel concurrency design and performance analysis**. * * * # Conclusion This `rwlock_t` demonstration shows how Linux kernel code can safely coordinate concurrent access to a shared linked list. The key property is simple but powerful: **multiple readers can execute concurrently, while writers require exclusive access**. The real value of the experiment lies beyond the API itself. It demonstrates how synchronization boundaries, critical-section duration, kernel-thread lifecycle, linked-list integrity, and object lifetime must work together. For Linux kernel programming, this is an important step toward understanding more advanced synchronization mechanisms such as **spinlocks, mutexes, semaphores, atomics, lockdep, and RCU**. GitHub Repository: 👉 [**linux\_kernel\_rw\_2**](https://github.com/aj333git/linux_kernel_rw_2) Explore the complete source code, build files, and module implementation on GitHub.