Introduction

Concurrency is one of the fundamental challenges in Linux kernel development. Multiple kernel threads, interrupt contexts, and subsystem components can access shared kernel state concurrently. Without proper synchronization, even a seemingly simple linked-list operation can result in race conditions, corrupted state, or use-after-free bugs.

This experiment demonstrates the use of rwlock_t to protect a shared kernel linked list.

The module creates:

• Two reader kernel threads

• One writer kernel thread

• A shared struct list_head based linked list

• An rwlock_t protecting the list

The goal is not simply to demonstrate an API, but to understand the reader-writer synchronization model, critical-section design, and module-lifecycle implications of protecting shared kernel data.

---

What Is rwlock_t?

Linux provides reader-writer locking for situations where shared data is accessed frequently for reading but less frequently for modification.

The basic semantics are:

• Multiple readers can hold the lock simultaneously.

• A writer requires exclusive ownership.

• Readers cannot execute the protected section while a writer owns the lock.

• Multiple writers cannot execute simultaneously.

The important distinction is therefore:

Reader + Reader → permitted

Reader + Writer → serialized

Writer + Writer → serialized

This makes rwlock_t fundamentally different from a traditional mutex.

---

The Shared Data Structure

The experiment uses a kernel linked list.

Each element contains an integer value and a Linux kernel list_head:

struct my_item {
    int value;
    struct list_head list;
};

The list itself is declared using:

static LIST_HEAD(my_list);

The list is shared by all three kernel threads, which means every traversal or modification must respect the synchronization rules.

---

Why Protect the Linked List?

A linked list is more than a collection of independent values.

Its correctness depends on the relationships between nodes and the internal next and prev pointers maintained by the kernel list infrastructure.

A writer inserting a node modifies the list structure.

At the same time, a reader may be traversing that structure.

Therefore, the synchronization requirement is not simply:

Protect the integer value.

It is:

Protect the consistency of the shared list while it is being accessed or modified.

This distinction becomes extremely important when moving from toy synchronization examples toward real kernel subsystem development.

---

Declaring the Reader-Writer Lock

The module uses:

static DEFINE_RWLOCK(my_rwlock);

This creates the reader-writer spinlock used to protect my_list.

Readers acquire the lock with read_lock() and release it with read_unlock().

Writers use write_lock() and write_unlock().

---

Reader-Side Synchronization

The reader threads traverse the list while holding the read lock.

The essential operation is:

read_lock(&my_rwlock);

list_for_each_entry(item, &my_list, list) {
    count++;
    sum += item->value;
}

read_unlock(&my_rwlock);

The critical section calculates two properties:

• Number of elements

• Sum of all element values

Because the operation is read-only, multiple reader threads can execute this section concurrently.

This is the central property being demonstrated by the experiment.

---

Concurrent Readers

Suppose both reader threads are executing simultaneously.

There is no requirement for reader 2 to wait merely because reader 1 already holds the read lock.

This allows the synchronization primitive to exploit read-side concurrency.

However, this does not mean readers execute without synchronization.

They are still synchronized with respect to writers.

A reader must not traverse the list while a writer is changing its structure.

---

Writer-Side Synchronization

The writer periodically adds a new node.

The shared-list modification occurs under the write lock:

write_lock(&my_rwlock);

list_add_tail(&item->list, &my_list);

write_unlock(&my_rwlock);

The write lock provides exclusive access.

While the writer holds this lock:

• Other writers cannot modify the list.

• Readers cannot enter their protected read-side critical section.

This creates the required serialization between list traversal and list modification.

---

Why Allocation Happens Before the Lock

The new object is allocated using kmalloc() before modifying the shared list.

Conceptually, the sequence is:

1. Allocate memory.

2. Initialize the object.

3. Acquire the write lock.

4. Insert the object into the shared list.

5. Release the write lock.

This is a useful critical-section design principle.

The lock should protect the shared-state transition, not unrelated preparation work.

Keeping unnecessary operations outside the lock can reduce lock hold time and therefore reduce contention.

---

rwlock_t Is a Spinlock-Based Primitive

One of the most important technical details in this example is that rwlock_t belongs to the spinlock-based synchronization family.

Therefore, the critical section must be short.

The module deliberately performs:

read_lock(&my_rwlock);

/* short list traversal */

read_unlock(&my_rwlock);

The thread then sleeps outside the lock.

This distinction matters because sleeping while holding a spinlock-based lock is not a valid synchronization design.

---

Why msleep() Is Outside the Critical Section

The reader thread periodically sleeps between iterations.

The writer also sleeps between insertions.

Those delays are intentionally placed outside the lock.

The design is effectively:

Acquire → operate on shared state → release → sleep

rather than:

Acquire → sleep → operate → release

The second pattern would unnecessarily extend lock ownership and block other participants.

For kernel synchronization, critical-section duration matters.

---

rwlock_t Versus mutex

Both rwlock_t and mutexes provide synchronization, but their execution models differ.

A mutex provides exclusive ownership to one thread.

An rwlock_t allows multiple readers while maintaining exclusive writer access.

For example, a read-heavy workload can conceptually have:

• Reader 1 reading

• Reader 2 reading

• Reader 3 reading

at the same time.

A writer must wait until the relevant readers have left their critical sections.

This makes reader-writer synchronization useful for particular access patterns, but it does not mean an rwlock_t should automatically replace a mutex.

The correct primitive depends on the execution context, contention characteristics, critical-section duration, and workload.

---

Kernel Threads and Shared State

The module uses kernel threads through the kthread infrastructure.

Each reader executes a loop controlled by:

while (!kthread_should_stop())

The writer follows the same lifecycle model.

This creates a simple concurrent environment in which multiple execution contexts interact with the same kernel data structure.

The experiment therefore combines two important concepts:

Thread lifecycle management

and

Shared-state synchronization

Understanding both is necessary for writing safe kernel modules.

---

Module Teardown Is Part of Synchronization

Synchronization does not end when the main operation finishes.

Module unloading also requires careful ordering.

The module first stops the kernel threads and only then frees the linked-list objects.

The lifecycle is therefore:

Stop threads → ensure thread termination → free shared objects

This prevents a reader from continuing to access a list node after the node has been released.

This is a fundamental kernel concept:

Resource lifetime and concurrency must be designed together.

---

Safe Linked-List Cleanup

The cleanup routine uses list_for_each_entry_safe() because the current node is removed during traversal.

The basic operation is:

list_for_each_entry_safe(item, tmp, &my_list, list) {
    list_del(&item->list);
    kfree(item);
}

The _safe variant preserves the information required to continue traversal after removing the current element.

This is different from merely traversing a list for read-only purposes.

---

What Happens During Execution?

Initially, the module creates several list elements.

The readers observe the initial list state.

The writer subsequently adds new elements.

As the list grows, the readers observe increasing element counts and sums.

The interesting part is not the arithmetic itself.

The important observation is that the readers and writer are accessing the same kernel data structure under a defined synchronization protocol.

This gives a practical demonstration of how a kernel lock establishes ownership and exclusion around shared state.

---

Why This Is More Than a Counter Example

A simple shared counter demonstrates atomicity or mutual exclusion, but it does not expose the structural problems involved in protecting a complex data structure.

Here, the protected object is a linked list.

The synchronization protects:

• List traversal

• List insertion

• Structural consistency

• Object visibility

• Object lifetime during teardown

This makes the example a useful bridge between basic locking experiments and more realistic kernel subsystem synchronization.

---

Important Design Lessons

1. Protect Shared State

Identify exactly which data is shared between execution contexts and define the synchronization boundary around it.

2. Keep Critical Sections Small

Lock hold time directly affects contention.

3. Do Not Sleep While Holding a Spinlock-Based Lock

rwlock_t is not a sleeping lock.

4. Synchronize Data Structure Operations

Protect structural modifications, not just individual fields.

5. Consider Object Lifetime

Stopping concurrent users before freeing shared objects is essential.

6. Choose the Primitive Based on the Workload

rwlock_t, mutexes, spinlocks, semaphores, atomics, and RCU solve different synchronization problems.

---

What This Experiment Teaches

This small kernel module connects several Linux kernel concepts:

• Kernel threads

• Reader-writer synchronization

• rwlock_t

• Kernel linked lists

• Dynamic memory allocation

• Critical-section design

• Concurrent readers

• Exclusive writers

• Thread termination

• Resource lifetime

• Module cleanup

The deeper lesson is that synchronization is not simply about preventing two threads from changing the same variable.

It is about maintaining the invariants of shared kernel state across concurrent execution.

---

Beyond This Example

Once the basic rwlock_t model is understood, several more advanced experiments become interesting.

Possible extensions include:

• Measuring lock contention

• Increasing the number of reader threads

• Increasing writer frequency

• Comparing rwlock_t with a mutex

• Investigating lock hold time

• Studying lock ordering

• Exploring lockdep

• Comparing rwlock_t with RCU

• Studying read-heavy scalability

• Examining synchronization under different kernel configurations

These experiments move the discussion from API usage toward kernel concurrency design and performance analysis.

---

Conclusion

This rwlock_t demonstration shows how Linux kernel code can safely coordinate concurrent access to a shared linked list.

The key property is simple but powerful: multiple readers can execute concurrently, while writers require exclusive access.

The real value of the experiment lies beyond the API itself. It demonstrates how synchronization boundaries, critical-section duration, kernel-thread lifecycle, linked-list integrity, and object lifetime must work together.

For Linux kernel programming, this is an important step toward understanding more advanced synchronization mechanisms such as spinlocks, mutexes, semaphores, atomics, lockdep, and RCU.

GitHub Repository: 👉 linux_kernel_rw_2 Explore the complete source code, build files, and module implementation on GitHub.