Skip to content

Commit 50308ce

Browse files
dbalutadbaluta
committed
Documentation: Add the documentation for assignment 3
Add SO2 assigment 3. Original is at: https://ocw.cs.pub.ro/courses/so2/teme/tema3 The assignment requests to create a software RAID1 kernel module. Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
1 parent b754618 commit 50308ce

File tree

2 files changed

+182
-0
lines changed

2 files changed

+182
-0
lines changed

‎Documentation/teaching/so2/assign3-software-raid.rst

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
===========================
2+
Assignment 3 - Software RAID
3+
===========================
4+
5+
- Deadline: :command:`Tuesday, May, 11, 2021, 23:00`
6+
- This assignment can be made in teams (max 2). Only one of them must submit the assignment, and the names of the student should be listed in a README file.
7+
8+
Implementing a software RAID module that uses a logical block device that will read and write data from two physical devices,
9+
ensuring the consistency and synchronization of data from the two physical devices. The type of RAID implemented will be similar to a `RAID 1`.
10+
11+
Assignment's Objectives
12+
=======================
13+
14+
* in-depth understanding of how the I/O subsystem works.
15+
* acquire advanced skills working with `bio` structures.
16+
* work with the block / disk devices in the Linux kernel.
17+
* acquire skills to navigate and understand the code and API dedicated to the I/O subsystem in Linux.
18+
19+
20+
Statement
21+
=========
22+
23+
Write a kernel module that implements the RAID software functionality. `Software RAID <https://en.wikipedia.org/wiki/RAID#Software-based_RAID>`__ provides an abstraction between
24+
the logical device and the physical devices. The implementation will use `RAID scheme 1 <https://en.wikipedia.org/wiki/RAID#Standard_levels>`__.
25+
26+
The virtual machine has two hard disks that will represent the physical devices: `/dev/vdb` and `/dev/vdc`. The operating system
27+
will provide a logical device (block type) that will interface the access from the user space. Writing requests to the logical device
28+
will result in two writes, one for each hard disk. Hard disks are not partitioned. It will be considered that each hard disk has a
29+
single partition that covers the entire disk.
30+
31+
Each partition will store a sector along with an associated checksum (CRC32) to ensure error recovery. At each reading, the related
32+
information from both partitions is read. If a sector of the first partition has corrupt data (CRC value is wrong) then the sector
33+
on the second partition will be read; at the same time the sector of the first partition will be corrected. Similar in the case of
34+
a reading of a corrupt sector on the second partition. If a sector has incorrect CRC values on both partitions, an appropriate error
35+
code will be returned.
36+
37+
Important to know
38+
-----------------
39+
40+
To ensure error recovery, a CRC code is associated with each sector. CRC codes are stored by LOGICAL_DISK_SIZE byte of the partition
41+
(macro defined in the assignment `header <http://elf.cs.pub.ro/so2/res/teme/ssr.h>`__). The disk structure will have the following layout:
42+
43+
44+
.. code-block:: console
45+
46+
+-----------+-----------+-----------+ +---+---+---+
47+
| sector1 | sector2 | sector3 |.....|C1 |C2 |C3 |
48+
+-----------+-----------+-----------+ +---+---+---+
49+
50+
where ``C1``, ``C2``, ``C3`` are the values CRC sectors ``sector1``, ``sector2``, ``sector3``. The CRC area is found immediately after the ``LOGICAL_DISK_SIZE`` bytes of the partition.
51+
52+
As a seed for CRC use 0(zero).
53+
54+
Implementation Details
55+
======================
56+
57+
- the kernel module will be named ``ssr.ko``
58+
- the logical device will be accessed as a block device with the major ``SSR_MAJOR`` and minor ``SSR_FIRST_MINOR`` under the name ``/dev/ssr`` (via the macro ``LOGICAL_DISK_NAME``)
59+
- the virtual device (``LOGICAL_DISK_NAME`` - ``/dev/ssr``) will have the capacity of ``LOGICAL_DISK_SECTORS`` (use ``set_capacity`` with the ``struct gendisk`` structure)
60+
- the two disks are represented by the devices ``/dev/vdb``, respectively ``/dev/vdc``, defined by means of macros ``PHYSICAL_DISK1_NAME``, respectively ``PHYSICAL_DISK2_NAME``
61+
- to work with the ``struct block _device`` structure associated with a physical device, you can use the ``blkdev_get_by_path`` and ``blkdev_put`` functions
62+
- for the handling of requests from the user space, we recommend not to use a ``request_queue``, but to do processing at :c:type:`struct bio` level
63+
using the ``submit_bio`` field of :c:type:`struct block_device_operations`
64+
- since data sectors are separated from CRC sectors you will have to build separate ``bio`` structures for data and CRC values
65+
- to allocate a :c:type:`struct bio` for physical disks you can use :c:func:`bio_alloc`; to add data pages to bio use :c:func:`alloc_page` and :c:func:`bio_add_page`
66+
- to free up the space allocated for a :c:type:`struct bio` you need to release the pages allocated to the bio (using the :c:func:`__free_page` macro ) and call
67+
:c:func:`bio_put`
68+
- when generating a :c:type:`struct bio` structure, consider that its size must be multiple of the disk sector size (``KERNEL_SECTOR_SIZE``)
69+
- to send a request to a block device and wait for it to end, you can use the :c:func:`submit_bio_wait` function
70+
- use :c:func:`bio_endio` to signal the completion of processing a ``bio`` structure
71+
- for the CRC32 calculation you can use the :c:func:`crc32` macro provided by the kernel
72+
- useful macro definitions can be found in the assignment support `header <http://elf.cs.pub.ro/so2/res/teme/ssr.h>`__
73+
- a single request processing function for block devices can be active at one time in a call stack (more details `here <https://elixir.bootlin.com/linux/v5.10/source/block/blk-core.c#L1048>`__).
74+
You will need to submit requests for physical devices in a kernel thread; we recommend using ``workqueues``.
75+
- For a quick run, use a single bio to batch send the read/write request for CRC values for adjacent sectors. For example,
76+
if you need to send requests for CRCs in sectors 0, 1, ..., 7, use a single bio, not 8 bios.
77+
- our recommendations are not mandatory (any solution that meets the requirements of the assignment is accepted)
78+
Testing
79+
=======
80+
In order to simplify the assignment evaluation process, but also to reduce the mistakes of the
81+
submitted assignments, the assignment evaluation will be done automatically with with the help of
82+
public tests that are in the new infrastructure. For local testing, use the following commands:
83+
84+
.. code-block:: console
85+
86+
$ git clone https://github.com/linux-kernel-labs/linux.git
87+
$ cd linux/tools/labs
88+
$ LABS=assignments/3-raid make skels
89+
$ #the development of the assignment will be written in the 3-raid directory
90+
$ make build
91+
$ make copy
92+
$ make boot
93+
94+
If, as a result of the testing process, the sectors on both disks contain invalid data, resulting in
95+
read errors that make the module impossible to use, you will need to redo the two disks in the
96+
virtual machine using the commands:
97+
98+
.. code-block:: console
99+
100+
$ dd if=/dev/zero of=/dev/vdb bs=1M
101+
$ dd if=/dev/zero of=/dev/vdc bs=1M
102+
103+
You can also get the same result using the following command to start the virtual machine:
104+
105+
.. code-block:: console
106+
107+
$ rm disk{1,2}.img; make
108+
109+
Tips
110+
----
111+
112+
To increase your chances of getting the highest grade, read and follow the Linux kernel
113+
coding style described in the `Coding Style document <https://elixir.bootlin.com/linux/v4.19.19/source/Documentation/process/coding-style.rst>`__.
114+
115+
Also, use the following static analysis tools to verify the code:
116+
117+
- checkpatch.pl
118+
119+
.. code-block:: console
120+
121+
$ linux/scripts/checkpatch.pl --no-tree --terse -f /path/to/your/file.c
122+
123+
- sparse
124+
125+
.. code-block:: console
126+
127+
$ sudo apt-get install sparse
128+
$ cd linux
129+
$ make C=2 /path/to/your/file.c
130+
131+
- cppcheck
132+
133+
.. code-block:: console
134+
135+
$ sudo apt-get install cppcheck
136+
$ cppcheck /path/to/your/file.c
137+
138+
Penalties
139+
---------
140+
141+
Information about assigments penalties can be found on the
142+
`General Directions page <https://ocw.cs.pub.ro/courses/so2/teme/general>`__.
143+
144+
In exceptional cases (the assigment passes the tests by not complying with the requirements)
145+
and if the assigment does not pass all the tests, the grade will may decrease more than mentioned above.
146+
147+
Submitting the assigment
148+
------------------------
149+
150+
The assignment archive will be submitted to vmchecker, according to the rules on the
151+
`rules page <https://ocw.cs.pub.ro/courses/so2/reguli-notare#reguli_de_trimitere_a_temelor>`__.
152+
153+
From the vmchecker interface choose the `Driver RAID` option for this assigment.
154+
155+
Resources
156+
=========
157+
158+
- implementation of the `RAID <https://elixir.bootlin.com/linux/v5.10/source/drivers/md>`__ software in the Linux kernel
159+
160+
We recommend that you use gitlab to store your homework. Follow the directions in
161+
`README <https://github.com/systems-cs-pub-ro/so2-assignments/blob/master/README.md>`__
162+
and on the dedicated `git wiki page <https://ocw.cs.pub.ro/courses/so2/teme/folosire-gitlab>`__.
163+
164+
The resources for the assignment can also be found in the `so2-assignments <https://github.com/systems-cs-pub-ro/so2-assignments>`__ repo on GitHub.
165+
The repo contains a `Bash script <https://github.com/systems-cs-pub-ro/so2-assignments/blob/master/so2-create-repo.sh>`__
166+
that helps you create a private repository on the faculty `GitLab <https://gitlab.cs.pub.ro/users/sign_in>`__ instance.
167+
Follow the tips from the `README <https://github.com/systems-cs-pub-ro/so2-assignments/blob/master/README.md>`__ and
168+
on the dedicated `Wiki page <https://ocw.cs.pub.ro/courses/so2/teme/folosire-gitlab>`__.
169+
170+
Questions
171+
=========
172+
173+
For questions about the assigment, you can consult the mailing `list archives <http://cursuri.cs.pub.ro/pipermail/so2/>`__
174+
or send an e-mail (you must be `registered <http://cursuri.cs.pub.ro/cgi-bin/mailman/listinfo/so2>`__).
175+
Please follow and follow `the tips for use of the list <https://ocw.cs.pub.ro/courses/so2/resurse/lista-discutii#mailing-list-guidelines>`__.
176+
177+
Before you ask a question, make sure that:
178+
179+
- you have read the statement of the assigment well
180+
- the question is not already presented on the `FAQ page <https://ocw.cs.pub.ro/courses/so2/teme/tema2/faq>`__
181+
- the answer cannot be found in the `mailing list archives <http://cursuri.cs.pub.ro/pipermail/so2/>`__

‎Documentation/teaching/so2/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,4 @@ Operating Systems 2
4444
assign0-kernel-api.rst
4545
assign1-kprobe-based-tracer.rst
4646
assign2-driver-uart.rst
47+
assign3-software-raid.rst

0 commit comments

Comments
 (0)