← Back to Projects

Linux Kernel Mentorship Program

Contributing to the linux kernel under the guidannce of Shuah Khan, David Hunter, and Khalid Aziz

linux kerneldebuggingbackporting

Experience

I'm grateful to be part of linux kernel mentorskip program(lkmp). This experience has taught me how to contribute to the linux kernel and the confidence to do so. I was able to confidently compile, test, and send patches when needed by the end of it.

Most of my experience came out of backporting and testing. Though I was always working on syzbot fixes on the side but was never really successful, but have learned a lot from it. I will continue to work on them until they are fixed.

Tools Used

  • LazyVim
    • For general code editing, and navigation
  • cscope
    • started out using this to look for files, but later fully relied of vim
  • scripts in kernel
    • scripts/checkpatch.pl
      • makes sure patch looks good
    • scripts/get_maintainer.pl
      • gets maintainers for files
    • scripts/gen_compile_commands.py
      • helps nvim with clangd Lsp
  • qemu
    • For testing syzbot reported bugs
  • gdb
    • For debuggint the kernel
  • virtme-ng
    • For quick code change tests

My workflow / What I've learned

Syzbot

To work on syzbot issues; Moon Hee Lee on LinkedIn has a really good post describing how to do this.

A good syzbot bug to work on is a bug then is relatively new, and has a reproducer. As I was told older bugs are what people have trouble fixing. Sorting by reproducer is generally a good idea while looking for a bug to work on.

I basically compile the kernel, download the required files and run them with my locally built kernel then run the repro.c. The hard part is actually understanding the problem then implementing the fix, I've learned that reading the code is really the only way to figure things out. With lazyvim gen_compile_commands script, I was able to jump to definitions learning about the structs and flow of functions to eventually figure out the problem. I still have trouble understanding bugs, and my way of learning to fix them was through reading older similar fixed syzbot bugs.

Backporting

Backporting was something I had a lot more luck with. Generally what you do is find a patch that is in upstream kernel that fixed something that stable kernels don't have. This could be from finding out that a fix magically appeared for one of your graphics card through dmesg or by stumbling into one. I personally have tried looking for it intentionally but it seems to have gotten ignored. I believe the backports that I was trying to send in likely is already in the stable queue. Since I tested every backport before I sent the patches however, I realized that one of the previous tests on kernel 6.6 hadn't been backported too, which is when I also made a patch for it which seems to have gotten accepted. Testing was mostly ran with virtme-ng and tools/testing in the kernel.

To check if patches are right, there are a bunch of backports examples in lore.kernel.org that I was able to reference and documentation online or in the kernel repository itself.

Fixing documentation

I found that fixing documentation is definitely a really good way to get used to the "linux kernel workflow". Making a change, commiting and writing the message then signing it, checkpatch it, then emailing it using get send-mail. I personally never sent one but has practiced making patches this way.

Patches

These are patches I worked on, sent in or not.

Patches I worked on

- <https://syzkaller.appspot.com/bug?extid=7f176adb30b21606c5fc>
  - ATTEMPTING
  - Subsystem was removed from mainline
  - Fixed by someone else
- net/wireless/util.c : ieee80211_get_channel_khz line 138  /*TODO: just handle MHz for now*/     ● MD013/line-l
  - ATTEMPTING
  - Read docs to figure out what this meant
  - Couldn't understand the TODO
- <https://syzkaller.appspot.com/bug?extid=1f1fbecb9413cdbfbef8>
  - ATTEMPTING
  - was able to use gdb
  - preempt issue discovered
  - Someone fixed it
- backport for c760bcda83571e07b72c10d9da175db5051ed971 from 6.18 to 6.1, 6.6, 6.12, 6.17
  - ATTEMPTING
  - Compiled
  - Ran
  - Sent
  - DONE
- <https://syzkaller.appspot.com/bug?extid=1f22cb1769f249df9fa0>
  - ATTEMPTING
  - following backtrace
  - learned structs
  - learned to navigate code a lot faster
  - Still attempting
- backport for 3fde60afe1f84746c1177861bd27b3ebb00cb8f5 5ace7ef87f059d68b5f50837ef3e8a1a4870c36e from 6.18 to 6.1, 6.6, 6.12, 6.17
  - ATTEMPTING
  - Compiled
  - Ran
  - Tested
  - Not all accepted.

Accepted patches

stable linux-6.1.y
2e696a06561e4c48ddad74a3fe94012fea947d92 drm/amd: Check whether secure display TA loaded successfully

stable linux-6.6.y
59a6e61863e20ea91acaf0d5bbd2fb510d28fc93 drm/amd: Check whether secure display TA loaded successfully

stable linux-6.12.y
03fe1647e26534d0b2c2a974f40dd5c5b208723c drm/amd: Check whether secure display TA loaded successfully

stable linux-6.17.y
d4ba1afb4109c5609b7abc9525e4aa2c7478d099 drm/amd: Check whether secure display TA loaded successfully

In stable linux-6.6.y queue
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/queue-6.6/selftests-openvswitch-fix-escape-chars-in-regexp.patch?id=e1458c20b600c61df39361b4b20e3301b75d81ea

To the next mentee

Be ready to learn a lot, especially by yourself. Some of the troubleshooting are crazy timesinks, I would suggest getting used to your tools and run linux bare-metal to avoid issues. For eg. I picked Fedora and had to install Dev Tool packages and sudo update-crypto-policies --set LEGACY to build older kernels. Little things here and there takes time and mental energy. Learning about the code was also harder for me as some functions, you just have to find to fix specific issues. I would defintely try to make time for this mentorship as, I was at a busy time of my life while doing this mentorship and it defintely wasn't easy. Also! build kernel baremetal then run tests on qemu/virtme-ng VM if it wasn't obvious to you! Focus on easy points to get the 5 patches then work on harder patches, you may get stuck like me and get demotivated.

Final thoughts

I'm really thankful for the opportunity Shuah Khan, David Hunter, Khalid Aziz, and LFX has given my along with all the guidance that was provided. This experience has truly given me the confidence and motivation to work on the linux kernel. This process is actually pretty fun after you get used to it. I plan to continue fixing syzbot bugs then eventually implementing features. A lot of the knowledge is already online but having a doc point me to certain task defintely helped focus on the learning. I had a lot of fun and would absolutely do it again!