BUD17-TR02: Upstreaming 101
-
Upload
linaro -
Category
Technology
-
view
76 -
download
0
Transcript of BUD17-TR02: Upstreaming 101
![Page 1: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/1.jpg)
Upstreaming 101Daniel Lezcano
![Page 2: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/2.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Introduction
● This presentation is focused on upstreaming the Linux code
● What means upstreaming ?● The Linux kernel development cycle● The Linux kernel code organization● Contributions● Writing one patch● Writing several patches● Conclusion● Introducing Upstreaming 201
![Page 3: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/3.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
What is upstreaming ?
● Bring your private changes to the mainstream kernel
● Be prepared to:○ restart from scratch○ change your approach○ be part of the OSS community
● No deadline, no schedule, no obligation to take a patch○ Linux is evolution, best proposed solution wins○ Consensus is the key
![Page 4: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/4.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Why upstream ?
● The Linux kernel has one rule: no regression○ The community won’t break your code
● The submitted code will be reviewed in detail○ Better quality
● The community will give support○ Better knowledge
● Stop porting out of tree code to newer kernel○ Save money and effort
![Page 5: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/5.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
The Linux kernel development cycle
1. Iterative release candidates: v4.10-rcX, up to -rc8○ Usually every Sunday
2. New release: v4.103. Merge window4. New release candidate: v4.11-rc15. … and so on
![Page 6: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/6.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
The Linux kernel development cycle ● A new release every 3 months● When to send patches?
○ Fixes: anytime○ New code and cleanup: depend on the maintainer
● How long is the merge window?○ The merge window lasts two weeks
● How to know if the merge window is happening?○ As soon as there is a new release
● How to know if the merge window is finished?○ As soon as there is a -rc1
![Page 7: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/7.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
The Linux kernel development cycle
● Automatic with a RSS feed:○ https://www.kernel.org/feeds/kdist.xml
● Manually by polling:○ https://www.kernel.org/
![Page 8: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/8.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Linux-next : catching the issues early
● Linux-next integrates all the different trees○ Kudos to Stephen Rothwell
● Image of the future Linux release ahead of the schedule
● Testing with this tree allows to catch the bugs before they hit mainline
![Page 9: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/9.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
How to start contributing ?
● Code review○ Cleanups○ Trivial fixes○ Potential issue
● Compilation test coverage○ Cross compile and fix errors / warnings○ Use rare compilation option (eg. headers_check, sparse, …)
● Communicate○ Dig into mailing lists to help people○ Review patches
● Test linux-next○ Boot and fix bugs○ Report compilation warnings
● Debug○ https://bugzilla.kernel.org
![Page 10: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/10.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linaro contribution for 4.9Total 16592 patches
kernel: 642
scripts: 65
arch: 4742
drivers: 9835
lib: 33
net: 463
mm: 34
block: 30
fs: 143
tools: 120
Source : http://www.remword.com/kps_result/
![Page 11: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/11.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Linux code organization
● Huge number of files, more than 40,000
● Organized in subsystems
● Posix implementation● Architecture specific● Frameworks● Drivers● Network stack
![Page 12: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/12.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
kernel
IPC
net
tools
scripts arch drivers
mm
lib
block
fs
net
usb
clock
arm64
mm
arm
![Page 13: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/13.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
kernel
IPC
net
mm
block
fs
● Generic code and framework● Scheduling● Process management● POSIX kernel implementation● Interrupt handling● Network protocols● Memory management● Block device management● File systems● Power management
![Page 14: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/14.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
lib ● Generic code● Integer math: CRC, div● List manipulation, red-black tree● String manipulation● … a lot of useful functions
![Page 15: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/15.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
tools
scripts ● Scripts to help making Linux better○ Patch checking, header check○ Kernel symbol size comparison○ ...
● Userspace tools with a dependency on the kernel sources
![Page 16: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/16.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
arch
arm64
mm
arm
● Architecture specific code● Memory management, locks, ...● Everything which is arch specific● Fills the gaps left by the generic code
![Page 17: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/17.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Linux kernel code organization
Topmost Linux dir
drivers
net
usb
clock
● All peripherals drivers● Contains a generic framework● Peripheral specific code based on the
generic framework above● Base object for the driver model
![Page 18: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/18.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
What is a maintainer ?
● Responsible of a part of the Linux kernel code
● Ensure the code complies with some rules:○ Coding style○ Consistency○ Consensus○ Technically relevant
● A maintainer is a gatekeeper
![Page 19: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/19.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Maintainership● Each directory falls under the umbrella of a maintainer● Each maintainer has its own tree● Each proposed change must stick to the relevant tree
● After a kernel release, all the maintainers’ tree are merged together : it is the merge window
● Topmost maintainer is Linus Torvalds
● All maintainers are listed in the MAINTAINERS file
![Page 20: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/20.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
netipv4
Linus Torvald’s tree - v4.10
v4.10+net v4.10+tip
v4.10+tip v4.10+tip
![Page 21: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/21.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
netipv4
Linus Torvald’s tree - v4.10
v4.10+net v4.10+tip
v4.10+tip v4.10+tip
patchpatch
patchpatch
![Page 22: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/22.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
netipv4
Linus Torvald’s tree - v4.10
v4.10+net v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
![Page 23: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/23.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 1
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 24: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/24.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 1
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 25: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/25.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 1
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10
v4.10+tip++
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 26: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/26.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 2
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 27: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/27.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 2
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10++++
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 28: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/28.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 2
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10++++
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 29: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/29.jpg)
ENGINEERS AND DEVICESWORKING TOGETHER
Example of a merge process - step 3
kernelirq
time
driversclocksource
driversirqchip
driversirqchip
driversclocksource
kernelirq
timedrivers
irqchipdrivers
clocksource
netipv4
Linus Torvald’s tree - v4.10-rc1
v4.10+tip
v4.10+tip+irqchip v4.10+tip+clocksource
netipv4
v4.10+net
![Page 30: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/30.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Communication
● All development communication through the mailing lists○ Important mailing lists for Linaro: lkml@, lakml@○ MAINTAINERS file gives the subsystem <-> mailing list○ http://vger.kernel.org/vger-lists.html
● Public discussion, no point-to-point○ Reply-all always
● Opensource events: Linux Plumbers Conference, Embedded Linux Conference, Linaro Connect
![Page 31: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/31.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Where to begin ?
● Send a simple contribution○ But don’t flood with too trivial patches
■ Read [email protected] rules
● One example with a checkpatch script○ Run it in a directory where you will be working in the future
■ Eg. drivers/acpi○ Target one ERROR spotted by checkpatch○ Beware of false positives
for i in $(ls drivers/acpi/utils.c); do./scripts/checkpatch -f $i
done
![Page 32: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/32.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
A simple contribution
● Compile, test first and then commit○ Even if the change is trivial
● Write a nice description:○ Why and what○ One simple sentence prefixed with the subsystem name ...○ … followed by a more detailed description
![Page 33: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/33.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
A simple contribution
● Read the Digital Certificate of Origin:○ Documentation/process/submitting-patches.rst
● Make sure you fully understand what that means○ You are legally responsible of your changes
● When committing, add your Signed-off-by
● Don’t send more than 2 trivial changes
![Page 34: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/34.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
A simple contribution
● A list of examples:
○ https://goo.gl/q7NqcZ : Fixing checkpatch errors
○ https://goo.gl/JCn2y : Fixing missing kfree
○ https://goo.gl/ASrbOU : Remove unused parameter
○ https://goo.gl/uiktEV : Remove pointless code
![Page 35: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/35.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
A more complex contribution
● Split the changes into several patches
● Bring the changes step by step, incremental changes
● Make sure the changes are git-bisect safe○ Not following this rule will hurt your karma in the community
● Set the scene by cleaning up the place before sending a complex contribution
![Page 36: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/36.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
A more complex contribution
● Some examples:○ Changing the loopback to be multi-instantiated:
■ https://goo.gl/q6w1Ay : Change the static variable to a pointer■ https://goo.gl/tdNth4 : Dynamically allocate the loopback
○ A cleanup to catch clocksource initialization error■ https://lkml.org/lkml/2016/6/16/781
○ A very complex change for CPU hotplug:■ https://goo.gl/c2NpDY : A long description of the changes
![Page 37: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/37.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be sure you have the right mindset
![Page 38: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/38.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be altruist
![Page 39: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/39.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be polite
![Page 40: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/40.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be patient
![Page 41: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/41.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be humble
![Page 42: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/42.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Be factual
![Page 43: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/43.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Always take into account the comments
![Page 44: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/44.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Always take into account the comments
![Page 45: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/45.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
Always take into account the comments
![Page 46: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/46.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
● Read all the documentation Documentation/process/*:○ Coding style rules○ Give all the details introduced in this presentation○ Digital Certificate of Origin
● More material at:○ https://kernelnewbies.org
![Page 47: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/47.jpg)
ENGINEERS AND DEVICES
WORKING TOGETHER
Before going to Upstreaming 201
● Ready to send patches for review ?○ No … I’m scared of what they’ll think of my code
● Be sure you have the right mindset:○ Be altruist○ Be polite○ Be patient○ Be humble○ Always take into account the comments
■ You can disagree, stick to technical reasons○ Comments can be harsh, stay polite and factual
■ Stick to technical reasons○ Don’t be afraid, increase self confidence
■ Read the kernel documentation■ Be altruist to encourage yourself to have a positive attitude
![Page 48: BUD17-TR02: Upstreaming 101](https://reader031.fdocuments.net/reader031/viewer/2022021813/58ce8ec01a28ab8c3b8b4e4d/html5/thumbnails/48.jpg)
Thank You#BUD17
For further information: www.linaro.orgBUD17 keynotes and videos on: connect.linaro.org