Very large development: How to manage over 1000 contributors in a Python open source project

19
9HU\ ODUJH GHYHORSPHQW +RZ WR PDQDJH RYHU FRQWULEXWRUV LQ D 3\WKRQ RSHQ VRXUFH SURMHFW -RH + 5DKPH HQJLQHHU # H1RYDQFH

description

OpenStack is a very large project with over 1.6 million lines of code, over 1000 active contributors and approximately 4000 commits per month. At least according to the latest stats from Ohloh.net. And this project is written (almost) completely in Python. This makes OpenStack one of the largest active open source projects written in this language. In this presentation I will to talk about the challenges of large scale collaboration in Python projects and the solutions set up by the OpenStack community to address them. By studying the typical life cycle of a patch, I will present the tools involved in testing, peer review, continuous integration, as well as the tools enforcing community policies like style checking or maintaining compatibility across different Python versions.

Transcript of Very large development: How to manage over 1000 contributors in a Python open source project

Page 1: Very large development: How to manage over 1000 contributors in a Python open source project

9HU\�ODUJH�GHYHORSPHQW+RZ�WR�PDQDJH�RYHU������FRQWULEXWRUV�LQ�

D�3\WKRQ�RSHQ�VRXUFH�SURMHFW

-RH�+��5DKPHHQJLQHHU�#�H1RYDQFH

Page 2: Very large development: How to manage over 1000 contributors in a Python open source project

:KR�DP�,"

Ɣ :RUN�DW�H1RYDQFH��D�ZHE�KRVWLQJ�FRPSDQ\�LQ�3DULV�

Ɣ :RUN�RQ�2SHQ6WDFN�

Ɣ 8VH�3\WKRQ�SURIHVVLRQDOO\�IRU�DERXW���\HDUV�

Page 3: Very large development: How to manage over 1000 contributors in a Python open source project

Ɣ 4XDOLW\�$VVXUDQFH�IRU�ODUJH�WHDPV��VSHFLILFDOO\�2SHQ6WDFN

Ɣ 3LWIDOOV�RI�3\WKRQ�DW�D�ODUJH�VFDOH

Ɣ ,QVSLUHG�E\�-RH�*RUGRQoV��

p9HU\�/DUJH�'HYHORSPHQW��+RZ�WR�UXQ�FRGH�UHYLHZ�IRU������2SHQ�6RXUFH�3\WKRQ�GHYHORSHUq

:KDW�LV�WKLV�WDON�DERXW"

Page 4: Very large development: How to manage over 1000 contributors in a Python open source project

:KDWoV�2SHQ6WDFN"

Ɣ &ORXG�&RPSXWLQJ�SURMHFW�SURYLGLQJ�,DD6Ɣ 2SHQ�6RXUFHG�E\�1$6$�DQG�5DFNVSDFH�LQ�����Ɣ 5XQ�E\�WKH�2SHQ6WDFN�)RXQGDWLRQƔ &RQWULEXWRUV�LQFOXGH��$77��$0'��&DQRQLFDO��&LVFR��'HOO��

H1RYDQFH��+3��,%0��,QWHO��5DFNVSDFH��5HG�+DW��f

Ɣ /RWV�DQG�ORWV�RI�3\WKRQ�FRGH

Page 5: Very large development: How to manage over 1000 contributors in a Python open source project

7LWOH

3XW�KHUH�VRPH�WH[W

Page 6: Very large development: How to manage over 1000 contributors in a Python open source project

7LWOH

3XW�KHUH�VRPH�WH[W

Page 7: Very large development: How to manage over 1000 contributors in a Python open source project

6RPH�QXPEHUV�DERXW�2SHQ6WDFN

Ɣ �������FRPPLWV�PDGH�E\������FRQWULEXWRUV�UHSUHVHQWLQJ�����������OLQHV�RI�FRGH

Ɣ �����FRPPLWV�PDGH�E\�����FRQWULEXWRUV��LQFOXGLQJ����QHZ��LQ�WKH�SDVW����GD\V

1XPEHUV�WDNHQ�IURP�KWWS���ZZZ�RKORK�QHW�

Page 8: Very large development: How to manage over 1000 contributors in a Python open source project
Page 9: Very large development: How to manage over 1000 contributors in a Python open source project

7KH�3LWIDOOV�RI�3\WKRQ

Ɣ /DFN�RI�VWDWLF�W\SLQJ

Ɣ 3RRU�VWDWLF�DQDO\VLV�WRROV

Ɣ 1R�LQWHUIDFHV��QR�HQFDSVXODWLRQ

Ɣ 1R�EXLOW�LQ�FRQFXUUHQF\

Page 10: Very large development: How to manage over 1000 contributors in a Python open source project

6HWWLQJ�XS�D�GHY�HQYLURQPHQW

Ɣ %XLOG�\RXU�HQYLURQPHQW�LQ�D�90ż ,I�\RX�GRQoW�NQRZ�KRZ��FKHFN�RXW�9DJUDQW�

Ɣ 8VH�JLW

Ɣ 'HYVWDFN

Page 11: Very large development: How to manage over 1000 contributors in a Python open source project

7HVWLQJ

Ɣ 8QLW�WHVWVƔ )XQFWLRQDO�WHVWVƔ 6W\OH

ż SHS�ż IODNH�ż S\OLQW

Ɣ WR[

Page 12: Very large development: How to manage over 1000 contributors in a Python open source project

*HUULW��&RPPLW�WLPH�

Ɣ :HE�EDVHG�FRGH�UHYLHZ�V\VWHPƔ %DVHG�RQ�WRS�RI�JLWƔ 2ULJLQDOO\�ZULWWHQ�E\�*RRJOHƔ 8VHG�WR�PDQDJH�$QGURLG��/LEUH2IILFH��0HGLD:LNL��4W��f

Ɣ (QDEOHV�D�PRUH�FHQWUDOL]HG�XVDJH�RI�JLW

Ɣ 'LIIHUHQW�WKDQ�*LWKXEoV�3XOO�5HTXHVW

Page 13: Very large development: How to manage over 1000 contributors in a Python open source project

:ULWH�GHVFULSWLYH�FRPPLW�PHVVDJHV

FRPPLW�G�IHIG�����FH�ED�E��FI�I���DD�D�EGEF��DE$XWKRU��-RKQ�*ULIILWK��MRKQ�JULIILWK#VROLGILUH�FRP!'DWH����0RQ�6HS�����������������������

����)L[�MVRQ�YHUVLRQ�RI�ERRWDEOH�FKHFN

����7KLV�LV�D�IROORZ�XS�WR�FKDQJH�LG�������,�D�E�I�FH�D�D���H����HD�F��D�E�IF���IH��

����7KH�LQLWLDO�FKDQJH�RQO\�DGGUHVVHG�WKH�[PO�DVSHFW�RI�WKH�EXJ����DQG�LJQRUHG�WKH�MVRQ�WHVWV�

����7KLV�FKDQJH�XSGDWHV�WKH�FKHFN�WR�FDWFK�WKH�VDPH�HUURUV�ZKHQ����SHUIRUPLQJ�WKH�MVRQ�YHUVLRQ�RI�WKH�FKHFNV�

����%RWK�RI�WKHVH�SDWFKHV�DUH�WHPSRUDU\�DQG�DIWHU�WKH�&LQGHU�XSGDWH����ODQGV�VKRXOG�EH�UHSODFHG�ZLWK�D�SURSHU�VWULFWO\�HQIRUFHG�FKHFN�RI����WUXH�DQG�IDOVH�YDOXHV�LQ�WKH�YROXPH>ERRWDEOH@�ILHOG�

����&KDQJH�,G��,IIH�G������I�F������D�F����F��DI��I�E��G

Page 14: Very large development: How to manage over 1000 contributors in a Python open source project

*LW�UHYLHZ

Ɣ 'HDOV�ZLWK�WKH�LQWULFDFLHV�RI�VHQGLQJ�SDWFKHV�WR�*HUULW

Ɣ :ULWWHQ�DV�D�JLW�VXEFRPPDQG

Ɣ 3XEOLVKHG�RQ�3\3,��DQG�XVHG�LQ�RWKHU�SURMHFWV�DV�ZHOO

Page 15: Very large development: How to manage over 1000 contributors in a Python open source project

&RQWLQXRXV�,QWHJUDWLRQ

Ɣ 5XQV�WKH�WHVWV�RQ�DOO�WKH�VXEPLWWHG�SDWFKHV

Ɣ *RRG��ż 'HWHFW�HUURUV�DV�HDUO\�DV�SRVVLEOHż ,QFUHDVH�FRQILGHQFH�LQ�LPPDWXUH�SURMHFWV

Ɣ %DG�ż 5HTXLUHV�D�ORW�RI�SURFHVVLQJ�SRZHUż %UHDNLQJ�WKH�JDWH�ZLOO�KDQGLFDS�HYHU\ERG\

Page 16: Very large development: How to manage over 1000 contributors in a Python open source project

0LUURU�RQ�*LWKXE

Ɣ *UHDW�SXEOLVKLQJ�SODWIRUP

Ɣ (DV\�WR�FORQH��VLQFH�LWoV�JLW

Ɣ ��DFFRXQWVż 2SHQVWDFNż 2SHQVWDFN�GHYż 6WDFNIRUJH

Page 17: Very large development: How to manage over 1000 contributors in a Python open source project

&RPPXQLFDWLRQ

Ɣ 0DLOLQJ�OLVWVƔ ,5&Ɣ /DXQFKSDGƔ (WKHUSDGƔ DVN�RSHQVWDFN�RUJƔ f

Page 18: Very large development: How to manage over 1000 contributors in a Python open source project

6WDFNIRUJH�DQG�LQFXEDWHG�SURMHFWV

Ɣ 8VHIXO�VFULSWV�EHFRPH�SURMHFWV�LQ�WKHLU�RZQ�ULJKWVƔ 0RVW�LPSRUWDQWO\��WKH\�JHW�DFFHVV�WR�WKH�4$�

LQIUDVWUXFWXUH�

Ɣ ��H[DPSOHV�ż SEU��3\WKRQ�%XLOG�5HDVRQDEOHQHVVż KDFNLQJ��3\WKRQ�VW\OH�JXLGHOLQHV

Page 19: Very large development: How to manage over 1000 contributors in a Python open source project

4XHVWLRQV�"

(�PDLO���MRH#HQRYDQFH�FRP,5&���UDKPX�#�IUHHQRGH7ZLWWHU���#KDNLPUDK