Modern FreeBSD (London Opentech 2010)

8/8/2019 Modern FreeBSD (London Opentech 2010)

http://slidepdf.com/reader/full/modern-freebsd-london-opentech-2010 1/18

Modern FreeBSDIndustrial Strength UNIX

Words: Robert Watson, University of Cambridge

Editing, Presentation: Mark Blackman

day, 11 September 2010



What is FreeBSD - Technical

Open source BSD UNIX-derived OS

ISP server network server platform

Yahoo!, Verio, NY Internet, ISC, ...

Appliance/product/embedded OS foundation

Juniper JunOS, Nokia, Panasas, Timing Solutions,...

VXWorks, Mac OS X, Netapp, Apple iOS

FreeBSD components in Solaris, BT Home hub.

One of the most successful open source projects

Focus on storage, networking, security




10/09/2010 22:2story-simple.svg

f x -

1969

71 to 1973

74 to 1975

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

01 to 2004

06 to 2010

2005

1969

1971 to 19

1974 to 19

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001 to 2

2006 to 20

2005

Open Source

Mixed/Shared Source

Closed Source

HP/UX1.0 to 1.2

HP/UX2.0 to 3.0

HP/UX6 to 11

HP/UX11i to 11i v3

OpenSolaris2008.05 and

later

Solaris2.1 to 10

System III

System VR1 to R2

System VR3

System VR4

Unixware1.x to 2.x

Unixware7.x

OpenServer6.x

OpenServer5.0.5 to 5.0.7

OpenServer5.0 to 5.04

SCO Unix3.2.4

SCO XenixW386

SCO XenixV386

SCO XenixW286

SCO Xenix

Xenix3.0

Xenix1.0 to 2.3

PWB/Unix

AIX1.0

AIX3.x to 6.x

OpenBSD2.3 to 4.x

OpenBSD1.0 to 2.2

Sun OS4

Sun OS1.2 to 3.0

Sun OS1 to 1.1

Unix 32v

UnixTSS1 to 4

UnixTSS5 to 6

UnixTSS7

Unics

BSD1.0 to 2.0

BSD3.0 to 4.1

BSD 4.2

BSD 4.3

UnixTSS8

UnixTSS(Time Sharing

System)9 to 10

NEXTSTEP/

OPENSTEP1.0 to 4.0

Mac OS XServer

Mac OS X10.0 to 10.6

(Darwin)

Minix1.x

Minix2.x

Minix3.x

Linux2.0 to 2.6.x

Linux0.95 to 1.2.x

Linux 0.0.1

BSD 4.3Tahoe

BSD 4.3Reno

BSD4.4 to

4.4 lite2

NetBSD0.8 to 1.0

NetBSD1.1 to 1.2

NetBSD 1.3

NetBSD1.3 to 5.x

FreeBSD3.3 to 8.0

FreeBSD3.0 to 3.2

FreeBSD1.0 to2.2.x

386BSD

BSD NET/2

Pedigree


http://commons.wikimedia.org/wiki/File:Unix_history-simple.svg






What is FreeBSD - Social

Global community of developers and users

FreeBSD.org web site, mailing lists, forums

Academic research platform: Cambridge, Berkeley, Rice and others

Developer community: Core team, Committers, Ports maintainers, Contributors

User communities

Some more or less involved in global community

Companies: find the BSD licence less demanding than GPL and contribute

back.




Modern Pressures

Multiprocessing now dominant paradigm

Convergence of hardware platforms

Granular security model requirements

Storage management

The evolving role and nature of open source


embedded now uses similar technologies to serversSMP dominance - (which motivates SMPng, GCD, ...)ZFS and UFS improvementswe've lead the way in identifying how open source projects should run, and what theyould do, but have done a lot of experimentation to get there



Threading

In the mid-1990's, it was unclear what concurrentprogramming model would win.

maturing thread models in FreeBSD over the last decade

pthreads project in the mid-1990's

followed by KSE, an M:N threading model

most recently, libthr.

Application models evolved, and simpler and less

concurrent 1:1 implementation offered greater benefit


1 - simplifying scheduler data structures andowing them to use more complex heuristics



Threading

FreeBSD engaged with Apple to adapt Grand Central

Dispatch (GCD)

builds on threads and the FreeBSD kqueue model

offers a more programmer-accessible, scalable,programming model


D - a new concurrent programming framework, answers mapping of M:N to 1:1switched from M:N to 1:1 to simply the scheduler / threading code, and because it

peared that application writers were generally choosing to use small thread pools (say,100 threads) rather than using very large numbers. In part because Linux used 1:1 and yuldn't get it to create very large thread counts



Tracking Processor Changes

transition from instruction-centric to cache-centric

performance

support for newer and widespread RISC architectures (ARM,

MIPS, PowerPC, Sparc64)

Embedded systems with virtual memory support makes

FreeBSD an ideal fit


ple’s FreeBSD-derived iOS on iPhone and IpadeeBSD-derived JunOS running on low-power switches



Parallel Hardware

Ten years ago, FreeBSD ran on two-core systems,

FreeBSD runs on 32-core systems, with 64,128-core

configurations in our immediate future

Giant locked kernel to granular, data-oriented locking

(the very successful 10-year SMPng project).Continous R&D on algorithms and approaches for high

core counts.


cent work on algorithms and approaches that scale to high core counts throughmplex work distribution, and hardware-assisted work distribution on high-performancetwork cards



Parallel Hardware

FreeBSD's performance in multiprocessing is cutting edge.

Now developing a highly scalable network stack able to run on dozens

(perhaps someday hundreds) of kernel threads. Collaborating with

network vendors such as BlueCoat and Juniper

hardware vendors such as Chelsio and Intel.

Likewise, NUMA is becoming a reality.


as benchmarks over the last few years have shown, and we continue to aggressively explw parallelism.and our kernel memory allocator has recently been updated to introduce NUMA-areness.



Virtualization

FreeBSD has lead and responded to trends

FreeBSD's Jail model, introduced around ten years

ago, led a revolution in OS-level virtualization, with

imitators such as

Solaris Zones

Linux Virtual Servers ( OpenVZ/Virtuozzo, Linux-

VServer, FreeVPS)

Runs on Xen, Virtualbox as well.




Security models

UNIX model was powerful, but limiting

TrustedBSD Project, announced roughly ten years

ago.

Our MAC Framework allows flexible and extensible access control.

Funded by DARPA, US Navy, Apple, McAfee and others.

extend the OS model without modifying the OS. ie. sandboxing

in iPhone OS apps, JunOS routers.

Relied on throughout military, industry, academia


for many evolving applications such as firewall appliances and smartphones.



Security Models

Companies like Apple, Juniper, and McAfee use MAC

framework extensively to

provide alternative access control models, such as

sandboxing, for third-party code,

build system assurance.

Capsicum: “capability” oriented security, out of Cambridge,

collaboration with Google, possible ChromeOS component.


Most recently, we've announced the Capsicum project, developed in collaboration withogle, to support capability-oriented OS security.capiscum cutting edge, best paper



Storage

advancing the UFS file system over the last decade

GEOM framework, developed under sponsorship

from DARPA in the early 2000's

ported Sun's ZFS file system to FreeBSD


-bit supportapshotsckground file system checkingtended attributesvanced security features such as ACLs and MACd most recently, journalingallows easily pluggable and extensible storage transforms,torage multipathingew RAID integrationull file-system journaling andeplication for fault tolerance.with its self-healing and management features



Development/Introspection

new clang/LLVM compiler suite

Coverity's Prevent static analysis tool

FreeBSD has ported Sun's DTrace

userspace Dtrace support coming.

hardware performance monitoring counters support

Kernel debugging tools


we even have our own Prevent server system and actively re-analyze our source codeghtly.of ering introspection tools for performance and behavioural analysisallows similar analysis of userspace applications the FreeBSD foundation is sponsoring Rulo to complete this.make FreeBSD one of the best OS platforms for kernel feature development, includingintegrated debuggingdynamic lock order analysislked to many developers who actually write their Linux kernel code on FreeBSD so that thn use the debugging tools, before porting to Linux!



Open Source Project

ManagementProject growthFreeBSD's social organization

Elected core team model

Huge OS development projects, c.f. SMPng

Delegated project leadership structure

No more “core team”

CVS to Subversion.

Perforce


Dealing with project growth has been a critical challenge, both socially and technically.. A social experiment that has proven extremely successful, allowing us to renew project

adership over time.Roughly ten years ago, we moved for the first time to anincluding the very interesting SMPng project, which involved dozens of developers (andmpanies) collaborating to improve multiprocessor scalability.in which chartered teams take responsibility for portions of the project's work: release

gineering, application porting, security advisories, system adminstration, bug-busting,cumentation, etc,Moving away from the idea that a small "core team" of developers does it all.CVS: many extensions --> Subversion: few extensionsas a way to improve our branched development methodology fore-projects with long life cycles, as well aspporting dozens of summer students each year sponsored by GoogleOur mostcent developer summit at BSDCan 2010 in Ottawa had over 100 attendeescluding developers and invited guests from various companies. That's a good

bigger than the 15-20 (?) folks at the first developer summit at the 2001

ENIX ATC in Boston.Warner’s audit of all involved licenses should beentioned also.

Modern FreeBSD (London Opentech 2010)

Documents

Transcript of Modern FreeBSD (London Opentech 2010)