nftables, far more than %s/ip/nf/g
Éric Leblond
Nefilter Coreteam
September 24, 2013
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 1 / 48
1 Introduction
2 Netfilter in 2013
3 Iptables limitations
4 Nftables, an Iptables replacement
5 Advantages of the approach
6 An updated user experience
7 Conclusion
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 2 / 48
Éric Leblond
Hacker and contractorIndependant Open Source and Security consultantStarted and developped NuFW, the authenticating firewallCore developer of Suricata IDS/IPS
Netfilter Coreteam memberWork on kernel-userspace interactionKernel hackingulogd2 maintainerPort of Openoffice firewall to Libreoffice
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 4 / 48
History
ipchains (1997)Linux 2.2 firewallingstatelessDevelopped by Paul ’Rusty’ Russel
iptables (2000)Linux 2.4 firewallingStateful tracking and full NAT supportin-extremis IPv6 support
Netfilter project’Rusty’ Russel developed iptables and funded Netfilter projectNetfilter coreteam was created to consolidate the community
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 6 / 48
Features
Filtering and loggingFiltering
on protocol fieldson internal state
Packet manglingChange TOSChange TTLSet mark
Connection trackingStateful filteringHelper to support protocol like FTP
Network Address TranslationDestination Network Address TranslationSource Network Address TranslationSupport of vicious protocol like FTPÉric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 7 / 48
Netfilter inside kernel
HooksHooks at different points of network stackVerdict can be issued and skb can be modifiedTo each hook correspond at least tableDifferent families
filterrawnatmangle
Loading a module create the table
Connection tracking tasksMaintain a hash table with known flowsDetect dynamic connection opening for some protocols
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 8 / 48
Major components
Netfilter filteringIn charge of accepting, blocking, transforming packetsConfigured by ioctl
Connection trackingAnalyse traffic and maintain flow tableCost in term of performanceIncrease security
iptablesConfiguration toolsUpdate ruleset inside kernel
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 9 / 48
The nfnetlink (r)evolution
NfnetlinkFirst major evolution of Netfilter (Linux 2.6.14, 2005)Netfilter dedicated configuration and message passingmechanism
New interactionsNFLOG: enhanced logging systemNFQUEUE: improved userspace decision systemNFCT: get information and update connection tracking entries
Based on Netlinkdatagram-oriented messaging systempassing messages from kernel to user-space and vice-versa
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 10 / 48
Netlink
Header format
Payload format
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 11 / 48
Components created following 2.6.14
conntrack-toolsconntrackd
connection tracking replication daemonprovide high availabilitydevelopped by Pablo Neira Ayuso
conntrack: command line tool to update and query connectiontracking
ulogd2logging daemonhandle packets and connections logging
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 12 / 48
Latest changes
ipsetEfficient set handlingAddress list or more complex setReach vanilla kernel in 2011 (Linux 2.6.39)
nfacctEfficient accounting systemAppeared in 2012
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 13 / 48
Kernel code
How much code70000 LOC reside in kernelspacearound 50000 LOC in user-space
Iptables extensions111 iptables extensions.Various tasks:
tcpclusterbpfstatistic
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 14 / 48
Performance
Adding a rule The problemAtomic replacementof ruleset
Sent from kernelto userspaceModified andsent back byuserspace
Huge performanceimpact
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 16 / 48
Dynamic ruleset
Network gets dynamicFirewall can’t be static anymore
CloudIP reputation
Combinatory explosion : one rule per-server and protocol
Set handlingSet handling is made via ipsetEfficient but not as integrated as possible
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 17 / 48
Code duplication
Different filtering familyNetfilter classic filteringBrigde filteringArp filteringIPv4 and IPv6
Matches and targetSimilar code in numerous Netfilter moduleNothing is sharedManual parsing
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 18 / 48
Problem due to binary blob usage
ABI breakageBinary exchange between userspace and kernelNo modification possible without touching kernel
Trusting userspaceKernel is parsing a binary blobPossible to break the internal parser
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 19 / 48
Integration via exec
Frontend and iptablesNo officially available libraryFrontend fork iptables command
libiptablesAvailable inside iptables sourcesNot a public libraryAPI and ABI breakage are not checked during version upgrade
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 20 / 48
Lack of flexible table and chains configurations
Module loading is the keyChains are created when module initInduce a performance cost even without rules
No configuration is possibleChains are hardcodedFORWARD is created on a server
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 21 / 48
Nftables
A new filtering systemReplace iptables and the filtering infrastructureNo changes in
HooksConnection trackingHelpers
A new languageBased on a grammarAccessible from a library
Netlink based communicationAtomic modificationNotification system
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 23 / 48
History
Introduced in 2008Developped andpresented by PatrickMcHardy atNFWS2008Presentation took 3hoursAlpha stage in 2008
Development did stopPatrick McHardy did not finish the code aloneNobody did join the effort
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 24 / 48
Video Interlude
The video
http://www.youtube.com/watch?v=DQp1AI1p3f8
Video generationVideo generated with gourceVarious git history have been mergedFile path has been prefixed with project name
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 25 / 48
What explanations ?
Should have "Release often release early" ?Started by Patrick McHardy onlyAlmost complete work presented during NFWS 2008Complex to enter the project
Too early ?No user were demanding for that explicitlyIpset was available and fixing the set issueSolution for dynamic handling was sufficient
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 26 / 48
Development restarted in 2012
Funding by Sophos/AstaroPablo Neira Ayuso get funded by AstaroWork restart in 2012
Gaining momemtumTomasz Bursztyka joined the development team
Work on ConnmanLack of libs was painful to himStart to hack on nftables
Google summer of code3 studentsSome good results
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 27 / 48
A filtering based on a pseudo-state machine
Inspired by BPF4 registers1 verdictA extensive instructions set
Add Some Magic ?reg = pkt.payload[offset, len]reg = cmp(reg1, reg2, EQ)reg = pkt.meta(mark)reg = lookup(set, reg1)reg = ct(reg1, state)
Easy creation of new matchesreg1 = pkt.payload[offset_src_port, len]reg2 = pkt.payload[offset_dst_port, len]reg = cmp(reg1, reg2, EQ)
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 28 / 48
Architecture
KernelTables: declared by user and attached to hookUser interface: nfnetlink socket
ADDDELETEDUMP
Userspacelibmnl: low level netlink interactionlibnftables: library handling low-level interaction with nftablesNetlink’s APInftables: command line utility to maintain ruleset
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 29 / 48
Dynamic chain loading
Chain are created on-demandChain are created via a specific netlink messageNon-user chain are:
Of a specific typeBound to a given hook
Current chain typefilter: filtering tableroute: old mangle tablenat: network address translation table
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 30 / 48
From userspace syntax to kernel
Converting user inputOperation is made via a netlink messageThe userspace syntax must be converted
From a text message following a grammarTo a binary Netlink message
LinearizeTokenisationParsingEvaluationLinearization
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 31 / 48
From kernel to userspace syntax
Kernel send netlink messageIt must be converted back to text
ConversionDeliniearizationPostprocessingTextify
Example
i p f i l t e r output 8 7[ payload load 4b @ network header + 16 => reg 1 ][ b i t w i s e reg 1 = ( reg=1 & 0 x 0 0 f f f f f f ) ^ 0x00000000 ][ cmp eq reg 1 0x00500fd9 ][ counter pkts 7 bytes 588 ]
is translated to:i p daddr 217.15.80.0 /24 counter packets 7 bytes 588 # handle 8
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 32 / 48
Kernel
Atomic ruleset updateatomically commit a set of rule-set updates incrementallybased on a generation counter/mask
00 active in the present, will be active in the next generation.01 active in the present, needs to zero its future, it becomes 00.10 inactive in the present, delete now.
xtables compatibilityPossible to use old extensionsNecessary to provide backward compatibility
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 33 / 48
Notification
Event based notificationEach rule update trigger an eventEvent is sent to userspace via nfnetlink
Userspace usageImplemented in libnftablesProgram can update his view on the ruleset without dump
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 34 / 48
A limited in-kernel sizeA limited set of operators and instructionsA state machineNo code dedicated to each match
One match on address use same code as a match on portNew matchs are possible without kernel modification
LOC count50000 LOC in userspaceonly 7000 LOC in kernel-space
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 36 / 48
Less kernel update
Pseudo state machine instructionCurrent instructions cover need found in previous 10 yearsNew instruction require very limited code
Development in userspaceA new match will not need a new kernelICMPv6 implementation is a single userspace patch
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 37 / 48
Example of ICMPv6
include/datatype.h | 2 ++include/payload.h | 14 +++++++++++src/parser.y | 33 +++++++++++++++++++++++++++---src/payload.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++src/scanner.l | 4 ++5 files changed, 109 insertions(+), 3 deletions(-)
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 38 / 48
Example of ICMPv6
s t a t i c const struct datatype icmp6_type_type = {. type = TYPE_ICMP6_TYPE,. name = " icmpv6_type " ,. desc = " ICMPv6 type " ,. by teorder = BYTEORDER_BIG_ENDIAN,. s ize = BITS_PER_BYTE,. basetype = &in teger_ type ,. sym_tbl = &icmp6_type_tbl ,
} ;
#define ICMP6HDR_FIELD(__name , __member ) \HDR_FIELD(__name , struct icmp6_hdr , __member )
#define ICMP6HDR_TYPE(__name , __type , __member ) \HDR_TYPE(__name , __type , struct icmp6_hdr , __member )
const struct payload_desc payload_icmp6 = {. name = " icmpv6 " ,. base = PAYLOAD_BASE_TRANSPORT_HDR,. templates = {
[ ICMP6HDR_TYPE] = ICMP6HDR_TYPE( " type " , &icmp6_type_type , icmp6_type ) ,[ ICMP6HDR_CODE] = ICMP6HDR_FIELD( " code " , icmp6_code ) ,[ICMP6HDR_CHECKSUM] = ICMP6HDR_FIELD( " checksum " , icmp6_cksum ) ,[ ICMP6HDR_PPTR] = ICMP6HDR_FIELD( " parameter−problem " , icmp6_pptr ) ,[ ICMP6HDR_MTU] = ICMP6HDR_FIELD( " packet−too−big " , icmp6_mtu ) ,[ ICMP6HDR_ID ] = ICMP6HDR_FIELD( " i d " , icmp6_id ) ,[ ICMP6HDR_SEQ] = ICMP6HDR_FIELD( " sequence " , icmp6_seq ) ,[ ICMP6HDR_MAXDELAY] = ICMP6HDR_FIELD( "max−delay " , icmp6_maxdelay ) ,
} ,} ;
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 39 / 48
Basic utilisation
File modenft -f ipv4-filter
Command line mode
nf t add rule i p f i l t e r i npu t tcp dpor t 80 dropnf t l i s t table f i l t e r −anf t delete rule f i l t e r output handle 10
CLI mode
# nf t − inft > l i s t table< c l i >:1:12−12: Er ro r : syntax er ro r , unexpected end of f i l e , expect ing s t r i n gl i s t table
^nft > l i s t table f i l t e rtable f i l t e r {
chain i npu t {i p saddr 1 . 2 . 3 . 4 counter packets 8 bytes 273
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 41 / 48
Set handling
Interests of setsOne single rule evaluationSimple and readable rulesetEvolution handling
Anonymous set
nf t add rule i p g loba l f i l t e r \i p daddr {192 .168 .0 .0 /24 , 192.168.1 .4 } \tcp dpor t {22 , 443} \accept
Named set
nf t add set g loba l ipv4_ad { type ipv4_address ; }nf t add element g loba l ipv4_ad { 192.168.1 .4 , 192.168.1 .5 }nf t delete element g loba l ipv4_ad { 192.168.1 .5 }nf t add rule i p g loba l f i l t e r i p saddr @ipv4_ad drop
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 42 / 48
Mapping
Principle and interestAssociative mapping linking two notionsA match on the key trigger the use of the valueUsing addresses, interfaces, verdicts
ExamplesAnonymous mapping:
# nf t add rule f i l t e r output i p daddr vmap \{192 .168 .0 .0 /24 =\ > drop , 192.168.0.1 =\ > accept }
Named mapping:
# nf t − inft > add map f i l t e r verdict_map { type ipv4_address => v e r d i c t ; }nft > add element f i l t e r verdict_map { 1 . 2 . 3 . 5 => drop }nft > add rule f i l t e r output i p daddr vmap @verdict_map
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 43 / 48
Usage example
set web_servers {type ipv4_addresselements = { 192.168.1.15 , 192.168.1 .5 }
}map admin_map {
type ipv4_address => v e r d i c telements = { 192.168.0.44 => jump logmetender , \
192.168.0.42 => jump logmetrue , 192.168.0.33 => accept }}chain forward {
c t s t a t e es tab l i shed accepti p daddr @web_servers tcp dpor t ssh i p saddr map @admin_mapi p daddr @web_servers tcp dpor t h t t p log accepti p daddr @web_servers tcp dpor t h t t ps acceptcounter log drop
}chain logmetender {
log l i m i t 10/ minute accept}chain logmetrue {
counter log accept}
}
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 44 / 48
Transition and evolution
A complete iptables compatibilityiptables-nftables
Binary compatible with iptablesUsing nftables framework
Same kernel can be used with two systemsA progressive update
A high level libraryTo be used by frontendsOr by network manager systems
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 45 / 48
Conclusion
A huge evolutionSolving iptables problemAn answer to new usages
Set handlingComplex matches
Availability for end 2013, beginning 2014Finalizing iptables compatibilityHigh level libraryDebug and some functionalities
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 47 / 48
Questions ?
Do you have questions ? Thanks toNetfilter teamAstaro/Sophos forfinancing the developmentGoogle for GSoC 2013
More informationNetfilter :http://www.netfilter.org
Nftables quick & dirty :https://t.co/cM4zogob8t
Contact meMail:[email protected]
Twitter: @Regiteric
Éric Leblond (Nefilter Coreteam) nftables, far more than %s/ip/nf/g September 24, 2013 48 / 48
Top Related