Changing Wireshark with Lua

Changing Wireshark with Lua:Writing a Lua Plug-in to Create a

Custom Decoder

Hadriel Kaplan128 Technology, Inc.

Questions for you

How many of you attended the Lua workshop at last year’s Sharkfest ‘14?

How many of you have written a Lua plugin already?

How many of you know Lua the language?

How many of you are in the wrong room?

Resources

http://wiki.wireshark.org/Luahttp://wiki.wireshark.org/Lua/ApiChangeshttp://wiki.wireshark.org/Lua/Examples

http://ask.wireshark.org

http://www.lua.org/docs.htmlhttp://lua-users.org/wiki/http://luarocks.org/

4

Agenda

Intro to Lua

What can you do with it?

Lua for Wireshark

Basics of a script

A small dissector script

What’s new in 1.12+

What’s coming in 2.0?

What’s Lua?Small but powerful scripting language

Interpreted, dynamically typed, etc.

Extremely fast

Size: ~200KB

Created by Pontifical Catholic University of Rio de Janeiro, Brazil

As a language, it’s very simple

Few keywords, data types, semantics

Runs on anything, in pure ANSI C

Popular for plugins in embedded systems, games, and of course Wireshark

6

Why Lua?I don’t know why Wireshark chose it originally, but…

There aren’t actually that many reasonable choices for a plugin-style language

Python, JavaScript, and Lua are common ones to choose from

There was a Python API for Wireshark, but it wasn't maintained so it’s been removed

If you want one, feel free to submit code

7

Why Lua vs. C?Easier/faster to learn Lua than C

Don’t have to deal with compiling Wireshark from source code, nor deal with git

Make changes and distribute to users as a plugin, vs. waiting for next Wireshark release

Much more stable API across Wireshark versions vs. C-based dynamic plugins

Backwards-compatibility is not guaranteed, but is very rarely broken

8

Why not Lua vs. C?#1 reason: support for bug fixes

Native C-based dissectors in the git repo can be fixed by other developers

There is no central repo for Lua scripts, nor a sufficiently large number of Lua developers to manage them

(well... There is wiki.wireshark.org/Contrib)

Other reasons:

C-code is faster

C-code can still do things Lua cannot

C-code distribution handled by wireshark.org

9

What can you do with Lua?Protocol dissectors

Any protocol type at any layer

Custom stats/counters

Advanced expert analysis

Save packets to pcap files, using custom criteria

Add new GUI menu items for actions

Read/write from/to custom packet file formats

Load log files and view entries as “frames”

Fileshark10

Lua for Wireshark

11

The life of a Lua script in Wireshark

When you start wireshark/tshark, the following happens (simplified view):1. C-code initializes most of the native code items,

including native protocol dissectors, taps, etc.2. Reads and executes the Lua scripts3. Registers tap listeners4. Reads preference file5. Parses command-line arguments6. etc…

Loading Lua plugins

Wireshark comes with a Lua script called “init.lua”, in <global-config-dir>

This init.lua script is always loadedThere’s a Lua variable to disable Lua inside this init.lua script

Then it loads <personal-config-dir>/init.luaThen it loads all “*.lua” Lua scripts in <personal-

plugins-dir>Then it loads any scripts from the command line-X lua_script:foo.lua

13

The init.lua scriptThe init.lua file in <global-config-dir> is important

Don’t muck with this file

It sets a lot of variables that are needed by your Lua scripts

Only change the “disable_lua” and “run_user_scripts_when_superuser” variables, if any

The init.lua in <personal-config-dir> is different, and should NOT contain the same variables/info as the one in <global-config-dir>

I.e., don’t just copy the file over

Because the same variables will overwrite the ones in <global-config-dir>14

Basics of a scriptIn this section we’ll switch back/forth with

example Lua scripts

Inside a scriptA Lua script is executed during startup of Wireshark, but

most of the things it does happen much later

That’s because a script usually does most of its work inside functions, and those functions only get invoked later by Wireshark

Wireshark knows to call those functions later, because they’re registered as callbacks

Usually by being a function of an object that is registered by the script

Either the object or function is registered explicitly, or added to a table

16

Listener taps

To access data of existing protocol dissectors we use Listener taps

Taps cannot add protocol fields, nor affect the GUI

Think of it as a “read-only” thing (hence “Listener”)

Their main use is generating statistics, performing analysis, etc.

They can be applied to just packets of certain protocols, even using display filters

Dissectors

To decode a new protocol we create a “dissector” script

Allows us to create new fields, affect the GUI, etc.

There are two ways to dissect packets:By registering a new protocol on another layerBy registering a post-dissector

There are two ways to register on another layer:By adding the new protocol to a table’s port numberBy registering it as a heuristic dissector

Basics of dissectors

To create a dissector, you actually create a new “Proto” object representing your new protocol

You then define the “dissector” function of the new Proto object to decode it

Wireshark invokes this function later

To create new fields for your protocol, you create “ProtoField” objects

Not to be confused with “Field” objects, used for existing fields

ProtoField objects

When you create them, you give them attributes:

Their display names, filter names

How big they are, how to display them

But they’re not part of your Proto’s protocol until you set them to the “field” member in a table

Once that’s done, Wireshark can use their attributes to auto-decode+display them when they’re used in TreeItem:add()/add_le() calls

The dissector function

You define the “dissector” function for Proto

To decode the packet, add to the tree, etc.

The function is passed 3 arguments:1. The Tvb object (the frame buffer)

2. The Pinfo object (packet info)

3. The TreeItem object (GUI tree)

You use these passed-in objects to decode the frame, add to the tree, etc.

TCP Dissection

Dissecting protocols in TCP is hardCould be partial message, or many, in one TCP pkt

It may be easier to do depending on your protocol encoding

If it has a length field early on, in a fixed location

Basically you need to handle:Invoking your dissector multiple timesGetting a partial messageStarting in the middle of a message

New stuff in 1.12+

Well… not all the new stuff

A lot of additions/enhancements in 1.11 - too many to enumerate all of them here

The following slides will cover the highlights

All changes are listed here:http://wiki.wireshark.org/Lua/ApiChanges

Also, the API docs now indicate what Wireshark version a new function appears in

For example, “Since: 1.11.3” 24

Heuristic dissector support

Previously, only specific dissectors or general post-dissectors could be implementedNow you can create a heuristic dissector

25

64-bit integer supportLua numbers are double-precision floating points (a C-code double), which means they are integer precise up to ~53bits

This is a problem if you need to count higher than that, or if you need to convert a 64-bit integer protocol field to a Lua number

So Wireshark now has full support for both signed Int64 and unsigned UInt64 objects

They now support math operations, comparisons, conversions, etc.

26

Binary structures: Struct library

Wireshark already had a ByteArray object to represent arrays of binary bytes

But it’s cumbersome to work with in Lua, and only supported going to/from packet contents

Now it also has the popular Struct libraryCan encode/decode Lua binary strings to/from Lua variables

Supports endianess control, padding, etc.

Since it is to/from a Lua string, it can be used on things other than packet contents

For example, file contents

27

True regex support

Lua has its own “pattern” matching language engine, which is similar but different to regular expressions

A simplistic implementation designed for a small code sizeIt’s slow and weak, so most people import a real regex library

The Glib Regex implementation has now been exposed into Lua, for PCRE-based regex support

Based on the popular Lrexlib library, with minimal differences

28

Command-line arguments

Previously in Wireshark and tshark you could load Lua scripts through the command-line:tshark -X lua_script:myscript.lua

Now you can pass the script command-line arguments as well:tshark -X lua_script:myscript.lua -X

lua_script1:foo -X lua_script1:bartshark -X lua_script:my.lua -X

lua_script:other.lua -X lua_script2:bar29

Custom file format read/write

Wireshark already natively supports numerous packet file formatsBut there are other files that contain packet content that might be useful to view as packets

For example, application-layer messages as received/sent by the application, not the lower-level

Or log files for activity, debug, etc. (logshark?)

30

Viewing file meta data

Since you can now read any file format, and you can create any “protocol” dissector, you can create fileshark in LuaThe concept of fileshark is to let you use wireshark to open various file formats, and view the file’s format information as frame decoded data

e.g., view an MPEG file’s internal format details, such as file header info, image info, etc.

31

Test suites

Not a user feature per se, but 1.11 added many test suites for Wireshark’s Lua APIThis should (hopefully) reduce the number of initial bugs as well as regressions

There weren’t very many to begin with, but with all the additions we’re hoping to keep it low

If you have Lua scripts you’d like to be included in automatic testing, let me know

email: hadrielk@yahoo.com32

What’s coming in 2.0

This is all tentative (i.e., “possible” but not done)Lua 5.3 support

Might mean removing support for BitopMore introspection of internals

Can view internal details of trees, protocolsLua GUI support in Qt-based GUI

The current GUI stuff is missing in QtGraphing exposed to Lua

Letting a Lua script affect a graph