Network Programming and Protocl Design - cse.tkk.fi · Nixu Ltd. 4/50Network Programming and...

Nixu Ltd.

Network Programming and Protocl Design

Nixu Ltd. 2/50 Network Programming and Protocl Design

What is a socket?

• Sockets (also known as Berkeley Sockets) is an application programming interface (API) to the operating system’s TCP/IP stack

• Used on all Unixes

• Windows Sockets are similar

• Some Unixes have XTI (X/Open Transport Interface) and TLI (Transport Layer Interface) (not covered here)

• Can be used for other protocol stacks than TCP/IP (not covered here)


Socket functions

• Socket functions:— Allocating resources: socket, bind— Opening connections: accept, connect— Closing connections: close, shutdown— Sending and receiving: read, write, recv, send, recvfrom, sendto,

recvmsg, sendmsg— Other: getsockopt, setsockopt, getsockname, getpeername

• Utilities— Network information (host names, addresses)— Byte order


Byte order

• Addresses and port number must be in network byte order — Also known as big-endian, most-significant byte first

• Conversion routines for 16 and 32-bit integers:— htons, htonl ("host to network short/long")— ntohs, ntohl ("network to host short/long")

• Null macros ("#define htons(x)(x)") on big-endian systems


Address structures

struct in_addr { in_addr_t s_addr; /* IPv4 address, network byte order */};

struct sockaddr_in { sa_family_t sin_family; /* AF_INET */ in_port_t sin_port; /* 16-bit port, network byte order */ struct in_addr sin_addr; /* IPv4 address */ char sin_zero[8]; /* always zero */};x


Address functions

• From string to address: unsigned long inet_addr(const char *cp)— returns -1 on error

• From address to string: char* inet_ntoa(struct in_addr)— return pointer to statically allocated buffer— surprisingly, is thread-safe (uses thread-specific data) ON SOME

UNIXES

• inet_aton (NOT ON ALL UNIXES) — from ascii "194.197.118.20" to struct in_addr— int inet_aton(const char *cp, struct in_addr *inp);


Creating a socket

• int socket(int domain, int type, int protocol)

• Domain is— AF_INET for TCP/IP protocols— AF_UNIX (AF_LOCAL) for Unix named pipes, others

• Type is— SOCK_STREAM (TCP)— SOCK_DGRAM (UDP)— SOCK_RAW (raw IPv4), others

• Protocol is usually zero

• Returns new socket descriptor, or -1 on error


Typical TCP client

socket

connect

read/write

close


Connecting

• int connect(int sockfd, struct sockaddr *serv_addr, int addrlen);

• Establishes a connection to server

• Return values are 0 on success, -1 on error (errno set accordingly)

• Typical errno values:— ECONNREFUSED: host is up, but no server listening— ETIMEDOUT: host or network is down?


Example: Simple HTTP client (1/4)

/* * Simple HTTP client program, version 1. * Written by [email protected]. */

#include <arpa/inet.h>#include <netinet/in.h>#include <stdio.h>#include <stdlib.h>#include <string.h>#include <sys/socket.h>#include <sys/types.h>#include <unistd.h>



void die(const char* message){ fprintf(stderr, "%s\n", message); exit(1);}

int main(int argc, char *argv[]){ int sockfd, n; struct sockaddr_in addr; unsigned char buffer[4096]; if (argc != 4) die("usage: geturl ip-address port local-url");



/* Open socket */ if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) die("socket error");

/* Parse address */ memset(&addr, 0, sizeof(addr)); addr.sin_family = AF_INET; addr.sin_port = htons(atoi(argv[2])); addr.sin_addr.s_addr = inet_addr(argv[1]); if (addr.sin_addr.s_addr == -1) die("bad address");

/* Connect to remote host */ if (connect(sockfd, (struct sockaddr*) &addr, sizeof(addr)) == -1) die("connect error");



/* Send HTTP request */ write(sockfd, "GET ", 4); write(sockfd, argv[3], strlen(argv[3])); write(sockfd, " HTTP/1.0\r\n\r\n", 13);

/* Read response */ while ((n = read(sockfd, buffer, sizeof(buffer))) > 0) write(STDOUT_FILENO, buffer, n);

/* Close and exit */ close(sockfd); return 0;}


Simple HTTP client in action

$ ./httpclient1 192.168.3.4 80 /HTTP/1.1 200 OKDate: Sat, 24 Apr 1999 17:08:25 GMTServer: Apache/1.3.4Last-Modified: Fri, 26 Feb 1999 15:28:20 GMTConnection: closeContent-Type: text/html

<html><head><title>Example Inc.</title></head><body><h1>Welcome to Example Inc’s web server!</h1>...

$


What is a Server

• Background process

• No user interface

• Handles service requests from network

• Can also send requests, for example DNS and NTP

• Typically must handle many requests concurrently ? ?some kind of multitasking needed


What is special in a server

• Concurrency & network I/O

• Protocol encoding/decoding

• Application-specific logic

• System interaction: Unix daemons, Windows NT services

• Logging

• Security


Network I/O on Unix

• Sockets are file descriptors

• Important I/O operations for TCP sockets:— accept— connect— read, write— close, shutdown

• Important I/O operations for UDP sockets:— sendto— recvfrom


Binding to specific port

• int bind(int sockfd, struct sockaddr *my_addr, int addrlen)

• bind assigns a specific local address and port to the socket— normally used for servers (where the port must be known)— IP address (or IPADDR_ANY)

• In clients, you don’t usually need bind— a random "ephemeral" port is chosen automatically— the correct interface and IP address are chosen automatically


bind() example

• Bind socket to local port 80 (for HTTP server)struct sockaddr_in my_addr;

memset(&my_addr, 0, sizeof(my_addr));my_addr.sin_family = AF_INET;my_addr.sin_port = htons(80);my_addr.sin_addr.s_addr = INADDR_ANY;

if (bind(sockfd, (struct sockaddr*) &my_addr, sizeof(my_addr)) == -1) die("bind failed");


bind() notes

• Bind can also bind to specific IP address!— Most hosts have at least two interfaces, loopback and Ethernet— One physical interface can have multiple addresses ("IP Aliasing")— Web servers need to bind to specific IP with IP Aliasing— INADDR_ANY binds to all addresses

• Binding to ports 1-1023 requires root priviledges on Unix— Traditionally used for security: do not trust this!


Getting remote host name

• gethostbyaddr() converts IP address to domain name

• Not all addresses have domain names

• Not secure: the owner of IP address can return any name he wants!

• Partial solution: Double DNS lookup— gethostbyaddr(IP_ADDRESS) ? NAME— gethostbyname(NAME) ? NAME_ADDRESSES (0...N)— check that IP_ADDRESS ? NAME_ADDRESSES


TCP connections

• When the server calls accept() it gets:— file descriptor for reading/writing data— remote IP + port (from getpeername)— local IP + port (from getsockname)


UDP "connections"

• UDP is not really connected

• When the server calls recvfrom() it gets— packet data— remote IP + port


Iterative UDP server

initialize

wait for packet

process request

send reply


Iterative UDP server

• Single thread of execution

• If processing doesn’t take long, works well!— Othervise the service is blocked

• Simple, easy to coordinate access to resources

• Must be careful not to use blocking operations:— CPU-intensive tasks— SQL database queries— gethostbyname, gethostbyaddr


Example: radiusd

• Potentially lots of requests

• Uses UDP

• Very little processing per request

• Solution: single-threaded UDP server.


Process-per-connection TCP server

initialize

wait for connection

fork

receive data

process request

send reply

close connection & exit


Process-per-connection TCP server

• New process started for each connection

• Good sides:— Easy to use, works!— Reliable: If one process dies, others continue

• Problems:— Starting new processes is slow— Co-operation between processes is limited or difficult

• Access to shared resources (log files, etc.) needs to be coordinated


Example: telnetd

• Each connection takes quite long, so process starting overhead is not a problem

• Asynchronous I/O would be very difficult

• Solution: process-per-connection TCP server


Process pre-allocation

• Since starting processes is slow, start all processes at the beginning

• Memory used by unused processes is wasted.

• Only a limited number of connections concurrently.

• Used very successfully!


Threads

• Creating threads is much faster than processes

• All modern Unixes and Windows NT have threads

• Shared memory makes co-operation easy

• Access to shared memory needs to be coordinated


Asynchronous TCP server

initialize

wait for events

accept new connection

receive data and process

send morereply data


Asynchronous TCP server

• Single thread of execution

• Event multiplexing using poll() or select()

• Easy to coordinate access to resources

• Must avoid blocking operations


Example: Bind DNS server

• Lots of requests

• Needs to be very fast

• Most requests are UDP, but some TCP

• Very little CPU processing

• In-memory database and cache(hard to share between processes)

• Needs to be portable to legacy systems ? no threads

• Solution: asynchronous I/O for both UDP and TCP


Concurrency in clients

• Typically clients have user interface, etc.

• Using separate processes is difficult, since there are communication needs between network process and user interface.

• Solution: threads.


Distributed Computing

• Generally a view of shared computing and data resources, transparent communications between programs and access to objects located in other hosts

• Sun RPC is the first popular protocol— CORBA is currently somewhat popular — Both are based on an abstraction layer that hides the network

• Web Services and .Net take a slightly different approach— XML is used to represent all kinds of objects

• Advantages are access to shared resources, transparent communications and flexibility

• Disadvantages are added complexity and security risks

• These technologies are also called middleware


RPC, Remote Procedure Calling

• From Sun Microsystem’s Open Networking Group

• A communications layer between transport and application levels— Used by NFS and NIS

• An API for software to communicate over the network

• Uses the client-server communications model— The client and the server communicate using "stub code" interfaces

that hide the actual communications layer— The stub code is created by a RPC protocol compiler


Object Request Broker (ORB) Architec-tures

• Object Management Architecture (OMA) of the Object Management Group:— Common Objet Request Broker Architecture (CORBA)— Common Object Services Specification (COSS)

• CORBA definitions are used to describe the communications mechanisms of distributed objects— Interface Description Language (IDL)

• COSS objects offer the basic services between applications’ objects

• The idea of an open software bus, on which object components from different vendors can interoperate on different networks and operating systems


… CORBA

• Instead of calling remote functions (RPC) you call methods in remote objects— Objects can have their own data store— Allows more dynamic operations

• The Object Request Broker (ORB) locates the objects to be called

• The Internet Inter-ORB Protocol (IIOP) connects the ORBs


Protocol design

• Implementing a protocol in software is only reasonably difficult

• Designing a good protocol is very difficult— But sometimes it must be done

• The design starts from the requirements— Define the probelm to be solved— Define why no existing protocol is good enough

• It is a good idea to supplement the requirements with practical usage cases

• Protocol design tools exist and should be used— Help to avoid some pitfalls, like deadlocks


Some notes on protocol design

• Watch out for overhead— Bandwidth— CPU— other

• Pretty good is hard to tell from optimal, but much more easier to implement

• Which is better, a big solution for all problems or a small solution for one problem?

• Things that are left for implementators to decide, are going to be implemented in as non-compatible way as possible— IPSec


Protocol life cycle

• How to prepare for future versions of the protocol— Version numbers— Reserved bit fields— Forwards and backwards compatibility

• Parameters— Leaving parameter values open creates more work for implementators

and can cause problems, but also allows for tuning the performance— Leaving values for implementators to set is the lazy way out and can

cause probelms, but can also be used to solve unforeseen problems


Desirable protocol features

• Autoconfiguration

• Robustness— Simple— Self-stabilization— Byzantine robustness

• Documentation and standardization— Remember, two freely available independent implementations for an

Internet standard


Protocol Design Folklore

• From "Interconnections, 2:nd ed." by Radia Perlman

• Flexibility, backward compliance and optimality create complexity, which makes the protocol implementation harder, which make the protocol less popular

• Each participant in a committee usually wants to add his own feature

• If the protocol becomes popular, it will be used for something that it is not intended for

• An underspecified protocol is not a protocol but a framework— "Content-type:" field in HTTP would be useless, unless MIME was

specified

• The goal should be simplicity, not complexity


… Protocol Design Folklore

• Solving the general problem and leaving exceptions to be handled by some external mechanism makes sometimes sense— Assuming that you know what the general problem is and what are the

exceptions— IPv6 is a sample of this— You can also optimize the most common cases

• Think about the boundary limits— Ethernet is physically limited, but does not need a router -> cheap

• Think about the address space— Limited like IPv4 or TCP port numbers— Unlimited like DNS or SNMP OIDs


… Protocol Design Folklore

• Design how to fail— How to run out of buffer space, capacity, etc.

• Plan an upgrade path for future versions— How do different versions talk to each other— "I talk" and "I accept" version fields— Make protocol select the highest common denominator isntead of the

lowes

• Respect the independence of layers

• Specify the handling of unknown options— "Skip and ignore"— "Skip and log"— "Stop parsing and generate error"


Implementation

• Be conservative in what you send, and liberal in what you accept— Always set reserved bits to zero

• For robustness, every line of code should be exercised frequently— Problems caused by changes in other parts of code or the environ-

ment become apparent faster, not just in exception cases

• Document

• Document your thinking


Practical Problems for Protocols

• Split networks

• Garbage data, 8-bit characters (not really anymore, but 16-bit...)

• Multiple copies of a network entity

• Overload, lack of resources

• Intentional attacks

• Upper/lower layer crashes/problems


The Trends of Data Communications

• IP over everything, everything over IP— Data, voice, pictures, reliability, security

• Mobility— Route and capacity variations— Weak terminals

• Ubiquitous computing— We are the Borg...— Always on, always connected, wearable computing

• Comprehensive security

• Nanocomputing, biocomputing...


So long

• Source and No Money will get you through the hard times better than Money and No Source.— Bob Kodner

• Facts without theory are useless - Theory without facts is bullshit— Unknown

Network Programming and Protocl Design - cse.tkk.fi · Nixu Ltd. 4/50Network Programming and...

Documents

Transcript of Network Programming and Protocl Design - cse.tkk.fi · Nixu Ltd. 4/50Network Programming and...