服务框架:Thrift & PasteScript

BPUG, Sep, 2010

为什么要服务？库不好吗？

云就是服务

• IaaS - Infrastructure as a Service

• PaaS - Platform as a Service

• SaaS - Software as a Service

适合服务的场景

• 分开部署• 独立维护• 错误隔离• 资源节省• 跨语言/跨平台

Service框架让常规的事情简单，让特殊的事情可能

HTTP + web.py

import web

urls = ('/add', 'add')

class add: def GET(self): i = web.input() r = int(i.a) + int(i.b) return str(r)

app = web.application(urls, globals)

if __name__ == '__main__': app.run()

import urllib

def add(a, b): f = urllib.urlopen('http://calculator.services.douban.com/add?a=%s&b=%s' % (a, b)) r = f.read() return int(r)

问题

• 纠结于格式转换• JSON有帮助，但有限

• HTTP和文本协议的overhead过重

Thrifthttp://incubator.apache.org/thrift/

Thrift

• 跨语言• C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, OCaml

• 可定义数据结构• struct, list, set, map

• 二进制传输协议• 支持oneway calling

• 自带RPC框架实现

Thrift Example

# Make an objectup = UserProfile(uid=1, name="Mark Slee", blurb="I'll find something to put here.")

# Talk to a server via TCP sockets, using a binary protocoltransport = TSocket.TSocket("localhost", 9090)transport.open()protocol = TBinaryProtocol.TBinaryProtocol(transport)

# Use the service we already definedservice = UserStorage.Client(protocol)service.store(up)

# Retrieve something as wellup2 = service.retrieve(2)

class UserStorageHandler : virtual public UserStorageIf { public: UserStorageHandler() { // Your initialization goes here }

void store(const UserProfile& user) { // Your implementation goes here printf("store\n"); }

void retrieve(UserProfile& _return, const int32_t uid) { // Your implementation goes here printf("retrieve\n"); }};

int main(int argc, char **argv) { int port = 9090; shared_ptr<UserStorageHandler> handler(new UserStorageHandler()); shared_ptr<TProcessor> processor(new UserStorageProcessor(handler)); shared_ptr<TServerTransport> serverTransport(new TServerSocket(port)); shared_ptr<TTransportFactory> transportFactory(new TBufferedTransportFactory()); shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory()); TSimpleServer server(processor, serverTransport, transportFactory, protocolFactory); server.serve(); return 0;}

struct UserProfile { 1: i32 uid, 2: string name, 3: string blurb}service UserStorage { void store(1: UserProfile user), UserProfile retrieve(1: i32 uid)}

Thrift通信架构

Transport(socket)

Protocol(binary)

Handler(app code)

Processor(dispatcher)

thrift package generated code

Thrift的问题

• 没有统一的部署方式• 调用代码复杂• 开发调试不方便

我们需要

• 快捷的创建项目方法• 方便的开发调试环境• 统一的部署方式

PasteScripthttp://pythonpaste.org/script/

Paste

• WSGI工具集

• 三部分组成• Paste

• PasteScript

• PasteDeploy

PasteScript

• paster serve

• templates

paster serve

• paster serve --reload --monitor-restart development.ini

[server:main]use = egg:Paste#httphost = 0.0.0.0port = 5000

[app:main]use = egg:MyWSGIApp

# Logging configuration[loggers]keys = root, access_log...

Thrift通信架构

Transport(socket)

Protocol(binary)

Handler(app code)

Processor(dispatcher)

server app

paster serve

• paster serve --reload --monitor-restart development.ini

[server:main]use = egg:DoubanService#thread_poolport = 9090pool_size = 10

[app:main]use = egg:CalculatorServer

# Logging configuration[loggers]keys = root, access_log...

setuptools entry points

entry_points = """[paste.server_runner]thread_pool = doubanservice.server:thread_pool_server_runner"""

DoubanService/setup.py:

entry_points = """[paste.app_factory]main = calculator_server.makeapp:make_app"""

CalculatorServer/setup.py:

app

from .gen.calculator import Iface, Processorfrom .app import Handler

def make_app(global_config, **local_conf): handler = Handler() processor = Processor(handler) return processor

server runnerdef thread_pool_server_runner(app, global_conf, **kwargs): for name in ['port', 'pool_size']: if name in kwargs: kwargs[name] = int(kwargs[name]) pool_size = kwargs.pop('pool_size') host = kwargs.pop('host', '0.0.0.0') transport = TSocket.TServerSocket(**kwargs) transport.host = host tfactory = TTransport.TBufferedTransportFactory() pfactory = TBinaryProtocol.TBinaryProtocolFactory() server = ThreadPoolServer(app, transport, tfactory, pfactory) if pool_size: server.threads = pool_size

server.serve()

我们得到了什么？

• 启动服务的命令 (paster serve)

• 可选的服务器 ([server:main] use)

• 可配置的服务器和应用参数• 代码更改会自动重启服务 (--reload)

• 可配置的日志 ([loggers])

Paste Template

entry_points = """[paste.server_runner]thread_pool = doubanservice.server:thread_pool_server_runner

[paste.paster_create_template]doubanservice = doubanservice.templates:DoubanServiceTemplatedoubanservice_server_py = doubanservice.templates:ServerPyTemplatedoubanservice_client_py = doubanservice.templates:ClientPyTemplate"""

DoubanService/setup.py:

paster create -t doubanservice calculator

doubanservice template

doubanservice/templates/doubanservice/├── +package+.thrift_tmpl├── create-client_tmpl├── create-server_tmpl└── gen_tmpl

vi calculator.thrift./create-server py./create-client py./gen

class DoubanServiceTemplate(paste.script.templates.Template): _template_dir = 'doubanservice' summary = "A DoubanService project"

def post(self, command, output_dir, vars): for filename in ['gen', 'create-server', 'create-client']: os.chmod(os.path.join(output_dir, filename), 0755)

server_py templatedoubanservice/templates/server-py/├── +package+│   ├── __init__.py│   ├── app.py_tmpl│   └── makeapp.py_tmpl├── development.ini_tmpl├── gen_tmpl├── production.ini_tmpl├── remote_tmpl├── scripts│   └── censor-server-py├── serve├── setup.cfg├── setup.py_tmpl└── tests

vi calculator_server/app.py./serve./remote add 1 2

client_py templatedoubanservice/templates/client-py├── +package+│   ├── __init__.py│   └── client.py_tmpl├── gen_tmpl├── setup.py_tmpl└── tests

from calculator_client import calculatorprint calculator.add(1, 2)

class Client(BaseClient): __metaclass__ = ClientMetaClass service_name = '${service}' thrift_module = ${service}_genmod port = 9090 connect_timeout = 1000 read_timeout = 5000

${service} = Client()

Demoa calculate service

广告时间A New OpenSource Project...

OneRingBuild Desktop Applications Using Web Technology

http://code.google.com/p/onering-desktop/

目标

• 用Web技术做桌面应用

• 使用HTML5+CSS3提供用户界面/本地存储

• 使用Javascript编写界面逻辑/业务逻辑

• 用AJAX方式调用本地代码

• 跨操作系统

• Qt (mac, windows, linux)

• Cairo (windows, linux)

• WebKit Framework (mac)

• 跨语言

• C API DLL

• Python - WSGI

#!/usr/bin/env pythonimport jsonimport webimport onering

urls = ( '/init', 'init', '/', 'index',)

class init: def GET(self): web.header('Content-Type', 'application/json') return json.dumps({'width': 400, 'height': 300, 'url': '/'})

class index: def GET(self): web.header('Content-Type', 'text/html') return """<html><head><script type="text/javascript" src="onering://onering/onering.js"></script></head><body><p>Hello, world!</p><button onclick="javascript:ONERING.exit()">Exit</button></body></html>"""

app = web.application(urls, globals())

if __name__ == '__main__': onering.register_wsgi_app("demo", app.wsgifunc()) onering.loop("demo")

Join Us If You Know...• C++ / C / Objective-C / Python

• Qt / win32 / Cocoa / GTK

• WebKit / Gecko

• Javascript / HTML5

• Python C API / py2exe / py2app

• 其他语言的C绑定和发布

• Win/Mac/Linux 下的动态库发布和软件发布

• 软件瘦身

你将得到

• 一个可能会被千万人使用的项目• 推动HTML5应用大潮

• 加入豆瓣的机会

ThanksQ & A