Ruby meets Go
-
Upload
ntt-communications-technology-development -
Category
Technology
-
view
15.132 -
download
0
Transcript of Ruby meets Go
About Me
● Masaki Matsushita● CRuby Committer
○ 138 Commits■ Mainly for performance improvement■ Marshal.load, Hash#flatten, etc.
● Software Engineer at NTT Communications○ Contribution to OpenStack○ Slide at OpenStack Summit Tokyo
http://goo.gl/OXTYor● Twitter: @_mmasaki Github: mmasaki
Today’s Topic
● Go 1.5 Feature: buildmode “c-shared”○ Cgo Basics
● Using Go Function from Ruby○ FFI and Fiddle without ruby.h
● Writing Extension Library with Go (and C)○ Define Functions Equivalent to C Macros○ Avoid Copy of Strings○ Propagate Reference from Ruby to Go○ Creating Gem including Go code
Buildmode “c-shared”
● Go 1.5 relased in August 2015● Buildmode “c-shared” was introduced
○ go build -buildmode c-shared○ Build C shared library with cgo
● cgo enables:○ Refer to C functions, types and variables○ Export Go functions for use by C
Cgo Example: Say hello with puts() in Cpackage main
/*
#include <stdlib.h>
#include <stdio.h>
*/
import "C"
import "unsafe"
func main() {
cstr := C.CString("Hello, world!")
defer C.free(unsafe.Pointer(cstr))
C.puts(cstr)
}
Include C header file
Convert Go string into C String
Cgo Example: define and use C functionpackage main
/*
char *hello(void) {
return "Hello, world!";
}
*/
import "C"
import "fmt"
func main() {
cstr := C.hello()
fmt.Println(C.GoString(cstr))
}
Define C Function
Convert into Go String
Call C Function from Go
Try c-shared: add.go
package main
import "C"
//export add
func add(a C.int, b C.int) C.int {
return a + b
}
func main() {}
● go build -buildmode c-shared -o add.so add.go
Export Go Function for use by C
Load c-shared Libraries
● ruby-ffi○ https://github.com/ffi/ffi○ gem install ffi
● fiddle○ Standard ext library
● useful to call Go functions simply(without ruby.h)
Call Go Function from Ruby: ruby-ffi
require "ffi"
module Int
extend FFI::Library
ffi_lib "int.so"
attach_function :add, [:int, :int], :int
end
p Int.add(15, 27) #=> 42
Load c-shared library
Add Go Function to Module
Call Go Function from Ruby: fiddle
require "fiddle/import"
module Int
extend Fiddle::Importer
dlload "int.so"
extern "int add(int, int)"
end
p Int.add(15, 27) #=> 42
Go String and C String: str.sopackage main
import "C"
import "fmt"
//export hello
func hello(cstr *C.char) {
str := C.GoString(cstr)
fmt.Println("Hello, " + str)
}
func main() {}
Receive C String
Convert to Go String
Returning String: ruby-ffi
require "ffi"
module Hello
extend FFI::Library
ffi_lib "str.so"
attach_function :hello, [:string], :void
end
Hello.hello("world") #=> "Hello, world"
Ruby String can be passed
Returning String: fiddle
require "fiddle/import"
module Hello
extend Fiddle::Importer
dlload "str.so"
extern "void hello(char *str)"
end
Hello.hello("world") #=> "Hello, world"
Cgo Functions to Convert String
● C.CString(goString string) *C.char
○ copy Go String to C String
○ Users are responsible to free C String
● C.GoString(cString *C.char) string
● C.GoStringN(cString *C.char, length C.int) string
○ copy C String to Go String
Writing Extension Library with Go
● Naruse-san’s Amazing Talk:“Writing extension libraries in Go”at OedoRubyKaigi 05https://speakerdeck.com/naruse/writing-extension-libraries-in-go
● gohttp: https://github.com/nurse/gohttp○ Implementation of extension library in Go
C Extension Library Basicsstatic VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo = rb_define_class("Foo");
rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0);
}
C Extension Library Basicsstatic VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo = rb_define_class("Foo");
rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0);
}
Method Implementation
C Extension Library Basicsstatic VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo = rb_define_class("Foo");
rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0);
}
Using C Macro
C Extension Library Basicsstatic VALUE
rb_magic_number(VALUE self)
{
return INT2NUM(42);
}
void
Init_test(void)
{
rb_cFoo = rb_define_class("Foo");
rb_define_method(rb_cFoo, "magic_number", rb_magic_number, 0);
}
Function Pointer
Minimal Go Ext Example?//export rb_magic_num
func rb_magic_num(self C.VALUE) C.VALUE {
return INT2NUM(42)
}
//export Init_foo
func Init_foo() {
rb_cFoo = rb_define_class("Foo", C.rb_cObject)
rb_define_method(rb_cFoo, "magic_num", C.rb_magic_num, 0)
}
Writing Extension Library with Go
● Wrapper Function equivalent to C Macro○ C macros can’t be used by Cgo
● Convert Go String into Ruby without Copy● Propagate Ruby Reference to Go● Create gem including Go code
○ Modify Rakefile and extconf.rb
C Macros for Ruby Extention Libraries
● Useful C macros are defined in ruby.h○ INT2NUM: C int to Ruby Numeric○ NIL_P: true if obj is nil○ RSTRING_PTR: pointer to buffer of String○ RSTRING_LEN: lengh of String
● These macros can’t be used from Cgo…● Define Go functions equivalent to C macros
○ Use equivalent C function○ Wrap C macros with C function
Use Equivalent C Function
func LONG2NUM(n C.long) C.VALUE {
return C.rb_long2num_inline(n)
}
func NUM2LONG(n C.VALUE) C.long {
return C.rb_num2long(n)
}
Wrap C macros with C functionpackage main
/*
long rstring_len(VALUE str) {
return RSTRING_LEN(str);
}
*/
import "C"
func RSTRING_LEN(str C.VALUE) C.long {
return C.rstring_len(str)
}
Convert Go String into Ruby without Copy
● Go String -> C String -> Ruby String
● C.CString(goString string) *C.char
○ copy Go String to C String
○ Users are responsible to free C String
● VALUE rb_str_new(const char *ptr, long len)
○ copy C String to Ruby String
Basic Usage of C.CString()
// go/doc/progs/cgo4.go
func Print(s string) {
cs := C.CString(s)
defer C.free(unsafe.Pointer(cs))
C.fputs(cs, (*C.FILE)(C.stdout))
}
● Call C func and discard C str soon
Basic Usage of C.CString()
str := "Hello, world!"
// Copy #1
cstr := C.CString(str) // will be discarded soon
// Copy #2
rbstr := C.rb_str_new(cstr, C.long(len(str)))
● Need to copy twice!
Avoid Copy of Strings
● Get *C.char from Go String without Copy
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte)(unsafe.Pointer(&str))
return (*C.char)(unsafe.Pointer(&bytes[0]))
}
// example of use
cstr := GOSTRING_PTR(str) C.rb_utf8_str_new
(cstr, C.long(len(str)))
Avoid Copy of Strings
● Technique to Get []byte from Go w/o Copy http://qiita.com/mattn/items/176459728ff4f854b165
func GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte)(unsafe.Pointer(&str))
return (*C.char)(unsafe.Pointer(&bytes[0]))
}
Avoid Copy of Strings
● Get *C.char from []bytefunc GOSTRING_PTR(str string) *C.char {
bytes := *(*[]byte)(unsafe.Pointer(&str))
return (*C.char)(unsafe.Pointer(&bytes[0]))
}Cast to char
Example Usage of GOSTRING_PTR()
func RbString(str string) C.VALUE {
if len(str) == 0 { return C.rb_utf8_str_new(nil, C.long(0)) }
return C.rb_utf8_str_new(GOSTRING_PTR(str), GOSTRING_LEN(str))
}
func rb_define_class(name string, parent C.VALUE) C.VALUE {
return C.rb_define_class(GOSTRING_PTR(name), parent)
}
func rb_define_method(klass C.VALUE, name string, fun unsafe.Pointer, args int) {
cname := GOSTRING_PTR(name)
C.rb_define_method(klass, cname, (*[0]byte)(fun), C.int(args))
}
Propagate Ruby Reference to Go
● Go’s GC doesn’t know refs from Ruby● Go obj referenced from Ruby can be collected● We have to propagate Ruby Refs to Go● Use Map to keep reference to Go Objects
Propagate Ruby Reference to Govar objects = make(map[interface{}]int)
//export goobj_retain
func goobj_retain(obj unsafe.Pointer) {
objects[obj]++ // increment reference count
}
//export goobj_free
func goobj_free(obj unsafe.Pointer) {
objects[obj]-- // decrement reference count
if objects[obj] <= 0 { delete(objects, obj) }
}
Propagate Ruby Reference to Gostatic const rb_data_type_t go_type = {
"GoStruct",
{NULL, goobj_free, NULL},
0, 0, RUBY_TYPED_FREE_IMMEDIATELY|RUBY_TYPED_WB_PROTECTED
};
VALUE
NewGoStruct(VALUE klass, void *p)
{
goobj_retain(p);
return TypedData_Wrap_Struct((klass), &go_type, p);
}
Increment Reference Count
Decrement Reference Count
Directory Structure
● Use “bundle gem --ext foo”
├── ext
│ └── foo
│ ├── extconf.rb // configured to use go build
│ ├── foo.c // helper functions for use by Go
│ └── foo.h // export helper functions
│ ├── foo.go // created by hand
│ └── wrapper.go // created by hand
└── lib
Rakefile
require 'bundler'
Bundler::GemHelper.install_tasks
require 'rake/extensiontask'
task :default => [:compile]
spec = eval File.read('foo.gemspec')
Rake::ExtensionTask.new('foo', spec) do |ext|
ext.lib_dir = File.join(*['lib', 'foo', ENV['FAT_DIR']].compact)
ext.ext_dir = 'ext/foo'
ext.source_pattern = "*.{c,cpp,go}"
end
● Add .go into source_pattern
extconf.rb
require 'mkmf'
find_executable('go')
$objs = []
def $objs.empty?; false ;end
create_makefile("memberlist/memberlist")
case `#{CONFIG['CC']} --version`
when /Free Software Foundation/
ldflags = '-Wl,--unresolved-symbols=ignore-all'
when /clang/
ldflags = '-undefined dynamic_lookup'
end
● Some techniques to build successful
extconf.rb
File.open('Makefile', 'a') do |f|
f.write <<-EOS.gsub(/^ {8}/, "\t")
$(DLLIB): Makefile $(srcdir)/memberlist.go $(srcdir)/wrapper.go
CGO_CFLAGS='$(INCFLAGS)' CGO_LDFLAGS='#{ldflags}' \
go build -p 4 -buildmode=c-shared -o $(DLLIB)
EOS
end
● Modify Makefile to use go build
Ruby meets Go
● Buildmode “c-shared” and Cgo Basics● Using Go Function from Ruby
○ FFI and Fiddle without ruby.h● Writing Extension Library with Go
○ Define Functions Equivalent to C Macros○ Avoid Copy of Strings○ Propagate Reference from Ruby to Go○ Creating Gem including Go code
● Let’s Hack Go for Ruby together!