Innovate – Tweak, Hack, and Bend Technology

Wolf Paulus

Wolf Paulus . com

Innovate: Tweak, Hack, and Bend Technology

© 2006 Wolf Paulus 2

Renaissance The period of European history at the close of the Middle Ages and the rise of the modern world; a cultural rebirth from the 14th through the middle of the 17th centuries.

Ren • ais • sance


Glass Gave Renaissance Paintings Their Glow

By looking beyond the limits of their usual practice and transforming materials from other trades, the great artists of the Renaissance created a palette that gave them an immediate and lasting reputation as brilliant colorists.


• To tweak, hack, and bend technology you need to know some of the Technology behind the Technology

• Look beyond the limits of your usual practice and transform materials and ideas from others.

Innovate -Tweak, Hack, and Bend Technology.


Hacking Nick Click, a digital toy camera.


Serial Port :

TXGroundRX

Power:9V Battery

Power

Shutter


9V Power

ShutterOn/Off

Serial Port :

TXGround

RX


Microcontroller vs. Microprocessor

• Microcontroller chips and most embedded devices embody the Harvard architecture, which has separate memories for programs and data.

• Microprocessor based machines usually have a von Neumann architecture, with a single memory for both.


Image Sensor, Key Features:

• Effective Resolution 160 x 120

• Color filters form a Bayer Color Pattern

• “Software is required to perform color processing.”

[Source: 50 pages ST VV5301 & VV6301 Mono and Color QSIF Digital Video CMOS Image Sensors]


Filter Array Sensor

• Only one color can be acquired per pixel.

• Most commonly used is the Bayer Pattern because of the eye’s higher sensitivity to green, 50% of the Bayer pattern consists of green information.


Decoding the Bayer Pattern


Shuffled Pixel Readout ..

Crosstalk: Parasitic capacitance between tracks can cause the fast edges of high-speed logic to propagate from one signal line to another.


A1 B1 C1 D1

E1 F1 G1 H1

A2 B2 C2 D2

E2 F2 G2 H2

8- 1x8 WDM

Silica-on-Silicon waveguides

A3 B3 C3 D3

E3 F3 G1 H3

A4 B4 C4 D4

E4 F4 G4 H4

A1

B1

C1 D3

E1 F1

G1

A2

B2

C2 D2

E2 F2

G2 H4

A3 B4

C3

E3 F3

G4 H1

A4 B3

C4 D1

E4 F4

G3

H3

D4

H2

Folded 128 x 128 Shuffle circuit

A 1

B 1

C 1

D 1

E 1

F 1

G 1

H 1

A 2

H 2

G 2

F 2

E 2

D 2

C 2

B 2

A 3

B 3

C 3

D 3

E 3

F 3

G 1

H 3

A 4

H 4

G 4

F 4

E 4

D 4

C 4

B 4

LiNbO3

64- 1x4switch array

A 1

B 1

C 1

D 1

E 1

F 1

G 1

H 1

A 2

H 2

G 2

F 2

E 2

D 2

C 2

B 2

A 3

B 3

C 3

D 3

E 3

F 3

G 1

H 3

A 4

H 4

G 4

F 4

E 4

D 4

C 4

B 4

Inputs

Outputs

.. leads to shorter Tracks


Artifacts due to color interpolation


The red, green, and blue photodetectors are located at different depths within the sensor.

Layered Image Sensor


For every pixel, three photodiodes are stacked like the three layers in color film. This arrangement utilizes the wavelength-dependent light absorption property of silicon to produce natural filters that use the incoming light to greatest advantage.

Layered Image Sensor

The red, green, and blue photodetectors are located at different depths within the sensor.


Filter Array vs. Layered Sensor


Communication Protocol

STX Command Data 1 ... Data n ETX

ACK STX Response Data 1 ... Data n ETX

• STX = ASCII character (02h)• Command = defined data byte• Response = defined response byte• Data = command specific parameter data• ETX = ASCII character (03h)• ACK = Acknowledge (06h)• NAK = (15h)


Example 1: Request Firmware Version

02h V 0 03h

06h 02h v 1 . 6 9 03h

Send Request via RS232 port at 57.6 kbps / 8bits / No Parity

Expected Response:


Example 2: Take a Picture

02h Cmd param 03h

06h 02h cmd error code response .. 03h

• Reset Image Counter to 0❷,A,0,❸ ⇒ ⇐ ❻,❷,a,0,❸

• Grab Image (Parameter defines delay and retries)❷,G,127,❸ ⇒ ⇐ ❻,❷,g,0,❸

• Upload Image (Parameter defines image index)❷,U,0,❸ ⇒ ⇐ ❻,❷,u,N1,..,N4, D1,..,Dn,❸


Java Code to drive the DigiCam

• cmd - array containing the command code, e.g.STX, CommandByte, ParameterByte(s), ETX

• size - number of data bytes expected on inputstream,excluding 4 bytes for ACK STX cmd ETX

• return - response bytes or null, if unsuccessful, e.g.ACK, STX, commandByte, DataBytes, ETX

public byte[] perform(byte[] cmd, int size);

© 2006 Wolf Paulus

private byte[] perform(final byte[] cmd, final int size) { boolean ok = false; boolean busy = false; int counter = 0; byte[] buffer = new byte[size + 4]; // data + ACK,STX,RESPONSE,ETX

try { // Flush the input stream while (in!=null && in.available() > 0) in.read(); } catch (IOException e) { // intented empty }

do { try { out.write(cmd); // Send Command out.flush(); busy = false;

int off = 0; // the number of bytes already read. int len = buffer.length; // the number of bytes still to read.

© 2006 Wolf Paulus

while (len > 0) { // Reads up to len-off bytes from the input stream into buffer. int k = in.read(buffer, off, len); if (k <= 0) // k is the number of bytes actually read. throw new IOException("Couldn't finish reading input file."); off += k; len -= k; }

if (buffer[0] != VV6301.ACK) // Expected: ACK,STX,cmd,..,ETX return null; ok = buffer[2] == (byte) (cmd[1] + RESPONSE_OFFEST); if (!ok) busy = buffer[2] == VV6301.RB_BUSY; } catch (IOException ioe) { System.err.println(ioe.getMessage()); } } while (!ok && busy && (++counter < VV6301.RETRIES)); return ok ? buffer : null; }


Serial Port, still a “must have”

• Keyspan High Speed USB-Serial Adapterhttp://www.keyspan.com/products/usb/USA19HS/

• Java communications libraries for OS Xhttp://prdownloads.sourceforge.net/jmri/JavaCommInstaller2.hqx

• Installation instructions for Mac OS Xhttp://jmri.sourceforge.net/install /MacOSX.html


Dallas Semiconductor: TINI board 390


TINI- Hardware

Ethernet Controller

512KFlash ROM

Two 512K SRAM

36 MHzCPU

Serial-IOController

384K Java Runtime 64K Shell

File Sys.Java Prog.Res.Files


• 8 Processes (incl. GC and Shell)

• 32 Threads per Process

• 24 Socket Connections

• A compiled class file cannot be larger than 32k

• 64k is the maximum size of any array

TINI’s Java Limitations


• A class is limited to 255 static and 255 instance member variables. (includes inherited members)

• A class method is limited to 63 local variables.

• Only UTF8 and ISO 8859_1 are supportedwith ByteToChar and CharToByte

• Only the default Locale is supported.

• There are a few more limitations that concern Classloading and Reflection.

more Java Limitations on the TINI


TINIConverter is a Java Program

Command line tool which outputs a *.tini file

StringBuffer sb = new StringBuffer(); sb.append(“this is a test”);

Create new object “java.lang.StringBuffer” Push reference to “this is a test” on the stackInvoke method “java.lang.StringBuffer.append(String)”

Create new object with class number 0x54 Push reference to “this is a test” on the stack Invoke method 0x12 of class 0x54

Reduces the constant pool and speeds up class loading

TINIConverterJa

vaP

seud

o B

yte

Cod

eTI

NI

File


Socket for the TINI board 390Null ModemSerial Port

5V Power10

-Bas

e-T

Eth

erne

t (+)

LEDR 330 Ω

(-)

zoom in on next slide


GREEN LED

330 Ω

VC

C =

5V

DC

Port 5

Bit 3 Por

t 5Bit

0

RED LED

330 Ω


Ohm’s law: U/R=I

3V / 330Ω = 9 mA

Tweaking Processor Port Pins

BitPort led = new BitPort(BitPort.Port5Bit0);led.clear(); // turns it onled.set(); // turns it off

• Driving an LED needs around 10 mA to make it clearly visible.

• A Port Pin can supply about 50 μA but can sink current up to 15 mA.

• LEDs have a voltage drop of about 2V. Therefore, the voltage across the Resistor is about 3V.


TINI in action ... (typical setup)

CAT5 Cable

TINI 390

Serial-Cable (3-wire)

Router

Nick Click


TINI in action ... (development setup)

CAT5 Cross Over Cable

5 VDC Power

Serial Cable TINI-390Nick Click

PowerBook

© 2006 Wolf Paulus

Demo

© 2006 Wolf Paulus

Demo

37

DS80C390: 192.168.200.6DS80C390: 192.168.200.5DS80C400: 192.168.200.4

Compact Wireless-G Broadband Router - WRT54GC Gateway 192.168.200.1 / 255.255.255.0


TiniNick -- Camera Driver & Image Server

Grabs Photos from Nick Click

Serial Port /Camera Driver Module

Access Properties Files

accepts client requests, adds new client sockets into Vector

streams new images to all Sockets in Vector


NickViewer -- Image Viewer Client

Converts 20KB bayer-encoded byte array in 60KB RGB-Bitmap.

Opens TCP Socket to TickNick Image Server.Updates Observer when new image arrives.

Builds Viewer GUI and observes SocketConnector for new images.


Shuffled Pixel Readout


Before Bayer Decoding


After Bayer Decoding


Image Resolutions Compared ...

Nick Click 160 x 120 1

Motorola V551 640 x 480 16

Canon PowerShot SD200

2048 x 1536 164

Canon EOS 350D / Digital Rebel XT

3456 x 2304 414


•Hacking a $10 Digital Toy Camera.

•Tweaking Micro-Controller Ports

Hack, Tweak, and Bend Technology

© 2006 Wolf Paulus

Mont Ventoux (1,909 m; 6,263 ft) is by far the biggest mountain in the Provence, a region of southern France.

On April 26, 1336, Francesco Petrarca climbed to the top of Mont Ventoux just to see the view, which at the time was rather unusual.

This event is often cited as the start of the Renaissance.

50


Resources• Textbooks

• Don Loomis - TINI Specification and Developer's Guide - ISBN: 0201722186PDF version online: http://www.maxim-ic.com/products/tini/devguide.cfm

• Ken Arnold - Embedded Controller Hardware Design - ISBN: 1878707523• DeMuth, Eisenreich, Designing Embedded Internet Devices, ISBN: 1-878707-98-1

• Image Processing / Bayer Pattern• Tommy Olsen, Jo Steinar Strand, Agder University College Faculty of Engineering:

"An improved image processing chain for mobile terminals", May 2002http://student.grm.hia.no/master/ikt02/ikt6400/g23/Report.htm

• Diagnostic Instruments, Inc.(http://www.diaginc.com)Elements of Sensor Resolutionhttp://www.diaginc.com/downloads/public/pdf/3ShotColorMosaic.pdf

• Data sheets and Drivers• CPiA webcam driver for Linux (http://webcam.sourceforge.net)

RS232 command protocol specs (scicomms.doc, 10 pages, 82Kb).http://webcam.sourceforge.net/barbie/scicomms.doc

• VV6301 Mono and Colour QSIF Digital Video CMOS Image Sensorshttp://www.datasheetarchive.com/search.php?search=vv6301&sType=part

• Tini DS80C390 and Tini DS80C400http://www.maxim-ic.com/pl_list.cfm/filter/tini

• TINI, DS80C400 Evaluation Kit• http://www.maxim-ic.com/getds.cfm?qv_pk=4983• http://www.maxim-ic.com/TINIplatform.cfm

http://www.diaginc.com

http://www.diaginc.com

http://www.maxim-ic.com/pl_list.cfm/filter/tini

http://www.maxim-ic.com/pl_list.cfm/filter/tini

TiffanyScreens


Tiffany Screens

The term "democracy" was coined in ancient Athens in the 5th century BC. All Athenian citizens were free to speak in the Assembly.

“PowerPoint-ation”

• Lecture Style Presentation

• One guy presents everybody else is asleep

• Changing Presenters severely interrupts a meeting, requiring change of location, unplugging, etc.

• Presenter need to adjust Laptop’s screen resolution and refresh-rate.

• Since a picture is worth a 1000 words, it’s hard to win an argument against the presenter, even if you have your facts right.

0

25

50

75

100

20042005

20062007

TiffanyScreens

• Detects who is running the same application. aka your Peer Group

• One peer at a time broadcasts her screen onto other watching peers’ screens

• Seamlessly changing broadcasters

http://TiffanyScreens.com/tiffany.jnlp

http://TiffanyScreens.com/tiffany.jnlp

Group Dictionary

Toolbar

• Show your screen to your peers - exit [ESC]

• Watch the presenter’s screen - exit [ESC]

• Pick your Conference room

• Look who’s currently where, doing what

• Change your network settings etc. - if you really think you have to -

Statusbar

• Status LED• blue = idle• green = ready to receive• orange = receiving

• heartbeat - sharing your apps attributes• Confidentiality features• Number of peers, running this app.• Preferred watching window resolution


TiffanyScreens.com


Thanks for coming

Innovate – Tweak, Hack, and Bend Technology

Technology

Transcript of Innovate – Tweak, Hack, and Bend Technology