Dirty Dish Detector _ Make_ DIY Projects, How-Tos, Electronics,

7/25/2019 Dirty Dish Detector _ Make_ DIY Projects, How-Tos, Electronics,

1/61

OpenCVto automate messages to your

Nick Normal andjkridner Difficulty: Moderate

46 6 3

46

6

3


2/61

For more on

microcontrollers

and wearables,

check out Make:

Volume 43.

Dont have thisissue? Get it in

the Maker Shed.

Does your makerspace have a sink that is always full of dirty

dishes? Or do you yourself require better discipline around

the home to stay on top of your chores? To automate your

home, office, or workshop, one of the first things youll require

is some kind of vision system to detect motion or objects. A

small computer capable of running OpenCV could be just the

tool you need.

46

6

3


3/61

OpenCV and Python libraries, among others.

Configure the BeagleBone Black to automatically connect with your WiFi network.

3-D print and modify an enclosure for the BeagleBone Black to accommodate additionalelements (webcam, USB hub, etc.).

Install the Dirty Dish Detector above your kitchen sink.

Use the Cloud9 IDE to interface with your Detector, programming the project using

readymade Python scripts.

Get notifications whenever dishes pile up! Or likewise whenever the dishes get done!

This project assumes some familiarity with the command line, along with the ability

to administer your local area network. A comfort with coding in general (be it

Python, PHP, or even HTML) will get you very far with this project. All of thenecessary code is supplied below and documented, but opportunities for you to

customize the code to your liking will arise.

46

6

3


4/61

The projects code will also be assembled sequentially. Once the physical project is

built and installed above your sink, we will first take a camera-test photo using a

simple script. Once confirmed, we will calibrate an empty sink image. That will give

OpenCV a reference image upon which to compare against when it processes

images, looking for culprit cups and dirty dishes.

46

6

3


5/61

Once the system knows what to compare against a clean sink well implement

the ability to send an email and/or MMS when an unclean sink is detected. These

notifications will not happen all the time, but rather with every status change in thesink. Thus when the sink goes from clean to dirty you will get a notification; and

likewise when it goes from dirty to clean you will get a notification! Thus the Dirty

Dish Detector is primarily a detector, and it will notify you accordingly. Lastly,

automate the system to take a photo every 5 minutes. The webcams status LED will

alight whenever the script runs, like so:

46

6

3


6/61

Youll want to locate the Dirty Dish Detector at an adequate distance to cover as

much of your sink or sinks as possible. Since kitchen cabinet installations follow

somewhat of a standard, I found the immediate underside of my cabinets to be ideal

for the webcams field of view. With the lens 23 1/2from the base of the sink, I was

able to see an area approximately 16x 12. Experiment with the webcams lens

before you install your Detector if youre unsure exactly what it will see.

46

6

3


7/61

3-D Print Your Enclosure

Print an enclosure case for your BeagleBone Black while youre reading the steps

below in anticipation of assembling your Dirty Dish Detector. I used Logic Supplys

enclosure available on Github. Print the enclosure in your favorite color/s or

assemble your Detector with a different design altogether if you do leave a note

below in the comments because Id like to see your mod!

itself. The sink where I tested the Detector gets plenty of indirect sunlight. So the

interior lighting available. Even during the day, overcast days or sudden cloud

tweaking of the software to suit your environment is almost a given. In the final

file look for the #match circles with drains comment and the

tolerance variable; adjust the tolerance as needed. A lower number will increase the

likelihood of false-positives, whereas a higher number will let the system be more

tolerant with what it defines as a dirty dish.

WiFi caveat

During the making of this project, configuring WiFi on the BeagleBone Black was a

headache, and the most time-consuming aspect of the build. Even now, the wireless

adapter only powers on and connects with my wireless network once in every three

boot-ups. I cant figure out why; the 3.6A power adapter supplies more than enough

current. I tested operations with several Netgear adapters but eventually chose the

G54 for the low-profile of the adapter package.

TIP: Never plug a wireless adapter directly into the BeagleBone Blacks USB port,

not even during prototyping. It will likely conflict with signals from the boards PCB.

Always use a hub.

46

6

3


8/61

PARTS

Getting Started with BeagleBone Black kit Maker Shed #MSGSBBK2

microSD card RadioShack #44-183

Enercell 5V/3.6 Amp AC Adapter with USB RadioShack #273-437

Enercell Adaptaplug M (Black) RadioShack #273-466

4-Port Stackable USB Hub RadioShack #26-1038

Netgear G54/N150 Wireless USB Micro Adapter RadioShack #25-2966

Webcam, Logitech C270 RadioShack #2500148

Browser caveat

As with most projects that involve web browsers, theres a caveat. This project is

tested with Chrome and Safari (on Windows and Mac respectively). Other browsers

specifically IE will likely not work.

Code Copy+Paste

Cloud9 IDE, I have embedded the necessary code using gists. I recommend right-

link in new tab. This will give you the raw, un-formatted text to copy to the Cloud9

I am indebted to Jason Kridner for the bulk of the code powering this build, who in

turn would like to thank Tom from London Hackerspace for sharing his solution

from several years ago.

46

6

3


9/61

TOOLS

Computer with USB port

Hot glue gun & hot glue

Nippy cutter RadioShack #64-064

3-D Printer Afinia H480 3D Printer from RadioShack, #277-224

zip ties RadioShack #55064935Hardware Various, needed for mounting the project above your sink, depending on your

3. Print Your Dirty Dish Detector Enclosure

4. Drill a Hole in the Enclosure Lid

5. Drill Out a Hole on the Enclosure Lid

6. Mount the USB Hub to the Enclosure Lid

7. Mount the Webcam to your Enclosure

8. Mount the Detector Above your Sink

9. Say 'Hello World' with Cloud9!

10. Camera Test

11. Capture a Clean Image of Your Sink to Configure the Detector

12. Capture a Dirty Image of Your Sink and Compare it to the Clean Image

46

6

3


10/61

ADVERTISEMENT

tep #1: Update your BeagleBone Black

13. Provide some notification

14. Success!

15. Only Perform Notification on a Status Change

16. The Final Code

46

6

3


11/61

The first thing you will want to do is update your BeagleBone Black so it is running the latest

version of Debian.

All that means is we want to update the operating system running on the BeagleBone Black. I

have written easy-to-follow guides for users of both OSX and Windows. Follow those

instructions and then return here when you are ready to proceed with your Dirty Dish Detector!

The instructions below were all built on top of the "2015-05-14" image available at

http://beagleboard.org/latest-images. If a more-recent image is available, use it, and follow

the same instructions for installing Debian on the BeagleBone Black.

tep #2: Configure WiFi

46

6

3


12/61

46

6

3


13/61

46

6

3


14/61

If you want to configure your BeagleBone Black to connect to a wireless network, follow this

simple how-to guide to do exactly that.

Note: That guide is written to work with the Netgear G54/N150 'micro' USB adapter. It should

also work with other wireless adapters. Please leave a comment on that project with your

configuration to let others know what works.

tep #3: Print Your Dirty Dish Detector Enclosure

46

6

3


15/61

46

6

3


16/61

46

6

3


17/61

-D print this Logic Supply enclosure for the BeagleBone Black in your favorite color

r so it blends in with your kitchen cabinets. The enclosure comes in two parts: a

ottom (body) and a top (lid).

tep #4: Drill a Hole in the Enclosure Lid

46

6

3


18/61

46

6

3


19/61

46

6

3


20/61

46

6

3


21/61

our enclosure lid has a 'notch' on one end. This is to accommodate the ethernet

dapter on the BeagleBone Black. Place the lid down on a workbench so the arrow in

e top-left of the lid's top side is also oriented in the top-left. From that arrow's corner,

easure 1 3/4" from the left, and 1 1/16" from the top (image 1). Find the point at which

ose two measurements meet and make a mark -- see image 3 for clarity.

tep #5: Drill Out a Hole on the Enclosure Lid

46

6

3


22/61

46

6

3


23/61

46

6

3


24/61

Get a 3/4" spade bit and carefully drill out a hole, placing the bit's center point on your mark.

With the hole drilled out, use a pair of nippy cutters to remove any plastic burs from the hole.

tep #6: Mount the USB Hub to the Enclosure Lid

46

6

3


25/61

46

6

3


26/61

46

6

3


27/61

46

6

3


28/61

The USB hub has a riser on one side for stacking the hub. Snap that riser into the 3/4" hole

drilled out from the enclosure lid; insert the riser from the top side of the lid, so when you close

the lid on the enclosure the hub is on top. The riser is ever-so-wider than 3/4", so a little bit of

force is needed, but it will snap in place (image 1).

Now orient the hub so it is flush with the edge of the lid (image 2). Press and hold the hub still i

place, flip the lid over, and hot glue the hub into place (image 3). I recommend holding this for

few minutes to ensure the hot glue dries without accidentally shifting.

We want the hub flush with the enclosure's edge because later the webcam will mount perfectl

around the enclosure and hub - the fit is serendipitous!

tep #7: Mount the Webcam to your Enclosure

46

6

3


29/61

46

6

3


30/61

46

6

3


31/61

46

6

3


32/61

Unpack the C270 webcam and remove the plastic film from around the 'neck' of the webcam

mount.

Plug the USB hub into the BeagleBone Black's USB type A port.

Plug the webcam's USB cable into the hub.

Holding the enclosure as I do in image 1, you'll see the webcam's mounting unit fits neatly

around the thickness of the enclosure and the USB hub! That's why I mounted the hub to be

flush with the edge of the enclosure.

Holding the webcam in place, orient an extra-long zip-tie as I do in image 2. Loop the zip tie

around enclosure, strapping the webcam to the enclosure securely (image 3).

You're now ready to mount the Dirty Dish Detector above your sink!

tep #8: Mount the Detector Above your Sink

46

6

3


33/61

46

6

3


34/61

46

6

3


35/61

46

6

3


36/61

How you mount the Dirty Dish Detector above your sink will be up to you. Some craftiness will

be necessary, depending on if you have cabinets above your sink or not, how many sink drains

you need to detect, what material your cabinets are made of, etc.

I used the zip tie as a strap to attach to some eye bolts mounted to the underside of my

cabinets (made from some composite material). I measured the distance either side of the

webcam mount (image 1), placed the webcam to the underside of the cabinet, and marked the

holes to drill. I accounted for a third eye bolt in the back.

In my case, I used two smaller (orange, see image 3) zip ties and secured the enclosure's zip ti

to the eye bolts. I threaded the velcro strap that comes on the webcam's wire through the eye

bolt in the rear, securing all the loose wires this way (image 3).

The webcam lens can swivel back and forth, while the housing unit is secured to the underside

of my cabinets.

46

6

3


37/61

tep #9: Say 'Hello World' with Cloud9!

46

6

3


38/61

46

6

3


39/61

46

6

3


40/61

Here we go. We're about to start interacting with the BeagleBone Black over our LAN, and

programming the board.

Now that your BeagleBone Black automatically connects to your wireless network with a boun

IP (in my case 192.168.1.108), open your web browser and enter the board's IP in the address

bar. Hit Enter. When connected, you'll see a green box at the top and 'Your board is connected

Great!

Scroll down and click the 'Cloud9 IDE' link. This will open up Cloud 9 running on the

BeagleBone Black. It consists of multiple tools you're probably already familiar with: a file

explorer, command line interface, and file editor, among others.

Let's get the BeagleBone Black to acknowledge its own existence. Click File -> New File to

create a new file. Type

#!/usr/bin/python

and hit Enter twice. Then type

print "Hello World!";

Click File -> Save As... and name the file 'helloworld.py' and save it to the main 'cloud9' folder.

Now click the 'Run' button. You'll see the script run in the lower pane, and it will print "Hello

World!" Success!

46

6

3


41/61

tep #10: Camera Test

46

6

3


42/61

46

6

3


43/61

46

6

3


44/61

Click File -> New Folderand create a sub-folder called "Dish-Detector" (image 1). Ensure there i

a hyphen between the two words, because the code we'll be running later has that hyphen in

the folder name.

Click File -> New File and create a new file in the Dish-Detector folder called camera-test.py;

copy the following code to that file, and save it.

https://gist.github.com/nicknormal/347c18d0b27d45e33f73

With the code copied to camera-test.py, click Run and the script will run in the lower pane. You

will see the following error printed many times: "VIDIOC_QUERYMENU: Invalid argument" -

dismiss this error (image 2). It does not effect the operation of the Dirty Dish Detector.

You will now see 'camera-test.jpg' in the Dish-Detector folder. Click it, and you'll see a picture o

your sink taken with the webcam! Yay!

tep #11: Capture a Clean Image of Your Sink to Confighe Detector

46

6

3


45/61

Note: Youll need to do some playing around here to get reliable detection of your drain(s).

Copy the entirety of the following code to a New File called sink-empty.pyand then Run it.

https://gist.github.com/nicknormal/b46775673ee654f4a262.js

You can see this code creates a JPEG titled 'sink-empty.jpg' and a text file called 'sink-empty.tx

that is like a configuration file for your 'empty' sink. If you have one sink drain the file will read

something like

790,382,64

whereas if you have two drains it will read something like

200,170,15 192,76,6

Compare it to the sink-empty.jpg file to understand how it orients according to X-Y coordinates

and circumference.

Be sure to keep this script around in case you need to run it again to reset how your sink looks

when empty or if the camera gets moved.

46

6

3


46/61

tep #12: Capture a Dirty Image of Your Sink and Compto the Clean Image

Make a meal (or in my case, coffee!) and when you're done put your dirty dishes in the sink. Do

not clean them!

Copy the following code to a New File and name it sink-latest.pyand then Run it.

https://gist.github.com/nicknormal/fc32f22c35f46c76044b.js"

tep #13: Provide some notification

46

6

3


47/61

46

6

3


48/61

46

6

3


49/61

46

6

3


50/61

Using a Google mail account or some other SMTP provider, you can send yourself an email

and/or text message. Once again copy the following code to a New File called sink-phone-

home.pyand save it. https://gist.github.com/nicknormal/760a20ba8f05b6c00539.js

You'll notice at the bottom of the code there are variables for your "[email protected]" username

and password, along with a phone MMS service option. If you change these to your Google

account details and Run the script, you will get a SMTPAuthenticationError because you haven'

associated this 'app' with your Gmail account's security clearance.

The error will prompt you to visit https://support.google.com/accounts/answer/185833 which

in turn will direct you to the App passwords for your Google Account.

Once there, click the drop-down menu for Select app and click Other (custom name). Type

"Dish-Detector" in the field box and click the "Generate" button (image 2). The service will

prompt you with "Your app password for your device" - this password will allow SMTP

authentication using your Gmail account for the 'app' "Dish-Detector."

Transfer the app password into the "password" variable in the code above, which again is now

called sink-phone-home.pyin your Cloud9 IDE. Save the file.

Now run it.

46

6

3


51/61

tep #14: Success!

ou will get an email sent from yourself, to yourself, without a Subject, but containing a

oname.jpg" snapshot of dirty dishes in your sink.

tep #15: Only Perform Notification on a Status Change

46

6

3


52/61

kay extend the code slightly so it will only send a notification upon a status change,

om clean to dirtyper the status.txt file in your Dish-Detector directory. Again copy the

ode below to a New File and name it sink-phone-home_change.pyand Run it.

ttps://gist.github.com/nicknormal/d64cb459bdfe7a8824f4.js

tep #16: The Final Code

46

6

3


53/61

46

6

3


54/61

46

6

3


55/61

And finally the complete code, with comments. Copy this code to a New File and name it "Dirty

Dish-Detective.py" and save it. https://gist.github.com/nicknormal/2a02718590992cd5f7af.js

When you Run it the prompt will return the location of "circular features" and also "drain not

found at" errors. This will inform the Detector whether the sink is dirty ("True") or not ("False").

Once confirmed that clean/dirty status changes send the appropriate email notifications, empty

your sink of dirty dishes and then proceed to automate the system.

tep #17: Implement Cron to Take a Picture Every 5inutes

46

6

3


56/61

46

6

3


57/61

46

6

3


58/61

One simple but lengthy command line is all that is needed to automate the Dirty Dish Detector'

primary Python script. Launch your preferred SSH client - or use the Cloud9 IDE bash! - and

connect with the BeagleBone Black as root.

The following command will initiate Cron, which can easily be thought of as a scheduler for

initiating repetitive tasks (in this case taking a photo and analyzing the image for dirty dishes).

These tasks are called crontabs.

Type

echo "*/5 6,7,8,9,10,11,12,13,14,15,16,17,18,19,20 * * * python /var/lib/cloud9/i!

hetector/irtyi!hetective#py" $ crontab

and hit Enter.

To confirm the cron is running, type

crontab l

and you will see the code you previously typed printed back.

Dirty Dish Detector is now automating the detection of dirty and clean kitchen sinks!

Crontab TIP: Admin's Choice has a simple breakdown on how crontabs are interpreted; and I

recommend plugging some data into this crontab generator to understand how crontab string

are generated and what each part of them mean.

46

6

3


59/61

ONCLUSIONs noted in the Introduction above, the general failure of the Detector to detect in low-light is ansue. Should anyone come along in the middle of the night for instance, and deposit dirty dishes i

doesn't have to be bright, but it does have to be direct, and the Detector should do the rest.

his form factor is pretty straightforward, using the parts of the project and simply strapping them

t? If you come up with any ideas for extending the software or inventing solutions to specific

lso read through and would like to thank the following blogs and websites for their advice and

nux-inspiration: the Linux Wireless Wiki, WikiDevi's documentation of the WNA1000M wireles

apter,AskUbuntu forums,Ayoub Zaki's Embedded Systems Blog, BeagleBone Notes, DLF,

d a special shout-out to Derek Molloy for his really rich BeagleBone Black documentation,

hich while I didn't use any of his steps to compile this project I really appreciate his thoroughnes

NICK NORMAL

I'm an artist & maker. A lifelong biblioholic, and advocate for all-things geekathon.

Home is Long Island City, Queens, which I consider the greatest place on Earth.

5-year former Resident of Flux Factory, co-organizer for World Maker Faire (NYC),

and blogger all over the net. Howdy!

[email protected]

46

6

3


60/61

nicknormal.com

JKRIDNERCo-founder of BeagleBoard.org

Jason Kridner is the co-founder of the BeagleBoard.org

Foundation, a US-based 501(c) non-profit corporation

existing to provide education in and promotion of the

design and use of open-source software and hardware in

BeagleBoard.org // Beagle Blogger

46

6

3


61/61

3 Comments 1

simin

Until now I did not notice these things. I'm glad you've mentioned. I

understand what you mean. I must thank you for this article

!"#$%&"(()*+,

Logic Supply

-reat ro/ect 0ake12 3e've em4edded the video on our download age for the 56

rinta4le case design " htt177insire.logicsuly.com...

Guest

-reat ro/ect 0ake12 3e went ahead and em4edded the video on our age for the

download age for 56 case design " htt177insire.logicsuly.com...

89

9

5

Dirty Dish Detector _ Make_ DIY Projects, How-Tos, Electronics,

Documents

Transcript of Dirty Dish Detector _ Make_ DIY Projects, How-Tos, Electronics,