Let's Make Robots! | RobotShop

Ping Pong Ball Juggling Robot

It Juggles a Ping Pong Ball with a Racket

This entry is about the quest to get a machine to juggle a ping pong ball. 

I started thinking about a ping pong ball juggler 2 years ago. And it was around that time that I built the first one of them. it uses LED - Photo transistor pairs to "see" when and approximately where the ball comes down. I used 4 micro servos to move a wooden plate.  It's the machine in the first YouTube video.

The next major update came when I decided it might be interesting to try to track the balls position by using the noise the ball makes when it hits the plate. I installed 4 mics and switched a FlipFlop on as soon as the sound level got over some threshold. The FlipFlop's value then gets read by a micro controller as fast as possible to then compute the time difference between the sound waves hitting the mic on the left in comparison to the mic on the right. The same goes for the front and back mics. This way 2 dimensional data about the ball position is gathered. It worked quite well. The machine on this stage is the one in the second YouTube video.

After that I did other things for a long time. But some months ago I wanted to rebuild the machine from scratch. This entry is about this rebuilt version. The 2 videos on the top are just meant for a quick history on how it started. So, what changed? I no longer use servos but Nema17 stepper motors for the movements. I bought some belt drive components and ball bearings. I used those to convert the axial motion of the steppers into linear motion. It's the same setup as in an inkjet printer. 

The main controller of this project is a Raspberry Pi 2. It has a Raspberry Pi camera module NoIR connected to it. Because it is NoIR, there's no IR filter on the thing. I originally planned to use infrared light to brighten up the ball in front of the camera. But it didn't work well. The IR elements I used were way to weak. The infrared noise in the background (sunlight etc.) would overweight the infrared light getting reflected from the ball. Another problem was, that the IR signal gets picked up on the red-channel of the camera. This means that even if there is no infrared noise from sunlight, the infrared light's intensity still has to surpass the normal red light's intensity to be recognizable. So that idea got dropped. I ended up using the green-channel on the camera with green light from a RGB Power LED pointed at the ping pong ball. Sure enough, normal white light would've done the job as well, but I just happened to have a RGB LED laying around so I ended up with green.

Another important thing is this: You might be asking yourself "Why did you use a light source anyway? Couldn't you just use OpenCV and get the thing to recognize the white ball with normal lightning conditions?". The answer to this is: Probably not. At least not with a Raspberry Pi 2. The reason being that we need 90 fps real time image processing to track that ball. The image processing has to be done fast. I didn't really use OpenCV for anything at this Point in time but my guess is, that it wouldn't work well, because I don't think I could get the processing rate up to 90 fps with OpenCV. Maybe I am wrong about this, but it was my gut feeling, so I went on to program a really simple algorithm that could take care of the image data really fast. The algorithm looks directly at the data stream, so no conversion to a different datatype (e.g. NumPy) is needed. This is one reason this algorithm is quite fast. The other reason is, of course, that it is a dumb algorithm. All it does is look for a clump of bright green pixels on a fairly black background and then compute the center of it.

After the algorithm extracted 3D coordinates out of the images it examines that data and, in the case the ball is going downwards and gets quite near to the racket, it uses that specific data set in a PID controller to get some useful correction values on the racket as the ball is moving towards it (the machine tries to get the ball into the center of the racket). The movement values are then sent to the Arduino Uno over the serial port. The Arduino Uno runs grbl (a program for running stepper motors very efficiently) and all it does most of the time is waiting for some new instructions from it's master; The Raspberry Pi.

Some lines about the tilting mechanism: The tilt in the racket is produced by different heights of the 3 belts on which the racket is mounted. The joint "belt to racket" is a rather elastic one. This is why a tilted racket can be accomplished without ruining the joints or other parts of the machine. 

It's worth mentioning that the thing isn't working perfectly at all. A 2 hour juggling session is definite not one of the things this machine is capable of doing. It is able to juggle the ball for about 2 mins. Then it makes some misjudgment and the ball drops. So it isn't great. But then again: It only uses one single camera to compute 3D coordinates of a flying object. using 2 cameras would improve the performance. But it would also make the thing more complicated.

Here are some pics of the machine:

The build of the Juggling Robot

Juggling Robot with all of it's components

 

Raspberry Pi camera module plus RBG Power LED

Raspberry Pi camera module plus RBG Power LED

 

Camera module view from back

Camera module view from back

 

The ball bearings mounted on the top

The ball bearings mounted on the top

 

The connection racket - belt drive

The connection racket - belt drive

 

The base built

The base build

 

Nema17 Stepper Motor

Nema17 Stepper Motor

 

And here some gifs made from image data of the Raspberry Pi camera module:

This gif was made to check if the ball in the image data looks crisp enough

This gif was made in the very early stages of this project. It was made to check if the ball in the image data looks crisp enough to allow an image processing algorithm to extract the ball position from it.

 

One ball Juggling

Here is some image data of actual ball juggling. The red light was put there to make the racket movements visible. The grid represents points at which the algorithm checks for pixel brigthness. If the value on a gridpoint is bright enough the algorithm computes the center of the clump of bright pixels at that specific position (this data is used as x and y coordinates). The z coordinate is extracted from the width of the diameter of the ball.

Here's the Raspberry Pi 2 Python code:
https://github.com/T-Kuhn/PingPongPi

And Here's the superb grbl code (the thing running on the Arduino Uno):
https://github.com/gnea/grbl

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

30.12.2016

What an interesting and well written article...excellent wok, indeed!

One thing: https://github.com/grbl/grbl   give a 404 Error.

Oh, Thank you Bonzadog.

I corrected that link!

So does the racket pivot to try to keep the ball centered? Is there just enough play in the belts to allow each belt drive to be at a different height?

Yes it tries to keep the ball centered (I really should have mentioned that somewhere... Will put it in there!). And yes, just as you suspected, there is enough play in the belt itself and in the joint "belt to racket". So, different heights result in a tilted racket. See picture "The connection racket - belt drive". The connection is actually made by using leftovers from the belt. Because the material seemed to have just about the right amount of elasticity. :)

Cool project. Here's a similar blind implementation: (http://www.idsc.ethz.ch/research-dandrea/research-projects/juggling-machines.html)

Hi ShuAux,
I read about that one. Thanks for the link!

Very impressive work and a great read.
Perhaps if you plan to expand on it, you could record the data and train a neural net?

Might be able to improve on your 2 mintes eventually (not that 2 minutes isn't impressive enough as it is!)

Thanks for the comment!
That would be sweet. Never done neural nets. Want to learn more about them. Is there a neural net library you can recommend for python?

Sure, i'd recommend using Keras, lots of tutorials online for it.
It uses tensorflow to do all of the hard work, but makes it really easy to get stuff up and running:

You can see docs for keras here:

https://keras.io/

You can find lots of great stuff for neural nets, deep learning and installing/working with keras here:

http://www.pyimagesearch.com/category/deep-learning-2/ 

 

Hope it helps, looking forward to what you come up with next!

Thank you very much!