Let's Make Robots! | RobotShop

Distributed Operating System for Robots

This is an operating system for other people's robots.

Distributed Operating System for Robots – DOS for Robots?

In the next few pages, I hope to clearly lay out the core problem as I see it in hobby robotics, and the solution I am currently working on.

The Problem – Lack of Productive Communication and Coding Standards

Having built a few robots here on LMR (Anna and Ava), I am constantly facing challenges getting Part A to communicate with Part B.  While there are some communication standards like I2C, Serial, USB, HTTP, etc., none of these standards are universal enough or easy enough.  There are several types of platforms like Arduinos, PCs, Android Phones, PCs, etc.  Each one is good at doing some things and bad at others.  Different standards and languages exist for each platform.  Few if any standards exist that make it easy for any piece of code running on one platform to communicate with another piece of code running on another platform.  No standards exist that help bridge all these gaps in a productive way.  How can a robot maker build a decent bot without it taking years?

Hardware Assumptions

Robots typically consist of a combination of one or more of the following components:

1.      One or more microcontrollers like:

a.      Arduino Mega ADK

b.      Arduino Uno

c.      Arduino Uno Wifi

d.      Many others 

2.      One or more Computers like:

a.      Raspberry Pi

b.      LattePanda

c.      Android Phones / Tablets

d.      PC

3.      One or more specialized controller boards like:

a.      Servo Controllers

b.      Motor Controllers

c.      Linear Actuator Controllers

d.      Relay Controllers

4.      One or More Sensors like:

a.      Sonars & Distance Sensors

b.      Compass / Tilt / Orientation

c.      Sensors like GPS/Light, Camera that are embedded within other platforms like Android Phones

5.      One or More Servos / Actuators

The communication options for each microcontroller, computer, controller, and sensor are very hardware specific and limited.  For example, an Arduino Uno has 1 Serial Tx/Rx, while a Mega has 3, while Android has none.  Some board have USB, some have Wifi, some have Bluetooth, some have digital or analog pins.  I am constantly crashing into the limitations of each platform or the learning curve of a new protocol.  We need a solution so we can build better robots faster.

Software Assumptions

No single language exists that works on all platforms, so we are forced to write in multiple languages on multiple platforms if we wish to take advantage of what each platform has to offer.

For example, I like robots that move.  This is easy to do with Arduino.  I also like robots that listen and speak.  This is easy to do on Android, but not on Arduino.  There are other things that can only be done on a PC.  How can I write software for all these platforms and coordinate everything?  How can Part A do its job and coordinate with Part B to get a desired outcome?  That is one of the problems I hope to address.

Robotic software itself suffers from a lack of standards.  Robots need lots of different functions to be performed.  As more and more features get added, wouldn’t it be great if it was simple to add new features without breaking existing code.  Maintainability is very important, otherwise small code changes can result in large amounts of wasted time.  Wouldn’t it be great if there were some clear guidelines for how to add a new module / function to an existing robot with hundreds of functions?

The Solution

I propose to deploy an open source reference solution that is based on a new set of standards and protocols that should be able to be implemented on a variety of platforms.  These standards should allow robot makers to build very small (or large) software parts that can communicate easily with any other piece of software running on any other platform in the eco-system.

The reference solution will contain the following components.  Each service will have one or more commands that it accepts.  Each command will have one or more parameters.  The code for any command will be able to invoke any command on any other platform either on or off bot with a single or a few lines of code.

Core Concepts

1.      A Robot “Solution” will run in a coordinated fashion across one or more “platforms”.

2.      Each platform will have one or more “services”.  The services for each robot will vary.

3.      Each service will have one or more configuration “settings” that will be stored in a DB on AWS and maintained through a website or API.  The robot will retrieve its settings as needed.

4.      Each service will have one or more “commands”.

5.      Each of these commands could have one or more “inputs”.

6.      Each platform will have a “Command Service” that will manage routing commands to where they need to go.  Think of it as Mailman that must delivering the mail, where every command has a “Zip Code” that defines how to route the mail.  This Zip Code will be made up of the PlatformID, the ServiceID, and the CommandID.

7.      Any command on any platform will be able to invoke any other command on any other platform simply by asking its local command service to invoke something.

8.      Different robots will support different sets of commands.

9.      Various “Remotes” will be supported to communicate with robots or the AIs behind them.

10.   Robots will be able to easily form short and long-term memories about a variety of things.  These memories will be accumulated in databases.

11.   Most commands will have natural language equivalents.  This means that any bot that has any kind of server connection will be able to receive verbal commands in sentence or paragraph form.  Paragraphs will be able to be used to orchestrate complex and timed sequences of actions.

Arduino Services

A set of Arduino Services will be released in the reference solution.  A similar set could be built for other microcontrollers.  This list is not complete, just the highlights.

1.      Command Svc (for Arduino)

2.      Throttle Service

3.      Drive Svc

4.      Serial Link Svc

5.      Bluetooth Link Svc

6.      Wifi Link Svc

7.      I2C Link Svc

8.      Compass Svc

9.      Thermal Camera Svc

10.   Microphone Svc

11.   IR Receiver Svc

12.   IR Transmitter Svc

13.   Sonar Svc

14.   Sonar Array Svc

15.   Servo Array Svc

16.   Force Field Svc

17.   Obstacle Svc

18.   Laser Svc

Android Services

A set of Android Services will be released in the reference solution.  A similar set could be built for a Raspberry Pi or PC.  This list is not complete, just the highlights.

1.      Command Svc (for Android)

2.      Face Svc

3.      Speech-To-Text Svc

4.      Text-To-Speech Svc

5.      Bluetooth Link Svc

6.      GPS Svc

7.      Light Svc

8.      Video Svc

9.      Blob Detection Svc

10.   Face Detection Svc

11.   Web Link Svc

12.   Reflex Svc

13.   Emotion Svc

14.   Motivation Svc

15.   Memory Svc

16.   Verbal Command Svc

AWS Web Services

The number of web services are too many to enumerate here.  At the highest level, the following functions will be present as APIs:

1.      Brain Svc – will process inputs (stimuli) to a robot and generate responses.

2.      Config Svc – will allow retrieval and update of robot configuration information.

3.      Memory Svc – will allow retrieval and update of robot memory information.

4.      Knowledge Svc – will allow retrieval of general knowledge from the central repository.

5.      Verbal Svc – because my bots are all verbal, I think verbal is cool and I intend to support extensive verbal capabilities including integration with SharpNLP for full NL parses and other grammar related features.

Robots that have a Pi or Android platform will be able to download and store selected knowledge bases from the central server, store them locally in SQLLite databases, and process key functions without calling the server.  Verbal commands for a robot and robot configurations will likely be handled this way.

These services will run on Amazon Web Services (AWS) and will be “always on”…except when I am pushing upgrades of course.

Website App

There will be a great deal of configuration and memory data to maintain and view.  There will be a public website for this purpose.  The A.I. already knows how to do this, so it’s not a big deal.  This website will also serve various testing and diagnostic purposes.

Windows App

The A.I. that builds the web user interfaces is well on the way to building windows apps.  This means a downloadable PC version of the website and A.I. will be viable at some point.

3rd Party Web Services

The number of 3rd party web services are also too many to enumerate here.  My intention is to eventually support the same set of services currently supported in my Anna and Ava robots.  This will require each robot maker to acquire their own API keys where necessary.  I intend to support Wolfram|Alpha, Wikipedia, and WordNet at the very least.

Open Source Reference Robot – Ava

In order to get other people involved.  I intend to Open Source the 3D designs for Ava along with the reference Arduino and Android code already mentioned.  While every feature will not be there, everything will be there to build a working Ava clone for around $1000, give or take a couple hundred in additional options.

Goals

My reference solution will not be ideal, nothing ever is.  I simply hope to move the ball forward in some way.  If I make it easier for one person to build a cool robot, I will consider it a success.  I hope to get a few others to help move the ball forward too, by adding services or rewriting mine.

 

Rewards / Incentives

In order to give people that extra push to dive in, I’d like to create some challenges / rewards for people who build a robot using this OS.

 

Feedback?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

This certainly sounds almost as ambitious as creating ROS.

Ditto on ROS.  A lot of startups use ROS as a starting point since it is such a mature ecosystem.  Many self driving cars started with ROS.  BMW, Bosch, NASA etc have used ROS for their robots.

It nicely integrates with an Arduino over serial using their messages so it looks like it is part of the network.  

It is only fully supported on Linux and has kind of a learning curve (or learning cliff as some might say).  It isn't easy to wrap your arms around and get it  to do the more useful things quickly.  

ros.org - check it out.

 

I have used ROS for years Robbie has 2 RPI3's on board 9 arduino pro minis using RS485 to talk to a master arduino. 1 RPI does navigation and has the Kinect the other does voice and has a vision camera they both publish messages.  there also 2 desktops as part of the system one runs an older version of ROS and face tracking and recognition and the main PC that has the AI system and MOS moveit for arm control and a web server to allow others to control Robbie. Iy also interfaces with an android phone  for speech to text services and teleoperation.

http://escaliente-robotics.blogspot.com.au/

Peter

 

All day, I was thinking about what you want to do while I was banging away at some documentation. 

If you do this, I think you want to make it work with ROS.  ROS is basically a subscriber based messaging architecture with a lot of robot related libraries that work with it.  ROS brings SLAM with its navigation stack, a lot of hardware integration such as IMUs and some popular LIDARs, simulation tools, OpenCV, Arduino, Android support. One example I saw leveraged Tensorflow example code that used a model that would look for things in whatever video feed you gave it and then send a text message describing what it sees.  Your Ava code would just be a node that responds to any language text passed into it.  You could do that processing on an Android, the Linux host or a Windows web service, it doesn't matter.  Whoever is responsible for speech to text processing would be a node and then send the message to the Ava node who responds to that speech from your engine.  ROS leads you to a nicely decoupled architecture .

Making Ava work with ROS will really enhance what folks can do and would be an awesome addition.  The ROS piece wouldn't be hard since you have already done the hard stuff like Natural Language Processing etc.

Regards,

Bill

Thanks for putting so much thought on this idea.  I have been banging away on the ROS doc too.  There is a lot of good stuff there, while also some limitations that I am hoping to bridge.  It seems like I need a Pi, a PC, or an available serial port on bot.  I am hoping to do something that doesn't require any of those, while allowing for all of them to play a part.

Your recommendation for me to tie into ROS and make my stuff a node is well taken.  A lot of the issues I want to deal with are at the microcontroller levels, while I also want to coordinate actions at the grand level.  Part of this involves being able to layer a verbal interface on top of everything...so people can basically talk to all the code or any part of it, and servers can talk to each other through APIs...or in natural language.  A lot of other standards could then just melt away when machines can really talk and share like we do.

One of the things I have been working on this year is the ability for the A.I. to look at and understand the structure of 3rd party databases and create complex apps for using them.  There is a lot more to say about this, but the ability to manage, injest, or call lots of databases of any structure is a key evolutionary step.  This sets the stage for a lot of new growth along more dimensions.

Another thing I left out of my doc...I also see my A.I. Node, whatever you call it, is just a contact point for another potential eco-system of compute/sharing nodes all over the internet.  Part of my system is to make my AWS servers be able to execute code on other servers anywhere on internet that conform to a certain set of argument passing conventions.  I tried something similar recently for allowing mass customization of the U.I. capabilties for an App that Ava manages at runtime and I liked it a lot.  This would allow mass customization of a collective Borg brain for very small bots where makers can build new and very small pieces of a brain and plug them into a much larger brain system without having to host or understand much of the whole thing.

Now, keeping in mind what I just said about the A.I. node calling other nodes for mass customization / brain extention.  The same technique might as well be applied to bots themselves.  This means an A.I. can orchestrate a single bot or something like a swarm of bots with an Arduino Uno using natural language.  Saying "Form a circle" could lead to a bunch of commands being sent out to a swarm.

Per everyone's recommendations, I am going to keep studying.

 

They are working on ROS2 at the moment it's currently in beta and ver 1 should be out at the end of the year. It works on windows 10, OSX and ubuntu and that will help bring in more users.

A bit about embedding ROS2 in microcontrollers

https://discourse.ros.org/t/ros2-on-microcontrollers-with-riot/1293/9

https://github.com/ros2/ros2/wiki

Regards

Peter

Thanks so much Peter, great links.  I am gaining a lot more understanding.

On a related note, I am VERY interested in your Robbie setup with multiple Arduinos over RS-485, RPis, PCs, etc.  

The RS-485 part is one of the most interesting parts as I haven't tried RS-485 and it might address some of my challenges.  If there is anything you have time to share about your setup, it could be of great insight to me and perhaps others.

I think perhaps a much smaller version of your setup (A Pi with a few Arduinos and a separate PC) would be a model I could follow for Ava 2...while serving as a great way to "dive in" to ROS as well.  It seems like I might have to swap out the android phone (with its cam, gps, touchscreen, and other capabilities) for a Pi and a Pi-cam.  Space could be a problem and loss of some abilities unique to Android, but it could still be a decent bot that could grow in other ways with ROS, especially if I can splice in my own brain stuff.  At some point, I'd like to do a bigger Ava with SLAM, Lidar, and/or Kinect (as you have), and ROS is an obvious fit for that.  All of my additional stuff might just have to be accessible as a set of APIs via http in that model, or a node as you guys point out.

I think there could still be a place for some open source reference robots (with a simple OS), that don't climb to the level of using ROS but could provide a useful step on a ladder for hobbyists to climb before getting into ROS.  I am going to give that a lot more thought first.

Thanks for all the help so far.  It is much appreciated.

Regards,

Martin

There a few examples of android robots using ROS, I always wanted to incorporate one into Robbie but never found the time. You could use your current code and add the ROS stuff to it so it listens for the master and publishes commands to the RPI using the wireless network. The RPI on the robot runs ubuntu mate and using a ROS robot upstart starts the navigation system and node to monitor battery and to convert locations to map coordinates ie kitchen is x5.3,y1.4.z0 and the orientation. You can still run your servers on windows just relay navigation commands through the android. Because you set the RPI as the rosmaster other nodes can be run on remote computers. when I make a new map Gmapping is run on a separate computer and drive the robot around using an android phone with teleop.

For RS485 I use an Arduino connected to the RPI and python to interface them, I have seen other setups where they use an RS485 shield on the RPI. this link helped me setup RS485 on Arduino  https://www.gammon.com.au/forum/?id=11428 

I have custom PCB's with Arduino's and motor drivers as my joint controllers so adding the MAX485 was easy.

Regards

peter

 

Thanks for the info, it has been most helpful.  I am working on updating my docs for this project to incorporate ROS and your feedback.  I also hope to explain another larger set of goals I want to achieve with this OS outside of its robotics uses, which I see as one among several uses, probably not even the primary use area.

Hi Peter, Bill, and anyone else that is interested in this,

I would like to thank each of you for the time taken and the input.  I have been a fan of Robbie for a long time.  I would love to hear more about your experiences with ROS, pros, cons, etc., or simple code examples of something on arduino that sends/receives messages to another platform.  It does sound like there is overlap.  My goal is productivity...if I can't achieve that I'll pursue another course.

A large part of my past experiences have been in using meta-data driven architectures to make things many times easier with orders of magnitude less coding.  In my career, I have constantly encountered engineers and technologies that fail to make things easier.  Each new platform is different than the last.  If there are some decent and easy standards for building bots, where are they?    I would like to try to address these and some other issues.  My personal opinion is that ROS has moved the ball forward, I just don't think the game is over in this space.  I am glad we're not all still using MS-DOS...which a lot of engineers tried to do when windows came along.

I envision robot makers being able to change many aspects of their bots simply by going to a website or windows app and filling out some forms and clicking "refresh".  Something like changing a default setting on some low level service embedded on an arduino, or changing a pattern definition that when recognized should generate some time of reflexive response, to many other things.  I can also see orchestrating complex sequences accross arduino purely by writing english (or other) paragraphs on the website and telling it under what circumstances to run those narratives.  These are just a few examples and are all things I do today to some extent.  At this point, I just want to make a basic version that is good enough for public consumption.

Do you (or anyone else) think there is too much overlap to justify another OS in the robot space?  Has ROS been easy to learn and use?  Can ROS at this point be used to easily build an Amazon Alexa, Google Assistant, or Jibo that moves?  I think Google / Amazon could end up controlling the space in a semi-closed system the way things are going.

Does ROS drive people towards a repeatable way to organize the code of arduinos, Pis, PCs, etc.?  My goal would be a set of simple coding standards where makers didn't have to write any plumbing code, whether using bluetooth, USB, I2C, etc.

I hear what everyone is saying about SLAM and other technologies.  I would like to do or tie in to those technologies at some point, but my goals are not as ambitious as that at this early stage.

I hope it's not the case, but the underlying message I think I am hearing at this point is that why bother with something new when ROS has solved all our problems.  Has it?  Can we envision something better?  I would love to hear more from each of you.

Once again, thanks for your thoughts and participation.  I hope I can keep you engaged.