Getting Started With PSchema In Python

This tutorial will take you through all the stages involved in creating a simple learning agent that can interact with a small simulated world that we’ve provided. We assume some familiarity with Python programming and the GLib framework, but all PSchema related concepts will be introduced from scratch.

The sandbox

PSchema Sandbox

Figure 1: The PSchema tutorial sandbox in its initial state.

To make this tutorial more interactive we have created a simple application called the “Sandbox”. This is an interactive world consisting of three buttons which have different effects on the environment. Inhabiting this world is our learning agent, a disembodied hand capable of moving around the world and manipulating objects.

The application is distributed with PSchema in the doc/tutorial/sandbox/ directory. Simply move to that directory and execute ./sandbox.py to start the application.

The agent in the sandbox communicates with us via a network socket, so can be controlled from any programming language, in this example we’ll be using C. However, before we start writing a controller for the agent we can get a feel for its commands and behaviour by directing it ourselves.

By default the sandbox listens on port 6000 for connections, first we connect to it via telnet:

$ telnet localhost 6000
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.

First lets ask it what can it currently see in the world, by sending the command:

GET_SENSORS

It then responds with a list of observations about the world:

POSITION 150, OBJECT ‘Hand’
POSITION 190, OBJECT ‘Blue Button’
POSITION 192, OBJECT ‘Green Button’
POSITION 188, OBJECT ‘Red Button’

Figure 2. The world is divided into a grid of 32x32 pixel squares, which are sequentially numbered.

The sandbox’s positioning system is handled via sequentially numbered grid squares (each 32×32 pixels) as illustrated in figure 2.

We can tell our hand to move to a new location by issuing the command:

MOVE 32

From the GET_SENSORS command that we issued earlier we know that there’s a blue button at position 190. If we move to that location and request the sensor information again:

MOVE 190
GET_SENSORS

We receive some new sensor information in the response, relating to the fact that we’re now touching the blue button:

POSITION 190, OBJECT ‘Hand’
POSITION 190, OBJECT ‘Blue Button’
TOUCHING ‘Blue Button’
POSITION 192, OBJECT ‘Green Button’
POSITION 188, OBJECT ‘Red Button’

There are two further commands available to us when controlling the hand, GRASP and RELEASE. These can be used for manipulating objects, some objects in the world can be picked up while others cannot. While it’s not possible to actually pick up the buttons, when we attempt to grasp them we instead press them causing different types of food to be dispensed into the world depending on which button was pressed. As before we can test this functionality by first sending a command to move the hand over the red button (position 188), then sending a grasp command.

MOVE 188
GRASP
GET_SENSORS

Pressing buttons in the sandbox

Figure 3. Upon pressing the red button a strawberry appears in the world.

And in the response we can now see a strawberry positioned above the button we just pressed. This can be seen visually in figure 3.

POSITION 192, OBJECT ‘Green Button’
POSITION 148, OBJECT ‘Strawberry’
POSITION 190, OBJECT ‘Blue Button’
POSITION 188, OBJECT ‘Red Button’
TOUCHING ‘Red Button’
POSITION 188, OBJECT ‘Hand’

In contrast to buttons, items of food can be picked up when grasped by the hand. Once held objects will move with the hand until a release command is issued. First we grasp the strawberry:

MOVE 148
GRASP
GET_SENSORS

And find that upon doing so we receive some new sensor feedback informing us that we’re now holding the strawberry:

POSITION 192, OBJECT ‘Green Button’
POSITION 148, OBJECT ‘Strawberry’
POSITION 190, OBJECT ‘Blue Button’
POSITION 188, OBJECT ‘Red Button’
POSITION 148, OBJECT ‘Hand’
HOLDING ‘Strawberry’

Finally we can move the hand, still holding the strawberry to a new location, release the strawberry and move our hand away:

MOVE 133
RELEASE
MOVE 150

At this point we know roughly how the simulated world works ourselves and what actions are available to our agent, next we can begin writing our control software so that our agent can start to act independently of us and learn about the world for itself.

Creating an application that uses PSchema

To start off with we create a very simple skeleton of an application that initialises the PSchema library and sets up a GTK window which we’ll use later for showing a few buttons. Because GTK and PSchema are both based around GLib they can share the same main loop, however it is not necessary to make use of GTK when writing the control software, instead a simple GLib main loop can be created without the need for GTK. Later on we’ll also be communicating with the simulator over a socket connection.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from gi.repository import Gtk, GObject, PSchema
import socket
 
class Controller:
 
        def __init__(self):
                # Create a window
                self.win = Gtk.Window()
                # Exit the GTK main loop when our window is closed
                self.win.connect("destroy", Gtk.main_quit)
                # Display our window
                self.win.show_all()
 
                # Create our schema memory
                self.memory = PSchema.Memory.new()
 
                # Start the GTK main loop
                Gtk.main()

Note that when we create our schema memory object instantiate it by running PSchema.Memory.new() rather than the more traditional PSchema.Memory(), this is a product of the way the dynamic bindings are created. It is important to remember this difference as using simply PSchema.Memory() will work, in that it will instantiate a Memory object, but it won’t run that class’s initialisation code which can result in unexpected behaviour later on.

Debugging

In addition to using a full-fledged debugger such as GDB for inspecting the inner workings of the framework it’s possible to request varying levels of debug information on different topics through a couple of environment variables:

  • PSCHEMA_DEBUG – This sets the amount of debugging information required, taken as an integer between 1 and 5. When set to 1 it will give very few high level messages, when set to 5 it will give very detailed information.
  • PSCHEMA_DEBUG_FILTER – This specifies which subsystem debugging information should be output for. If unspecified then all debugging output information will be displayed. For example it can be set to “excitation” for debugging messages relating directly to the excitation calculations.

We can see this in action with our own application:

$ PSCHEMA_DEBUG=5 ./Controller.py
1326745281.562089 [1 - general] Debugging output enabled (level 5).
1326745281.562185 [1 - general] Showing all debug messages.

Communicating with the simulator

Now that we have all the basics set up we can start interacting with the simulator. In this example we communicate with our simulated agent through a simple TCP socket. This could easily be replaced with a custom agent control library, or robotics middleware such as Player or YARP, to allow for communication with robots in the real world or agents in other virtual environments.

First can set up our connection to the simulator in our general initialisation code.

17
18
19
20
21
22
23
24
25
26
27
                # Create our schema memory
                self.memory = PSchema.Memory.new()
 
                # Make a connection to the simulator
                self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                self.sock.connect(("localhost", 6000))
                # Call our data_received() function whenever new network messages come in
                GObject.io_add_watch(self.sock, GObject.IO_IN, self.data_received)
 
                # Start the GTK main loop
                Gtk.main()

Next we can add a skeleton implementation of our self.data_received() function:

30
31
32
33
        def data_received(self, sock, *args):
                data = sock.recv(1024)
                print data
                return True

For now this function just prints out any data it receives from the simulator, later on we’ll modify this so it can create a representation of the world state out of the sensor values sent to it from the simulator.

Now that we can receive messages from the simulator we can try sending some commands. First we’ll implement a move_hand() function:

36
37
        def move_hand(self, position):
                self.sock.send("MOVE %d\n" % position)

Finally we need to implement our function for requesting sensor data. This’ll be very similar to the move_hand() function, as it’s just sending the GET_SENSORS message over the network. All actually processing of the sensor information that the simulator returns will be handled in the data_received() function.

40
41
        def get_sensors(self):
                self.sock.send("GET_SENSORS\n")

To test our new functions we can try calling them immediately before starting the GTK main loop:

39
40
41
42
43
44
45
46
                # Call our data_received() function whenever new network messages come in
                GObject.io_add_watch(self.sock, GObject.IO_IN, self.data_received)
 
                self.move_hand(100)
                self.get_sensors()
 
                # Start the GTK main loop
                Gtk.main()

This should move the hand to position 100 then request the sensor data. When the simulator responds to the sensor request this will be processed by our data_received() function, which for now will simply print the information to stdout:

$ ./Controller.py
POSITION 192, OBJECT ‘Green Button’
POSITION 190, OBJECT ‘Blue Button’
POSITION 188, OBJECT ‘Red Button’
POSITION 100, OBJECT ‘Hand’

Observations

Now that we’re receiving sensor information from the simulator we need to convert it into a form that the PSchema framework can reason about. We do this by first creating some custom observation classes, inheriting from the abstract Observation class defined by PSchema. While there are already a few basic observation types built into the framework for this tutorial we will create our own custom class to more precisely represent the information we’re being supplied with.

There is a certain amount of boiler plate code involved in defining a new GLib based class, it’s not necessary to understand this in detail, instead the main focus will be placed on the aspects relating specifically to PSchema observations.

Since the sensor information we’re receiving at the moment is about the positions of objects we’ll create a new observation class called MyObjectObservation which can store this information. To begin with we need to create a python file, MyObjectObservation, to store our new class. We’ll begin by writing our initialisation functions:

1
2
3
4
5
6
7
8
9
10
11
from gi.repository import PSchema
 
class MyObjectObservation(PSchema.Observation):
 
        position = 0
        name = ""
        position_var = None
        name_var = None
 
        def __init__(self):
                PSchema.Observation.__init__(self)

This create our new observation based on the abstract Observation class and sets up a few properties for us to use later.

Next we’ll write simple method for checking to see if two instances of MyObjectObservation are equal:

13
14
15
16
        def do_equals(self, o2):
                if type(o2) != type(self):
                        return False
                return self.position == o2.position and self.name == o2.name

Note that when overriding methods from the Observation class (or any other PSchema class) it is necessary to prepend their method name with do_.

When an observation is loaded from a previously saved XML file its XML node is passed to the relevant class so that any properties specific to that observation class can be parsed. The Observation base class can handle all
of the XML parsing for us, but it needs a way of setting the concrete and generalised properties in our class, it does this via the set_concrete_var() and set_property_var() methods:

18
19
20
21
22
23
24
25
26
27
28
        def do_set_property_var(self, propname, value):
                if propname == "position":
                        self.position_var = value
                elif propname == "name":
                        self.name_var == value
 
        def do_set_concrete_var(self, propname, value):
                if propname == "position":
                        self.position = int(value)
                elif propname == "name":
                        self.name == name

These methods are also made use of by the generaliser for creating new generalised values and for instantiating existing generalised values at execution/planning time.

Next a function for creating a copy of the current observation:

30
31
32
33
34
35
36
        def do_copy(self):
                o2 = MyObjectObservation()
                o2.position = self.position
                o2.position_var = self.position_var
                o2.name = self.name
                o2.name_var = self.name_var
                return o2

Finally a function for retrieving all the properties of our observation, either in generalised or concrete form:

38
39
40
41
42
43
44
45
46
47
48
        def do_get_properties(self):
                props = {}
                if self.position_var:
                        props["position"] = self.position_var
                else:   
                        props["position"] = str(self.position)
                if self.name_var:
                        props["name"] = self.name_var
                else:   
                        props["name"] = self.name
                return props

Amongst other uses this function is used by the base Observation class when creating XML representations of our observation. We can test this functionality out in the python console:

Python 2.7.3 (default, Apr 20 2012, 22:44:07)
[GCC 4.6.3] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>> from MyObjectObservation import MyObjectObservation
>>> moo = MyObjectObservation()
>>> moo.name = “Cow”
>>> moo.position_var = “$x”
>>> moo.to_xml()
“<observation type=’MyObjectObservation+MyObjectObservation’ name=’Cow’ position=’$x’ successes=’0.000000′ activations=’0.000000′ sensor_id=’0′ />”

There are a few things to notice here, firstly it’s not necessary to instantiate our custom class via MyObjectObservation.new() because its not part of the dynamically generated bindings. We then assign a concrete value to the name, but a generalised value to the position. When we run the to_xml() method the underlying Observation class that we inherited from creates an XML representation for us and correctly identifies that the position property has a generalised property thanks to the order of preference in our do_get_properties() method

This completes the MyObjectObservation class, however to fully represent the information we get back about the world we’ll need two other Observation types, MyTouchObservation and MyHoldingObservation. Using MyObjectObservation as an example try writing these two Observations. MyTouchObservation should have one boolean property “touching” and MyHoldingObservation should have one string property “object”. If you have difficulty implementing these you can find a reference implementation in the doc/tutorial/python/controller/ directory.

Representing the world

With our new Observation classes we can now create world states representing the robot’s current view of the world. First we need to include our new observations in our controller.

1
2
3
4
5
6
#!/usr/bin/env python
from gi.repository import Gtk, GObject, PSchema
import socket
from MyObjectObservation import MyObjectObservation
from MyTouchObservation import MyTouchObservation
from MyHoldingObservation import MyHoldingObservation

We already have a stub of a method for receiving sensor data, so we can expand this to process the data we’re being sent from the simulator.

36
37
38
def data_received(self, sock, *args):
        data = sock.recv(1024)
        return True

Actions

Using the Observation classes we wrote earlier we’re now able to represent everything our sensors tell us about the world. However we’re not yet able to represent the actions our agent itself takes, to do this we need some Action classes. This should be very familiar to you after creating the Observation classes, so we’ll simply provide the full source for the MyMovementAction class here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
from gi.repository import PSchema, GObject
 
class MyMovementAction(PSchema.Action):
 
        __gsignals__ = {
                "move_signal" : (GObject.SIGNAL_RUN_LAST, GObject.TYPE_NONE, (GObject.TYPE_INT,))
        }
 
        position = 0
 
        def __init__(self):
                PSchema.Action.__init__(self)
 
        def do_equals(self, a2):
                if type(a2) != type(self):
                        return False
                return a2.position == self.position
 
        def do_execute(self):
                # Emit a signal for the controller to act on
                PSchema.debug(5, "Executing movement action to position %d\n" % self.position, "action")
                self.emit("move_signal", self.position)
 
        def do_set_concrete_var(self, name, val):
                if name == "position":
                        self.position = int(val)
 
        def do_copy(self):
                a2 = MyMovementAction()
                a2.position = self.position
                return a2
 
        def do_get_properties(self):
                props = {}
                props['position'] = str(position)
                return props
 
GObject.type_register(MyMovementAction)

The only main difference from the Observation classes that deserves additional extra attention is the introduction of signals. Rather than directly implementing the code used to make our agent carry out these action within our Action class we send a signal back to the controller software. This means that our action types can be portable across many different agents, all with different controllers. We’ll look at this in more detail when we connect to the signal to the move() function within our main Controller.py file.

It will now be necessary for you to create a couple of additional Action classes based on this; MyGraspAction and MyReleaseAction. Again if you have difficulties there are reference implementations in doc/tutorial/python/controller/.

Bootstrapping

 

Saving

 

Loading

 

Source code

The final version of the controller developed throughout this tutorial can be found in the doc/tutorial/python/controller/ directory, however it is recommended that you construct the controller yourself whilst following the tutorial so that each concept can be studied in isolation and then built upon.