An Introduction to Schema Learning

What are schemas?

Schemas are units of knowledge associating perceptions, actions and predictions. If the environment is perceived to be in a certain state then taking an action associated with this state should cause the environment to change to match the sensor values specified in that schema’s prediction.

In its simplest form a schema consists of a set of pre-conditions, an action and a set of post-conditions (often represented in the form pre-conditions/action/post-conditions), providing a basic forward learning model.

Schema chaining

The linking of pre-conditions and post-conditions from different schemas (‘schema chaining’ ) creates a traversable network representing different world states and the actions required to move between these states, as illustrated in figure 3.8.

Within PSchema chaining is achieved by applying Dijkstra’s algorithm to this network of world states. We treat each state as a vertex in the network of potential world states and each action as an edge between two states. These paths are then weighted based on the probability of success, allowing the system to determine the shortest chain of actions required to achieve a goal in the manner most likely to succeed.

The potential states that make up the vertices are determined by matching the post-conditions of the last schema tested against the pre-condition of the next candidate. Any generalised schemas have their values populated based upon the
prior schema in a potential chain before being tested for compatibility.

During the execution of a chain PSchema evaluates the success of the predictions made by each schema in the chain after the action component of that schema has been executed. If the execution of one of the schemas results in an unexpected outcome which is not compatible with the next schema’s pre-conditions then the chaining process begins anew, finding a chain of schemas from the new world state to the target.

Continual evaluation of the environment during the execution of a chain allows the system to adapt to both incorrect or unreliable learning and to other agents acting upon the environment.

Schema generalisation

Schema generalisation allows the system to go beyond simply being able to predict and form action plans based around previously experienced outcomes, giving it the ability to make informed decisions about scenarios it hasn’t encountered yet but which are similar to past experiences.

The generalisation mechanism implemented within PSchema produces schemas containing parameters (or ‘slots’ ) which can be populated based upon the current experiences of the robot when being executed. Beyond simply determining which aspects of the schema may be interchangeable with other values as many existing schema systems do, this mechanism attempts to find generalisable relationships between the pre-conditions, the action and the post-conditions of a schema. This allows the generalised schemas which are produced as a result of this process, to represent the agent’s hypotheses about how an interaction may work at a more abstract level.

Custom generalisers can be implemented as part of your application and made use of by PSchema in place of the default generaliser, allowing for the investigation of alternative generalisation mechanisms.