Principles of holistic control

Followup to: Where map meets the territory, Levels of structure, Keeping the target in sight, Vague questions and precise answers, Causal rules and unpredictable actions.

When mind is supporting a picture of environment, it is not being passive. Some of the future events in the environment are determined by this picture, they happen because they are in the picture. The chain of events flows from representation in the mind to the outcome through the actions, which lie on the both sides, in the mind and in the environment at the same time.

It is trivial that whatever state the low-level output event assumes in the mind, the same state will appear in reality, because it is the same event on both sides. For other events in the mind, it is not necessarily so; changing such events in the mind will make them represent environment incorrectly, rather than making them determine corresponding state of environment. On the other hand, every change that does leave representation correct, leads to the environment complying with it.

This perspective can be turned around: if representation is restricted to only assume correct states, it can be freely varied within that bound, and whatever picture of environment it chooses to draw will automatically come true.

One way of constructing an accurate picture of environment is through following the rules of thumb, filling the gaps in the picture based on known elements of any kind. When it is known that there is a chair in a room, it is possible to infer that the chair is likely to stand there, instead of flying in the midair, even though original description doesn’t include information about how the room floor relates to the chair.

Since this picture is centered around the actions of the agent that supports it, rules of thumb need to be sufficiently strong (if not individually then cumulatively) not to break down under the surprising changes of context that may result from the actions.

Each element of the representation asks a question, adds a constraint on the set of possible causal patterns that satisfy it. Additional elements help filling the gaps in the model of environment, initiating inference that weaves the structure. Not all details of the environment can be supported in the mind at once, even if they can be inferred from known facts. A tiniest hint may bring to attention many precise details, reconstructing intermediate elements of structure. Inferred from robust rules of thumb, state of mind would indirectly correspond to the state of environment, and giving different hints will lead to the state of mind corresponding to different aspects of environment.

The uncertainty about the future state of environment that is determined by the state of mind may be resolved in many ways, by assuming one of the allowed states of mind. On the side of the mind, the process of resolution of this uncertainty starts from making a decision, from introducing a fact in the model that doesn’t otherwise follow, and checking if the model assembles to a coherent state, if this element leaves the picture in the mind corresponding to environment, thus causing the environment to assume the state corresponding to the decision. If the decision consists in a relatively vague hint about the future, there are good chances that there is in fact a state of environment that satisfies the hint.

Much like drawing the attention to an aspect of environment, asserting a certain vague property in the future state of environment leads to construction of the detailed representation of the state of environment that has that property, driven by a multitude of specific inferred facts about the state of environment (both in the past and in the future) and general direction specified by the property. Where attention draws the details of representation from available factual information, model of the future may make up some of the details when they can be determined by the model. Thus constructed model will include events in the past and the future, but also the present, in particular low-level action events. Choosing a certain property in the future leads to construction of the model of environment that has that property, and the model of environment includes specific state of low-level actions in the present, which causes these actions to be carried out, which in turn determines the future to have required property, to be in accordance with the model.

The plan formed by the model of the future chosen to lead to a certain outcome doesn’t need to be very detailed, for example it doesn’t need to contain the whole sequence of low-level actions from the current point on. Plan gets refined as it unwinds, as more accurate factual information becomes available about events in the environment that were only modeled based on the goal at the start. At each moment, low-level action is chosen according to the best current guess included in the current model.

This approach allows to view the process of control in intelligent agent as a result of two cognitive pressures acting on representation of environment supported by its mind. The first pressure compels the representation in the mind to be correct, to depict the state of environment (past, future and present) as accurately as possible. The second pressure biases the representation to see the future state of environment that is as close as possible to the goal.

I call this perspective on how control algorithm could operate “holistic control”, to reflect the way plans get constructed. Inference operates across the levels of representation and in both directions in time, it is neither bottom-up nor top-down, it is not forward chaining or backward chaining. Control algorithm doesn’t contain clear-cut feedback loops, processing doesn’t happen in feed-forward fashion. The model of environment is held together by heuristic rules that aren’t organized in any kind of hierarchy, the model itself is “flat”, not modular except for the structure inherited from environment it represents. The operation of control algorithm is focused on the support of model of environment, not on action and perception. Action and perception are only peripheral (although indispensable) aspects of control, with low-level input binding the model of environment to reality at one tiny point, supplying new facts and showing the mistakes, and low-level output giving the model ability to participate in the causal web of environment.

Where map meets the territory

Followup to: Levels of structure, The dynamics of mind.

Events in the mind of intelligent agent indicate events in the environment. A configuration of events in the mind represents the current context, knowledge about which events are known to be present. Given a context, some additional events can be inferred according to known rules of thumb, supplementing the context with events on different levels of representation and resolving uncertain events.

The breakup of the current knowledge about environment on separate general events is a property of data structure, knowledge representation, designed for the purpose of being manipulated by inference algorithm. General events that don’t change (don’t become irrelevant or invalidated), and therefore don’t need to be manipulated by inference, don’t need to be separately represented in the mind. Events in the mind show what is different in the current knowledge about environment, they are active elements resulting from inference, starting from sensory input and trains of deliberative computation.

Most of the events in the mind are separate from events in the environment they represent. The property of representing (indicating) events in the environment results from using inference process that, through many intermediate steps, creates events in the mind in the right way. Some of the events, on the other hand, may be thought of as lying both in the mind and in the environment, and representing themselves. These are events of low-level input and output, boundary of the mind, where map meets the territory.

State of the environment is reflected in the mind, events outside the agent are warped around the border to fit inside its mind. These events include elements of future as well as past, elements independent on agent’s actions and determined by them. Two pictures of environment work by different laws: events in the environment itself are constrained by laws of physics and feature excruciating level of detail, while events in the mind are simple caricatures constrained by rules of thumb, mimicking the former. Both pictures meet at the identity of boundary events, at input and output. Events at the boundary are identical in both forms, and events further and further away from the boundary in both directions are traced through chains of relations holding in each of the pictures, reaching high-level events in the mind and events far away, both in space and time, in the environment. Captured in correspondence, pictures determine the elements in each other.

Unpredictability of actions

Followup to: The flow of reality.

Effects produced by optimization processes are much less predictable than other events happening in the environment. This happens because optimization processes can produce novel causal patterns, which break rules of environment that worked well in the past. When optimization process is set up with a known goal, some rules of the optimized environment may be known in advance. But specific path towards the goal is usually unknown. Leaving the details of the path unknown in advance may be the whole point of launching an optimization process with known goal: you describe the goal, perhaps only vaguely, and out comes a precise and efficient plan for achieving it. Along this unknown path the optimization process may produce all kinds of unknown causal patterns breaking the old rules.

Ordinary causal patterns appear in ordinary contexts. It is reflected both in their origin, where a pattern is produced through a usual kind of interaction between usual causal patterns, and in the form of rules of thumb that capture its operation, where the conditions of applicability include just a few surface properties. Zooming in on optimization process as an intelligent agent, the choice of actions depends on representation of environment that includes lots of contextual information. Unlike an ordinary causal pattern, a mind isn’t limited to any few of the configurations appearing in the environment; it absorbs as much of them as possible. As a result, any action that doesn’t take an obvious step towards the goal can depend on many contextual features, and so isn’t amenable to being captured by a simple rule.

Unpredictability of actions doesn’t necessarily refer to creation of novel configurations that break the semantics of old events. It can consist merely in the failure to predict or interpret specific actions, where they perform a choice between known causal patterns. You get in the taxi in unknown city, and you don’t know where driver will turn, even if you know the destination.

Forming cached interpretations for the actions of intelligent agent may be unreliable: they don’t work by the rules of natural causal patterns and may defy normal classification. An actual action is chosen by the agent for the specific context, which can place it in any of the context-insensitive bins an observer might have. It may be marked as “stupid”, “brilliant”, “careless” or “disastrous” and not actually be one.

The flow of reality

Followup to: Rules of thumb.

If we are to study the mind, the structure that drives the environment towards the target by choosing the actions that lead there, we need to look at the structure of environment, and the ways in which its overall dynamics can be predictable.

The environment develops in time, the past determines the future. Physical laws establish a relation that holds between the physical configurations in the past and in the future. Given an event (set of configurations), these laws show which configurations are allowed at different times. Literally applying laws of physics to events is no use: an object that at first looks like a door, but then grows legs and runs away is not a physical impossibility, configuration that does that is included in the event of there being a door-like object. But it’s not actually realized.

Each actual physical configuration descends from the deep causal history. A certain object can only be realized when it is preceded by the process that leads to the creation of that object. The form that a particular event takes in reality is determined by its causal past, and not just by physical restrictions that apply to it.

Events that fit together either arise by coincidence, or as a result of sharing the causal past. Complex dependencies are very unlikely to result from a coincidence, but producing or preserving dependencies also requires a special kind of process that doesn’t degenerate into random noise. Causal patterns not only establish the relationship between surface properties that are associated with them, but also act as elements in the overall flow of reality, determined by causal patterns that precede them, and determining causal pattern that follow.

There are many general kinds of causal patterns that are useful in describing the environment. One-time events result from certain combinations of patterns and dissolve afterwards, casting the ripples of their structure down the stream, in the future. Persistent physical objects are patterns that preserve their structure, generating the patterns in the future that hold the same properties as patterns in the past, forming steady streams in the flow. By far the most interesting kind of pattern is optimization process. Where stability can keep the event from dissolving away, propagating its structure over time, optimization makes a step further and drives the dynamics of environment towards a state that was never encountered before. The structure of the state that optimization process targets is implicit in the structure of the process, and as such, it can be much more intricate than the structures that can appear explicitly by coincidence. Optimization process is a knot on the stream of reality that tightens over time, ever closer to revealing the hidden structure.

Just as optimization process that arises by accident may be quite unlike the target state, other patterns may also pass through implicit stages, where the causal pattern only exists in a form of the process that eventually converges on it. Persistent patterns, such as living organisms, may pass through quite a metamorphosis before restoring the pattern in original form. Events may interact with each other using stable pathways, establishing robust dependencies even when confronted with the full richness of the real world. Patterns that repeat and persist build themselves on top of the stable processes. The actions that an intelligent agent chooses may seem to oppose its goals, or look random and irrelevant, but a reliable plan adds up to the intended outcome.