Focus of attention

Followup to: Principles of holistic control, Event detector as experimental setup.

Event detectors can be regarded as experimental devices for discovering properties of past and future. State of detector is the result of the experiment, so if detector is designed to answer only one specific question, the answer should also stay the same, as constant state of the detector. But semantics of detectors changes over time, and the questions are more generally context-sensitive, so state doesn’t stay the same, and interpretation needs to keep up. When interpretation of detector changes, so do interpretations of dependent detectors, and detectors depending on them.

Locally and globally, each moment the mind changes which properties of environment it indicates, and correspondingly changes its state. Interpretation of possible states of the detectors (external description) and states themselves (actual implementation) are interrelated: current states form a context for reaching following states, and correspondingly interpretation of current states determines interpretation of possible following states.

Let’s call interpretation of (properties of environment indicated by) current state of mind, focus of attention. Events indicated by next state of mind are directly indicated by currently indicated events in environment (including current sensory input). Thus, focus of attention follows indicator chains in the environment. Since mutually indicating properties propagate together, inference running for a while without drastic disturbances will lead to a robust model of aspect of environment, with elements reinforcing and double-checking each other.

Focus of attention is driven by several related pressures. First, directions of inference from properties of environment in the focus of attention form the direction of change in overall attention, if aligned sufficiently together. Second, events in the focus of attention tend to form a coherent picture, with mutually reinforcing events staying longer together and separate events gradually dissipating. Third, low-level input and output have fixed semantics, thus binding focus of attention to the agent in several points. Even though the focus can stretch for light-years in high-level representation, it never completely leaves action and perception, which emanate waves of attention through the first pressure, by immediate inference, in all directions around the agent. This same pressure drags the attention forward in time, changing the representation to reflect current events in the environment. And fourth, focus of attention is driven to seek desirable properties in environment, forming a substrate for preparing and following goal-reaching plans.

Depending on context, individual detectors can have a great variety of interpretations. As a result, separately, detectors have a very distributed semantics, indicating a certain texture of configurations located anywhere and at any time. But together, they add up to an inferentially localized picture, in which each individual event makes much more clear-cut distinction, being focused by other events.

Principles of holistic control

Followup to: Where map meets the territory, Levels of structure, Keeping the target in sight, Vague questions and precise answers, Causal rules and unpredictable actions.

When mind is supporting a picture of environment, it is not being passive. Some of the future events in the environment are determined by this picture, they happen because they are in the picture. The chain of events flows from representation in the mind to the outcome through the actions, which lie on the both sides, in the mind and in the environment at the same time.

It is trivial that whatever state the low-level output event assumes in the mind, the same state will appear in reality, because it is the same event on both sides. For other events in the mind, it is not necessarily so; changing such events in the mind will make them represent environment incorrectly, rather than making them determine corresponding state of environment. On the other hand, every change that does leave representation correct, leads to the environment complying with it.

This perspective can be turned around: if representation is restricted to only assume correct states, it can be freely varied within that bound, and whatever picture of environment it chooses to draw will automatically come true.

One way of constructing an accurate picture of environment is through following the rules of thumb, filling the gaps in the picture based on known elements of any kind. When it is known that there is a chair in a room, it is possible to infer that the chair is likely to stand there, instead of flying in the midair, even though original description doesn’t include information about how the room floor relates to the chair.

Since this picture is centered around the actions of the agent that supports it, rules of thumb need to be sufficiently strong (if not individually then cumulatively) not to break down under the surprising changes of context that may result from the actions.

Each element of the representation asks a question, adds a constraint on the set of possible causal patterns that satisfy it. Additional elements help filling the gaps in the model of environment, initiating inference that weaves the structure. Not all details of the environment can be supported in the mind at once, even if they can be inferred from known facts. A tiniest hint may bring to attention many precise details, reconstructing intermediate elements of structure. Inferred from robust rules of thumb, state of mind would indirectly correspond to the state of environment, and giving different hints will lead to the state of mind corresponding to different aspects of environment.

The uncertainty about the future state of environment that is determined by the state of mind may be resolved in many ways, by assuming one of the allowed states of mind. On the side of the mind, the process of resolution of this uncertainty starts from making a decision, from introducing a fact in the model that doesn’t otherwise follow, and checking if the model assembles to a coherent state, if this element leaves the picture in the mind corresponding to environment, thus causing the environment to assume the state corresponding to the decision. If the decision consists in a relatively vague hint about the future, there are good chances that there is in fact a state of environment that satisfies the hint.

Much like drawing the attention to an aspect of environment, asserting a certain vague property in the future state of environment leads to construction of the detailed representation of the state of environment that has that property, driven by a multitude of specific inferred facts about the state of environment (both in the past and in the future) and general direction specified by the property. Where attention draws the details of representation from available factual information, model of the future may make up some of the details when they can be determined by the model. Thus constructed model will include events in the past and the future, but also the present, in particular low-level action events. Choosing a certain property in the future leads to construction of the model of environment that has that property, and the model of environment includes specific state of low-level actions in the present, which causes these actions to be carried out, which in turn determines the future to have required property, to be in accordance with the model.

The plan formed by the model of the future chosen to lead to a certain outcome doesn’t need to be very detailed, for example it doesn’t need to contain the whole sequence of low-level actions from the current point on. Plan gets refined as it unwinds, as more accurate factual information becomes available about events in the environment that were only modeled based on the goal at the start. At each moment, low-level action is chosen according to the best current guess included in the current model.

This approach allows to view the process of control in intelligent agent as a result of two cognitive pressures acting on representation of environment supported by its mind. The first pressure compels the representation in the mind to be correct, to depict the state of environment (past, future and present) as accurately as possible. The second pressure biases the representation to see the future state of environment that is as close as possible to the goal.

I call this perspective on how control algorithm could operate “holistic control”, to reflect the way plans get constructed. Inference operates across the levels of representation and in both directions in time, it is neither bottom-up nor top-down, it is not forward chaining or backward chaining. Control algorithm doesn’t contain clear-cut feedback loops, processing doesn’t happen in feed-forward fashion. The model of environment is held together by heuristic rules that aren’t organized in any kind of hierarchy, the model itself is “flat”, not modular except for the structure inherited from environment it represents. The operation of control algorithm is focused on the support of model of environment, not on action and perception. Action and perception are only peripheral (although indispensable) aspects of control, with low-level input binding the model of environment to reality at one tiny point, supplying new facts and showing the mistakes, and low-level output giving the model ability to participate in the causal web of environment.

Causal rules and unpredictable actions

Followup to: Causation as robust indication, Unpredictability of actions.

Changes considered for causal rules to handle are usually brought up in the context of decision making. Decisions lead to actions, and actions can create radical changes in context. Being the most unpredictable, actions of intelligent agents serve as the hardest test for robustness of causal rules and filter out many rules of thumb that are not normally considered causal. The more significant changes are allowed to occur, the less rules of thumb are general enough to endure them.

Causal rules don’t need to be preserved for all physically possible contexts, since only contexts that follow from causal history of environment are actually realized, and only optimization processes that currently exist in the environment can change contexts in novel ways. Causal rules serve as a tool that works where the set of significantly different causal patterns remains approximately the same, but within that domain of applicability they allow to draw accurate conclusions. Thus, causal rules only need to persist during sufficiently insignificant changes in global context that preserve the semantics of events, through local changes in context that can be enacted by existing optimization processes.

For example, grass is usually wet when it’s raining. Wet grass is a good indicator of rain, it is a reliable rule of thumb in natural context. Yet if an intelligent agent creates a new context, by specifically pouring water on the grass, in this new context the rule will no longer hold, wet grass doesn’t cause rain. Colliding stones produce a sound in natural context, and if an intelligent agent collides stones in artificial context, there is still a sound, which shows that colliding stones cause a sound.

Causal rules are rarely infallible, the same principle that tests them and distinguishes them from mere correlations can break them as well. There usually is a way to create a contrived context in which a given causal rule will no longer hold. Colliding stones in vacuum or colliding specifically crafted elastic “stones” will produce no audible sound. Administering symptom relief medicine leads to disease not causing symptoms. Only causal rules that follow immediately from laws of physics or those guarded by limitations in available technology can withstand a directed attack. Thus, most of the causal rules have additional conditions, specifying the kinds of changes that they are able to withstand.

This places various correlations on the same scale with causal rules, distinguishing causal rules only by generality, applicability in wider context. Depending on the context, different strength is required from rules to carry out accurate inferences, and weaker rules can contribute considerably to determining the details not captured by stronger causal rules.

Unpredictability of actions

Followup to: The flow of reality.

Effects produced by optimization processes are much less predictable than other events happening in the environment. This happens because optimization processes can produce novel causal patterns, which break rules of environment that worked well in the past. When optimization process is set up with a known goal, some rules of the optimized environment may be known in advance. But specific path towards the goal is usually unknown. Leaving the details of the path unknown in advance may be the whole point of launching an optimization process with known goal: you describe the goal, perhaps only vaguely, and out comes a precise and efficient plan for achieving it. Along this unknown path the optimization process may produce all kinds of unknown causal patterns breaking the old rules.

Ordinary causal patterns appear in ordinary contexts. It is reflected both in their origin, where a pattern is produced through a usual kind of interaction between usual causal patterns, and in the form of rules of thumb that capture its operation, where the conditions of applicability include just a few surface properties. Zooming in on optimization process as an intelligent agent, the choice of actions depends on representation of environment that includes lots of contextual information. Unlike an ordinary causal pattern, a mind isn’t limited to any few of the configurations appearing in the environment; it absorbs as much of them as possible. As a result, any action that doesn’t take an obvious step towards the goal can depend on many contextual features, and so isn’t amenable to being captured by a simple rule.

Unpredictability of actions doesn’t necessarily refer to creation of novel configurations that break the semantics of old events. It can consist merely in the failure to predict or interpret specific actions, where they perform a choice between known causal patterns. You get in the taxi in unknown city, and you don’t know where driver will turn, even if you know the destination.

Forming cached interpretations for the actions of intelligent agent may be unreliable: they don’t work by the rules of natural causal patterns and may defy normal classification. An actual action is chosen by the agent for the specific context, which can place it in any of the context-insensitive bins an observer might have. It may be marked as “stupid”, “brilliant”, “careless” or “disastrous” and not actually be one.

Keeping the target in sight

Followup to: The dynamics of mind.

When constructing an action, it is not enough to know that the action is a good indicator of target outcome. It is also useful to know if the action is required to get the outcome: maybe it’ll just come about anyway. Performing a sun-worshiping ritual every evening is a good indicator of an event that sunrise will occur tomorrow morning, probability of sunrise given that you performed the ritual is high, but the ritual was hardly useful for achieving this outcome. This seems to require a separate notion of causation, only choosing actions that do contribute to the outcome, that cause it. But there is also a way to do without causation, at least in this sense.

The trick is that preparing a supper in the evening is also a good indicator of sunrise. The decision making can be thought of not as a sequence of specific solutions to specific problems, but as a process that preserves the target state of environment in the role of an event indicated by current state of mind. If model of the environment that assesses indication is accurate enough, having the target outcome indicated by actual state of mind shows that target outcome will likely in fact happen. If performing a sun-worshiping ritual is a good indicator of sunrise, and sunrise is desirable, it can as well be performed, just as bones are allowed to be white instead of green, so long as they perform their function. On the other hand, if preparing a supper leads to even better outcome than performing a ritual, it is a preferable action to establish. Accurate decisions are self-fulfilling prophecies, and right decisions foretell good outcomes.

The dynamics of mind

Followup to: The flow of reality.

A configuration confined within a narrow event (causal pattern) plays a role of a program in the physical world: its structure (partially) determines what will follow in which contexts, what will be the output for each input, the future for each possible past. As a part of the output, causal pattern determines its successor, the pattern that will implement the next phases of the process initiated by previous pattern.

Let’s consider the dynamics of a causal pattern that implements a mind. First, some imagery to fuel intuition with. Mind is an optimization process, a converging stream in the flow of reality. In its wake, this stream warps the environment by applying little nudges calculated for higher impact. To calculate the appropriate nudges, the stream tunes itself to react to the relevant properties of the environment. The nudges are determined by common causes to become concerted with other causal patterns, so as to direct the flow towards the target. The structure of the stream gets refined over time to become more receptive to current properties of the environment and to turn them into actions to a greater effect. The flow of causality gets captured by the stream, and, transformed by its structure, it is released to shape the future.

Knowing the environment means identification of causal patterns present in it. For the knowledge to be applied to decision-making, it needs to be reflected in the state of mind. In other words, the different possible causal patterns to be found in the environment need to cause different functional states of mind (different causal patterns in the mind). The state of mind acts as a way to disambiguate the possible states of environment. Where the environment is sparse, it is sufficient for the mind to be set so as to assume different states depending on the wide events that contain the possible causal patterns in the environment. As well as representing the long-term rules by which the environment happens to play, mind needs to represent the current state of the environment, or, in other words, the state of the environment relative to the time and place where the causal pattern implementing the mind happens to be. The structure of the environment around the mind is important, because it’s where the actions are initiated. The long-term rules are used to support a representation that robustly integrates different pieces of information that the mind comes across and allows it to run deep queries about the state of the environment that are not immediately accessible to it, or even about the states of environment in the future.

Armed with representation that reflects the structure and state of the environment, the dynamics of mind causes the actions that are predicted to lead to the goal. Goal is not necessarily very fine-tuned, it may simply consist of certain events that are targeted, although that alone could be sufficient to launch the optimization process that leads to the state of environment much more intricate than anything that could’ve been created by chance. An action, or a decision to implement a high-level plan, consists in the state of mind, a belief, in the fact that the objective will be achieved, that itself causes it to be achieved. A reliable decision acts as an indicator of the outcome in the environment, in the same way as representation of other knowledge about the environment, and so it is constructed from the representation. Amusingly, making good decisions consists in the mind dreaming up self-fulfilling prophecies foretelling good fortune.

The flow of reality

Followup to: Rules of thumb.

If we are to study the mind, the structure that drives the environment towards the target by choosing the actions that lead there, we need to look at the structure of environment, and the ways in which its overall dynamics can be predictable.

The environment develops in time, the past determines the future. Physical laws establish a relation that holds between the physical configurations in the past and in the future. Given an event (set of configurations), these laws show which configurations are allowed at different times. Literally applying laws of physics to events is no use: an object that at first looks like a door, but then grows legs and runs away is not a physical impossibility, configuration that does that is included in the event of there being a door-like object. But it’s not actually realized.

Each actual physical configuration descends from the deep causal history. A certain object can only be realized when it is preceded by the process that leads to the creation of that object. The form that a particular event takes in reality is determined by its causal past, and not just by physical restrictions that apply to it.

Events that fit together either arise by coincidence, or as a result of sharing the causal past. Complex dependencies are very unlikely to result from a coincidence, but producing or preserving dependencies also requires a special kind of process that doesn’t degenerate into random noise. Causal patterns not only establish the relationship between surface properties that are associated with them, but also act as elements in the overall flow of reality, determined by causal patterns that precede them, and determining causal pattern that follow.

There are many general kinds of causal patterns that are useful in describing the environment. One-time events result from certain combinations of patterns and dissolve afterwards, casting the ripples of their structure down the stream, in the future. Persistent physical objects are patterns that preserve their structure, generating the patterns in the future that hold the same properties as patterns in the past, forming steady streams in the flow. By far the most interesting kind of pattern is optimization process. Where stability can keep the event from dissolving away, propagating its structure over time, optimization makes a step further and drives the dynamics of environment towards a state that was never encountered before. The structure of the state that optimization process targets is implicit in the structure of the process, and as such, it can be much more intricate than the structures that can appear explicitly by coincidence. Optimization process is a knot on the stream of reality that tightens over time, ever closer to revealing the hidden structure.

Just as optimization process that arises by accident may be quite unlike the target state, other patterns may also pass through implicit stages, where the causal pattern only exists in a form of the process that eventually converges on it. Persistent patterns, such as living organisms, may pass through quite a metamorphosis before restoring the pattern in original form. Events may interact with each other using stable pathways, establishing robust dependencies even when confronted with the full richness of the real world. Patterns that repeat and persist build themselves on top of the stable processes. The actions that an intelligent agent chooses may seem to oppose its goals, or look random and irrelevant, but a reliable plan adds up to the intended outcome.

Goals and means

Followup to: Context-specific actions.

At each step, alternative available actions can lead to different consequences. The choice of action (state of mind) translates into the choice of properties of the future environment. If each choice is performed according to the same preference of properties of future environment, the cumulative pressure of all choices driving the environment in the same direction becomes very strong, even if each individual action can influence very little. The trick is to know the right timing and right direction, to nudge the environment according to its own rhythm, and world-changing results can follow.

The direction in which the agent steers the environment is called the agent’s goal. The goal can be specified in different ways, notably in terms of utility function that ranks each possible state of the environment with a numeric value. From this perspective, intelligent agent is an optimization process that optimizes the environment to assume the state of higher utility, according to a specific utility function that governs the process. Note that the goal (ranking of possible states of the environment) is in general arbitrary, and it is erroneous to apply characteristics of human goals to goals-in-general. Since different goals can favor completely different states of the environment, for every “obvious” preference there is, theoretically, a mind that prefers otherwise (and optimizes in that direction).

Agent’s goal is the sole focus of all of its functionality. The particular way in which the agent is implemented, the ways in which it seeks out the information about the environment, or a particular ritual of rationality that it follows — all are instrumental and are there only to facilitate the optimization of environment according to the goal. (Which doesn’t mean that the goal is external to implementation or that implementation is inherently unimportant. Goal may include the clauses about environment containing an implementation with particular characteristics, and goal is embodied by a particular implementation of the agent.)

Just as perception is only necessary to find the actions available in the current context, actions only need to be constructed according to the agent’s goal. If it is expected that a particular action 1 will be ranked lower than action 2, there is no point in considering action 1. Thus, goal specifies which future states of the environment need to be optimized for, actions are selected to lead to target states of the environment, and perception allows to find out which actions lead to which states of the environment.

Goal, perception and action are not necessarily explicit in the implementation. Basically, goal plays the same role as perception, by limiting available actions. Perception limits the actions according to the state of the environment, and goal limits the actions according to agent’s preferences. Since action itself plays the role similar to perception, the distinctions may disappear altogether. From this perspective, agent operates only through high-level perception that is initiated by sensory input, biased by agent’s goal and translated into low-level action output. This picture doesn’t help in terms of clear semantics of agent’s operation, but it is useful to see how the fundamental building blocks of the agent may blur into each other in some implementations, and also in identifying these building blocks in implementation that doesn’t explicitly contain them (such as human brain).