Learning factored representation

Followup to: Balancing context with conceptual slippages, Summarizing structure in new labels, Independence of patterns.

Repeated contexts and context transitions become compressed over use, losing variability in their compressed form. Any distinguishing characteristics of particular instances of such repeated contexts can be extracted as separate properties. Commonalities get compressed in a central pattern, and variations become properties of that central pattern. For example, typical objects, such as cups, have certain common characteristics, but properties of a particular cup can be expressed as additional patterns showing where it differs from typicality.

Central pattern of an object extracts mutual information from features describing the object, and as a result remaining patterns of object properties become more independent from each other, given the object pattern. A change in one property of an object doesn’t usually call for changes in other properties, and if it does, the dependent properties should probably again be summarized by a new single property. Individual slippages of object properties don’t affect most of the scene.

Resulting representation shouldn’t be strictly hierarchical, as limiting the representation to a hierarchy significantly reduces its expressive power. Center of a natural category can consist of a collection of interfering patterns, encoding the object’s structure and instantiated depending on context, whereas more rare characteristics are much more independent, given any compatible state of the object’s center.

Learning factored representations of transformations of the scene may result in formation of procedural patterns, with the center of transformation becoming procedure itself, and peripheral variations in transformation’s properties becoming arguments of the procedure.

Independence of patterns

Followup to: Structural representation of uncertainty, Interference of patterns

In a given scene, two patterns are called independent, if changes in one of them don’t lead to changes in another, if they don’t interfere directly or through short enough sequence of changes in the scene. Independence is conditional on context, so two patterns can be independent in one scene, but not in a different scene, and a change to a third pattern can make them interfere.

Interference makes scene a whole, connects its parts, translates the presence of additional patterns into influence on behavior of existing patterns. Independence allows modular composition of elements of the scene, “keeps everything from happening all at once”. It could also be fundamental to scalable implementation, since it makes interactions between patterns local on each given step.

Groups of patterns, where patterns in each group are mostly independent of patterns in other groups, can function in parallel, so that the whole processes in inference within each group are independent from other processes. Such configurations could be used to compute answers to subproblems, to divide a bigger problem on a collection of smaller ones (using procedural patterns), or to model a bigger system by a collection of models of its parts.