Episodic and semantic memory

Followup to: Fragments of structure, Summarizing structure in new labels.

In this model, structure restoration can be interpreted as remembering, and context balancing as focusing attention on contextually relevant facts and memories. Depending on the character of restored structure and restoration process, some memories can be considered episodic or semantic. Episodic memory restores a significant part of a single past scene, allowing to situate the restored structure relative to known locations and times. Semantic memory plays out a semantic rule of thumb, filling in a property that can be discerned from the context, and even though this property can have complex structure, it isn’t associated with a particular past scene.

Before an episode is first recalled, the pattern of that episode is unique in the memory. This theoretically allows to recall every single bit of the old episode, there is no ambiguity in the details, for example if an unique label belonging to that episode gets triggered by a cue. But once a part of an episode gets recalled, its content is associated with two episodes: the original one, and the episode of recall. The recalled part, or episodic memory, becomes a weaker cue for the parts of the episode that were not recalled the first time. After many recalls, the episodic memory that gets restored in the context of recall is formed more as a reconstruction of previous episodes of recall, than of the original episode. The details that are not usually recalled get forgotten, and the details that by some reason get distorted during recalls stay distorted in the subsequent recalls. These effects, following from simple considerations about associative memories, are known to occur with human memory, as retrieval-induced forgetting and memory distortion, and can be mimicked by very simple models.

The first episode in agent’s experience to which a rule encoded in semantic memory applies, plays the same role as the original episode of episodic memory. The only difference is in what kind of content gets the emphasis during the recall. Episodic memory retains the relations to many details, even as retrieval-induced forgetting tries to shut them out, with rarity of recall events helping the matter, while semantic memory focuses on few properties and gets applied over and over. This allows to view semantic memory as a special case of episodic memory, that through many cases of recall abstracted out all of the episode-specific details, leaving only what’s usually important in the contexts of recall.

When a fact is learned declaratively, there is an episode in which it’s first stated, but the details of that episode are irrelevant to the fact itself. When the fact is recalled in the future, the recalling process can stop on the fact, without going into the details of the episode in which the fact was learned, even though those details are available. The limited part of episodic memory becomes semantic memory. Alternatively, a fact can be abstracted out as a regularity present in many scenes, without ever being a part of episodic memory, as the only unambiguous inference that follows from cumulative memory of many past episodes.

Another interesting effect is that not every episode can form an episodic memory. If an episode is so ordinary that no cue can uniquely point to it, there is no way to recall it as an episode. When you commute to work a hundredth time, you don’t usually pay attention to details, each action results from a known rule, there is nothing to learn from the process. New memories capture reusable novelty, contexts that are expected to repeat sometime in the future, but did not appear in the past.

Summarizing structure in new labels

Followup to: Structural representation of uncertainty, Continuous balancing of changing structure.

Labels allow to represent states of knowledge in the most compact form. Expressive power of structural contexts and whole scenes allows to construct representations of elaborate combinations of previously encountered states of knowledge. These more complex representations are harder to manage than simple labels, and so when certain structural pattern (or map) becomes common enough, it can be assigned a new unique label of its own.

New labels allow to compactly represent frequently encountered states of knowledge, to form a language adapted to the environment. Multiple generations of labels can represent the most salient aspects of bigger and bigger structures in the scene. Smaller representation allows more robust processing. Structure restoration can function with fewer errors because structures that need to be restored become smaller. Maps can capture more global concepts in the scene without needing to consider more labels at the same time, because common combinations of labels are summarized by new labels. New labels form a basis for new levels of representations for structures that are already represented in the scenes.

A label by itself is good for nothing: if it isn’t in any map, it’ll never get restored. If a new label appears once in an ordinary context, it’ll never be restored again, because this context is matched by old maps better than a new map that also includes a new label.

A new label can get remembered if it appears in a novel structural context consisting of old labels. New maps that capture this new context can be restored in the future by right combinations of the old labels, and as a result restore the new label. From now on, the new label appears in all scenes containing this novel structural context, at the same time new maps representing this context do. As a result, it becomes possible to represent this context and associated maps just by the new label, and this label gets learned by other maps to reflect the presence of regularity represented by it. It’s no longer owned by the context in which it was bootstrapped and with which it was originally associated, and in the future it can even get completely disassociated from it, gradually shifting its semantics elsewhere.

Thus, it’s unnecessary to create a separate algorithm to manage new labels and rigidly assign them to maps they are supposed to represent, map learning takes care of it. It’s sufficient to create a new label for each new map, and if it turns out to be useful, it’ll get learned by other maps. Relatively useless labels get abstracted out of maps, the same way relatively useless maps get discarded or merged with other maps.