Followup to: Representation of structure.
Let’s say, we have a set of known scenes of similar nature, given in graph representation. The task is to reconstruct the missing properties in a new scene, based on known scenes.
On a scene graph, let structural context of given node be a local subgraph centered on that node, that includes only nodes and links within a certain small distance from it. One simplifying approach to reconstruction of missing properties is adding nodes and edges locally and incrementally, looking only at one structural context at a time (it requires some assumptions about the scene graphs). If current scene is extended locally, we only need to look at the local structural contexts of known scenes. The algorithm, roughly, would select a node in the current scene, and compare its structural context to all structural contexts of all known scenes. If a certain subgraph is encountered in sufficient portion of sufficiently similar structural contexts, algorithm inserts that subgraph in the current structural context, connecting it to the rest of the structural context in the same way it was connected in known scenes.
This process is basically inference. Known scenes establish the rules of thumb, and new incomplete scene is an initial focus of attention, collection of facts to be built upon according to known rules of thumb. Structural contexts limit the complexity and sensitivity of each individual inferential step. At this point, nothing fades away or gets repaired, elements are only added.
Big complicated structures from the past scenes can be restored and adaptively connected to existing scene using multilevel representation. Adding high-level description of the structure sketches its new location, and then gradually details can be elaborated, all within structural contexts, with details at each level of description connecting to corresponding details in the current scene at the neighboring levels of description. Each new detail adds to the context for selection of the next detail. To avoid local inconsistencies, where top-down rebuilding of new structure starts going the wrong way and comes to a halt in inconsistent state, sufficient amount of high-level representation is required, binding details of the scene together, establishing robust structural framework. This is one of the assumptions for the format of scene graphs.
The process of structure restoration can also be seen as growth of structure at the boundaries of the specified scene, according to the “natural” way structure used to grow in the past scenes. Would the scene be a simple regular crystal, there would be a single rule to be applied repeatedly to each shallow layer of the structure. In a real-world scene graph, growth simultaneously happens of multiple levels, which guide each other according to many established rules. Growth happens locally, seeing limited amount of structure on each step, but acts on many levels, which have different notions of locality. High-level representation gives general guidance, while low-level representation captures the little details, allowing the new structure to naturally extend beyond the boundary, instead of being crudely fastened on top.
