That scream of horror and embarrassment is the sound that rationalists make when they level up.
I wasn’t updating this blog since February, and the reason for that is that I understood a couple of things that prompted a change in the perspective on how to do and communicate research, as well as the direction of research.
What I’ve been doing before was top-down formalization of intuition, something akin to philosophy: start with a vague idea about a phenomenon, and then iteratively clarify it, step by step rendering parts of the idea more explicit, in turn using the clearer understanding to train intuition, and so on. Throughout the process, there are almost no stand-alone technically understood components, everything is only held together in the mind. The intermediate product of this process is a set of mental tools allowing to better understand the phenomenon under study.
There is a number of related difficulties to this approach. As most of the concepts are fuzzy, there is a temptation to neglect epistemic hygiene. This shows in attempts to cover inferential distances with explanations that misuse technical terms, saying only something similar to the truth, but not really true, as it’s easier this way. This plagued the first sequence I ran on the blog, in June-July 2008, with term “probability”. What it really takes to describe a complex idea that you don’t yet understand technically is probably a book-length description, that won’t be an easy read either (with much of philosophy being the primary example). More importantly, it’s easier to engage in sloppy thinking, creating the illusion of progress while going in circles, and to start chasing lost purposes, solving problems that don’t need to be solved.
While research is informed by both facts and tools from the literature, in the “fuzzy” mode there is really very little that generalizes to something helpful on a not-directly-related problem. The most helpful thing is the methodology, a set of tricks for managing concepts as they develop, separating meaningful ones from the trivial, grounding in the existing body of science, and so on.
What I discovered when I started to look into the mathematics on topics related to intelligence (machine learning, graphical models, decision theory, game theory, formal semantics, logic, model checking, etc.) is that the intuitions forming in the mind once you understand these topics are vastly superior to those I was able to gather before, both from reading “fuzzy-grade” research (descriptions of “ad-hoc” AI approaches, neural nets, cognitive science, neuroscience), and from developing my own structures. At that point, I was down this “fuzzy” path for about year and a half, starting from no knowledge in the related fields; the material I described on the blog is what I constructed in the first half a year, a year before writing it up, since reduced to a kind of recurrent neural networks, with experimental implementations and so on. It took only a couple of months to comprehend the hands-down superiority of math, even for the ideas that aren’t reduced to math yet.
And then I saw that the problem I was solving doesn’t really develop in the direction of Friendly AI (FAI), that all my previous activity was mostly a lost purpose, apart from educational value. I was acting from a vague idea that understanding AGI is a step in the direction of understanding FAI, since FAI is a kind of AGI. This idea turned out to be misguided for a number of reasons, that should become clear from the following posts.
I leave the existing posts be, despite not really approving of them, and will resume blogging here.
Posted by Vladimir Nesov 