The Hierarchical Temporal Memory (HTM) theory of cognition

In the article Why Can’t A Computer Be More Like A Brain? (spectrum.ieee.org) author Jeff Hawkins proposes a theory of how the brain works and how it could be implemented in computers:

“Memory of what a dog looks like is not stored in one location. Low-level visual details such as fur, ears, and eyes are stored in low-level nodes, and high-level structure, such as head or torso, are stored in higher-level nodes.the low-level nodes learn first. Representations in high-level nodes then share what was previously learned in low-level nodes.

each node in the hierarchy learns common, sequential patterns, analogous to learning a melody. When a new sequence comes along, the [lower level] node matches the input to previously learned patterns, analogous to recognizing a melody. Then the [lower level] node outputs a constant pattern representing the best matched sequences, analogous to naming a melody. Given that the output of nodes at one [lower] level becomes input to nodes at the next [higher] level, the hierarchy learns sequences of sequences of sequences.”

A graphic (PDF) from the article demonstrates the point:

What the article sort of talks about is the intrinsic role of time based feedback loops as part of how a node recognizes a pattern. I’m guessing that rather than process a wide bit pattern directly, “nodes” process the wide bit pattern in chunks remembering the node’s state based on the previous sequence of chunks and determining the new state based on the previous state and the current new chunk (and maybe neighboring/parent node states)

Author: Ben Slade

I'm a software technologist with a political bent. My views tend toward the contrarian and slightly curmudgeonly end of the spectrum.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.