Laypeople say that two people can “see” the same thing — an abstract painting or a cloud — differently. Cognitive scientists like newly appointed assistant professor Joseph Austerweil must be more precise. What really differs, they note, is the “representation” that each individual forms in the brain in response to that visual stimulus. In his research, Austerweil is modeling that process with the especially precise rigor of statistics so that he can develop theory and testable hypotheses about how people construct interpretations of what they see and think about.

The way people arrive at these representations in cases of uncertainty is an important basic research question in psychology that also has applications in education. It is not limited to literally nebulous images. In a small, basic, and quite telling experiment, Austerweil played on how the symbols for addition and multiplication vary only by 45 degrees of rotation.

In the experiment with 20 online volunteers, Austerweil presented what on its own would appear to be the multiplication symbol. He arranged the operator between two numeral fives in a 45-degree diagonal. Half the participants saw the fives straight up, while the other half saw them rotated along with the diagonal. He then asked each group to solve the math problem. Their answer indicated which operator they used to represent the multiplication symbol. By answering “25” a participant indicated a representation of the symbol as the multiplication operator, while an answer of “10” indicated a representation of addition.

The answers depended strongly on whether those fives were rotated. Arranging the symbol in a diagonal with straight up fives led seven in 10 people to represent it as multiplication, but when the fives were rotated, 9 of the other 10 people took it to be addition.

“I was surprised at how strong the effect was and how well it worked,” Austerweil recalled.

What the experiment makes clear is that people don’t develop representations solely based on the object’s features, even when they have prior experience with the object. Instead, they represent an object based on its context; in this case, the context of nearby objects.

“Cognitive models must account for the fact that the representation of an object can flexibly change depending on the context, prior knowledge, and experience,” Austerweil said. “To do this, people must be using prior knowledge to distinguish between the multiple representations consistent with an object and its context.”

Austerweil uses probabilistic models — Bayesian, to be exact — because that requires explicitly specifying the knowledge that people use to solve problems that have more than one solution.

Austerweil is adding other factors to his model, too. An example is that previously learned features can influence how an object is represented. Take the white lettering in the image at left: One representation is that it’s a somewhat skewed “M” with a hockey stick; another is that it depicts a smooshed-together “N” and a hockey-stick shaped “Y” (the designer’s intent). Showing someone the “N” first and the “Y” second and the whole image third makes it more likely that the observer will infer the intended representation. Another way to make this more likely is to give them the appropriate context, which in this case, is that it’s the logo for a New York ice hockey team.

What’s useful about a mathematical model of a psychological phenomenon is not only that it formalizes knowledge, but also that it can make specific predictions that can in turn generate ideas for behavioral experiments, Austerweil said. It also allows for productive translations back and forth between human behavior and machine learning, which was Austerweil’s original passion as an undergraduate at Brown.

Austerweil, whose appointment will be in the Department of Cognitive, Linguistic, and Psychological Sciences, came to Brown in 2003 to concentrate in computer science. Under Eugene Charniak he studied how computer software could formulate a coherent ordering of sentences, given a set of sentences, but he couldn’t help but notice how much better people were at the task. Learning how people work could inform machine learning, which could in turn inform psychology.

“I started thinking about the types of problems people try to solve using machine learning and I started thinking, ‘Well we solve these problems seemingly effortlessly and if we’re so good at it, why? How do we do it?’” he said. “How can we make computational systems that also do that in the same sort of ways and how can we use that to better understand how we solve these problems?”

His studies were influenced by Thomas Griffiths, a former Brown psychology professor. When Griffiths left Brown for the University of California–Berkeley, Austerweil followed for a masters degree in statistics and then a Ph.D. in psychology.

Since January 2013, Austerweil has been a postdoctoral researcher at Stanford, where he recently began a collaboration to learn about and model how people construct representations of events. He is also working with a collaborator in England on a separate project to study how people with autism form representations.

But when he saw a fateful posting for a new faculty member, he decided to come back to Brown.

Austerweil said he fondly remembers Brown as a place where he felt encouraged to focus on learning and exploring new ideas, rather than competing and stressing over grades.

Not coincidentally, Austerweil’s return to the classroom will be to teach one of the last classes he took here, “CLPS 1200 — Thinking.” According to the bulletin, it’s a course where “the focus is on the relation between empirical evidence, theories, and models of cognitive process and structure.”

Based on his research, that’s a subject that Austerweil can “represent” well.