Phillip Wolff

Phillip Wolff is a Professor of Psychology at Emory University. His research focuses on the use of language semantics and machine learning to predict human thinking and mental health. Additional areas of interests include causal reasoning, future thinking, intentionality, and cross-linguistic semantics. Research in his Concept Mining Lab investigates how machine learning and analyses of language can provide insights into human thinking and mental health. The lab pursues research examining how Natural Language Processing and Big Data analyses can, in effect, data mine the mind.

Interview

‍‍

OASIS: Thanks for sitting down to talk with us. What's changed for you in your field since you started? ‍

I think right now the field has really achieved what G.W. Bush wanted, which was that the NIH decided that everything has to have a purpose and it had to address some practical problem. And it had to provide a concrete solution. It makes it harder to justify projects that are focused more on understanding the nature of the cognitive and linguistic systems. 

‍OASIS: Who are your professional heroes?

PW: I can think of one person, Geoff Hinton. He's been this person that has been like the second and third author on important papers, seemingly in the background. And you start to see over the length of a career that he may have been the secret sauce to a lot of other people's shining moment. That modesty, that humility, impresses me, and I'm really happy he won the Nobel. 

‍OASIS: Oh yeah, I saw some physicists complaining that it wasn't physics. 

PW: Yeah, it wasn't! He's someone who kept pursuing what he thought was the right answer and  no one was with him for many years. So he had that confidence and certainty. It led at the very least to a provocative set of outcomes in the form of these large language models. ‍

OASIS: What would you like to tell students or early career researchers?

PW: There's going to be this tension. You want to make your unique mark, something you can say is yours. But at the same time, the problems have gotten pretty difficult at this point for single players to make a dent. And you have to find a partner that's doing more of the modeling side of things. And if you're a modeler, you definitely need someone who understands the content area. I'd be on the lookout for that person that could complement you because I think at this point it's too challenging, especially if you want funding.‍

OASIS: What is the strangest hill you would die on?

PW: People flirt with the idea of the end of theory. The problems are going to be solved with modeling methods that are driven by data, and the outcomes will be those we don't really understand, and we need to let go of the idea that we need to understand it. This approach is contrasted with the one where people come up with a theory and then, if you will, look for the confirmation empirical data. Now, in that situation prediction was dependent on theory. But we don't need that anymore. 

But I'm not going to give up on theory. So the hill is, we should pursue grand theories even of cognition and of language and in philosophy, even though we now have the capability to achieve predictions without understanding how. We can re-express what was achieved in a data-driven way in terms that look like a theory. I still believe there's a calculus of meaning composition!

So I think the data-driven approaches are awesome and we ought to pursue them. Because they will help us develop those theories. And I think especially for cognition and for language. One reasonably might ask, well, why do you need theory, really, other than maybe it's just a habit of training. It's just in my experience that the reason it's important is because if you use a data-driven approach, the project ends, if you will, at the point when you do pretty good prediction. And then there's nothing more to do almost.‍

OASIS: Aren't these models called "generative"? Shouldn't they generate what's next?

PW: Yes, but in terms of our own efforts, like what do you do next? Why do you do it? Theory is pretty key to filling in or explaining why you should do it. It's maybe more for us, to help guide our efforts. But with only a black box, you have a system that does the prediction, a calculator that does the prediction, what do you do next?‍

OASIS: What do you feel optimistic about? 

PW: A long-term motivation or question was how do you get mind out of matter? Like cognition or emotion or certain behaviors, out of physical processes in neurons? That has always struck many people as really beyond what we'll ever achieve, but now I think we're coming closer to a solution. These larger systems are giving someone like me more confidence that this is something that might be solvable. So it's moved from being a total mystery to being a potential project. 

Another thing that's kind of cool is that we have systems now that can do a trillion operations per second, and the human brain is estimated around 60 trillion connections given the number of neurons. But computers' compute capabilities are approaching or maybe have already met the human compute potential. And that's a pretty big landmark. And that's happened roughly around now, probably. So I'm excited about that. 

And if that's true, then the limitation, like being able to understand the system is no longer a problem. We have the compute, we just don't have the science yet. And I think that I'm excited by that. We are in a new period. We have enough processors to test our theories, but we don't have the science yet. 

We know these systems are not learning like people learn. So what could we add to them that would help make that difference? To give an example, I'm intrigued with how the language system is laid out.in both sort of at a theory level as well as sort of like at the cortical level. If we look at it in terms of cortex, it looks as if let's say areas relevant to verb processing are close to areas relevant to the processing of motion and spatial relations.

That observation is very compatible with linguistic proposals about what many verbs mean.And so it raises, I think, some really cool ideas or possibilities that the human system is really set up to take advantage of this non-linguistic hardware. You might even call it parasitic. But that's a kind of negative! But it like it's glomming on to structures that were there prior to language.

And those structures are imposing pressures on the learning system. That maybe could accelerate the learning process or enable it. And so, the future looks exciting in this regard, because right now the large language models are not designed with this kind of architectural constraint built in. And if they were, we could see really different kinds of performance.‍

OASIS: Thanks for that optimistic outlook. We're looking forward to hearing your talk at OASIS 4. 

PW: Thanks for the invitation, I'm looking forward to it too.

‍

‍