Some thoughts on Pearl 2019 + Dunbar 2019

I think these two commentaries (mine and Dunbar’s) pair together pretty nicely -- my key thought can be summed up as “if we can interpret neural networks, maybe they can build things we didn’t think to build with the same pieces and that would be cool”; Dunbar’s key thought is something like “we really need to think carefully about how interpretable those networks are…” So, we both seem to agree that it’s great to advance linguistic theory with neural networks, but only if you can in fact interpret them.

More specific thoughts on Dunbar 2019:

(1) Dunbar highlights what he calls the “implementational mapping problem”, which is basically the interpretability problem. How do we draw “a correspondence between an abstract linguistic representational system and an opaque parameter vector”? (Of course, neurolinguists the world over are nodding their heads vigorously in agreement because exactly the same interpretability problem arises with human neural data.)

To draw this correspondence, Dunbar suggests that we need to know what representations are meant to be there. What’s the set of things we should be looking for in those hard-to-interpret network innards? How do we know if a new something is a reasonable something (where reasonable may be “useful for understanding human representations”)?

(2) For learnability:  Dunbar notes that to the extent we believe networks have approximated a theory well enough, we can test learnability claims (such as whether the network can learn from the evidence children learn from or instead requires additional information). I get this, but I still don’t see why it’s better to use this over a symbolic modeling approach (i.e., an approach where the theory is transparent).

Maybe if we don’t have an explicit theory, we generate a network that seems to be human-like in its behavior. Then, we can use the network as a good-enough theory approximation to test learnability claims, even if we can’t exactly say what theory it’s implementing? So, this would focus on the “in principle” learnability claims (i.e., can whatever knowledge be learned from the data children learn from, period).