One-sentence SOP: I want to build computational models that understand human’s language in an explainable and intelligent way.

Discourse

Discourse concerns the organization of text in a coherent and logical manner. It is a perfect discipline to study language understanding because diverse linguistic devices exist and complex reasoning is required.

The problem is decomposed as:

• Can we find a computational proxy for understanding discourse? Despite the strong performance, pretrained language models (PLMs) are said to NOT understand the language from text alone. But can we say certain understanding is achieved when the models behave like human?
• Intermediate guidance for better understanding? PLMs are also prone to adversarial input and spurious features. Can we guide the model with human’s intelligence about decision-making as intermediate input?
• Neurons for discourse functions? Can we causally estimate different components in PLMs that are responsible for different discourse functions?
• Better pretraining strategy? Popular PLMs (e.g. BERT) are trained in a [mask] way ([mask]=(simple, stupid, …)). Are there better pretraining strategies to make it understand the language better? Can the PLMs self-improve?

We have a paper under submission. Please contact me if you’re interested in reading it!