EXTRACTING DISCRIMINATING FEATURES FROM CONTEXTUAL DOMAIN KNOWLEDGE FOR ENHANCING TOPIC DISCOVERY OF ONLINE FORUMS

Abstract

Topic models are popular in discovering latent topics from document corpora. Traditional topic models such as Latent Dirichlet Allocation (LDA) are usually fully unsupervised, which are often found to generate ambiguous or incoherent topics based on human evaluations. Many extensions of topic models have been proposed to address this issue by incorporating prior knowledge into topic model to guide the topic discovery. Such prior knowledge can usually be defined as the discriminating words or cannot-links between two words indicating that these words should not be put into the same topic.

However, discovering the domain prior knowledge is not an easy job. Common practice usually requires domain experts to manually define the discriminating words, which require laborious efforts and may even fail to fit the data closely. In this thesis, we explore the possibility to automatically extract discriminating words from relevant informal domain knowledge. For example, in epidemics online course forum, the discussion is usually related to the syllabus of the course. The course syllabus can serve as a source of informal domain knowledge that contains a structural information of topics and reflects the high level knowledge of the forum. We can call this kind of contextual domain knowledge as domain contexts.

To extract the discriminating words from the domain contexts automatically, we first extend the information distance measures to discover discriminating words with respect to the domain contexts. Specifically, we introduce KL Distance and Loss of Mutual Information to determine the most discriminating words that should be separated.

Using these discriminating words, we developed an agglomerative algorithm to construct discriminating feature sets, which can be easily encoded as the cannot-link constraints to improve topic models. Then we incorporate the constraints into a state-of-the-art Multi-generalized Po´lya Urn model to extract topics. Finally, we introduce a technique to discover the relationships between the discovered topics and the original domain contexts by mapping the top-k topics to each domain context elements. Our experiments on three large forum data sets show that even without any user input, it is possible to extract meaningful discriminating words automatically and discover coherent topics.

Chapter 1 Introduction

As Web 2.0 applications developed, we have witnessed a lot of popular online social medias (such as forums, news groups, blogs, etc.) emerge and thrive. Among these technologies, online forums are a very unique type of information exchange platform. Typically, online forums have a particular set of topics of interest where users can share information or ask questions in a thread structure. Users can also comment or reply to the thread to respond and communicate to the each other. In such an environment, analyzing the topics in the discussion is usually a very interesting and important task. For example, in a discussion forum of a Massive Open Online Course (MOOC), students can ask questions and help each other to better learn the course. Sometimes, it is even beneficial for the teachers to know the topics of those discussions so that they can address those questions in the following classes or reply to the thread.

Traditionally, researchers have utilized topic models such as Latent Dirichlet Allocation (LDA) [8] and Probabilistic Latent Semantic Analysis (pLSA) [25] to discover hidden topics in text documents. Many extensions [6, 34, 52] of LDA and pLSA, are introduced during the past decades and showed their values. However, these traditional topic models often tend to mingle several different concepts with close meanings into a single topic. This phenomenon gets worse when the scope of the data is a narrow topic itself. As a matter of fact, most unsupervised topic models may not be able to produce coherent topics that are consistent with human judgments[11] .

To help topic models to create more meaningful topics, there have been a lot of studies that include auxiliary domain prior knowledge into topic models, either by incorporating the authorship of the documents[48] , label and tags [47] , syntax[9] , or document classes[7] . Similarly, some knowledge-based models allow users to define domain prior knowledge as constraints to regulate the topic generation process[30, 2, 29] .

However, few of the above methods can leverage the contextual knowledge of online forums to improve topic qualities. Unlike other conversational texts, the topics in online forums are usually narrow and highly focused on a certain domain, which brings unique challenges to topic modeling. For example, in an epidemics course forum, it is often difficult to distinguish different topics in such a narrow domain where almost all topics are about infectious diseases. Fortunately, some domain contexts such as the course syllabus that correlate with the topics of online forum discussions are often available. These domain contexts are usually created by domain experts to reflect the structure of the related domain, which has high information value. Therefore, it is our goal to leverage valuable domain knowledge from the domain contexts to enhance topic models.

Consider the following real world example in an epidemic course forum. The syllabus of the course outlines eight weeks of the course materials. The first week of the course provides an overview of epidemics with key words including ”pathogen”, ”disease” and ”virus”. The following weeks focus on various aspects such as ”hosts”, ”vaccination” and ”disease control”. However, we observe that the topics generated by LDA often fail to separate different concepts and form mixed topics such as {disease, host, virus}, {vaccination, control}, which does not help the lecturers and students to better understand the forum structure. It is expected that the generated topics are more fine grained and have specific meanings. Furthermore, only after we correctly discover the topics that correlate with the course content, we can then find emerging topics in the forum that are beyond the scope of domain contexts, which could help to facilitate discussions of these topics in the future.

The above example inspired us to construct some constraints in topic models and create a separation of certain key words. For example, we may construct a set of discriminating features ”{disease, host, vaccine, control}” to specify that any two of these features should not appear in a same topic. In this way, the generated topics could be more focused on a sub-domain. Similar constraints has been proven to be effective in the literature [2, 60, 15] .

Although incorporating constraints into topic models has shown its potentials, obtaining the proper domain prior knowledge is usually not an easy job. Common practices require domain experts to define the constraints manually [2, 30, 3, 60] , but it takes laborious work from the domain experts and is hard to scale. It becomes even harder to acquire prior knowledge when dealing with incremental data and evolving topics.

In this thesis, we propose to extract domain constraints from domain contexts automatically using information distance measures, which consists of four steps:

First, to learn whether two words should be separated into different topics, we propose to use a consistent information distance measure to extract the discriminating features. Clearly, learning the distance without the contexts of a domain could be mistaken especially in a narrow domain. We explored the information measures Kullback-Leibler Distance and Loss of Mutual Information to learn the information distance between words, using the domain contexts as input. Intuitively, by conditioning the information distance on domain context, the proposed measures can quantify the differences of probability distribution of domain contexts induced by the word pairs, which can be used to discover the discriminating words that are the most representative on different concepts by maximizing the information distance. This is further explained in chapter 4.

Second, while simply considering the maximum information distance between word pairs can be useful, it will also give us a lot of redundant information. Imagine if we want to distinguish the concepts in a syllabus for 10 weeks of a course, and if we choose one representative word from each week, we will end up with C(10,2) = 45 different combinations between these word pairs, which is difficult for human to interpret and adjust. To tackle this problem, we introduce discriminating feature sets such that each word pairs within each set are mutually discriminating. We introduce an agglomorative algorithm based on the idea of abstract features[31, 61, 50] to construct discriminating feature sets greedily. Such domain constraints can be easily incorporated to split discriminating words into different topics and enhance topic modeling.

Third, we leverage a state-of-the-art Multi-generalized Po´lya Urn Model (M- GPU)[12] to incorporate the extracted domain constraints into topic models. M-GPU model is similar to LDA but uses a simple logic during the parameter inference process. When a word is assigned to a topic, then the model will try to throw the words that shares a cannot-link with this word out of this topic and put them into the topics that are more suitable. With this mechanism, we can safely incorporate the domain constraints as cannot-links into topic models.

Finally, we will need to extract the relationships between the derived topics and domain contexts. For example, when the teachers of the Epidemic forum provided the syllabus and derived a more clearly separated topical structure, they may want to know that which topic is correspond to which week in the syllabus and whether there are some emerging topics that the students are interested in but not taught in class. This mapping relationships can be extracted with a simple Gibbs sampling technique. We propose to use a simple Gibbs Sampling technique to inference the distribution of topics in each domain context element. And we select the top five topics for each domain context element to create a link and derive the mapping relationships.

In summary, this paper makes two main contributions:

• It leverages domain contexts for online forums to enhance topic models. To our knowledge, this is the first time that documents with small size but high information value are being used to enhance topic modeling.
• It employs information measures to automatic discover discriminating features that needs to be separated into different topics, which is not well studied before. This allows us to further construct discriminating feature sets where any pairs of features are mutually discriminating.

Our experiments on epidemics course forum and breast cancer forum show that the propose method can effectively discover domain constraints as discriminating feature sets and can generate significantly more coherent topics.

EXTRACTING DISCRIMINATING FEATURES FROM CONTEXTUAL DOMAIN KNOWLEDGE FOR ENHANCING TOPIC DISCOVERY OF ONLINE FORUMS

Sharing is caring!