Modeling virtual organizations with Latent Dirichlet Allocation: A case for natural language processing

Gross, Alexander and Murthy, Dhiraj. 2014. Modeling virtual organizations with Latent Dirichlet Allocation: A case for natural language processing. Neural Networks, 58, pp. 38-49. ISSN 0893-6080 [Article]

Modeling virtual organizations with Latent Dirichlet Allocation_Final_After_Review.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (700kB) | Preview

Abstract or Description

This paper explores a variety of methods for applying the Latent Dirichlet Allocation (LDA) automated topic modeling algorithm to the modeling of the structure and behavior of virtual organizations found within modern social media and social networking environments. As the field of Big Data reveals, an increase in the scale of social data available presents new challenges which are not tackled by merely scaling up hardware and software. Rather, they necessitate new methods and, indeed, new areas of expertise. Natural language processing provides one such method. This paper applies LDA to the study of scientific virtual organizations whose members employ social technologies. Because of the vast data footprint in these virtual platforms, we found that natural language processing was needed to ‘unlock’ and render visible latent, previously unseen conversational connections across large textual corpora (spanning profiles, discussion threads, forums, and other social media incarnations). We introduce variants of LDA and ultimately make the argument that natural language processing is a critical interdisciplinary methodology to make better sense of social ‘Big Data’ and we were able to successfully model nested discussion topics from forums and blog posts using LDA. Importantly, we found that LDA can move us beyond the state-of-the-art in conventional Social Network Analysis techniques.

Item Type:


Identification Number (DOI):


Natural language processing; Latent Dirichlet Allocation; Big Data; Social media; Virtual organizations

Departments, Centres and Research Units:



October 2014Published
1 June 2014Published Online

Item ID:


Date Deposited:

16 Jun 2014 08:40

Last Modified:

21 Apr 2021 15:03

Peer Reviewed:

Yes, this version has been peer-reviewed.


View statistics for this item...

Edit Record Edit Record (login required)