Interactive Visualization of Topic Models

Tools to create an interactive web-based visualization of a topic model that has been fit to a corpus of text data using Latent Dirichlet Allocation (LDA). Given the estimated parameters of the topic model, it computes various summary statistics as input to an interactive visualization built with D3.js that is accessed via a browser. The goal is to help users interpret the topics in their LDA topic model.


R package for interactive topic model visualization.

LDAvis is designed to help users interpret the topics in a topic model that has been fit to a corpus of text data. The package extracts information from a fitted LDA topic model to inform an interactive web-based visualization.

  • Stable version on CRAN:
install.packages("LDAvis")
  • Development version on GitHub (with devtools):
devtools::install_github("cpsievert/LDAvis")

Once installed, we recommend a visit to the main help page:

library(LDAvis)
help(createJSON, package = "LDAvis")

The documentation and example on the bottom of that page should provide a quick sense of how to create (and share) your own visualizations. If you want more details about the technical specifications of the visualization, see the vignette:

vignette("details", package = "LDAvis")

Note that LDAvis itself does not provide facilities for fitting the model (only visualizing a fitted model). If you want to perform LDA in R, there are several packages, including mallet, lda, and topicmodels.

If you want to perform LDA with the R package lda and visualize the result with LDAvis, our example of a 20-topic model fit to 2,000 movie reviews may be helpful.

LDAvis does not limit you to topic modeling facilities in R. If you use other tools (MALLET and gensim are popular), we recommend that you visit our Twenty Newsgroups example to help quickly understand what components LDAvis will need.

To share a visualization that you created using LDAvis, you can encode the state of the visualization into the URL by appending a string of the form:

"#topic=k&lambda=l&term=s"

to the end of the URL, where "k", "l", and "s" are strings indicating the desired values of the selected topic, the value of lambda, and the selected term, respectively. For more details, see the last section of our Movie Reviews example, or for a quick example, see the link here:

http://cpsievert.github.io/LDAvis/reviews/vis/#topic=3&lambda=0.6&term=cop

To read about the methodology behind LDAvis, see our paper, which we presented at the 2014 ACL Workshop on Interactive Language Learning, Visualization, and Interfaces in Baltimore on June 27, 2014.

We included one data set in LDAvis, 'TwentyNewsgroups', which consists of a list with 5 elements:

  • phi, a matrix with the topic-term distributions
  • theta, a matrix with the document-topic distributions
  • doc.length, a numeric vector with token counts for each document
  • vocab, a character vector containing the terms
  • term.frequency, a numeric vector of observed term frequencies

We also created a second data-only package called LDAvisData to hold additional example data sets. Currently there are three more examples available there:

  • Movie Reviews (a 20-topic model fit to 2,000 movie reviews)
  • AP (a 40-topic model fit to approximately 2,246 news articles)
  • Jeopardy (a 100-topic model fit to approximately 20,000 Jeopardy questions)

News

CHANGES IN LDAvis VERSION 0.3.2

BUG FIX

In some cases, the widths of the red topic-term bars did not decrease (as they should have) from term #1 to term #R under the relevance ranking with $\lambda = 1$. In other words, when $\lambda = 1$, there were topics in which a narrow red bar was displayed above a wider red bar, which should never happen. The issue had to do with the way topic-term bar widths are computed, and is discussed in detail in #32.

In the end, we implemented a quick fix in which we compute term frequencies implicitly, rather than using those supplied in the createJSON() function. The upside is that the red bar widths are now explicitly controlled to produce the correct visualization. The downside is that the blue bar widths do not necessarily match the user-supplied term frequencies exactly -- in fact, the new version of LDAvis ignores the user-supplied term frequencies entirely. In a few experiments, the differences are small, and decrease (as a proportion of the true term frequencies) as the true term frequencies increase. Thanks to @bmabey for leading us to this bug.

CHANGES IN LDAvis VERSION 0.3.1

BUG FIX

  • See #33

CHANGES IN LDAvis VERSION 0.3

NEW FEATURES

  • LDAvis output can now be integrated in shiny apps and interactive rmarkdown documents using the functions renderVis & visOutput.

CHANGES IN LDAvis VERSION 0.2

MAJOR CHANGES

  • The function runShiny was deprecated. Please use createJSON to create visualizations instead.

    CHANGES IN LDAvis VERSION 0.1

NEW FEATURES

  • first version of LDAvis

MISC

  • in this NEWS file, #n means the issue number on GitHub, e.g. #1 is https://github.com/cpsievert/pitchRx/issues/1

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("LDAvis")

0.3.2 by Carson Sievert, a year ago


https://github.com/cpsievert/LDAvis


Report a bug at https://github.com/cpsievert/LDAvis/issues


Browse source code at https://github.com/cran/LDAvis


Authors: Carson Sievert [aut, cre], Kenny Shirley [aut]


Documentation:   PDF Manual  


MIT + file LICENSE license


Imports proxy, RJSONIO, parallel

Suggests mallet, lda, topicmodels, gistr, servr, shiny, knitr, rmarkdown, digest, htmltools


Suggested by stm, textmining.


See at CRAN