Implementing Random Indexing on GPU

• http://dl.acm.org/citation.cfm?id=2048577.2048595

random indexing, word space models, term co-occurence, GPGPU 

Vector space models (also word space models or term space models) are algebraic models, used for representing text documents as vectors of terms. They have received much attention recently as they have wide spectrum of applications, including information filtering, information retrieval, indexing and relevancy ranking. They can be advantageous over the other representations because vector spaces are mathematically well defined and there's large set of tools for manipulating them. Random Indexing is one of methods used for calculating vector space models from set of documents, based on distributional statistics of term cooccurrences. To  produce useful results it may therefore require large amounts of data and significant computational power. We present an efficient implementation of Random Indexing on GPU, allowing fast training even on large datasets. It is only limited by amount of memory available on GPU, some techniques to overcome this limitation are suggested. Speedups in magnitude of tens are achieved for training from random seed vectors, and even much higher for retraining. Implementation scales well with both term vector dimension and seed length.