0.13.0rc1 Tutorials reworked, Word Movers Distance

Changes

0.12.5, 2016

• Tutorials migrated from website to ipynb (@j9chan, #721), (@jesford, #733, #725, 716)
• New doc2vec intro tutorial (@seanlaw, #730)
• Gensim Quick Start Tutorial (@andrewjlm, #727)
• Add export_phrases(sentences) to model Phrases (hanabi1224 #588)
• SparseMatrixSimilarity returns a sparse matrix if maintain_sparsity is True (@davechallis, #590)
• added functionality for Topics of Words in document - i.e, dynamic topics. (@bhargavvader, #704)
  • also included tutorial which explains new functionalities, and document word-topic coloring.
• Made normalization an explicit transformation. Added 'l1' norm support (@sQuareindia, #649)
• added term-topics API for most probable topic for word in vocab. (@bhargavvader, #706)
• build_vocab takes progress_per parameter for smaller output (@zer0n, #624)
• Control whether to use lowercase for computing word2vec accuracy. (@alantian, #607)
• Easy import of GloVe vectors using Gensim (Manas Ranjan Kar, #625)
  • Allow easy port of GloVe vectors into Gensim
  • Standalone script with command line arguments, compatible with Python>=2.6
  • Usage: python -m gensim.scripts.glove2word2vec -i glove_vectors.txt -o output_word2vec_compatible.txt
• Add similar_by_word() and similar_by_vector() to word2vec (@isohyt, #381)
• Convenience method for similarity of two out of training sentences to doc2vec (@ellolo, #707)
• Dynamic Topic Modelling Tutorial updated with Dynamic Influence Model (@bhargavvader, #689)
• Added function to filter 'n' most frequent words from the dictionary (@abhinavchawla, #718)
• Raise warnings if vocab is single character elements and if alpha is increased in word2vec/doc2vec (@dsquareindia, #705)
• Tests for wikidump (@jonmcoe, #723)
• Mallet wrapper sparse format support (@RishabGoel, #664)
• Doc2vec pre-processing script translated from bash to Python (@andrewjlm, #720)

0.12.4, 29/01/2016

• Word2vec in line with original word2vec.c (Andrey Kutuzov, #538)
• Same default values. See diff akutuzov@6456cbc
• Standalone script with command line arguments matching those of original C tool.
  Usage ./word2vec_standalone.py -train data.txt -output trained_vec.txt -size 200 -window 2 -sample 1e-4
• load_word2vec_format() performance (@svenkreiss, #555)
  • Remove init_sims() call for performance improvements when normalized vectors are not needed.
  • Remove norm_only parameter (API change). Call init_sims(replace=True) after the load_word2vec_format() call for the old norm_only=True behavior.
• Better internal handling of job batching in word2vec (#535)
  • up to 300% speed up when training on very short documents (~tweets)
• Word2vec allows non-strict unicode error handling (ignore or replace) (Gordon Mohr, #466)
• Doc2Vec model.docvecs[key] now raises KeyError for unknown keys (Gordon Mohr, #520)
• Fix DocvecsArray.index_to_doctag so most_similar() returns string doctags (Gordon Mohr, #560)
• On-demand loading of the pattern library in utils.lemmatize (Jan Zikes, #461)
  • utils.HAS_PATTERN flag moved to utils.has_pattern()
• Threadsafe Word2Vec/Doc2Vec finish-check to avoid hang/unending Word2Vec/Doc2Vec training (Gordon Mohr, #571)
• Tuned TestWord2VecModel.test_cbow_hs() against random failures (Gordon Mohr, #531)
• Prevent ZeroDivisionError when default_timer() indicate no elapsed time (Gordon Mohr, #518)
• Forwards compatibility for NumPy > 1.10 (Matti Lyra, #494, #513)
  • LdaModel and LdaMulticore produce a large number of DeprecationWarnings from
    .inference() because the term ids in each chunk returned from utils.grouper
    are floats. This behaviour has been changed so that the term IDs are now ints.
  • utils.grouper returns a python list instead of a numpy array in .update() when
    LdaModel is called in non distributed mode
  • in distributed mode .update() will still call utils.grouper with as_numpy=True
    to save memory
  • LdaModel.update and LdaMulticore.update have a new keyword parameter
    chunks_as_numpy=True/False (defaults to False) that allows controlling
    this behaviour

0.12.3

0.12.3rc1, 05/11/2015

• Make show_topics return value consistent across models (Christopher Corley, #448)
  • All models with the show_topics method should return a list of
    (topic_number, topic) tuples, where topic is a list of
    (word, probability) tuples.
  • This is a breaking change that affects users of the LsiModel, LdaModel,
    and LdaMulticore that may be reliant on the old tuple layout of
    (probability, word).
• Mixed integer & string document-tags (keys to doc-vectors) will work (Gordon Mohr, #491)
  • DocvecsArray's index2doctag list is renamed/reinterpreted as offset2doctag
  • offset2doctag entries map to doctag_syn0 indexes after last plain-int doctag (if any)
  • (If using only string doctags, offset2doctag may be interpreted same as index2doctag.)
• New Tutorials on Dynamic Topic Modelling and Classification via Word2Vec (@Arttii #471, @mataddy #500)
• Auto-learning for the eta parameter on the LdaModel (Christopher Corley, #479)
• Python 3.5 support
• Speed improvements to keyword and summarisation methods (@erbas #441)
• OSX wheels (#504)
• Win build (#492)

0.12.3rc1

0.12.3rc1, 05/11/2015

• Make show_topics return value consistent across models (Christopher Corley, #448)
  • All models with the show_topics method should return a list of
    (topic_number, topic) tuples, where topic is a list of
    (word, probability) tuples.
  • This is a breaking change that affects users of the LsiModel, LdaModel,
    and LdaMulticore that may be reliant on the old tuple layout of
    (probability, word).
• Mixed integer & string document-tags (keys to doc-vectors) will work (Gordon Mohr, #491)
  • DocvecsArray's index2doctag list is renamed/reinterpreted as offset2doctag
  • offset2doctag entries map to doctag_syn0 indexes after last plain-int doctag (if any)
  • (If using only string doctags, offset2doctag may be interpreted same as index2doctag.)
• New Tutorials on Dynamic Topic Modelling and Classification via Word2Vec (@Arttii #471, @mataddy #500)
• Auto-learning for the eta parameter on the LdaModel (Christopher Corley, #479)
• Python 3.5 support
• Speed improvements to keyword and summarisation methods (@erbas #441)
• OSX wheels (#504)
• Win build (#492)