Virtual Core Distribution Issue Error When Importing/Assigning Very Large Word2Vec Encodings

Description

On fully updated Ubuntu 16.04.2 LTS, I created a word2vec model object using the .vec file of a 8GB fasttext encoding (I know it's not best practices, but it was good enough for what I was trying to do) in ipython to do a few quick similarities look ups. On a whim, I went to my system monitor utility to watch my ram being consumed during the import (I have 32GB or RAM, this is not an issue with that), and saw this: http://i.imgur.com/LsR37sc.png . My CPU is an intel i7 7700k (with well documented stability and no overclocking, issue isn't there either). For some reason, one virtual core is used at 100% while the others stay nearly idle randomly switching between different ones. I repeated this across reboots, new kernels, etc. The encoding was built off the the English file .vec from here: https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md .

Code to Reproduce

import gensim path='/home/station/Documents/wiki.en/wiki.en.vec' model = gensim.models.KeyedVectors.load_word2vec_format(path, binary=False)

Expected Results

Distributing load across all virtual cores

Actual Results

See above, code ran didn't print anything so I can't give you that.

Versions

Everythings up to date according to pip3 as of 15 minutes ago and my systems up to date according to apt as of an hour ago. Your code:

import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import gensim; print("gensim", gensim.version) from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)

Linux-4.8.0-58-generic-x86_64-with-Ubuntu-16.04-xenial Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] NumPy 1.13.1 SciPy 0.19.1 gensim 2.2.0 FAST_VERSION 1

If you want me to test anything with this due to developer machine ram limits or something, please let me know and I'd be happy to help.