deep-siamese-text-similarity

This project is a prototype for experimental purposes only and production grade code is not released here.

Deep LSTM siamese network for text similarity

It is a tensorflow based implementation of deep siamese LSTM network to capture phrase/sentence similarity using character embeddings.

This code provides architecture for learning two kinds of tasks:

• Phrase similarity using char level embeddings [1]

• Sentence similarity using word level embeddings [2]

For both the tasks mentioned above it uses a multilayer siamese LSTM network and euclidian distance based contrastive loss to learn input pair similairty.

Capabilities

Given adequate training pairs, this model can learn Semantic as well as structural similarity. For eg:

Phrases :

• International Business Machines = I.B.M
• Synergy Telecom = SynTel
• Beam inc = Beam Incorporate
• Sir J J Smith = Johnson Smith
• Alex, Julia = J Alex
• James B. D. Joshi = James Joshi
• James Beaty, Jr. = Beaty

For phrases, the model learns character based embeddings to identify structural/syntactic similarities.

Sentences :

• He is smart = He is a wise man.
• Someone is travelling countryside = He is travelling to a village.
• She is cooking a dessert = Pudding is being cooked.
• Microsoft to acquire Linkedin ≠ Linkedin to acquire microsoft

(More examples Ref: semEval dataset)

For Sentences, the model uses pre-trained word embeddings to identify semantic similarities.

Categories of pairs, it can learn as similar:

• Annotations
• Abbreviations
• Extra words
• Similar semantics
• Typos
• Compositions
• Summaries

Training Data

• Phrases:

  • A sample set of learning person name paraphrases have been attached to this repository. To generate full person name disambiguation data follow the steps mentioned at:

  https://github.com/dhwajraj/dataset-person-name-disambiguation

  "person_match.train" : https://drive.google.com/open?id=1HnMv7ulfh8yuq9yIrt_IComGEpDrNyo-

• Sentences:

  • A sample set of learning sentence semantic similarity can be downloaded from:

  "train_snli.txt" : https://drive.google.com/open?id=1itu7IreU_SyUSdmTWydniGxW-JEGTjrv

  This data is generated using SNLI project :

  https://nlp.stanford.edu/projects/snli/

  • word embeddings: any set of pre-trained word embeddings can be utilized in this project. For our testing we had used fastText simple english embeddings from https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md

  alternate download location for "wiki.simple.vec" is : https://drive.google.com/open?id=1u79f3d2PkmePzyKgubkbxOjeaZCJgCrt

Environment

• numpy 1.11.0
• tensorflow 1.2.1
• gensim 1.0.1
• nltk 3.2.2

How to run

Training

…

Evaluation

$ python eval.py --model graph#.pb

Performance

Phrases:

• Training time: (8 core cpu) = 1 complete epoch : 6min 48secs (training requires atleast 30 epochs)
  • Contrastive Loss : 0.0248
• Evaluation performance : similarity measure for 100,000 pairs (8core cpu) = 1min 40secs
  • Accuracy 91%

Sentences:

• Training time: (8 core cpu) = 1 complete epoch : 8min 10secs (training requires atleast 50 epochs)
  • Contrastive Loss : 0.0477
• Evaluation performance : similarity measure for 100,000 pairs (8core cpu) = 2min 10secs
  • Accuracy 81%

References

1. Learning Text Similarity with Siamese Recurrent Networks
2. Siamese Recurrent Architectures for Learning Sentence Similarity