texar

Texar is a toolkit aiming to support a broad set of machine learning, especially natural language processing and text generation tasks. Texar provides a library of easy-to-use ML modules and functionalities for composing whatever models and algorithms. The tool is designed for both researchers and practitioners for fast prototyping and experimentation.

Texar was originally developed and is actively contributed by Petuum and CMU in collaboration with other institutes. A mirror of this repository is maintained by Petuum Open Source.

Key Features

• Two Versions, (Mostly) Same Interfaces. Texar-TensorFlow (this repo) and Texar-PyTorch have mostly the same interfaces. Both further combine the best design of TF and PyTorch:
  • Interfaces and variable sharing in PyTorch convention
  • Excellent factorization and rich functionalities in TF convention.
• Rich Pre-trained Models, Rich Usage with Uniform Interfaces. BERT, GPT2, XLNet, etc, for encoding, classification, generation, and composing complex models with other Texar components!
• Fully Customizable at multiple abstraction level -- both novice-friendly and expert-friendly.
  • Free to plug in whatever external modules, since Texar is fully compatible with the native TF/PyTorch APIs.
• Versatile to support broad tasks, models, algorithms, data processing, evaluation, etc.
  • encoder(s) to decoder(s), sequential- and self-attentions, memory, hierarchical models, classifiers...
  • maximum likelihood learning, reinforcement learning, adversarial learning, probabilistic modeling, ...
• Modularized for maximal re-use and clean APIs, based on principled decomposition of Learning-Inference-Model Architecture.
• Distributed model training with multiple GPUs.
• Clean, detailed documentation and rich examples.

Library API Example

Builds an encoder-decoder model, with maximum likelihood learning:

…

The same model, but with adversarial learning:

helper = tx.modules.GumbelSoftmaxTraingHelper( # Gumbel-softmax decoding
    start_tokens=[BOS]*batch_size, end_token=EOS, embedding=embedder)
outputs, _ = decoder(helper=helper)            # automatic re-use of the decoder variables

discriminator = tx.modules.BertClassifier(hparams=hparams_bert)        # pre-trained model

G_loss, D_loss = tx.losses.binary_adversarial_losses(
    real_data=data['target_text_ids'][:, 1:],
    fake_data=outputs.sample_id,
    discriminator_fn=discriminator)

The same model, but with RL policy gradient learning:

agent = tx.agents.SeqPGAgent(samples=outputs.sample_id,
                             logits=outputs.logits,
                             sequence_length=batch['target_length']-1,
                             hparams=config_model.agent)

Many more examples are available here

Installation

(Note: Texar>0.2.3 requires Python 3.6 or 3.7. To use with older Python versions, please use Texar<=0.2.3)

Texar requires:

• tensorflow >= 1.10.0 (but < 2.0.0). Follow the tensorflow official instructions to install the appropriate version
• tensorflow_probability >= 0.3.0 (but < 0.8.0). Follow the tensorflow_probability official instractions to install.

After tensorflow and tensorflow_probability are installed, install Texar from PyPI:

pip install texar

To use cutting-edge features or develop locally, install from source:

git clone https://github.com/asyml/texar.git
cd texar
pip install .

Getting Started

• Examples
• Documentation

Reference

If you use Texar, please cite the tech report with the following BibTex entry:

…

License

Apache License 2.0

Companies and Universities Supporting Texar