百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
D

DrQA

> 编程语言
开源

一个基于 PyTorch 的实现,用于阅读维基百科以回答开放域问题。

400 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

一个基于 PyTorch 的实现,用于阅读维基百科以回答开放域问题。

DrQA

A pytorch implementation of the ACL 2017 paper Reading Wikipedia to Answer Open-Domain Questions (DrQA).

Reading comprehension is a task to produce an answer when given a question and one or more pieces of evidence (usually natural language paragraphs). Compared to question answering over knowledge bases, reading comprehension models are more flexible and have revealed a great potential for zero-shot learning.

SQuAD is a reading comprehension benchmark where there's only a single piece of evidence and the answer is guaranteed to be a part of the evidence. Since the publication of SQuAD dataset, there has been fast progress in the research of reading comprehension and a bunch of great models have come out. DrQA is one that is conceptually simpler than most others but still yields strong performance even as a single model.

The motivation for this project is to offer a clean version of DrQA for the machine reading comprehension task, so one can quickly do some modifications and try out new ideas. Click here to see the comparison with what's described in the original paper and with two "official" projects ParlAI and DrQA.

Requirements

  • python >=3.5
  • pytorch >=0.4. Tested on pytorch 0.4 and pytorch 1.10. Historical versions:
    • DrQA with pytorch 0.3
    • DrQA with pytorch 0.2
  • numpy
  • msgpack
  • spacy 3.x

Quick Start

Setup

  • download the project via git clone https://github.com/hitvoice/DrQA.git; cd DrQA
  • make sure python 3, pip, wget and unzip are installed.
  • install pytorch matched with your OS, python and cuda versions.
  • install the remaining requirements via pip install -r requirements.txt
  • download the SQuAD datafile, GloVe word vectors and Spacy English language models using bash download.sh.

Train

bash
# prepare the data
python prepro.py
# train for 40 epochs with batchsize 32
python train.py -e 40 -bs 32

Warning: Running prepro.py takes about 9G memory when using 8 threads. If there's not enough memory on your machine, try reducing the number of threads used by the script, for example, python prepro.py --threads 2

Predict

bash
python interact.py

Example interactions:

…

The last example is a randomly picked product description from Amazon (not in SQuAD).

Results

EM & F1

EM F1
in the original paper 69.5 78.8
in this project 69.64 78.76
offical(Spacy) 69.71 78.94
offical(CoreNLP) 69.76 79.09

Compared with the official implementation:

Detailed Comparisons

Compared to what's described in the original paper:

  • The grammatical features are generated by spaCy instead of Stanford CoreNLP. It's much faster and produces similar scores.

Compared to the code in facebookresearch/DrQA:

  • This project is much more light-weighted and focusing solely on training and evaluating on SQuAD dataset while lacking the document retriever, the interactive inference API, and some other features.
  • The implementation in facebookresearch/DrQA is able to train on multiple GPUs, while (currently and for simplicity) in this implementation we only support single-GPU training.

Compared to the code in facebookresearch/ParlAI:

  • The DrQA model is no longer wrapped in a chatbot framework, which makes the code more readable, easier to modify and is faster to train. The preprocessing for text corpus is performed only once, while in a dialog framework raw text is transmitted each time and preprocessing for the same text must be done again and again.
  • This is a full implementation of the original paper, while the model in ParlAI is a partial implementation, missing all grammatical features (lemma, POS tags and named entity tags).
  • Some minor bug fixes. Some of them have been merged into ParlAI.

About

Maintainer: Runqi Yang.

Credits: thank Jun Yang for code review and advice.

Most of the pytorch model code is borrowed from Facebook/ParlAI under a BSD-3 license.

Issues· 9 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Pythondrqapytorchsquad

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言