C++ library and cmdline tools for parsing and manipulating VCF files with python and zig bindings
C++ library and cmdline tools for parsing and manipulating VCF files with python and zig bindings
Vcflib and related tools are the workhorses in bioinformatics for processing the VCF variant calling format. See
A spectrum of free software tools for processing the VCF variant call format: vcflib, bio-vcf, cyvcf2, hts-nim and slivar. Garrison E, Kronenberg ZN, Dawson ET, Pedersen BS, Prins P (2022), PLoS Comput Biol 18(5): e1009123. https://doi.org/10.1371/journal.pcbi.1009123
(bibtex reference at the end of this page)
May 2025: fixes, updates and downloadable binaries. See RELEASE_NOTES.md
November 2024: a fresh release with, mostly, pangenome related fixes.
January 2023: this is vcflib's first Humpty Dumpty release: vcfcreatemulti is the natural companion to vcfwave. Often variant callers are not perfect.
vcfwave with its companion tool vcfcreatemulti can take an existing VCF file that contains multiple complex overlapping and even nested alleles and, unlike Humpty Dumpty, take them apart and put them together again. Thereby, hopefully, creating sane VCF output that is useful for analysis and getting rid of false positives.
We created these tools by including the state-of-the-art biWFA wavefront aligner. The tools are particularly useful for the output from structural variation callers and pangenome genotypers, such as used by the Human Pangenome Reference Consortium (HPRC) because of overlapping ALT segments.
May 2022: the vcflib paper has been published on PLoS Computational Biology!
April 2022: vcflib has just gone pangenome!
By introducing the wavefront algorithm we can now realign long sequences and reduce call complexity (and FPs!) introduced by pangenome variant callers using the new vcfwave tool.
See also RELEASE_NOTES.md
The Variant Call Format (VCF) is a flat-file, tab-delimited textual format that describes reference-indexed variations between individuals. VCF provides a common interchange format for the description of variation in individuals and populations of samples, and has become the de facto standard reporting format for a wide array of genomic variant detectors.
vcflib provides methods to manipulate and interpret sequence variation described by VCF. It is both:
vclib is both a library (with an API) and a collection of useful tools. The API provides a quick and extremely permissive method to read and write VCF files. Extensions and applications of the library provided in the included utilities (*.cpp) comprise the vast bulk of the library's utility.
We have also added infrastructure to write Python bindings. See below.
Short index:
For latest updates see RELEASE NOTES.
Conda installs in user land without root access
conda install -c bioconda vcflibHomebrew installs on Linux and Mac OSX
brew install brewsci/bio/vcflibFor Debian and Ubuntu
apt-get install libvcflib-tools libvcflib-devWe develop against guix and vcflib is packaged as
guix package -i vcflibSee also the Guix shell below.
Images are available from files.genenetwork.org.
For singularity simply run someting like:
singularity exec vcflib-version-singularity.pack.gz vcfcreatemulti -hUsers are encouraged to drive the utilities in the library in a streaming fashion, using Unix pipes to fully utilize resources on multi-core systems. Piping provides a convenient method to interface with other libraries (vcf-tools, BedTools, GATK, htslib, bio-vcf, bcftools, freebayes) which interface via VCF files, allowing the composition of an immense variety of processing functions. Examples can be found in the scripts, e.g. ./scripts.
| filter command | description |
|---|---|
| vcfuniq | List unique genotypes. Similar to GNU uniq, but aimed at VCF records. vcfuniq removes records which have the same position, ref, and alt as the previous record on a sorted VCF file. Note that it does not adjust/combine genotypes in the output, but simply takes the first record. See also vcfcreatemulti for combining records. |
| vcfuniqalleles | List unique alleles For each record, remove any duplicate alternate alleles that may have resulted from merging separate VCF files. |
| vcffilter | VCF filter the specified vcf file using the set of filters |
| metrics command | description |
|---|---|
| vcfhethomratio | Generates the het/hom ratio for each individual in the file |
| vcfhetcount | Calculate the heterozygosity rate: count the number of alternate alleles in heterozygous genotypes in all records in the vcf file |
| vcfdistance | Adds a tag to each variant record which indicates the distance to the nearest variant. (defaults to BasesToClosestVariant if no custom tag name is given. |
| vcfentropy | Annotate VCF records with the Shannon entropy of flanking sequence. Anotates the output VCF file with, for each record, EntropyLeft, EntropyRight, EntropyCenter, which are the entropies of the sequence of the given window size to the left, right, and center of the record. Also adds EntropyRef and EntropyAlt for each alt. |
| vcfcheck | Validate integrity and identity of the VCF by verifying that the VCF record's REF matches a given reference file. |
| phenotype command | description |
|---|---|
| permuteGPAT++ | permuteGPAT++ is a method for adding empirical p-values to a GPAT++ score. |
| genotype command | description |
|---|---|
| hapLrt | HapLRT is a likelihood ratio test for haplotype lengths. The lengths are modeled with an exponential distribution. The sign denotes if the target has longer haplotypes (1) or the background (-1). |
| normalize-iHS | normalizes iHS or XP-EHH scores. |
| abba-baba | abba-baba calculates the tree pattern for four indviduals. This tool assumes reference is ancestral and ignores non abba-baba sites. The output is a boolian value: 1 = true , 0 = false for abba and baba. the tree argument should be specified from the most basal taxa to the most derived. |
| transformation command | description |
|---|---|
| vcfintersect | VCF set analysis |
| vcfleftalign | Left-align indels and complex variants in the input using a pairwise ref/alt alignment followed by a heuristic, iterative left realignment process that shifts indel representations to their absolute leftmost (5') extent. |
| vcfglxgt | Set genotypes using the maximum genotype likelihood for each sample. |
| vcfkeepsamples | outputs each record in the vcf file, removing samples not listed on the command line |
| vcfallelicprimitives | Note that this tool is considered legacy and will emit a warning! Use vcfwave instead. |
| vcfcommonsamples | Generates each record in the first file, removing samples not present in the second |
| vcfcreatemulti | Go through sorted VCF and if overlapping alleles are represented across multiple records, merge them into a single record. Currently only for indels. |
| vcfld | Compute LD |
| vcfsamplenames | List sample names |
| vcfsample2info | Take annotations given in the per-sample fields and add the mean, median, min, or max to the site-level INFO. |
| vcflength | Add length info field |
| vcfstreamsort | Sorts the input (either stdin or file) using a streaming sort algorithm. Guarantees that the positional order is correct provided out-of-order variants are no more than 100 positions in the VCF file apart. |
| dumpContigsFromHeader | Dump contigs from header |
| vcfgeno2haplo | Convert genotype-based phased alleles within --window-size into haplotype alleles. Will break haplotype construction when encountering non-phased genotypes on input. |
| vcfinfo2qual | Sets QUAL from info field tag keyed by [key]. The VCF file may be omitted and read from stdin. The average of the field is used if it contains multiple values. |
| vcfevenregions | Generates a list of regions, e.g. chr20:10..30 using the variant density information provided in the VCF file to ensure that the regions have even numbers of variants. This can be use to reduce the variance in runtime when dividing variant detection or genotyping by genomic coordinates. |
| vcfbreakmulti | If multiple alleles are specified in a single record, break the record into multiple lines, preserving allele-specific INFO fields. |
| vcfwave | Realign reference and alternate alleles with WFA, parsing out the primitive alleles into multiple VCF records. New records have IDs that reference the source record ID. Genotypes are handled. Deletions generate haploid/missing genotypes at overlapping sites. |
| vcfafpath | Display genotype paths |
| vcfremoveaberrantgenotypes | strips samples which are homozygous but have observations implying heterozygosity. Remove samples for which the reported genotype (GT) and observation counts disagree (AO, RO). |
| vcf2tsv | Converts VCF to per-allelle or per-genotype tab-delimited format, using null string to replace empty values in the table. Specifying -g will output one line per sample with genotype information. When there is more than one alt allele there will be multiple rows, one for each allele and, the info will match the 'A' index |
| vcfqual2info | Puts QUAL into an info field tag keyed by [key]. |
| [vcfn |
No open issues yet, or sync has not completed.