High-level batteries-included neural network training library for Pytorch
High-level batteries-included neural network training library for Pytorch
Pywick is a high-level Pytorch training framework that aims to get you up and running quickly with state of the art neural networks. Does the world need another Pytorch framework? Probably not. But we started this project when no good frameworks were available and it just kept growing. So here we are.
Pywick tries to stay on the bleeding edge of research into neural networks. If you just wish to run a vanilla CNN, this is probably going to be overkill. However, if you want to get lost in the world of neural networks, fine-tuning and hyperparameter optimization for months on end then this is probably the right place for you :)
Among other things Pywick includes:
Hey, check this out, we now have docs! They're still a work in progress though so apologies for anything that's broken.
Another great improvement to the framework - docker! You can now run the 17flowers demo right out of the box!
docker pull achaiah/pywick:latest. Pytorch 1.8 and cuda dependencies are pre-installed.docker run --rm -it --ipc=host -v your_local_out_dir:/jobs/17flowers --init -e demo=true achaiah/pywick:latest docker run --rm -it \
--ipc=host \
-v : \
-v : \
--init \
achaiah/pywick:latestpywick/examples/17flowers_split.py to convert)configs/train_classifier.yaml file to fit your workspacepython3 train_classifier.py configs/train_classifier.yaml and watch it train!torch.hub! See docs for all the variants!Pywick requires pytorch >= 1.4
pip install pywick
or specific version from git:
pip install git+https://github.com/achaiah/[email protected]
The ModuleTrainer class provides a high-level training interface which abstracts away the training loop while providing callbacks, constraints, initializers, regularizers,
and more.
See the train_classifier.py example for a pretty complete configuration example. To get up and running with your own data quickly simply edit the configs/train_classifier.yaml file with your desired parameters and dataset location(s).
Note: Dataset needs to be organized for classification where each directory name is the name of a class and contains all images pertaining to that class
PyWick provides a wide range of callbacks, generally mimicking the interface found in Keras:
CSVLogger - Logs epoch-level metrics to a CSV fileCyclicLRScheduler - Cycles through min-max learning rateEarlyStopping - Provides ability to stop training early based on supplied criteriaHistory - Keeps history of metrics etc. during the learning processLambdaCallback - Allows you to implement your own callbacks on the flyLRScheduler - Simple learning rate scheduler based on function or supplied scheduleModelCheckpoint - Comprehensive model saverReduceLROnPlateau - Reduces learning rate (LR) when a plateau has been reachedSimpleModelCheckpoint - Simple model saverTensorboardLogger is incredibly easy to implement via tensorboardX (now part of pytorch 1.1 release!)from pywick.callbacks import EarlyStopping
callbacks = [EarlyStopping(monitor='val_loss', patience=5)]
trainer.set_callbacks(callbacks)PyWick also provides regularizers:
L1RegularizerL2RegularizerL1L2Regularizerand constraints:
UnitNormMaxNormNonNegBoth regularizers and constraints can be selectively applied on layers using regular expressions and the module_filter
argument. Constraints can be explicit (hard) constraints applied at an arbitrary batch or
epoch frequency, or they can be implicit (soft) constraints similar to regularizers
where the constraint deviation is added as a penalty to the total model loss.
from pywick.constraints import MaxNorm, NonNeg
from pywick.regularizers import L1Regularizer
# hard constraint applied every 5 batches
hard_constraint = MaxNorm(value=2., frequency=5, unit='batch', module_filter='*fc*')
# implicit constraint added as a penalty term to model loss
soft_constraint = NonNeg(lagrangian=True, scale=1e-3, module_filter='*fc*')
constraints = [hard_constraint, soft_constraint]
trainer.set_constraints(constraints)
regularizers = [L1Regularizer(scale=1e-4, module_filter='*conv*')]
trainer.set_regularizers(regularizers)You can also fit directly on a torch.utils.data.DataLoader and can have
a validation set as well :
from pywick import TensorDataset
from torch.utils.data import DataLoader
train_dataset = TensorDataset(x_train, y_train)
train_loader = DataLoader(train_dataset, batch_size=32)
val_dataset = TensorDataset(x_val, y_val)
val_loader = DataLoader(val_dataset, batch_size=32)
trainer.fit_loader(loader, val_loader=val_loader, num_epoch=100)No open issues yet, or sync has not completed.