Safely store secrets in Git/Mercurial/Subversion
THIS PROJECT IS ABANDONED. DO NOT USE. DO NOT SUBMIT BUG REPORTS OR PRs.
There are plenty of other, better, projects. A partial list is listed under "Alternatives" below.
========
Safely store secrets in a VCS repo (i.e. Git, Mercurial, Subversion or Perforce). These commands make it easy for you to Gnu Privacy Guard (GPG) encrypt specific files in a repo so they are "encrypted at rest" in your repository. However, the scripts make it easy to decrypt them when you need to view or edit them, and decrypt them for use in production. Originally written for Puppet, BlackBox now works with any Git or Mercurial repository.
WARNING: The goal of this project is to be a simple wrapper around gpg so you and your coworkers don't have to remember its all those inscrutable and confusing flags. It is not intended to be a sophisticated encryption system that solves all problems or supports a large numbers of files. The ideal use-case is to keep secrets in a secure service such as Conjur, AWS KMS, Azure Key Vault or GCP KMS; then use Blackbox for safely storing the API keys needed to access that system. That way you are encrypting a single, tiny, file. Feature-requests for anything more will be rejected; do not expect or even request "enterprise features". If this disappoints you, please consider a competiting project such as https://www.agwa.name/projects/git-crypt
A slide presentation (about an older release) is on SlideShare.
Join our mailing list: https://groups.google.com/d/forum/blackbox-project
Suppose you have a VCS repository (i.e. a Git or Mercurial repo) and certain files contain secrets such as passwords or SSL private keys. Often people just store such files "and hope that nobody finds them in the repo". That's not safe.
With BlackBox, those files are stored encrypted using GPG. Access to the VCS repo without also having the right GPG keys makes it worthless to have the files. As long as you keep your GPG keys safe, you don't have to worry about storing your VCS repo on an untrusted server. Heck, even if you trust your server, now you don't have to trust the people that do backups of that server, or the people that handle the backup tapes!
Rather than one GPG passphrase for all the files, each person with access has their own GPG keys in the system. Any file can be decrypted by anyone with their GPG key. This way, if one person leaves the company, you don't have to communicate a new password to everyone with access. Simply disable the one key that should no longer have access. The process for doing this is as easy as running 2 commands (1 to disable their key, 1 to re-encrypt all files.)
Automated processes often need access to all the decrypted files. This is easy too. For example, suppose Git is being used for Puppet files. The master needs access to the decrypted version of all the files. Simply set up a GPG key for the Puppet master (or the role account that pushes new files to the Puppet master) and have that user run blackbox_postdeploy after any files are updated.
If you don't have a GPG key, set it up using instructions such as:
Set up GPG key.
Now you are ready to go.
cd into a Git, Mercurial, Subversion or Perforce repository and run blackbox_initialize.
If a file is to be encrypted, run blackbox_register_new_file and you are done.
Add and remove keys with blackbox_addadmin and blackbox_removeadmin.
To view and/or edit a file, run blackbox_edit;
this will decrypt the file and open with whatever is specified by
your $EDITOR environment variable.
When you close the editor the
file will automatically be encrypted again and the temporary plaintext
file will be shredded.
If you need to leave the file decrypted while
you update you can use theblackbox_edit_start to decrypt the file
and blackbox_edit_end when you want to "put it back in the box."
OBVIOUSLY we don't want secret things like SSL private keys and passwords to be leaked.
NOT SO OBVIOUSLY when we store "secrets" in a VCS repo like Git or Mercurial, suddenly we are less able to share our code with other people. Communication between subteams of an organization is hurt. You can't collaborate as well. Either you find yourself emailing individual files around (yuck!), making a special repo with just the files needed by your collaborators (yuck!!), or just deciding that collaboration isn't worth all that effort (yuck!!!).
The ability to be open and transparent about our code, with the exception of a few specific files, is key to the kind of collaboration that DevOps and modern IT practitioners need to do.
make copy-install will copy the bin files into $PREFIX/bin, default is /usr/local (uninstall with make copy-uninstall).make symlinks-install will make symlinks of the bin files into $PREFIX/bin, default is /usr/local (uninstall with make copy-uninstall) (useful when doing development)sudo port install vcs_blackboxbrew install blackboxmake packages-rpm; now you can distribute the RPM via local methods. (Requires fpm.)make packages-deb; now you can distribute the DEB via local methods. (Requires fpm.)antigen bundle StackExchange/blackbox to your .zshrczgenom load StackExchange/blackbox to your .zshrc where you're loading your other plugins.nix-shell -p blackboxpkgin in scm-blackboxblackbox_edit <file>
Decrypt, run $EDITOR, re-encrypt a file
blackbox_edit_start <file>
Decrypt a file so it can be updated
blackbox_edit_end <file>
Encrypt a file after blackbox_edit_start was used
blackbox_cat <file>
Decrypt and view the contents of a file
blackbox_view <file>
Like blackbox_cat but pipes to less or $PAGER
blackbox_diff
Diff decrypted files against their original crypted version
blackbox_initialize
Enable blackbox for a GIT or HG repo
blackbox_register_new_file <file>
Encrypt a file for the first time
blackbox_deregister_file <file>
Remove a file from blackbox
blackbox_list_files
List the files maintained by blackbox
blackbox_list_admins
List admins currently authorized for blackbox
blackbox_decrypt_file <file>
Decrypt a file
blackbox_decrypt_all_files
Decrypt all managed files (INTERACTIVE)
blackbox_postdeploy
Decrypt all managed files (batch)
blackbox_addadmin <gpg-key>
Add someone to the list of people that can encrypt/decrypt secrets
blackbox_removeadmin <gpg-key>
Remove someone from the list of people that can encrypt/decrypt secrets
blackbox_shred_all_files
Safely delete any decrypted files
blackbox_update_all_files
Decrypt then re-encrypt all files. Useful after keys are changed
blackbox_whatsnew <file>
show what has changed in the last commit for a given file
BlackBox automatically determines which VCS you are using and does the right thing. It has a plug-in architecture to make it easy to extend to work with other systems. It has been tested to work with many operating systems.
git -- The Githg -- Mercurialsvn -- SubVersion (Thanks, Ben Drasin!)p4 -- Perforce.blackbox directory is intactTo add or fix support for a VCS system, look for code at the end of bin/_blackbox_common.sh
To add or fix support for a new operating system, look for the case statements in bin/_blackbox_common.sh and bin/_stack_lib.sh and maybe tools/confidence_test.sh
BlackBox can be used with Cygwin, MinGW or WSL2.
BlackBox assumes that blackbox-admins.txt and blackbox-files.txt will have
LF line endings. Windows users should be careful to configure Git or other systems
to not convert or "fix" those files.
If you use Git, add the following lines to your .gitattributes file:
**/blackbox-admins.txt text
No open issues yet, or sync has not completed.