kompute

Kompute

The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends)

Blazing fast, mobile-enabled, asynchronous, and optimized for advanced GPU acceleration usecases.

[Join the Discord & Community Calls](https://kompute.cc/overview/community.html) [Documentation](https://kompute.cc) [Blog Post](https://medium.com/@AxSaucedo/machine-learning-and-data-processing-in-the-gpu-with-vulkan-kompute-c9350e5e5d3a) ⌨ [Examples](#more-examples)

---

##### Kompute is backed by the Linux Foundation as a hosted project by the LF AI & Data Foundation. ## Principles & Features * [Flexible Python module](#your-first-kompute-python) with [C++ SDK](#your-first-kompute-c) for optimizations * [Asynchronous & parallel processing](#asynchronous-and-parallel-operations) support through GPU family queues * [Mobile enabled](#mobile-enabled) with examples via Android NDK across several architectures * BYOV: [Bring-your-own-Vulkan design](#motivations) to play nice with existing Vulkan applications * Explicit relationships for GPU and host [memory ownership and memory management](https://kompute.cc/overview/memory-management.html) * Robust codebase with [90% unit test code coverage](https://kompute.cc/codecov/) * Advanced use-cases on [machine learning ](https://towardsdatascience.com/machine-learning-and-data-processing-in-the-gpu-with-vulkan-kompute-c9350e5e5d3a), [mobile development ](https://towardsdatascience.com/gpu-accelerated-machine-learning-in-your-mobile-applications-using-the-android-ndk-vulkan-kompute-1e9da37b7617) and [game development ](https://towardsdatascience.com/supercharging-game-development-with-gpu-accelerated-ml-using-vulkan-kompute-the-godot-game-engine-4e75a84ea9f0). * Active community with [monthly calls, discord chat and more](https://kompute.cc/overview/community.html) ## Projects using Kompute ❤️ * [GPT4ALL](https://github.com/nomic-ai/gpt4all) - An ecosystem of open-source on-edge large language models that run locally on your CPU and nearly any GPU. * [llama.cpp](https://github.com/ggerganov/llama.cpp) - Port of Facebook's LLaMA model in C/C++ (now decomissioned). * [tpoisonooo/how-to-optimize-gemm](https://github.com/tpoisonooo/how-to-optimize-gemm) - row-major matmul optimization. * [vkJAX](https://github.com/alexander-g/vkJAX) - JAX interpreter for Vulkan. ## Getting Started Below you can find a GPU multiplication example using the C++ and Python Kompute interfaces. You can [join the Discord](https://discord.gg/MaH5Jv5zwv) for questions / discussion, open a [github issue](https://github.com/KomputeProject/kompute/issues/new), or read [the documentation](https://kompute.cc/). ### Your First Kompute (C++) The C++ interface provides low level access to the native components of Kompute, enabling for [advanced optimizations](https://kompute.cc/overview/async-parallel.html) as well as [extension of components](https://kompute.cc/overview/reference.html). ``` … ``` ### Your First Kompute (Python) The [Python package](https://kompute.cc/overview/python-package.html) provides a [high level interactive interface](https://kompute.cc/overview/python-reference.html) that enables for experimentation whilst ensuring high performance and fast development workflows. ``` … ``` ### Interactive Notebooks & Hands on Videos You are able to try out the interactive Colab Notebooks which allow you to use a free GPU. The available examples are the Python and C++ examples below: Try the interactive C++ Colab from Blog Post Try the interactive Python Colab from Blog Post You can also check out the two following talks presented at the FOSDEM 2021 conference. Both videos have timestamps which will allow you to skip to the most relevant section for you - the intro & motivations for both is almost the same so you can skip to the more specific content. Watch the video for C++ Enthusiasts Watch the video for Python & Machine Learning Enthusiasts ## Architectural Overview The core architecture of Kompute includes the following: * [Kompute Manager](https://kompute.cc/overview/reference.html#manager) - Base orchestrator which creates and manages device and child components * [Kompute Sequence](https://kompute.cc/overview/reference.html#sequence) - Container of operations that can be sent to GPU as batch * [Kompute Operation (Base)](https://kompute.cc/overview/reference.html#algorithm) - Base class from which all operations inherit * [Kompute Tensor](https://kompute.cc/overview/reference.html#tensor) - Tensor structured data used in GPU operations * [Kompute Algorithm](https://kompute.cc/overview/reference.html#algorithm) - Abstraction for (shader) logic executed in the GPU To see a full breakdown you can read further in the [C++ Class Reference](https://kompute.cc/overview/reference.html). Full Architecture Simplified Kompute Components

(very tiny, check the full reference diagram in docs for details)

## Asynchronous and Parallel Operations Kompute provides flexibility to run operations in an asynrchonous way through vk::Fences. Furthermore, Kompute enables for explicit allocation of queues, which allow for parallel execution of operations across queue families. The image below provides an intuition on how Kompute Sequences can be allocated to different queues to enable parallel execution based on hardware. You can see the [hands on example](https://kompute.cc/overview/advanced-examples.html#parallel-operations), as well as the [detailed documentation page](https://kompute.cc/overview/async-parallel.html) describing how it would work using an NVIDIA 1650 as an example. ## Mobile Enabled Kompute has been optimized to work in mobile environments. The [build system](#build-overview) enables for dynamic loading of the Vulkan shared library for Android environments, together with a working [Android NDK wrapper](https://github.com/KomputeProject/kompute/tree/master/vk_ndk_wrapper_include) for the CPP headers.

For a full deep dive you can read the blog post "Supercharging your Mobile Apps with On-Device GPU Accelerated Machine Learning". You can also access the end-to-end example code in the repository, which can be run using android studio.

## More examples ### Simple examples * [Simple multiplication example](https://kompute.cc/overview/advanced-examples.html#simple-shader-example) * [Record batch commands with a Kompute Sequence](https://kompute.cc/overview/advanced-examples.html#record-batch-commands) * [Run Asynchronous Operations](https://kompute.cc/overview/advanced-examples.html#asynchronous-operations) * [Run Parallel Operations Across Multiple GPU Queues](https://kompute.cc/overview/advanced-examples.html#parallel-operations) * [Create your custom Kompute Operations](https://kompute.cc/overview/advanced-examples.html#your-custom-kompute-operation) * [Implementing logistic regression from scratch](https://kompute.cc/overview/advanced-examples.html#logistic-regression-example) ### End-to-end examples * [Machine Learning Logistic Regression Implementation](https://towardsdatascience.com/machine-learning-and-data-processing-in-the-gpu-with-vulkan-kompute-c9350e5e5d3a) * [Parallelizing GPU-intensive Workloads via Multi-Queue Operations](https://towardsdatascience.com/parallelizing-heavy-gpu-workloads-via-multi-queue-operations-50a38b15a1dc) * [Android NDK Mobile Kompute ML Application](https://towardsdatascience.com/gpu-accelerated-machine-learning-in-your-mobile-applications-using-the-android-ndk-vulkan-kompute-1e9da37b7617) * [Game Development Kompute ML in Godot Engine](https://towardsdatascience.com/supercharging-game-development-with-gpu-accelerated-ml-using-vulkan-kompute-the-godot-game-engine-4e75a84ea9f0) ## Python Package Besides the C++ core SDK you can also use the Python package of Kompute, which exposes the same core functionality, and supports interoperability with Python objects like Lists, Numpy Arrays, etc. The only dependencies are Python 3.5+ and Cmake 3.4.1+. You can install Kompute from the [Python pypi package](https://pypi.org/project/kp/) using the following command. ``` pip install kp ``` You can also install from master branch using: ``` pip install git+git://github.com/KomputeProject/kompute.git@master ``` For further details you can read the [Python Package documentation](https://kompute.cc/overview/python-package.html) or the [Python Class Reference documentation](https://kompute.cc/overview/python-reference.html). ## C++ Build Overview The build system provided uses `cmake`, which allows for cross platform builds. The top level `Makefile` provides a set of optimized configurations for development as well as the docker image build, but you can start a build with the following command: ``` cmake -Bbuild ``` You also are able to add Kompute in your repo with `add_subdirectory` - the [Android example CMakeLists.txt file](https://github.com/KomputeProject/kompute/blob/7c8c0eeba2cdc098349fcd999102bb2cca1bf711/examples/android/android-simple/app/src/main/cpp/CMakeLists.txt#L3) shows how this would be done. For a more advanced overview of the build configuration check out the [Build System Deep Dive](https://kompute.cc/overview/build-system.html) documentation. ## Kompute Development We appreciate PRs and Issues. If you want to contribute try checking the "Good first issue" tag, but even using Kompute and reporting issues is a great contribution! ### Contributing #### Dev Dependencies * Testing + GTest * Documentation + Doxygen (with Dot) + Sphynx #### Development * Follows Mozilla C++ Style Guide https://www-archive.mozilla.org/hacking/mozilla-style-guide.html + Uses post-commit hook to run the linter, you can set it up so it runs the linter before commit + All dependencies are defined in vcpkg.json * Uses cmake as build system, and provides a top level makefile with recommended