百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
G

GenAIComps

> 编程语言
开源

GenAI 组件在微服务级别; GenAI 服务合成器用于创建巨型服务

199 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

GenAI 组件在微服务级别; GenAI 服务合成器用于创建巨型服务

Generative AI Components (GenAIComps)

Build Enterprise-grade Generative AI Applications with Microservice Architecture

This initiative empowers the development of high-quality Generative AI applications for enterprises via microservices, simplifying the scaling and deployment process for production. It abstracts away infrastructure complexities, facilitating the seamless development and deployment of Enterprise AI services.

GenAIComps

GenAIComps provides a suite of microservices, leveraging a service composer to assemble a mega-service tailored for real-world Enterprise AI applications. All the microservices are containerized, allowing cloud native deployment. Check out how the microservices are used in GenAIExamples or Getting Start with OPEA to deploy the ChatQnA application from OPEA GenAIExamples across multiple cloud platforms.

Installation

  • Install from Pypi
bash
pip install opea-comps
  • Build from Source
bash
git clone https://github.com/opea-project/GenAIComps
cd GenAIComps
pip install -e .

MicroService

Microservices are akin to building blocks, offering the fundamental services for constructing RAG (Retrieval-Augmented Generation) and other Enterprise AI applications.

Each Microservice is designed to perform a specific function or task within the application architecture. By breaking down the system into smaller, self-contained services, Microservices promote modularity, flexibility, and scalability.

This modular approach allows developers to independently develop, deploy, and scale individual components of the application, making it easier to maintain and evolve over time. Additionally, Microservices facilitate fault isolation, as issues in one service are less likely to impact the entire system.

The initially supported Microservices are described in the below table. More Microservices are on the way.

MicroService Framework Model Serving HW Description
Embedding LangChain/LlamaIndex BAAI/bge-base-en-v1.5 TEI-Gaudi Gaudi2 Embedding on Gaudi2
Embedding LangChain/LlamaIndex BAAI/bge-base-en-v1.5 TEI Xeon Embedding on Xeon CPU
Retriever LangChain/LlamaIndex BAAI/bge-base-en-v1.5 TEI Xeon Retriever on Xeon CPU
Reranking LangChain/LlamaIndex BAAI/bge-reranker-base TEI-Gaudi Gaudi2 Reranking on Gaudi2
Reranking LangChain/LlamaIndex BAAI/bge-reranker-base TEI Xeon Reranking on Xeon CPU
ASR NA openai/whisper-small NA Gaudi2 Audio-Speech-Recognition on Gaudi2
ASR NA openai/whisper-small NA Xeon Audio-Speech-Recognition on Xeon CPU
TTS NA microsoft/speecht5_tts NA Gaudi2 Text-To-Speech on Gaudi2
TTS NA microsoft/speecht5_tts NA Xeon Text-To-Speech on Xeon CPU
Dataprep Qdrant sentence-transformers/all-MiniLM-L6-v2 NA Gaudi2 Dataprep on Gaudi2
Dataprep Qdrant sentence-transformers/all-MiniLM-L6-v2 NA Xeon Dataprep on Xeon CPU
Dataprep Redis BAAI/bge-base-en-v1.5 NA Gaudi2 Dataprep on Gaudi2
Dataprep Redis BAAI/bge-base-en-v1.5 NA Xeon Dataprep on Xeon CPU
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 TGI Gaudi Gaudi2 LLM on Gaudi2
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 TGI Xeon LLM on Xeon CPU
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 Ray Serve Gaudi2 LLM on Gaudi2
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 Ray Serve Xeon LLM on Xeon CPU
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 vLLM Gaudi2 LLM on Gaudi2
LLM LangChain/LlamaIndex Intel/neural-chat-7b-v3-3 vLLM Xeon LLM on Xeon CPU

A Microservices can be created by using the decorator register_microservice. Taking the embedding microservice as an example:

python
from comps import register_microservice, EmbedDoc, ServiceType, TextDoc

@register_microservice(
    name="opea_service@embedding_tgi_gaudi",
    service_type=ServiceType.EMBEDDING,
    endpoint="/v1/embeddings",
    host="0.0.0.0",
    port=6000,
    input_datatype=TextDoc,
    output_datatype=EmbedDoc,
)
def embedding(input: TextDoc) -> EmbedDoc:
    embed_vector = embeddings.embed_query(input.text)
    res = EmbedDoc(text=input.text, embedding=embed_vector)
    return res

MegaService

A Megaservice is a higher-level architectural construct composed of one or more Microservices, providing the capability to assemble end-to-end applications. Unlike individual Microservices, which focus on specific tasks or functions, a Megaservice orchestrates multiple Microservices to deliver a comprehensive solution.

Megaservices encapsulate complex business logic and workflow orchestration, coordinating the interactions between various Microservices to fulfill specific application requirements. This approach enables the creation of modular yet integrated applications, where each Microservice contributes to the overall functionality of the Megaservice.

Here is a simple example of building Megaservice:

…

self.gateway = ChatQnAGateway(megaservice=self.megaservice, host="0.0.0.0", port=self.port)

bash

## Check Mega/Micro Service health status and version number

Use the command below to check Mega/Micro Service status.

```bash
curl http://${your_ip}:${service_port}/v1/health_check\
  -X GET \
  -H 'Content-Type: application/json'

Users should get output like below example if Mega/Micro Service works correctly.

bash
{"Service Title":"ChatQnAGateway/MicroService","Version":"1.0","Service Description":"OPEA Microservice Infrastructure"}

Contributing to OPEA

Welcome to the OPEA open-source community! We are thrilled to have you here and excited about the potential contributions you can bring to the OPEA platform. Whether you are fixing bugs, adding new GenAI components, improving documentation, or sharing your unique use cases, your contributions are invaluable.

Together, we can make OPEA the go-to platform for enterprise AI solutions. Let's work together to push the boundaries of what's possible and create a future where AI is accessible, efficient, and impactful for everyone.

Please check the [C

Issues· 91 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言