Description

Synopsis

The current Apache Seata Server supports the Raft cluster mode, but the performance and throughput of the cluster are significantly limited due to the single leader in a single Raft group. Therefore, the goal is to extend Seata Server to support multi-raft capability.

Benefits to Community

Due to the characteristics of Raft, requests are processed on the leader node and the results are submitted to the followers through the Raft consensus protocol. As a result, a significant amount of computational load is placed on the leader node, while followers only need to receive the final computed result. This causes the CPU, memory, and other metrics of the leader to be much higher than those of the followers. Additionally, the throughput of a single leader is limited by the machine configuration of the highest-spec node in the cluster, making it difficult to balance the traffic effectively. Therefore, supporting multi-raft would make the load distribution more balanced across all nodes in the cluster, improving throughput and performance, while also reducing the waste of machine resources.

Deliverables

The expected delivery goal is to apply the multi-raft capability of the sofa-jraft component to Seata Server through detailed learning and practice

The step expected are the following:

• Learning and using the sofa-jraft component

• Understanding and practicing the transaction grouping capability in Seata

• Gaining a certain level of understanding of Seata's communication protocol

• Gaining a certain level of understanding of Seata's storage model, especially the Raft mode

• Ensuring compatibility between different versions

Useful links

• seata-raft-detailed-explanation

• transaction-group

• sofa-jraft

Mentor

• • Mentor: Jianbin Chen, Apache Seata(Incubating) PPMC Member jianbin@apache.org

Background

Apache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with Redis protocol.
 
In the past, basic Web UI capabilities have been provided for Apache Kvrocks Controller, including features such as cluster creation and migration. In the future, we aim to offer a better and more modern UI experience, also enhancing centralized visualization capabilities.

Objectives

The key objectives of the project include the following:

• Refactor the existing UI pages

• Enhance the visualization capabilities for cluster migration

• Provide a cluster Overview dashboard

Recommend Skills

1. Familiar with next.js & tailwind
2. Have a basic understanding of RESTFul
3. Have an experience of Apache Kvrocks

Mentor: Hulk Lin, Apache Apache Kvrocks PMC,  hulk@apache.org
Mailing List: dev@kvrocks.apache.org
Please leave comments if you want to be a mentor

Apache Beam's YAML DSL provides a powerful and declarative way to define data processing pipelines. In particular, many users want to use Beam for machine learning use cases like feature generation, embedding generation, and retrieval augmented generation (RAG). Today, however, Beam integrates with a relatively limited set of feature stores and vector DBs for these use cases. This project aims to build out a rich ecosystem of connectors to systems like Pinecone and Tecton to enable these ML use cases.

Apache Beam's YAML DSL provides a powerful and declarative way to define data processing pipelines. However, its adoption for complex use cases like Machine Learning (ML) and Managed IO (specifically Apache Iceberg and Kafka) is hindered by a lack of comprehensive documentation and practical examples. This project aims to significantly improve the Beam YAML documentation and create illustrative examples focused on ML workflows and Iceberg/Kafka integration, making these advanced features more accessible to users.

The Apache Beam JupyterLab Sidepanel provides a valuable tool for interactive development and visualization of Apache Beam pipelines within the JupyterLab environment. This project aims to significantly enhance the sidepanel by achieving full compatibility with the latest JupyterLab 4.x release and implementing substantial UI/UX improvements. This will ensure seamless integration with modern JupyterLab workflows and provide a more intuitive and user-friendly experience for Apache Beam developers.

Background

Apache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with Redis protocol.
 
In the past, basic Web UI capabilities have been provided for Apache Kvrocks Controller, including features such as cluster creation and migration. In the future, we aim to offer a better and more modern UI experience, also enhancing centralized visualization capabilities.

Objectives

The key objectives of the project include the following:

• Refactor the existing UI pages

• Enhance the visualization capabilities for cluster migration

• Provide a cluster Overview dashboard

Recommend Skills

1. Familiar with next.js & tailwind
2. Have a basic understanding of RESTFul
3. Have an experience of Apache Kvrocks

Mentor: Hulk Lin, Apache Apache Kvrocks PMC,  hulk@apache.org
Mailing List: dev@kvrocks.apache.org
Please leave comments if you want to be a mentor