You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 18 Next »

This page is auto-generated! Please do NOT edit it, all changes will be lost on next update

Contents

Apache Dubbo

GSoC 2025 - Service Discovery

Background and Goal

Service Discovery

  1. Well organized logs
  2. Actuator endpoints
  3. Tools

Relevant Skills

  1. Familiar with Java
  2. Familiar with Microservice architecture

Potential Mentors

  1. Jun Liu, Apache Dubbo PMC Chair, junliu@apache.org
  2. dev@dubbo.apache.org

     


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Jun Liu, mail: liujun (at) apache.org
Project Devs, mail:

GSoC 2025 - Add more traffic management rule support for Dubbo Proxyless Mesh

Background and Goal

The concept of[ Proxyless Mesh|https://istio.io/v1.15/blog/2021/proxyless-grpc/] was first introduced in this blog. Please read it to learn more concept details.

We have started the development of Dubbo Proxyss Mesh for a while, so that means you don't have to start the project from scratch, anyone who gets involved can start with a specific task at hand. 

In this specific GSoC project, we need developers to mainly focus on implementing more traffic management features of Istio for Dubbo.

Relevant Skills

  1. Familiar with Java
  2. Familiar with Service Mesh, istio and Microservice architectures
  3. Familiar with Kubernetes

Potential Mentors

  1. Jun Liu, Apache Dubbo PMC Chair, junliu@apache.org


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Jun Liu, mail: liujun (at) apache.org
Project Devs, mail:

GSoC 2025 - Dubbo Admin traffic management feature

Background and Goal

Dubbo is an easy-to-use, high-performance microservice framework that provides both RPC and rich enterprise-level traffic management features.

The community has been working on the improvement of Dubbo's traffic management abilities, to make it support rich features like traffic spliting, canary release, a/b testing, circuit breaker, mocking, etc. The complete traffic management architecture in Dubbo consists of two major parts, Control Plane and Data Plane. In Dubbo, Control Plane refers to Dubbo Admin, with source code in apache/dubbo-kubernetes. Dubbo Data Plane is implemented by Dubbo sdk (Java, Go, etc)

The traffic management rules Dubbo ueses now is compatible with the rules in Istio. That means the rules generated by Dubbo Admin and sent to SDK is Istio compatible rules. In this project, we need developers to work mainly on Dubbo Admin to make sure it generates and sends those rules correctly.


Relevant Skills

  1. Familiar with Golang
  2. Familiar with Service Mesh, istio and Microservice architectures
  3. Familiar with Kubernetes

Potential Mentors

  1. Jun Liu, Apache Dubbo PMC Chair, junliu@apache.org
  2. dev@dubbo.apache.org
Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Jun Liu, mail: liujun (at) apache.org
Project Devs, mail:

GSoC 2025 - Enhancing Dubbo Python Serialization

Background and Goal

Currently, Dubbo Python exposes a serialization function interface that requires users to implement their own serialization methods. For commonly used serialization formats such as JSON and Protobuf, users must manually configure them each time. To streamline this process, we aim to build a built-in serialization layer that provides support for these common serialization formats by default.

Goal

We recommend using Pydantic to achieve this. Therefore, we expect the implementation to:

1. an internal serialization layer based on Pydantic, with support for at least JSON and Protobuf.

2. Leverage Pydantic's additional features, including data validation and other useful functionalities.

Relevant Skills

1. Familiar with Python

2. Familiar with RPC

Potential Mentors

  1. Albumen Kevin, Apache Dubbo PMC, albumenj@apache.org
  2. dev@dubbo.apache.org

     

     


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Albumen Kevin, mail: albumenj (at) apache.org
Project Devs, mail:

GSoC 2025 - Dubbo triple protocol for go language implementation

Background and Goal

Dubbo is an easy-to-use, high-performance microservice framework that provides both RPC and rich enterprise-level traffic management features.

  1. keep-alive
  2. connection management
  3. programming api
  4. error code

Relevant Skills

  1. Familiar with Golang
  2. Familiar with RPC
  3. Familiar HTTP/1/2/3 protocol

Potential Mentors

  1. Jun Liu, Apache Dubbo PMC Chair, junliu@apache.org
  2. dev@dubbo.apache.org

     

     

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Jun Liu, mail: liujun (at) apache.org
Project Devs, mail:

GSoC 2025 - Dubbo Gradle IDL Plugin

Background and Goal

In the API-First design paradigm, IDL (Interface Definition Language) and its corresponding generation tools have become essential. IDL files are the specifications for defining service interfaces, and generation tools can convert IDL files into executable code, thereby simplifying the development process and improving efficiency.
Currently, Apache Dubbo only provides a Maven IDL generation plugin, lacking a Gradle plugin. This brings inconvenience to developers using Gradle to build projects.

Necessity

  1. Unify Build Tools: Gradle is the preferred build tool for Android projects and many Java projects. Providing a Dubbo Gradle IDL plugin can maintain the consistency of build tools and reduce the cost for developers to switch between different build tools.
  2. Simplify Configuration: Gradle plugins can simplify the configuration and generation process of IDL files. Developers only need to add plugin dependencies and simple configurations in the `build.gradle` file to complete the generation of IDL files without manually executing complex commands.
  3. Integrate Development Process: Gradle plugins can be better integrated with IDEs (Integrated Development Environments). Developers can directly execute Gradle tasks in the IDE, thereby realizing the automatic generation of IDL files and improving development efficiency.

Implementation Plan

  1. Plugin Development: Develop a Gradle plugin that encapsulates the Dubbo IDL generation tool and provides a concise configuration interface.
  2. Configuration: In the `build.gradle` file, developers can configure parameters such as the path of the IDL file and the directory of the generated code.
  3. Task: The plugin provides a Gradle task for executing the generation of IDL files. Developers can execute the task through the command line or the IDE.
  4. Dependency Management: The plugin can automatically manage the dependencies of the Dubbo IDL generation tool, ensuring that developers do not need to manually download and configure it.

Expected Results

  • Developers can use Gradle to build Dubbo projects and easily generate the code corresponding to the IDL.
  • Simplify the configuration and generation process of IDL files, and improve development efficiency.
  • Better integration with IDEs to achieve automatic generation of IDL files.

Potential Mentors

  1. Albumen Kevin, Apache Dubbo PMC, albumenj@apache.org
  2. dev@dubbo.apache.org

     

     


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Albumen Kevin, mail: albumenj (at) apache.org
Project Devs, mail:

Seata

GSoC 2025 - Apache Seata(Incubating) Extend multi-raft cluster mode

Description

Synopsis

The current Apache Seata Server supports the Raft cluster mode, but the performance and throughput of the cluster are significantly limited due to the single leader in a single Raft group. Therefore, the goal is to extend Seata Server to support multi-raft capability.


Benefits to Community

Due to the characteristics of Raft, requests are processed on the leader node and the results are submitted to the followers through the Raft consensus protocol. As a result, a significant amount of computational load is placed on the leader node, while followers only need to receive the final computed result. This causes the CPU, memory, and other metrics of the leader to be much higher than those of the followers. Additionally, the throughput of a single leader is limited by the machine configuration of the highest-spec node in the cluster, making it difficult to balance the traffic effectively. Therefore, supporting multi-raft would make the load distribution more balanced across all nodes in the cluster, improving throughput and performance, while also reducing the waste of machine resources.


Deliverables

The expected delivery goal is to apply the multi-raft capability of the sofa-jraft component to Seata Server through detailed learning and practice


The step expected are the following:

  • Learning and using the sofa-jraft component
  • Understanding and practicing the transaction grouping capability in Seata
  • Gaining a certain level of understanding of Seata's communication protocol
  • Gaining a certain level of understanding of Seata's storage model, especially the Raft mode
  • Ensuring compatibility between different versions

Useful links

Mentor

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Jianbin Chen, mail: jianbin (at) apache.org
Project Devs, mail: dev (at) seata.apache.org

Kvrocks

[GSOC][Kvrocks] Improve the controller UI

Background

Apache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with Redis protocol.
 
In the past, basic Web UI capabilities have been provided for Apache Kvrocks Controller, including features such as cluster creation and migration. In the future, we aim to offer a better and more modern UI experience, also enhancing centralized visualization capabilities.

Objectives

The key objectives of the project include the following:

  • Refactor the existing UI pages
  • Enhance the visualization capabilities for cluster migration
  • Provide a cluster Overview dashboard


Recommend Skills

  1. Familiar with next.js & tailwind
  2. Have a basic understanding of RESTFul
  3. Have an experience of Apache Kvrocks


Mentor: Hulk Lin, Apache Apache Kvrocks PMC,  hulk@apache.org
Mailing List: dev@kvrocks.apache.org
Please leave comments if you want to be a mentor

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Hulk Lin, mail: hulk (at) apache.org
Project Devs, mail: dev (at) kvrocks.apache.org

[GSOC][Kvrocks] Support database backup to cloud storage

Backgroud:

Kvrocks is a key-value database that provides a Redis-compatible API on top of RocksDB. Currently, Kvrocks lacks a built-in mechanism for database backup to cloud storage, which is crucial for data durability, disaster recovery, and scalability in cloud environments.

This project aims to implement a robust backup system that allows users to store Kvrocks backups directly in cloud storage services such as Amazon S3, Google Cloud Storage, and/or Azure Blob Storage. The solution will integrate with the existing Kvrocks backup and restore mechanisms while ensuring efficient and secure data transfer.

Deliverables:

  1. Cloud Storage Integration: Implement backup storage support for Amazon S3, Google Cloud Storage, and Azure Blob Storage using SDKs, REST APIs or libraries (e.g. Apache OpenDAL).
  1. Backup & Restore Commands: Extend Kvrocks’ backup functionality to allow exporting and importing database snapshots from cloud storage.
  1. Configuration & Authentication: Provide user-configurable options to specify storage credentials and backup parameters.
  1. Incremental Backup Support (Stretch Goal): Optimize storage usage by implementing differential or incremental backup capabilities.
  1. Documentation & Tests: Comprehensive documentation and test coverage to ensure reliability and ease of use.

Recommended Skills:

  1. Good at coding in C++;
  2. Knowledge about database internals and cloud storage;
  3. Knowledge about Kvrocks or Redis.

Mentor: Mingyang Liu, Apache Kvrocks PMC member,  twice@apache.org
Mailing List: dev@kvrocks.apache.org

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Mingyang Liu, mail: twice (at) apache.org
Project Devs, mail: dev (at) kvrocks.apache.org

Beam

Simplify management of Beam infrastructure, access control and permissions via Platform features

This project consists in a series of tasks that build a sort of 'infra platform' for Beam. Some tasks include:


A quality proposal will include a series of features beyond the ones listed above. Some ideas:

  • Detection of policy breakages, and nagging to fix
  • Security detections based on cloud logging
  • others?
Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Pablo Estrada, mail: pabloem (at) apache.org
Project Devs, mail: dev (at) beam.apache.org

Enhancing Apache Beam JupyterLab Sidepanel for JupyterLab 4.x and Improved UI/UX

The Apache Beam JupyterLab Sidepanel provides a valuable tool for interactive development and visualization of Apache Beam pipelines within the JupyterLab environment. This project aims to significantly enhance the sidepanel by achieving full compatibility with the latest JupyterLab 4.x release and implementing substantial UI/UX improvements. This will ensure seamless integration with modern JupyterLab workflows and provide a more intuitive and user-friendly experience for Apache Beam developers.

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
XQ Hu, mail: xqhu (at) apache.org
Project Devs, mail: dev (at) beam.apache.org

Beam YAML ML, Iceberg, and Kafka User Accessibility

Apache Beam's YAML DSL provides a powerful and declarative way to define data processing pipelines. However, its adoption for complex use cases like Machine Learning (ML) and Managed IO (specifically Apache Iceberg and Kafka) is hindered by a lack of comprehensive documentation and practical examples. This project aims to significantly improve the Beam YAML documentation and create illustrative examples focused on ML workflows and Iceberg/Kafka integration, making these advanced features more accessible to users.

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
XQ Hu, mail: xqhu (at) apache.org
Project Devs, mail: dev (at) beam.apache.org

Beam ML Vector DB/Feature Store integrations

Apache Beam's Python SDK provides a powerful way to define data processing pipelines. In particular, many users want to use Beam for machine learning use cases like feature generation, embedding generation, and retrieval augmented generation (RAG). Today, however, Beam integrates with a relatively limited set of feature stores and vector DBs for these use cases. This project aims to build out a rich ecosystem of connectors to systems like Pinecone and Tecton to enable these ML use cases.

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Danny McCormick, mail: damccorm (at) apache.org
Project Devs, mail: dev (at) beam.apache.org

RocketMQ

Optimizing Apache RocketMQ's POP Orderly Consumption Process

Background

 
Apache RocketMQ is a distributed messaging and streaming platform that supports various messaging protocols. One of the key features of RocketMQ is its orderly message consumption capability, which guarantees that messages are processed in the order they are sent. However, there are existing issues with the POP Orderly consumption process that need to be addressed to enhance its reliability and performance.

 Current Challenges


Currently, the POP Orderly feature faces several shortcomings, particularly in scenarios where network instability leads to the loss of the attemptId carried by the consumer from the previous round. This issue can result in message consumption getting stuck until the acknowledgment response (ack) for the previous message pull times out. Such situations hinder the efficient processing of messages and reduce the overall effectiveness of the messaging system.

 Objectives

The primary objectives of this project are as follows:
●Refactor the POP Orderly Code: Analyze and redesign the existing codebase to improve its structure, maintainability, and performance.
●Optimize Performance: Implement performance enhancements that allow the POP Orderly feature to cope with network fluctuations and reduce the likelihood of consumption halting.
●Elegant Process Resolution: Develop a more graceful approach to handling the issue of consumption stalling, ensuring that the system can recover more smoothly from failures.

 Recommended Skills


1. Proficiency in Java programming.
2. Strong understanding of concurrent programming.
3. Excellent logical thinking and problem-solving skills.
4. Familiarity with message queue systems, particularly Apache RocketMQ.

 
 

Mentor


Rongtong Jin, Apache RocketMQ PMC, jinrongtong@apache.org

Potential Mentor

Juntao Ji, 3160102420@zju.edu.cn


Difficulty: Major
Project Size: ~350 hours (large)


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Rongtong Jin, mail: jinrongtong (at) apache.org
Project Devs, mail: dev (at) rocketmq.apache.org

Refactoring the RocketMQ Dashboard UI and Enhancing Usability

Background


Apache RocketMQ is renowned as a cloud-native messaging and streaming platform, enabling the creation of event-driven applications with simplicity and flexibility. The RocketMQ Dashboard is a crucial component that provides users with insight into system performance and client interactions through intuitive graphs and statistical data. Despite its fundamental role, the current user interface (UI) of the RocketMQ Dashboard is outdated, affecting user experience and interaction efficiency. Additionally, while the Dashboard offers valuable functionalities, there is a pressing need to enhance its usability and ensure robust security. This project aims to refactor the RocketMQ Dashboard by redesigning its UI with a more contemporary and user-friendly approach, improving overall usability, and introducing effective security measures to safeguard data and user interactions.

Relevant Skills

  • Strong Java development skills.
  • Experience with modern front-end technologies and frameworks
  • Proficiency in Spring Boot development.
  • Understanding of UX/UI design principles. - Knowledge of security best practices in web applications.
  • A keen interest in open-source projects and a willingness to learn and adapt.

Tasks


  • Launch and experiment with the RocketMQ Dashboard to understand current functionalities.
  • Refactor the UI of the RocketMQ Dashboard to align with modern user interface standards, ensuring it is intuitive and visually appealing.  
  • Improve usability by streamlining workflows, enhancing navigation, and incorporating responsive design. 
  • Integrate security features to protect user data, prevent unauthorized access, and mitigate potential vulnerabilities.
  • Maintain compatibility with existing RocketMQ functionalities while focusing on enhancements. 

Learning Material



Mentor

Rongtong Jin, Apache RocketMQ PMC, jinrongtong@apache.org

Potential Mentor

Juntao Ji, 3160102420@zju.edu.cn

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Rongtong Jin, mail: jinrongtong (at) apache.org
Project Devs, mail: dev (at) rocketmq.apache.org

SkyWalking

SkyWalking BanyanDB Extend remote.FS with Object Storage Support for AWS, Google Cloud, and Azure

Overview:
The current implementation of the remote.FS interface only supports a local file system (via the implementation in local.go). This GSOC2025 project proposes to extend remote.FS with popular object storage services—namely AWS S3, Google Cloud Storage, and Azure Blob Storage. This enhancement will allow the project to support robust cloud-based backup and restore operations in addition to local storage.

Proposed Features:

  1. AWS S3 Implementation:
    • Implement methods for Upload, Download, List, and Delete operations using the AWS S3 API.
  2. Google Cloud Storage Implementation:
    • Provide a module that integrates with Google Cloud Storage to perform similar operations.
  3. Azure Blob Storage Implementation:
    • Develop functionality to access and manage Azure Blob Storage via the remote.FS interface.

Implementation Details:

  • Interface Compliance:
    Each object storage implementation must adhere to the remote.FS interface defined in remote.go.
  • Error Handling & Resilience:
    Implement robust error handling, logging, and retry mechanisms to ensure reliable operations across different cloud services.
  • Testing:
    Develop comprehensive unit and integration tests to cover edge cases and guarantee compatibility and stability.
  • Documentation:
    Update the project documentation to detail configuration, deployment, and usage of each cloud storage option.


Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Hongtao Gao, mail: hanahmily (at) apache.org
Project Devs, mail: dev (at) skywalking.apache.org

HugeGraph

[GSoC][HugeGraph] Implement Agentic GraphRAG Architecture

Apache HugeGraph(incubating) is a fast-speed and highly-scalable graph database/computing/AI ecosystem. Billions of vertices and edges can be easily stored into and queried from HugeGraph due to its excellent OLTP/OLAP ability.
 
Website: https://hugegraph.apache.org/
GitHub:

Currently, we have implemented a basic GraphRAG that relies on fixed processing workflows (e.g., knowledge retrieval & graph structure updates using the same execution pipeline), leading to insufficient flexibility and high overhead in complex scenarios. The proposed task introduces an Agentic architecture based on the principles of "dynamic awareness, lightweight scheduling, concurrent execution," focusing on solving the following issues:

  1. Rigid Intent Recognition: Existing systems cannot effectively distinguish between simple retrievals (e.g., entity queries) and complex operations (e.g., multi-hop reasoning), often defaulting to BFS-based template subgraph searches.
  2. Coupled Execution Resources: Memory/computational resources are not isolated based on task characteristics, causing long-tail tasks to block high-priority requests.
  3. Lack of Feedback Mechanisms: Absence of self-correction capabilities for erroneous operations (e.g., automatically switching to similar vertices/entities after path retrieval failures).

The task will include three core parts:

1. Dynamic Awareness Layer

  • Implement an LLM-based real-time (as of February 14, 2025) intent classifier that categorizes tasks (L1 simple retrieval/L2 path reasoning/L3 graph computation/L4+ etc.) based on semantic features (verb types/entity complexity/temporal modifiers).
  • Build a lightweight operation cache to generate feature hashes for high-frequency requests, enabling millisecond-level intent matching.

2. Task Orchestration Layer

  • Introduce a suitable workflow/taskflow framework emphasizing low coupling, high performance, and flexibility.
  • Adopt a preemptive scheduling mechanism allowing high-priority tasks to pause non-critical phases of low-priority tasks (e.g., suspending subgraph preloading without interrupting core computations).

3. Concurrent Execution

  • Decouple traditional RAG pipelines into composable operations (entity recall → path validation → context enhancement → result refinement), with dynamic enable/disable support for each component.
  • Implement automatic execution engine degradation, triggering fallback strategies upon sub-operation failures (e.g., switching to alternative methods if Gremlin queries timeout).

Recommended Skills

  1. Proficiency in Python and familiarity with at least one open/closed-source LLM.
  2. Experience with one LLM RAG/Agent framework like LangGraph/RAGflow/LLamaindex/Dify.
  3. Knowledge of LLM optimization techniques and RAG construction (KG extraction/construction experience is a plus).
  4. Strong algorithmic engineering skills (problem abstraction, algorithm research, big data processing, model tuning).
  5. Familiarity with VectorDB/Graph/KG/HugeGraph read-write workflows and principles.
  6. Understanding of graph algorithms (e.g., community detection, centrality, PageRank) and open-source community experience preferred.

Task List

  • Develop a hierarchical triggering mechanism for the intent classifier to categorize L1~LN tasks within milliseconds (accuracy >90%).
  • Semi-automatically generate Graph Schema/extraction prompts.
  • Support dynamic routing and query decomposition.
  • Design an execution trace tracker to log micro-operation resource consumption and generate optimization reports.
  • Enhance retrieval with graph algorithms: Apply node importance evaluation, path search, etc., to optimize knowledge recall.
  • Implement a dialogue memory management module for context-aware state tracking and information reuse.

Size

  • Difficulty: Hard
  • Project size: ~350 hours (full-time/large)

Potential Mentors

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Imba Jin, mail: jin (at) apache.org
Project Devs, mail:

DolphinScheduler

Enhancing Apache DolphinScheduler with Generalized OIDC Authentication

Background

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform designed to orchestrate complex data processing tasks. It provides a user-friendly interface for defining, scheduling, and monitoring workflows, making it easier to manage and automate data pipelines. DolphinScheduler supports various types of tasks, including shell scripts, SQL queries, and custom scripts, and integrates seamlessly with popular big data ecosystems.

Currently, the Apache DolphinScheduler system supports user login via Password, LDAP, Casdoor SSO, and OAuth. However, as a data platform, it frequently needs to integrate with enterprise - internal user accounts to achieve unified identity authentication, which is crucial for ensuring system security and unified user account management. The existing implementation of Casdoor has a high degree of dependence on the Casdoor project, and the OAuth implementation lacks universality and flexibility.

Our objective is to implement a more generalized OIDC (OpenID Connect) login authentication mechanism. This will enable users to make better use of unified login authentication. Moreover, popular open source login authentication projects like Dexidp, Keycloak, and OAuthProxy all support OIDC. By supporting OIDC, users can integrate with both internal and third-party login authentication methods, such as Feishu Login and WeChat Work Login.

Relevant Skills

Strong proficiency in Java development.
Experience in modern frontend technologies and frameworks.
Highlevel expertise in Spring Boot development.
Thorough familiarity with OIDC and OAuth2 protocols.
Keen interest in opensource projects and eagerness to learn and adapt.

Tasks

Initiate and conduct experiments with Apache DolphinScheduler to comprehensively understand its current functionalities.
Implement and support a more generalized OIDC (OpenID Connect) login authentication mechanism.
Compose corresponding E2E test cases.
Create corresponding documentation for third-party login integrations, covering Keycloak, Dexidp, OAuthProxy, as well as Feishu Login and WeChat Work Login.
Optimize the UI of the Apache DolphinScheduler login page.
Ensure compatibility with the existing functionalities of Apache DolphinScheduler during the process of focusing on enhancements.

Learning Material

 
Apache DolphinScheduler HomePage: https://dolphinscheduler.apache.org
Apache DolphinScheduler GitHub Repository: https://github.com/apache/dolphinscheduler
Sprint OAuth 2.0 Client: https://docs.spring.io/spring-security/reference/reactive/oauth2/client/index.html

pac4j OIDC: https://www.pac4j.org/docs/clients/openid-connect.html
OIDC (OpenID Connect): https://openid.net/developers/how-connect-works/

Mentor

Gallardot, Apache DolphinScheduler committer, gallardot@apache.org

SbloodyS, Apache DolphinScheduler PMC, zihaoxiang@apache.org

Difficulty: Medium
Project Size: ~150 hours (medium)

Difficulty: Major
Project size: ~175 hour (medium)
Potential mentors:
Hengliang Tan, mail: gallardot (at) apache.org
Project Devs, mail: dev (at) dolphinscheduler.apache.org

Lucene.NET

Apache Lucene.NET Replicator and Dependency Injection Enhancements

Background and Goal

Apache Lucene.NET is a .NET port of the Apache Lucene search engine (originally written in Java). This powerful library enables indexing and searching of documents with custom queries, making it a core component in many production environments. With over 100 million NuGet downloads, Lucene.NET is utilized in diverse scenarios, from local search functionality in mobile apps to supporting large-scale cloud infrastructures.

Lucene.NET already provides a foundation for replicating a search index from a primary node to one or more replica nodes, enabling High Availability (HA) and scalability through load balancing. Currently, our Lucene.Net.Replicator.AspNetCore project offers minimal replication support for ASP.NET Core servers, but it remains unpublished on NuGet and lacks the robustness required for most use cases. Your focus for this project will be to enhance and finalize the ASP.NET Core library, ensuring a seamless user experience by adhering to best practices and making replication setup as straightforward as possible – ideally requiring just one line of code.

Additionally, users may need replication support for applications outside ASP.NET Core, such as cloud-based distributed architectures, Windows services, or command-line tools running on Linux. To address this, we propose creating modular intermediate libraries using Microsoft.Extensions.DependencyInjection.Abstractions, enabling flexible and reusable replication configurations. This approach should also ensure that essential components like IndexWriter and IndexReader are configured in a straightforward and user-friendly manner.

Your task will also include creating one or more sample projects that demonstrate how to effectively use the enhanced replication functionality. These projects should serve as practical, real-world examples for the community, showcasing best practices and ease of use. Additionally, you will be responsible for thoroughly testing the code changes to ensure they work as intended in real-world scenarios. This includes writing comprehensive unit tests to guarantee the reliability and quality of the solution.

We plan for this to be a hands-on mentorship, and we will set up any infrastructure for you. As a contributor, your responsibilities will include analyzing the problem, developing a detailed plan, refining it with input from the project team, and collaborating regularly to implement the solution through pull requests and code reviews.

Relevant Skills

  • Familiarity with C# and unit testing
  • Strong grasp of design patterns and practices, such as dependency injection and i.e. the fluent builder and abstract factory patterns
  • Basic understanding of HTTP(S) and networking
  • Not required, but good to have:
    • Familiarity with ASP.NET Core 5 or later
    • Understanding of distributed architectures
    • Familiarity with Lucene(.NET) search indexes
Difficulty: Normal
Project size: ~175 hour (medium)
Potential mentors:
Paul Irwin, mail: paulirwin (at) apache.org
Project Devs, mail: dev (at) lucenenet.apache.org

CloudStack

Apache CloudStack DRS improvements

As a Operator I would like to have the loads on my systems more evenly/centrally distributed. At the moment there is a simple DRS for clusterwide distribution of loads, this is however not applying zone wide distribution or based on automated queries/improvements.

In addition we should add historic data for the VM in planning possible migrations.
At the moment allocated metrics are used. An first improvement would be to use actual metrics.


ref: cloudstack issue: https://github.com/apache/cloudstack/issues/10397

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

verification of LDAP connection

When a new ldap connection is added there is no diagnostics to verify the validity/usability of the connection, making trouble shooting troublesome. This issue aims to facilitate ldap configuration.

ref. cloudstack issue: https://github.com/apache/cloudstack/issues/6934

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

SSL - LetsEncrypt the Console Proxy

New Global Option For Letsencrypt enable on console proxy. Letsencrypt domain name option for letsencrypt ssl auto renew

ref. cloudstack issue: https://github.com/apache/cloudstack/issues/3141

Difficulty: Major
Project size: ~175 hour (medium)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

Autodetect IPs used inside the VM on L2 networks

With regards to IP info reporting, Cloudstack relies entirely on it's DHCP data bases and so on. When this is not available (L2 networks etc) no IP information is shown for a given VM.

I propose we introduce a mechanism for "IP autodetection" and try to discover the IPs used inside the machines by means of querying the hypervisors. For example with KVM/libvirt we can simply do something like this:
 {{root@fedora35 ~]# virsh domifaddr win2k22 --source agent
Name MAC address Protocol Address
-------------------------------------------------------------------------------
Ethernet 52:54:00:7b:23:6a ipv4 192.168.0.68/24
Loopback Pseudo-Interface 1 ipv6 ::1/128

  • - ipv4 127.0.0.1/8}}
     
    The above command queries the qemu-guest-agent inside the Windows VM. The VM needs to have the qemu-guest-agent installed and running as well as the virtio serial drivers (easily done in this case with virtio-win-guest-tools.exe ) as well as a guest-agent socket channel defined in libvirt.

ref. cloudstack issue: https://github.com/apache/cloudstack/issues/7142

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

eBPF-based Network Observability for CloudStack

CloudStack’s network monitoring is mostly based on logs and external agents, making real-time traffic analysis difficult. This project will integrate eBPF-based network observability to capture per-VM traffic metrics, detect anomalies, and improve tenant isolation.

Benefits to CloudStack

  • Enhanced security: Detect suspicious activity at the kernel level.
  • Real-time traffic monitoring: Gain deep insights into VM networking.
  • Better tenant isolation: Identify cross-tenant traffic issues.

Deliverables

  • Develop eBPF probes to capture:
    • Per-VM network traffic metrics (packets, bytes, latency)
    • Connection tracking for detecting unauthorized access patterns
    • Packet drops and retransmission rates
  • Expose network metrics via CloudStack’s API.
  • Provide visualization through Prometheus/Grafana.
  • Document setup, usage, and performance benchmarks.

Expected Outcome

An eBPF-based solution that improves network observability in CloudStack, providing security and performance insights with minimal resource usage.


Difficulty: Major
Project size: ~175 hour (medium)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

Enhancing CloudStack Monitoring with eBPF

Apache CloudStack currently relies on traditional monitoring tools, which may lack deep visibility into kernel-level events and networking performance. This project aims to integrate eBPF-based monitoring into CloudStack to provide lightweight, real-time performance analysis and security auditing.

Benefits to CloudStack

  • Improved observability: Gain fine-grained insights into VM performance metrics.
  • Lower overhead: eBPF runs in the kernel and avoids the performance penalties of user-space monitoring tools.
  • Enhanced security auditing: Detect and log anomalies in system behavior.

Deliverables

  • Implement eBPF programs to track:
    • VM CPU usage
    • Memory consumption
    • Disk I/O metrics
    • Network traffic analysis
  • Develop a CloudStack-compatible API or CLI for retrieving eBPF-generated insights.
  • Provide visualization support using Prometheus/Grafana.
  • Write documentation for setup and usage.

Expected Outcome

A robust eBPF-based monitoring solution integrated into CloudStack, offering real-time performance insights with minimal overhead.

ref. cloudstack issue: https://github.com/apache/cloudstack/issues/10415


This project is marked as part-time, but the scope can be extended to full-time. This depends largely on whether the full amount of metrics to track is implemented or only one, as a proof of concept.

Difficulty: Major
Project size: ~175 hour (medium)
Potential mentors:
Daan, mail: dahn (at) apache.org
Project Devs, mail: dev (at) cloudstack.apache.org

StreamPipes

Extend visualization capabilities of Apache StreamPipes

Background

 
Apache StreamPipes is a self-service Industrial IoT toolbox which helps users to connect, analyze and exploit industrial data streams. StreamPipes offers a variety of tools which help users to interact with data from industrial sources such as PLCs. An adapter library allows to get real-time data from industrial controllers or other systems, a pipeline editor allows to build stream processing pipelines using either graphical or code-based flow modeling, and a data explorer allows to quickly create visualizations based on connected adapters.

 Current Challenges

The StreamPipes data explorer consists of a chart view, where users can create charts based on live data, and a dashboard view, where users can create live dashboards based on charts.

The data explorer provides a set of charts, which are mainly based on Apache ECharts. The currently available chart library includes time-series line/bar charts, heatmaps, scatter plots, density charts and others. To improve the user experience and add additional capabilities, we plan to extend this chart library with additional charts that are useful for industrial data analytics. 

 Objectives

The primary objectives of this project are as follows:

  • Explore the Apache ECharts library and identify useful additional charts for industrial data analytics
  • Improve the StreamPipes data explorer by adding new chart types using Apache ECharts
  • Add a more advanced table visualization
  • Extend existing charts with additional configurations (e.g., axis configurations, labels, data transformations)
  • Add a data preview for all charts, which is shown below the actual chart in the chart view
  • Design and implement end-to-end-tests using Cypress

 Recommended Skills


  1. Proficiency in TypeScript programming + testing 
  2. Proficiency in Angular
  3. Excellent logical thinking and problem-solving skills.
  4. Good sense for beautifully looking user interfaces 


Mentor


Dominik Riemer, Apache StreamPipes PMC, riemer@apache.org

Difficulty: Major
Project Size: ~350 hours (large)

Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Dominik Riemer, mail: riemer (at) apache.org
Project Devs, mail: dev (at) streampipes.apache.org

Comdev GSOC

[GSOC][HertzBeat] AI Agent Based on the MCP Protocol for Monitoring Info Interaction

Website: https://hertzbeat.apache.org/

Github: http://github.com/apache/hertzbeat/

*Background*

Apache HertzBeat is an open-source real-time monitoring tool that supports a wide range of monitoring targets, including web services, databases, middleware, and more. It features high performance, scalability, and security.

With the advancement of artificial intelligence (AI) technologies, integrating AI with monitoring systems can significantly enhance their usability and interactivity. By developing an AI Agent based on the Model Context Protocol (MCP), we aim to enable conversational interaction for querying monitoring information, adding new monitoring tasks, and retrieving monitoring metrics. This will provide a more user-friendly and intelligent monitoring management experience.

*Objectives*

1. Research and Implementation: Develop an AI Agent based on Apache HertzBeat and the MCP protocol to enable conversational interaction with users.

2. Functional Implementation:

  • Query Monitoring And Alarm Information: Allow users to query the status of monitoring targets (e.g., normal, abnormal) and retrieve metrics data (e.g., CPU usage, memory usage, response time), alarm data through conversational commands.
  • Add New Monitoring Tasks: Enable users to add new monitoring targets (e.g., web services, databases, middleware) and configure alert thresholds via conversational commands.
  • Retrieve Monitoring Metrics Data: Allow users to obtain metrics data for specific monitoring targets and support data visualization via conversational commands.

*Requirements Analysis*

  • Apache HertzBeat: As the core backend for the monitoring system, it provides functions for data collection, storage, and management.
  • MCP Protocol: An open protocol that enables seamless integration between LLM applications and external data sources and tools.
  • Front-end Interaction: Develop a user-friendly interface that supports voice or text input and displays monitoring information and interaction results.

*Recommended Skills*

  • Java + TypeScript: Apache HertzBeat is developed based on this technology stack. Therefore, mastering these technologies is crucial for integrating with HertzBeat.
  • SpringAi: It is recommended to use SpringAi to build the AI agent.
  • LLM + MCP: You need to have an understanding of LLM (Large Language Models) and the MCP protocol. SpringAi seem supports the MCP protocol or consider use the mcp-sdk directly.

*Size*

  • Difficulty: Hard
  • Project size: ~350 hours

*Potential Mentors*

  • Chao Gong: gongchao@apache.org 
     
  • Shenghang Zhang: shenghang@apache.org
Difficulty: Major
Project size: ~350 hour (large)
Potential mentors:
Chao Gong, mail: gongchao (at) apache.org
Project Devs, mail:
  • No labels