Apache HugeGraph(incubating) is a fast-speed and highly-scalable graph database/computing/AI ecosystem. Billions of vertices and edges can be easily stored into and queried from HugeGraph due to its excellent OLTP/OLAP ability.
 
Website: https://hugegraph.apache.org/
GitHub:

• https://github.com/apache/incubator-hugegraph/
• https://github.com/apache/incubator-hugegraph-ai/
   

  Description

Currently, we have implemented a basic GraphRAG that relies on fixed processing workflows (e.g., knowledge retrieval & graph structure updates using the same execution pipeline), leading to insufficient flexibility and high overhead in complex scenarios. The proposed task introduces an Agentic architecture based on the principles of "dynamic awareness, lightweight scheduling, concurrent execution," focusing on solving the following issues:

1. Rigid Intent Recognition: Existing systems cannot effectively distinguish between simple retrievals (e.g., entity queries) and complex operations (e.g., multi-hop reasoning), often defaulting to BFS-based template subgraph searches.
2. Coupled Execution Resources: Memory/computational resources are not isolated based on task characteristics, causing long-tail tasks to block high-priority requests.
3. Lack of Feedback Mechanisms: Absence of self-correction capabilities for erroneous operations (e.g., automatically switching to similar vertices/entities after path retrieval failures).

The task will include three core parts:

1. Dynamic Awareness Layer

• Implement an LLM-based real-time (as of February 14, 2025) intent classifier that categorizes tasks (L1 simple retrieval/L2 path reasoning/L3 graph computation/L4+ etc.) based on semantic features (verb types/entity complexity/temporal modifiers).
• Build a lightweight operation cache to generate feature hashes for high-frequency requests, enabling millisecond-level intent matching.

2. Task Orchestration Layer

• Introduce a suitable workflow/taskflow framework emphasizing low coupling, high performance, and flexibility.
• Adopt a preemptive scheduling mechanism allowing high-priority tasks to pause non-critical phases of low-priority tasks (e.g., suspending subgraph preloading without interrupting core computations).

3. Concurrent Execution

• Decouple traditional RAG pipelines into composable operations (entity recall → path validation → context enhancement → result refinement), with dynamic enable/disable support for each component.
• Implement automatic execution engine degradation, triggering fallback strategies upon sub-operation failures (e.g., switching to alternative methods if Gremlin queries timeout).

Recommended Skills

1. Proficiency in Python and familiarity with at least one open/closed-source LLM.
2. Experience with one LLM RAG/Agent framework like LangGraph/RAGflow/LLamaindex/Dify.
3. Knowledge of LLM optimization techniques and RAG construction (KG extraction/construction experience is a plus).
4. Strong algorithmic engineering skills (problem abstraction, algorithm research, big data processing, model tuning).
5. Familiarity with VectorDB/Graph/KG/HugeGraph read-write workflows and principles.
6. Understanding of graph algorithms (e.g., community detection, centrality, PageRank) and open-source community experience preferred.

Task List

• Develop a hierarchical triggering mechanism for the intent classifier to categorize L1~LN tasks within milliseconds (accuracy >90%).

• Semi-automatically generate Graph Schema/extraction prompts.

• Support dynamic routing and query decomposition.

• Design an execution trace tracker to log micro-operation resource consumption and generate optimization reports.

• Enhance retrieval with graph algorithms: Apply node importance evaluation, path search, etc., to optimize knowledge recall.

• Implement a dialogue memory management module for context-aware state tracking and information reuse.

Size

• Difficulty: Hard

• Project size: ~350 hours (full-time/large)

Potential Mentors

• Imba Jin: jin@apache.org (Apache HugeGraph PPMC)
• Simon: ming@apache.org (Apache HugeGraph PPMC)
   

Background

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform designed to orchestrate complex data processing tasks. It provides a user-friendly interface for defining, scheduling, and monitoring workflows, making it easier to manage and automate data pipelines. DolphinScheduler supports various types of tasks, including shell scripts, SQL queries, and custom scripts, and integrates seamlessly with popular big data ecosystems.

Currently, the Apache DolphinScheduler system supports user login via Password, LDAP, Casdoor SSO, and OAuth. However, as a data platform, it frequently needs to integrate with enterprise - internal user accounts to achieve unified identity authentication, which is crucial for ensuring system security and unified user account management. The existing implementation of Casdoor has a high degree of dependence on the Casdoor project, and the OAuth implementation lacks universality and flexibility.

Our objective is to implement a more generalized OIDC (OpenID Connect) login authentication mechanism. This will enable users to make better use of unified login authentication. Moreover, popular open source login authentication projects like Dexidp, Keycloak, and OAuthProxy all support OIDC. By supporting OIDC, users can integrate with both internal and third-party login authentication methods, such as Feishu Login and WeChat Work Login.

Relevant Skills

Strong proficiency in Java development.
Experience in modern frontend technologies and frameworks.
Highlevel expertise in Spring Boot development.
Thorough familiarity with OIDC and OAuth2 protocols.
Keen interest in opensource projects and eagerness to learn and adapt.

Tasks

Initiate and conduct experiments with Apache DolphinScheduler to comprehensively understand its current functionalities.
Implement and support a more generalized OIDC (OpenID Connect) login authentication mechanism.
Compose corresponding E2E test cases.
Create corresponding documentation for third-party login integrations, covering Keycloak, Dexidp, OAuthProxy, as well as Feishu Login and WeChat Work Login.
Optimize the UI of the Apache DolphinScheduler login page.
Ensure compatibility with the existing functionalities of Apache DolphinScheduler during the process of focusing on enhancements.

Learning Material

 
Apache DolphinScheduler HomePage: https://dolphinscheduler.apache.org
Apache DolphinScheduler GitHub Repository: https://github.com/apache/dolphinscheduler
Sprint OAuth 2.0 Client: https://docs.spring.io/spring-security/reference/reactive/oauth2/client/index.html
OIDC (OpenID Connect): https://openid.net/developers/how-connect-works/

Mentor

Gallardot, Apache DolphinScheduler committer, gallardot@apache.org

Difficulty: Medium
Project Size: ~150 hours (medium)