Apache HugeGraph is a fast-speed and highly-scalable graph database/computing/AI ecosystem. Billions of vertices and edges can be easily stored into and queried from HugeGraph due to its excellent OLTP ability. ​​

Description

Currently, the HugeGraph core query engine is built on Java 11 + TinkerPop 3.5.x + Groovy 3. While this stack provides fundamental graph query capabilities, it lags behind in security, performance optimization, and support for modern features. Specifically, the built-in Groovy engine relies on complex, high-maintenance black/whitelist mechanisms for script security, which poses potential bypass risks.

The goal of this task is to comprehensively upgrade HugeGraph's underlying dependencies to Java 17 + TinkerPop 3.7/3.8 + Groovy 4. This is not just a version iteration, but a modern architectural transformation:

1. Groovy 4 & TinkerPop 3.7/3.8: Introduce improved syntax features and security designs. We aim to refactor HugeGraphSecurity using native, efficient sandboxing mechanisms to replace the legacy blacklist logic.
2. Java 17/21 Support: Adapt to the new JDK to fully leverage features like ZGC/Shenandoah GC, Records, and Virtual Threads, significantly improving throughput and reducing long-tail latency in large-scale graph queries.

Applicants are expected to handle the full lifecycle, from dependency upgrades and code refactoring to unit test fixes and final performance benchmarking.

Recommended Skills

1. Java Core: Proficiency in Java development with a solid understanding of Java 17+ new features.
2. HugeGraph Architecture: Basic understanding of HugeGraph's storage structure (KV Store), Schema design, and specifically the Gremlin query execution flow.
3. Graph Computing & Compilers: Familiarity with the TinkerPop Gremlin framework architecture; knowledge of AST (Abstract Syntax Tree) parsing or Functional Programming (FP) mindset is a plus.
4. AI Coding: Proficiency in using AI Coding tools (e.g., Codex, Claude Code, Copilot) to assist in code refactoring, test case optimization, and source code interpretation is highly preferred.
5. Security Awareness: Awareness of code security, understanding of how to prevent Script Injection, and experience designing secure sandbox environments.

💡 Important Notes for Applicants

1. Authenticity Matters: While we encourage the use of AI for coding efficiency, please strictly control and reasonably limit the use of LLMs when writing your project proposal/emails. We value genuine communication and mutual respect.
2. Proactive Engagement: We highly recommend participating in community Mini Tasks early. Demonstrating your hands-on ability within the community will significantly increase your chances of selection and help build trust with mentors.

Task List

• Dependency Analysis & Upgrade:
  • Analyze Breaking Changes from TinkerPop 3.5 to 3.7/3.8.
  • Complete core dependency version upgrades and API adaptations following mentor confirmation.

• Java 17 Environment Adaptation:
  • Resolve compile-time and runtime compatibility issues (e.g., reflection restrictions, module access) to ensure the Server module runs correctly on Java 17 (Java 21 is even better).
  • Update Docker configurations to migrate the default runtime to Java 17 (while exploring backward compatibility with Java 11).

• PD & Store Module Upgrade (New):
  • Extend the upgrade scope to the PD (Placement Driver) and Store modules after completing the core Server upgrade.
  • Ensure these modules are adapted to Java 17 to unify the runtime environment across the HugeGraph ecosystem.

• Security Module Refactoring:
  • Refactor the HugeGraphSecurity component based on Groovy 4 features.
  • Design a lightweight, secure script execution strategy and remove the performance-heavy legacy blacklist logic.

• Testing & Fixes:
  • Fix Unit Test (UT) failures caused by the upgrade.
  • Ensure all core functions (CRUD, complex Gremlin queries) pass verification.

• Performance Benchmarking:
  • Produce a performance comparison report: Java 11 (Old) vs. Java 17 (New) using the Twitter-14B public dataset.
  • Quantify improvements in Latency reduction and Throughput increases.

References

• New Contributor Guide: HugeGraph Contribution Guide (Issue #2212) - Environment setup & basics.

• Upgrade Docs: TinkerPop Upgrade Documentation

• Reference Implementation: JanusGraph Upgrade PR (For reference only)

• Gremlin Learning: Practical Gremlin Guide

• Project Wiki: HugeGraph Deepwiki

Project Size

• Difficulty: Medium (Similar references available)

• Estimated Time: ~250 Hours (~15 Weeks)

Mentors

• Yan Zhang: vaughn@apache.org (Apache HugeGraph PMC)

• Imba Jin: jin@apache.org (Apache HugeGraph PMC)

Background

Apache Airflow’s Breeze environment is the de facto way to reproduce CI, run tests, and verify changes locally. It encapsulates complex tooling (Docker, integrations, static checks, tests, system verification) behind a single, consistent developer interface.

However, modern AI coding tools (e.g. Claude Code, Gemini CLI, GitHub Copilot–style agents) currently treat Airflow’s repo like any generic Python project. They rarely:

• Understand whether they are running inside or outside Breeze.
• Choose the correct commands for host vs. container.
• Follow the same workflows that Airflow contributors actually use (e.g. prek, breeze shell, breeze start-airflow).

We already expose some information through docs (e.g. AGENTS.md), but this mostly inflates the context window rather than giving agents a structured, machine-usable interface to Breeze.

This project aims to bridge that gap by creating an “Airflow Breeze Contribution / Contribution Verification” AI skill (final name TBD) that systematically encodes common contribution workflows and makes them reliably executable and testable by AI agents.

Goal

The overarching goal is to make AI tools:

Breeze-aware: able to detect whether they are running inside or outside Breeze and act accordingly.

In practice, this means that for a typical contributor PR, an AI agent can:

• Run the right static checks.
• Run the right subset of tests in Breeze.
• Spin up Airflow and verify system behavior for a Dag representing the change (nice-to-have).
• Do all of the above while respecting host/container boundaries.

Additionally, the solution should be consistency-focused, meaning that we want to keep Breeze CLI as the single source of truth for agent skills. This can be achieved by auto-syncing CLI docstrings and behaviors into the AI skill using existing tooling (e.g. prek), ensuring that the skill definitions always reflect the current state of the Breeze CLI.

Core Tasks

1. Environment Awareness & Detection

• Design and implement a simple, robust mechanism for the agent skills to detect:
  • “Host” vs “inside Breeze container”.
  • Relevant environment variables, markers, or file paths that indicate context.
• Encode decision logic for when to run:
  • Host-only commands (e.g. breeze shell, breeze start-airflow, git operations).
  • Container-only commands (e.g. pytest, airflow ...).
• Provide a clear API/contract that AI tools can call to query current context and get recommended commands.

Note: Maybe we need to add some explicit markers, files in the repo, or write a small helper script that can be called to determine context in a reliable way. Or maybe we can rely on existing environment variables or filesystem cues. This is an open design question to explore.

2. Modeling Core Contributor Workflows as Skills

Based on the three scenarios described, define and implement skills that represent common contribution flows:

Scenario 1: Static checks pass

• Stage changes (git add ...).
• Run prek.
• Collect and surface failures in a structured way so that an agent can fix them.

Scenario 2: Unit tests in Breeze

• Start or attach to a Breeze container with breeze shell or breeze exec.
• Run pytest with a targeted module/test path (not the whole suite).
• Then the agent can inspect results and decide on next steps (e.g. fix code, exit Breeze).

3. Syncing with Breeze CLI as Source of Truth (via prek)

• Investigate existing Breeze CLI docstrings and structure.
• Define a mapping from Breeze commands (and their docstrings) to skill definitions, paths, and parameters.
• Implement a prek hook that:
  • Generates or updates the agent skills definition files from Breeze CLI docstrings.
  • Fails when drift is detected (e.g. a command changed but the skill spec was not updated).
• Integrate these checks into existing static check pipelines so the skills stay in sync automatically.

4. Evaluation & Test Harness

• Design a testable user scenario or “exam” that simulates a typical contribution workflow (e.g. fixing a simple bug, adding a small feature) to verify that the added skills work as intended.
• Add unit tests for any additional scripts or helper functions created.

5. Documentation & Developer Guide

• Add or extend documentation (e.g. AGENTS.md, Breeze docs) to:
  • Describe the new Breeze-aware skills.
  • Show example workflows for human contributors and AI tools.
  • Document how other tools can integrate with the skills (e.g. path to spec file, key commands).

Advanced Tasks (Optional / Stretch Goals)

Scenario: System behavior verification

• Write a Dag representing the feature/bugfix being contributed (or use an existing one).
• Run breeze start-airflow (with --integration when needed).
• Trigger the Dag via CLI (instead of UI) and wait for completion.
• Inspect logs/status to determine success/failure from the TaskInstance logs.
• Inspect logs/status from all the component services (scheduler, api-server, triggerer, etc) to determine if there are any underlying issues.
• The agent can then decide to fix code, fix the Dag, or exit Breeze based on the results.

Expected Outcome

By the end of the project, we expect:

• A Breeze-aware AI skill that can:
  • Detect host vs. container context.
  • Choose appropriate commands and environment transitions.
• The AI toolings will be "smart-enough" to handle the core workflows for contributions, including:
  • Static checks with prek.
  • Targeted unit tests in Breeze.
  • Continue iterating based on results (e.g. fix code, fix tests, exit).
• A sync mechanism (likely using prek) that:
  • Keeps Breeze CLI and the skill definitions in sync.
  • Fails CI when they diverge, ensuring Breeze remains the single source of truth.
• Initial evaluation “exam(s)” and test harnesses that:
  • Verify that an implementation of the skill behaves correctly on at least the core scenarios.
• Updated documentation explaining how contributors and AI tools can make use of the new capability.

A successful project will make it much easier for future AI tooling (IDEs, CLIs, bots) to interact with Breeze in a reliable and Airflow-native way, increasing contributor productivity and lowering the barrier to entry.

Recommended Skills

• Programming & Tooling
  • Solid Python skills (CLI tools, packaging, basic testing).
  • Familiarity with Docker and containerized development environments.
  • Experience with writing or using CLIs and handling subprocesses.
• Dev Workflow & CI
  • Understanding of typical open source contribution workflows (git, PRs, static checks, unit tests, pre-commit).
  • Exposure to CI systems and concepts of reproducible environments.
• AI/Agents
  • Interest in or experience with AI coding assistants, Agent Skills, tool-calling, or agent frameworks.
  • Comfort reasoning about what “smart enough” means in terms of concrete, testable behaviors.
• Airflow/Breeze (Nice to Have)
  • Basic knowledge of Apache Airflow concepts (Dags, tasks, operators).
  • Prior use of Breeze for development or testing is a plus, but not strictly required.

Motivation to work at the intersection of developer experience, tooling, and AI is more important than prior deep expertise in all of these areas.

Mentors

• Jason Liu (GitHub: @jason810496)
• Jarek Potiuk (GitHub: @potiuk)

Learning Materials

• Airflow Breeze documentation: https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst
• Recent Airflow Dev Mailing List discussion regarding Agent Skills/ Agents:
  • [CONTRIBUTION OPPORTUNITY] Help Define Translation Agent Skills for Airflow Terminology
  • New: AGENTS.md overhaul + Copilot catches AI-slop PRs automatically
• Airflow prek (pre-commit) hooks entrypoint: https://github.com/apache/airflow/blob/main/.pre-commit-config.yaml
• Modern Python monorepo for Apache Airflow (by Jarek): https://medium.com/apache-airflow/modern-python-monorepo-for-apache-airflow-part-1-1fe84863e1e1
• pre-commit: https://pre-commit.com/
• prek: https://github.com/j178/prek

Tracked GitHub Issue

https://github.com/apache/airflow/issues/62500