Background

Apache IoTDB is a high-performance, IoT-native time-series database designed to manage massive volumes of time-series data generated by industrial IoT devices. It addresses challenges including high ingestion rates, complex out-of-order data handling, and real-time analytical requirements. IoTDB-AINode represents an endogenous node type in the IoTDB ecosystem, extending the database with native machine learning capabilities. IoTDB-AINode enables seamless integration of time series machine learning algorithms directly within the database engine, allowing users to register, manage, and execute inference tasks using simple SQL statements (e.g., CREATE MODEL ..., SELECT * FROM FORECAST (...)). This architecture eliminates costly data migration to external ML platforms, accelerates processing pipelines, and enhances data security by keeping computations close to the data. Currently, AINode includes built-in time series foundation models such as the Timer and Chronos for time series forecasting task.
Tensor Processing Units (TPUs) are Google-developed AI accelerators specifically designed for neural network computations. Offering high-throughput matrix operations and energy efficiency, TPUs provide a compelling alternative to GPUs for deploying large foundation models. PyTorch/XLA enables PyTorch models to leverage TPU hardware through the XLA (Accelerated Linear Algebra) compiler, supporting both single-device and distributed training scenarios.
Time Series Foundation Models have emerged as powerful tools for temporal analysis. These models demonstrate superior performance across diverse domains—from industrial sensor data to financial forecasting—making them ideal candidates for integration into IoTDB's analytical pipeline.

Goal

This project aims to enhance IoTDB-AINode with TPU hardware acceleration capabilities and integrate cutting-edge time series foundation models into the database's model inference pipeline. Specifically, the project will: 

• Enable IoTDB-AINode to recognize and leverage Google TPU devices for model deployment and inference.
• Adapt the AINode packaging and compilation workflow (Maven/Java and Poetry/Python) to support TPU-specific releases.
• Survey and integrate 1-2 SOTA time series foundation models (e.g., TimesFM) into AINode's SQL-accessible model registry.
• Establish comprehensive CI pipelines for TPU environments to ensure long-term maintainability.

The ultimate outcome will empower IoTDB users to execute high-performance time series analysis on TPU hardware using state-of-the-art foundation models through simple SQL interfaces, significantly enhancing the database's analytical capabilities for industrial AI applications.

Core Tasks（Mandatory）

1. TPU Adaptation. Implement TPU device recognition and tensor management within the AINode Python runtime. This involves:
   1. Integrating PyTorch/XLA (torch_xla) to detect available TPU devices during AINode initialization.
   2. Implementing device abstraction layers to handle model loading and tensor operations on TPU hardware.
   3. Ensuring automatic fallback mechanisms to CPU/GPU when TPU is unavailable.
2. Packaging for TPU Version. Extend the existing build infrastructure to support TPU-enabled distributions:
   1. Update Poetry configuration to manage PyTorch/XLA and TPU-specific Python dependencies.
   2. Create automated packaging scripts that bundle XLA compilers and TPU runtime libraries.
   3. Ensure the TPU version can be deployed directly in Google Cloud TPU environments and on-premise TPU pods without manual dependency resolution.
3. Model Survey. Conduct a comprehensive technical survey of SOTA time series foundation models available at project commencement. The deliverable will be a technical document analyzing each model's architecture, input requirements, computational complexity, zero-shot capabilities, and suitability for IoTDB's SQL-based inference pipeline. The survey will conclude with a justified selection of 1–2 models for integration based on deployability, inference latency, licensing, and compatibility with IoTDB’s SQL-based workflow.
4. Model Integration. Integrate 1-2 selected foundation models into IoTDB-AINode's model inference framework:
   1. Implement model wrappers conforming to AINode's model registration interface.
   2. Adapt models to process IoTDB's time series data format.
   3. Ensure compatibility with AINode's inference pipeline, supporting SQL syntax such as SELECT * FROM FORECAST (...).
   4. Support both built-in model usage and custom model registration for integrated architectures.
5. Integration Testing & CI. Establish robust testing infrastructure for TPU functionality:
   1. Design and implement integration tests covering device detection, model loading, tensor operations, and end-to-end inference workflows.
   2. Build TPU-specific CI environments using Google Cloud TPUs or TPU simulators.

Advanced Tasks (Optional)

• Distributed Large Model Deployment. As an optional stretch goal, this task explores distributed deployment of large time series foundation models across multiple TPU devices. This involves:
  • Enabling distributed inference where large models are partitioned across TPU pods.
  • Developing SQL extensions to specify distributed compute resources (e.g., LOAD MODEL ... TO DEVICES ...).
  • Optimizing communication patterns between DataNodes and AINode for high-throughput industrial scenarios involving thousands of time series streams.

Deliverables

1. Fully Functional Source Code.
   1. Pull requests to Apache IoTDB repository containing TPU adaptation modules.
   2. Integration code for SOTA time series foundation models.
   3. Extended build configurations (Maven/Poetry/PyInstaller) supporting TPU distributions.
2. Comprehensive Integration Tests.
   1. Automated test suites for TPU device detection and model execution.
   2. CI pipeline configurations for TPU environments.
3. User Documentation.
   1. Deployment guide for TPU-enabled AINode (e.g. Google Cloud TPU).
   2. SQL reference extensions for new model types and TPU-specific configuration options.
   3. Tutorial documentation demonstrating time series analysis workflows using the integrated foundation models.

Recommended Skills

• Python >= 3.11. Including asynchronous programming and ML pipeline development.

• Poetry & PyInstaller. Experience with Python dependency management and executable packaging.

• PyTorch. Known about the PyTorch/XLA integration for TPU support.

• Java & Maven. Knowledge of multi-module Java projects, build profiles, and dependency management.

Learning Material

• Apache IoTDB. https://iotdb.apache.org/

• IoTDB AINode. https://iotdb.apache.org/UserGuide/latest-Table/AI-capability/AINode_apache.html

• Time series forecasting models in HuggingFace. https://huggingface.co/models?pipeline_tag=time-series-forecasting&sort=trending

• PyTorch TPU support. https://docs.pytorch.org/xla/master/accelerators/tpu.html

 
Difficulty: medium
Mentor: Yongzao Dan (Apache IoTDB PMC Member) (yongzao@apache.org)

Background

Apache IoTDB is an open-source IoT-native time-series database designed for high-performance storage, ingestion, and analysis of massive time-series data from IoT devices. It supports deep integration with big data ecosystems like Apache Hadoop, Spark, and Flink, enabling seamless data processing workflows. IoTDB traditionally uses a tree-based data model for organizing time-series data hierarchically (e.g., root.group.device.sensor), which is efficient for device-centric IoT scenarios.
Starting with IoTDB 2.0, a dual-mode SQL architecture was introduced, adding a table mode alongside the tree mode. The table mode allows users to manage time-series data using SQL-like table structures, where each table represents a device type, with columns for timestamps, tags, and fields (e.g., measurements like temperature or humidity). This mode enhances flexibility for data analysis, supports standard SQL queries, and improves interoperability with relational tools. It is particularly useful for scenarios involving heterogeneous devices or advanced analytics, as it supports table-level schema management and retention-related configurations (e.g., TTL).
Apache Flink is a powerful stream and batch processing framework for real-time data analytics. IoTDB already provides a Flink connector (flink-iotdb-connector) for reading from and writing to IoTDB using the tree mode, including IoTDBSource for data ingestion and IoTDBSink for output. There is also a Flink SQL connector (flink-sql-iotdb-connector) for SQL-based interactions and change data capture (CDC). However, these connectors primarily target the tree mode and lack full support for the table mode's features, such as table-specific metadata handling, SQL table mappings in Flink Table API, and optimized read/write operations for table-structured data. As a result, Flink users cannot natively treat IoTDB table-mode data as first-class tables in Flink SQL or the Table API. This gap limits the ability to leverage Flink's processing capabilities with IoTDB's modern table mode, especially in real-time IoT applications like predictive maintenance or anomaly detection.
This project aims to bridge this gap by developing a dedicated Flink connector for IoTDB's 2.X table mode, enabling efficient, real-time integration between Flink and IoTDB tables.

Goal

The primary goal is to create a robust, production-ready Flink connector that supports reading from and writing to IoTDB tables using the 2.X table mode. This will allow Flink users to process IoT time-series data stored in table format, perform transformations, aggregations, and joins in real-time, and sink results back into IoTDB tables. The connector should align with Flink's DataStream and Table APIs, support fault tolerance, and handle table-specific features like tags, fields, and TTL. Ultimately, this will enhance IoTDB's ecosystem integration, making it easier for developers to build scalable IoT data pipelines.
 

Core Tasks (Mandatory)

1. Research and Design: Analyze the existing flink-iotdb-connector and flink-sql-iotdb-connector to identify limitations with the table mode. Design the connector architecture, including schema and type mappings between Flink Table/RowData and IoTDB table-mode concepts (e.g., time column, tags, and fields). Define APIs for source and sink functions compatible with Flink 1.18+.

1. Implement IoTDB Table Source: Develop a Flink source connector (e.g., IoTDBTableSource) that reads data from IoTDB tables. Support filtering by time ranges, tags, and fields using IoTDB's SQL interface. Ensure it handles schema inference and dynamic table changes.

1. Implement IoTDB Table Sink: Create a Flink sink connector (e.g., IoTDBTableSink) for writing processed data back to IoTDB tables. Support batch and streaming modes, automatic schema creation (if enabled in IoTDB), and error handling for constraints like TTL or data types.

1. Testing and Documentation: Write unit and integration tests using Flink's testing utilities and IoTDB test clusters. Document usage examples, configuration options, and deployment guides in the IoTDB repository.

1. Community Contributions: Submit pull requests to upstream repositories for any required changes, and create example Flink jobs demonstrating the use cases.

Advanced Tasks (Optional)

• Performance Optimization: Implement optimizations like parallel reading/writing.

• Benchmarking and Comparison: Develop benchmarks comparing the new connector's performance with the existing tree-mode connector, focusing on throughput, latency, and resource usage in IoT scenarios.

Deliverables

• 
  Source code for the Flink connector for IoTDB table mode, including Maven artifacts (e.g., flink-iotdb-table-connector).
• 
  Comprehensive documentation, including API references, setup guides, and usage examples integrated into the IoTDB website.
• 
  Test suites covering core functionality, edge cases, and integration with Flink.
• 
  A demo application showcasing a complete Flink pipeline reading from/writing to IoTDB tables.
• 
  Optimization reports, benchmarks, and any upstream PRs.

Recommended Skills

• Programming Language: Proficiency in Java, as both Flink and IoTDB are primarily Java-based projects.

• Database Knowledge: Understanding of SQL and fundamental database concepts. Knowledge of time-series data is a plus.

• System Integration: Interest or experience in connecting different systems and understanding data flows.

• Learning and Communication: Ability to quickly understand the codebases of two open-source projects and willingness to actively collaborate with community mentors and members.

Learning Material

• 
  Apache IoTDB Official Website: https://iotdb.apache.org/
• 
  Apache Flink Official Documentation: https://flink.apache.org/
• 
  Integrated Reference: https://github.com/apache/iotdb-extras/tree/master/connectors/flink-iotdb-connector
• 
  IoTDB Table Mode Concepts: https://iotdb.apache.org/UserGuide/latest/Background-knowledge/Data-Model-and-Terminology_apache.html
• 
  IoTDB Table Mode Query Syntax: https://iotdb.apache.org/UserGuide/latest-Table/SQL-Manual/overview_apache.html

 
Difficulty: medium
Mentor: Haonan Hou (Apache IoTDB PMC member) (haonan@apache.org)
 

Background

Apache IoTDB is a high-performance, open-source time-series database optimized for data management and analysis in Internet of Things (IoT) scenarios, while ThingsBoard is an open-source IoT platform for device management, data visualization, and rule-based automation.
With the release of IoTDB 2.X introducing a dual-mode architecture (tree and table), significant opportunities arise to enhance this integration. The table mode supports standard SQL syntax, JOIN operations, and user-defined functions, enabling more complex queries and analytics. This project proposes to develop an enhanced storage backend for ThingsBoard based on IoTDB's 2.X table mode, providing improved flexibility and performance for IoT data storage and analysis.

Goal

The primary goal of this project is to design and implement a new, enhanced storage backend for ThingsBoard that strategically leverages key features of Apache IoTDB 2.X’s table mode to improve flexibility, query expressiveness, and performance for core IoT telemetry workloads. This enhancement aims to provide ThingsBoard users with more powerful SQL querying capabilities (including complex multi-device joins and time-window aggregations) and improved performance for specific workloads. Furthermore, the project seeks to strengthen the open-source ecosystem by providing a deeper, more capable integration between ThingsBoard and the Apache IoTDB project, resulting in a more robust end-to-end IoT solution for the community.
 

Core Tasks (Mandatory)

1. In-depth Analysis and Design: Conduct a thorough analysis of the existing ThingsBoard-IoTDB integration architecture and ThingsBoard's storage backend interfaces (e.g., TimeseriesDao). Then, design an optimal strategy for mapping the ThingsBoard data model (devices, assets, telemetry, attributes, labels) to the IoTDB 2.X table mode. A key focus will be utilizing IoTDB's TAGS column to efficiently store and manage static device attributes (e.g., location, device type), enabling flexible device filtering and grouping based on these tags .
2. Implementation of Storage Backend Connector:
   1. Data Access Layer: Based on the design, implement the relevant ThingsBoard storage backend interfaces to connect with IoTDB.
   2. Write Path: Develop efficient data writing logic that transforms device telemetry data received by ThingsBoard and performs batch writes to the corresponding tables in IoTDB.
   3. Read/Query Path: Implement query interfaces that translate data requests from ThingsBoard dashboards or the rule engine into efficient SQL queries that take full advantage of IoTDB 2.X table mode features.
3. Performance Benchmarking and Comparison: Design and execute standardized performance test cases (e.g., high-concurrency data ingestion, complex conditional queries, large-scale range queries). Produce a detailed performance comparison report between the new IoTDB 2.X table mode-based backend and ThingsBoard's existing data storage options, This report should quantify improvements in metrics like write throughput and query latency.
4. Testing and Documentation: Write comprehensive integration tests to ensure the correctness and stability of the new functionality. Create detailed user documentation, including installation/configuration instructions, data model explanations, API usage guidelines, and best practices.
5. Community Collaboration and Upstream Contribution: Actively communicate with the ThingsBoard open-source community at key project milestones to discuss designs and gather feedback. Submit high-quality Pull Requests (PRs) to the official ThingsBoard repository, adhering to its coding standards, with the goal of getting the implementation merged.

Advanced Tasks (Optional)

• Leverage IoTDB UDFs: Explore the integration of IoTDB's User-Defined Functions (UDFs) within ThingsBoard's rule engine. This could allow for performing more complex data processing and analysis (e.g., anomaly detection) directly within the database before data is pulled into ThingsBoard.

• Enhanced Data Modeling for Assets: Extend the data mapping design to optimally support ThingsBoard's assets and the relations between entities (devices, assets, customers), exploiting the relational capabilities of the IoTDB table mode for more complex queries.

• Comprehensive Dashboard Demo: Build a detailed ThingsBoard dashboard that showcases the advanced querying capabilities made possible by the new integration, such as visualizations based on multi-device joins or complex aggregations.

Deliverables

1. A fully functional storage backend plugin/implementation, including source code, build scripts, and configuration examples.
2. A detailed design document explaining the data mapping and integration architecture between ThingsBoard and the IoTDB 2.X table mode.
3. A comprehensive performance benchmark report comparing the new solution with existing options.
4. Complete user and developer documentation.
5. A Pull Request submitted to the ThingsBoard community containing the implementation, tests, and relevant documentation.
6. A final project report summarizing work, technical challenges, learnings, and future possibilities.

Recommended Skills

• Programming Language: Proficiency in Java, as both ThingsBoard and IoTDB are primarily Java-based projects.

• Database Knowledge: Understanding of SQL and fundamental database concepts. Knowledge of time-series data is a plus.

• System Integration: Interest or experience in connecting different systems and understanding data flows.

• Learning and Communication: Ability to quickly understand the codebases of two open-source projects and willingness to actively collaborate with community mentors and members.

Learning Material

• Apache IoTDB Official Website: https://iotdb.apache.org/
• ThingsBoard Official Documentation: https://thingsboard.io/docs/
• Integrated Reference: https://github.com/thingsboard/thingsboard/pull/11476
• IoTDB Table Mode Concepts: https://iotdb.apache.org/UserGuide/latest/Background-knowledge/Data-Model-and-Terminology_apache.html
• IoTDB Table Mode Query Syntax: https://iotdb.apache.org/UserGuide/latest-Table/SQL-Manual/overview_apache.html

 
Difficulty: medium
Mentor: Xuan Wang (Apache IoTDB Committer) (critas@apache.org)
 

Background

Apache Beam is a unified programming model for user developing data processing pipelines capable running in distributed systems. Apache Beam SDK officially supports Java, Python, and Go. While Java SDK was historically dominant, Python SDK is increasingly popular thanks to Beam ML. Python APIs are crucial for developers. We plan to port highly anticipated basic streaming transforms made convenient for Beam Python developers.

Tasks

1. Python UnboundedSource (https://github.com/apache/beam/issues/19137)

While Splittable DoFn has been introduced as a Beam primitive transform handling IO sources, UnboundedSource arguably remains an easier API for users to author their own IOs. In the Java SDK, UnboundedSource/UnboundedReader has been (re)implemented as a wrapper of Splittable DoFn, we can follow the Java implementation and add it to Python.

Stretch goal: implement a native Python streaming IO based on UnboundedSource.

2. Python Watch Transform (https://github.com/apache/beam/issues/21521)

Currently we have a Watch transform in the Java SDK that is very useful when periodically polling for new input to a pipeline. We would like a parallel transform in Python.

Stretch goal: Update Python FileIO.readContinuously to use watch transform

Deliverables

• Implementation of Python UnboundedSource: A functional wrapper API for UnboundedSource and UnboundedReader built on Splittable DoFn (a merged pull request to the Apache Beam repo).
• Implementation of Python Watch Transform: A parallel transform to the Java Watch API for periodic polling (a merged pull request to the Apache Beam repo).
• Unit and Integration Tests: tests for both features, specifically covering watermarks, checkpointing, and polling termination conditions.
• User Documentation: Updated SDK guides and Docstrings explaining how to author custom IOs using UnboundedSource and how to use the Watch transform in pipelines.
• Refactored FileIO.readContinuously (Stretch Goal): A pull request updating FileIO.read_continuously to utilize the new Watch transform logic.

Recommended Skills

• Proficiency in Python, experience with pytest
• Java-to-Python Porting: Ability to read and interpret Java source code
• Version control: Git, development with GitHub
• nice to have: exposure to streaming data processing tools (e.g. Apache Beam/Flink/Spark, etc)

Apache DolphinScheduler

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Website: https://dolphinscheduler.apache.org/en-us/index.html

GitHub: https://github.com/apache/dolphinscheduler

Linked GitHub Issue: https://github.com/apache/dolphinscheduler/issues/8975

Background

Currently, DolphinScheduler requires a separate alert-server to handle workflow and task alerts. Although the alert-server is lightweight, maintaining and deploying it separately adds operational complexity.

We aim to remove the standalone alert-server and embed its alerting functionality directly into the API server.

Task

Integrate the alert-server functionality into the API server so that it can handle workflow and task alerts natively.

Deliverables

• Remove the standalone alert-server.

• Enable the API server to handle all alerting tasks.
• Add Integration test case.

Recommended Skills

• Proficiency in Java.
• Familiarity with microservice, e.g. spring-boot.

• Familiarity with DolphinScheduler’s architecture and alerting mechanisms is a plus.

Mentors

• Wenjun Ruan(Apache DolphinScheduler PMC member), wenjun@apache.org
• Zihao Xiang(Apache DolphinScheduler PMC member),   zihaoxiang@apache.org