Background

Apache IoTDB is a high-performance, IoT-native time-series database designed to manage massive volumes of time-series data generated by industrial IoT devices. It addresses challenges including high ingestion rates, complex out-of-order data handling, and real-time analytical requirements. IoTDB-AINode represents an endogenous node type in the IoTDB ecosystem, extending the database with native machine learning capabilities. IoTDB-AINode enables seamless integration of time series machine learning algorithms directly within the database engine, allowing users to register, manage, and execute inference tasks using simple SQL statements (e.g., CREATE MODEL ..., SELECT * FROM FORECAST (...)). This architecture eliminates costly data migration to external ML platforms, accelerates processing pipelines, and enhances data security by keeping computations close to the data. Currently, AINode includes built-in time series foundation models such as the Timer and Chronos for time series forecasting task.
Tensor Processing Units (TPUs) are Google-developed AI accelerators specifically designed for neural network computations. Offering high-throughput matrix operations and energy efficiency, TPUs provide a compelling alternative to GPUs for deploying large foundation models. PyTorch/XLA enables PyTorch models to leverage TPU hardware through the XLA (Accelerated Linear Algebra) compiler, supporting both single-device and distributed training scenarios.
Time Series Foundation Models have emerged as powerful tools for temporal analysis. These models demonstrate superior performance across diverse domains—from industrial sensor data to financial forecasting—making them ideal candidates for integration into IoTDB's analytical pipeline.

Goal

This project aims to enhance IoTDB-AINode with TPU hardware acceleration capabilities and integrate cutting-edge time series foundation models into the database's model inference pipeline. Specifically, the project will: *  

• Enable IoTDB-AINode to recognize and leverage Google TPU devices for model deployment and inference.
• Adapt the AINode packaging and compilation workflow (Maven/Java and Poetry/Python) to support TPU-specific releases.
• Survey and integrate 1-2 SOTA time series foundation models (e.g., TimesFM) into AINode's SQL-accessible model registry.
• Establish comprehensive CI pipelines for TPU environments to ensure long-term maintainability.

The ultimate outcome will empower IoTDB users to execute high-performance time series analysis on TPU hardware using state-of-the-art foundation models through simple SQL interfaces, significantly enhancing the database's analytical capabilities for industrial AI applications.

Core Tasks（Mandatory）

1. TPU Adaptation. Implement TPU device recognition and tensor management within the AINode Python runtime. This involves:
   1. Integrating PyTorch/XLA (torch_xla) to detect available TPU devices during AINode initialization.
   2. Implementing device abstraction layers to handle model loading and tensor operations on TPU hardware.
   3. Ensuring automatic fallback mechanisms to CPU/GPU when TPU is unavailable.
2. Packaging for TPU Version. Extend the existing build infrastructure to support TPU-enabled distributions:
   1. Update Poetry configuration to manage PyTorch/XLA and TPU-specific Python dependencies.
   2. Create automated packaging scripts that bundle XLA compilers and TPU runtime libraries.
   3. Ensure the TPU version can be deployed directly in Google Cloud TPU environments and on-premise TPU pods without manual dependency resolution.
3. Model Survey. Conduct a comprehensive technical survey of SOTA time series foundation models available at project commencement. The deliverable will be a technical document analyzing each model's architecture, input requirements, computational complexity, zero-shot capabilities, and suitability for IoTDB's SQL-based inference pipeline. The survey will conclude with a justified selection of 1–2 models for integration based on deployability, inference latency, licensing, and compatibility with IoTDB’s SQL-based workflow.
4. Model Integration. Integrate 1-2 selected foundation models into IoTDB-AINode's model inference framework:
   1. Implement model wrappers conforming to AINode's model registration interface.
   2. Adapt models to process IoTDB's time series data format.
   3. Ensure compatibility with AINode's inference pipeline, supporting SQL syntax such as SELECT * FROM FORECAST (...).
   4. Support both built-in model usage and custom model registration for integrated architectures.
5. Integration Testing & CI. Establish robust testing infrastructure for TPU functionality:
   1. Design and implement integration tests covering device detection, model loading, tensor operations, and end-to-end inference workflows.
   2. Build TPU-specific CI environments using Google Cloud TPUs or TPU simulators.

Advanced Tasks (Optional)

• Distributed Large Model Deployment. As an optional stretch goal, this task explores distributed deployment of large time series foundation models across multiple TPU devices. This involves:
  • Enabling distributed inference where large models are partitioned across TPU pods.
  • Developing SQL extensions to specify distributed compute resources (e.g., LOAD MODEL ... TO DEVICES ...).
  • Optimizing communication patterns between DataNodes and AINode for high-throughput industrial scenarios involving thousands of time series streams.

Deliverables

1. Fully Functional Source Code.
   1. Pull requests to Apache IoTDB repository containing TPU adaptation modules.
   2. Integration code for SOTA time series foundation models.
   3. Extended build configurations (Maven/Poetry/PyInstaller) supporting TPU distributions.
2. Comprehensive Integration Tests.
   1. Automated test suites for TPU device detection and model execution.
   2. CI pipeline configurations for TPU environments.
3. User Documentation.
   1. Deployment guide for TPU-enabled AINode (e.g. Google Cloud TPU).
   2. SQL reference extensions for new model types and TPU-specific configuration options.
   3. Tutorial documentation demonstrating time series analysis workflows using the integrated foundation models.

Recommended Skills

• Python >= 3.11. Including asynchronous programming and ML pipeline development.

• Poetry & PyInstaller. Experience with Python dependency management and executable packaging.

• PyTorch. Known about the PyTorch/XLA integration for TPU support.

• Java & Maven. Knowledge of multi-module Java projects, build profiles, and dependency management.

Learning Material

• Apache IoTDB. https://iotdb.apache.org/

• IoTDB AINode. https://iotdb.apache.org/UserGuide/latest-Table/AI-capability/AINode_apache.html

• Time series forecasting models in HuggingFace. https://huggingface.co/models?pipeline_tag=time-series-forecasting&sort=trending

• PyTorch TPU support. https://docs.pytorch.org/xla/master/accelerators/tpu.html

 
Difficulty: medium
Mentor: Yongzao Dan (Apache IoTDB PMC Member) (yongzao@apache.org)

Background

Apache IoTDB is an open-source IoT-native time-series database designed for high-performance storage, ingestion, and analysis of massive time-series data from IoT devices. It supports deep integration with big data ecosystems like Apache Hadoop, Spark, and Flink, enabling seamless data processing workflows. IoTDB traditionally uses a tree-based data model for organizing time-series data hierarchically (e.g., root.group.device.sensor), which is efficient for device-centric IoT scenarios.
Starting with IoTDB 2.0, a dual-mode SQL architecture was introduced, adding a table mode alongside the tree mode. The table mode allows users to manage time-series data using SQL-like table structures, where each table represents a device type, with columns for timestamps, tags, and fields (e.g., measurements like temperature or humidity). This mode enhances flexibility for data analysis, supports standard SQL queries, and improves interoperability with relational tools. It is particularly useful for scenarios involving heterogeneous devices or advanced analytics, as it supports table-level schema management and retention-related configurations (e.g., TTL).
Apache Flink is a powerful stream and batch processing framework for real-time data analytics. IoTDB already provides a Flink connector (flink-iotdb-connector) for reading from and writing to IoTDB using the tree mode, including IoTDBSource for data ingestion and IoTDBSink for output. There is also a Flink SQL connector (flink-sql-iotdb-connector) for SQL-based interactions and change data capture (CDC). However, these connectors primarily target the tree mode and lack full support for the table mode's features, such as table-specific metadata handling, SQL table mappings in Flink Table API, and optimized read/write operations for table-structured data. As a result, Flink users cannot natively treat IoTDB table-mode data as first-class tables in Flink SQL or the Table API. This gap limits the ability to leverage Flink's processing capabilities with IoTDB's modern table mode, especially in real-time IoT applications like predictive maintenance or anomaly detection.
This project aims to bridge this gap by developing a dedicated Flink connector for IoTDB's 2.X table mode, enabling efficient, real-time integration between Flink and IoTDB tables.

Goal

The primary goal is to create a robust, production-ready Flink connector that supports reading from and writing to IoTDB tables using the 2.X table mode. This will allow Flink users to process IoT time-series data stored in table format, perform transformations, aggregations, and joins in real-time, and sink results back into IoTDB tables. The connector should align with Flink's DataStream and Table APIs, support fault tolerance, and handle table-specific features like tags, fields, and TTL. Ultimately, this will enhance IoTDB's ecosystem integration, making it easier for developers to build scalable IoT data pipelines.
 

Core Tasks (Mandatory)

1. Research and Design: Analyze the existing flink-iotdb-connector and flink-sql-iotdb-connector to identify limitations with the table mode. Design the connector architecture, including schema and type mappings between Flink Table/RowData and IoTDB table-mode concepts (e.g., time column, tags, and fields). Define APIs for source and sink functions compatible with Flink 1.18+.

1. Implement IoTDB Table Source: Develop a Flink source connector (e.g., IoTDBTableSource) that reads data from IoTDB tables. Support filtering by time ranges, tags, and fields using IoTDB's SQL interface. Ensure it handles schema inference and dynamic table changes.

1. Implement IoTDB Table Sink: Create a Flink sink connector (e.g., IoTDBTableSink) for writing processed data back to IoTDB tables. Support batch and streaming modes, automatic schema creation (if enabled in IoTDB), and error handling for constraints like TTL or data types.

1. Testing and Documentation: Write unit and integration tests using Flink's testing utilities and IoTDB test clusters. Document usage examples, configuration options, and deployment guides in the IoTDB repository.

1. Community Contributions: Submit pull requests to upstream repositories for any required changes, and create example Flink jobs demonstrating the use cases.

Advanced Tasks (Optional)

• Performance Optimization: Implement optimizations like parallel reading/writing.

• Benchmarking and Comparison: Develop benchmarks comparing the new connector's performance with the existing tree-mode connector, focusing on throughput, latency, and resource usage in IoT scenarios.

Deliverables

• 
  Source code for the Flink connector for IoTDB table mode, including Maven artifacts (e.g., flink-iotdb-table-connector).
• 
  Comprehensive documentation, including API references, setup guides, and usage examples integrated into the IoTDB website.
• 
  Test suites covering core functionality, edge cases, and integration with Flink.
• 
  A demo application showcasing a complete Flink pipeline reading from/writing to IoTDB tables.
• 
  Optimization reports, benchmarks, and any upstream PRs.

Recommended Skills

• Programming Language: Proficiency in Java, as both Flink and IoTDB are primarily Java-based projects.

• Database Knowledge: Understanding of SQL and fundamental database concepts. Knowledge of time-series data is a plus.

• System Integration: Interest or experience in connecting different systems and understanding data flows.

• Learning and Communication: Ability to quickly understand the codebases of two open-source projects and willingness to actively collaborate with community mentors and members.

Learning Material

• 
  Apache IoTDB Official Website: https://iotdb.apache.org/
• 
  Apache Flink Official Documentation: https://flink.apache.org/
• 
  Integrated Reference: https://github.com/apache/iotdb-extras/tree/master/connectors/flink-iotdb-connector
• 
  IoTDB Table Model Mode Concepts: https://iotdb.apache.org/UserGuide/latest/Background-knowledge/Data-Model-and-Terminology_apache.html
• 
  IoTDB Table Model Mode Query Syntax: https://iotdb.apache.org/UserGuide/latest-Table/SQL-Manual/overview_apache.html

 
Difficulty: medium
Mentor: Haonan Hou (Apache IoTDB PMC member) (haonan@apache.org)