https://www.mail-archive.com/server-dev@james.apache.org/msg71462.html

A good long term objective for the PMC is to drop RabbitMQ in
favor of pulsar (third parties could package their own components using
RabbitMQ if they wishes...)

This means:

• Solve the bugs that were found during the Pulsar MailQueue review
• Pulsar MailQueue need to allow listing blobs in order to be
  deduplication friendly.
• Provide an event bus based on Pulsar
• Provide a task manager based on Pulsar
• Package a distributed server backed by pulsar, deprecate then replace
  the current one.
• (optionally) support mail queue priorities

While contributions would of course be welcomed on this topic, we could
offer it as part of GSOC 2022, and we could co-mentor it with mentors of
the Pulsar community (see [3])

[3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259

Would such a plan gain traction around here ?

Why ?

Alternatives like Postfix...

• Do not offer a unified view of the mail queue across nodes
• Requires statefull persistent storage

Given Apache James recent push to adopt a distributed mail queue based on Pulsar supporting delays (JAMES-3687), it starts making sense developing tooling for MX related tooling.

I propose myself to mentor a Gsoc on this topic.

Benefits for the student

At the end of this GSOC you will...

• Have a solid understanding of email relaying and associated mechanics
• Understand James modular architecture (mailet/ matcher / routes)
• Have a hands-on expertise in SQL / NoSQL working with technologies like Cassandra, Redis, JPA...
• Identify fix and solve architecture problems.
• Conduct performance tests and develop an operational mindset

Inventory...

James ships a couple of MX related tools within smtp-hooks/mailets in default packages. It would make sense to me to move those as an extension.

James supports today...

checks agains DNS blacklists. `DNSRBLHandler` or `URIRBLHandler` smtp hook for instance. This can be moved as an extension IMO.

We would need a little performance benchmark to document performance implications of activating DNS-RBL.

Finally as quoted by a gitter guy: it would make more sens to have this done as a MailHook rather as a RcptHook as it would avoid doing the same job again and over again for each recipients. See JAMES-3820 .

Grey listing. There's an existing implementation using JDBC as an underlying storage.

Move it as an extension.

Remove JDBC storage, propose 2 storage possibilities: in-memory for single node, REDIS for a distributed topology.

Some work around whitelist mailets? Move it as an extension, propose JPA, Cassandra, and XML configured implementations ? With a route to manage entries in there for JPA + Cassandra ?

I would expect a student to do his own little audit and come up with extra suggestions!

James today provides a command line tool to do administration tasks like creating a domain, listing users, setting quota, etc.
It requires access to JMX port and even if lot of admins are confortable with such tools, to make our user base broader, we probably should expose the same commands in Rest and provide a fancy default web ui.
The task would need some basic skills on frontend tools to design an administration board, knowledge on what REST mean and enough Java understanding to add commands to existing Rest backend.
In the team, we have a strong focus on test (who want a mail server that is not tested enough ?) so we will explain and/or teach the student how to have the right test coverage of the features using modern tools like Cucumber, Selenium, rest-assured, etc.

Why ?

Alternatives like Postfix...

• Do not offer a unified view of the mail queue across nodes
• Requires statefull persistent storage

Given Apache James recent push to adopt a distributed mail queue based on Pulsar supporting delays (JAMES-3687), it starts making sense developing tooling for MX related tooling.

I propose myself to mentor a Gsoc on this topic.

Benefits for the student

At the end of this GSOC you will...

• Have a solid understanding of email relaying and associated mechanics
• Understand James modular architecture (mailet/ matcher / routes)
• Have a hands-on expertise in SQL / NoSQL working with technologies like Cassandra, Redis, JPA...
• Identify fix and solve architecture problems.
• Conduct performance tests and develop an operational mindset

Inventory...

James ships a couple of MX related tools within smtp-hooks/mailets in default packages. It would make sense to me to move those as an extension.

James supports today...

checks agains DNS blacklists. `DNSRBLHandler` or `URIRBLHandler` smtp hook for instance. This can be moved as an extension IMO.

We would need a little performance benchmark to document performance implications of activating DNS-RBL.

Finally as quoted by a gitter guy: it would make more sens to have this done as a MailHook rather as a RcptHook as it would avoid doing the same job again and over again for each recipients. See JAMES-3820 .

Grey listing. There's an existing implementation using JDBC as an underlying storage.

Move it as an extension.

Remove JDBC storage, propose 2 storage possibilities: in-memory for single node, REDIS for a distributed topology.

Some work around whitelist mailets? Move it as an extension, propose JPA, Cassandra, and XML configured implementations ? With a route to manage entries in there for JPA + Cassandra ?

I would expect a student to do his own little audit and come up with extra suggestions!

https://www.mail-archive.com/server-dev@james.apache.org/msg71462.html

A good long term objective for the PMC is to drop RabbitMQ in
favor of pulsar (third parties could package their own components using
RabbitMQ if they wishes...)

This means:

• Solve the bugs that were found during the Pulsar MailQueue review
• Pulsar MailQueue need to allow listing blobs in order to be
  deduplication friendly.
• Provide an event bus based on Pulsar
• Provide a task manager based on Pulsar
• Package a distributed server backed by pulsar, deprecate then replace
  the current one.
• (optionally) support mail queue priorities

While contributions would of course be welcomed on this topic, we could
offer it as part of GSOC 2022, and we could co-mentor it with mentors of
the Pulsar community (see [3])

[3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259

Would such a plan gain traction around here ?

James today provides a command line tool to do administration tasks like creating a domain, listing users, setting quota, etc.
It requires access to JMX port and even if lot of admins are confortable with such tools, to make our user base broader, we probably should expose the same commands in Rest and provide a fancy default web ui.
The task would need some basic skills on frontend tools to design an administration board, knowledge on what REST mean and enough Java understanding to add commands to existing Rest backend.
In the team, we have a strong focus on test (who want a mail server that is not tested enough ?) so we will explain and/or teach the student how to have the right test coverage of the features using modern tools like Cucumber, Selenium, rest-assured, etc.

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org/
Github: https://github.com/apache/shardingsphere shardingsphere 

Background

The community just added CDC (change data capture) feature recently. Change feed will be published in created network connection after logging in, then it could be consumed.

Since Kafka is popular distributed event streaming platform, it's useful to import change feed into Kafka for later processing.

Task

1. Familiar with ShardingSphere CDC client usage, create publication and subscribe change feed.
2. Familiar with Kafka connector development, develop source connector, integrate with ShardingSphere CDC. Persist change feed to Kafka topics properly.
3. Add unit test and E2E integration test.

Relevant Skills

1. Java language

2. Basic knowledge of CDC and Kafka

3. Maven

References

There is a proposal about new CRD Cluster and ComputeNode as belows:

• WIP: [New Feature] Introduce new CRD Cluster #167
• [Feat] Introduce new CRD as ComputeNode for better usability #166

Currently we try to promote ComputeNode as major CRD to represent a special ShardingSphere Proxy deployment. And plan to use Cluster indicating a special ShardingSphere Proxy cluster.

Task

This issue is to enhance ComputeNode reconciliation availability. The specific case list is as follows.

•  Add IT test case for Deployment spec volume
•  Add IT test case for Deployment spec template init containers
•  Add IT test case for Deployment spec template spec containers
•  Add IT test case for Deployment spec volume mounts
•  Add IT test case for Deployment spec container ports
•  Add IT test case for Deployment spec container image tag
•  Add IT test case for Service spec ports
•  Add IT test case for ConfigMap data serverconfig
•  Add IT test case for ConfigMap data logback
   
  Notice, these issues can be a good example.
• chore: add more Ginkgo tests for ComputeNode #203

Relevant Skills

1. Master Go language, Ginkgo test framework
2. Have a basic understanding of Apache ShardingSphere Concepts
3. Be familiar with Kubernetes Operator, kubebuilder framework

Targets files

ComputeNode IT - https://github.com/apache/

shardingsphere-on-cloud/blob/main/shardingsphere-operator/pkg/reconcile/computenode/compute_node_test.go

Mentor

Liyao Miao, Committer of Apache ShardingSphere,  miaoliyao@apache.org^{Image Added}

Chuxin Chen

Mentor

Hongsheng Zhong, PMC of Apache ShardingSphere, zhonghongsheng@apache.org

Xinze Guo, Committer of Apache ShardingSphere, azexin@apache tuichenchuxin@apache.org^{Image Added}

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org
Github: https://github.com/apache/shardingsphere 

Background

The ShardingSphere SQL federation engine provides support for complex SQL statements, and it can well support cross-database join queries, subqueries, aggregation queries and other statements. An important part of SQL federation engine is to convert the SQL statement parsed by ShardingSphere into SqlNode, so that Calcite can be used to implement SQL optimization and federated query.

Task

This issue is to solve the MySQL exception that occurs during SQLNodeConverterEngine conversion. The specific case list is as follows.

• select_char
• select_extract
• select_from_dual
• select_from_with_table
• select_group_by_with_having_and_window
• select_not_between_with_single_table
• select_not_in_with_single_table
• select_substring
• select_trim
• select_weight_string
• select_where_with_bit_expr_with_ampersand
• select_where_with_bit_expr_with_caret
• select_where_with_bit_expr_with_div
• select_where_with_bit_expr_with_minus_interval
• select_where_with_bit_expr_with_mod
• select_where_with_bit_expr_with_mod_sign
• select_where_with_bit_expr_with_plus_interval
• select_where_with_bit_expr_with_signed_left_shift
• select_where_with_bit_expr_with_signed_right_shift
• select_where_with_bit_expr_with_vertical_bar
• select_where_with_boolean_primary_with_comparison_subquery
• select_where_with_boolean_primary_with_is
• select_where_with_boolean_primary_with_is_not
• select_where_with_boolean_primary_with_null_safe
• select_where_with_expr_with_and_sign
• select_where_with_expr_with_is
• select_where_with_expr_with_is_not
• select_where_with_expr_with_not
• select_where_with_expr_with_not_sign
• select_where_with_expr_with_or_sign
• select_where_with_expr_with_xor
• select_where_with_predicate_with_in_subquery
• select_where_with_predicate_with_regexp
• select_where_with_predicate_with_sounds_like
• select_where_with_simple_expr_with_collate
• select_where_with_simple_expr_with_match
• select_where_with_simple_expr_with_not
• select_where_with_simple_expr_with_odbc_escape_syntax
• select_where_with_simple_expr_with_row
• select_where_with_simple_expr_with_tilde
• select_where_with_simple_expr_with_variable
• select_window_function
• select_with_assignment_operator
• select_with_assignment_operator_and_keyword
• select_with_case_expression
• select_with_collate_with_marker
• select_with_date_format_function
• select_with_exists_sub_query_with_project
• select_with_function_name
• select_with_json_value_return_type
• select_with_match_against
• select_with_regexp
• select_with_schema_name_in_column_projection
• select_with_schema_name_in_shorthand_projection
• select_with_spatial_function
• select_with_trim_expr
• select_with_trim_expr_from_expr

You need to compare the difference between actual and expected, and then correct the logic in SQLNodeConverterEngine so that actual can be consistent with expected.

After you make changes, remember to add case to SUPPORTED_SQL_CASE_IDS to ensure it can be tested.

provides two adapters: ShardingSphere-JDBC and ShardingSphere-Proxy.

Now, ShardingSphere uses logback for logging, but consider the following situations:

• Users may need to switch the logging framework to meet special needs, such as log4j2 can provide better asynchronous performance;
• When using the JDBC adapter, the user application may not use logback, which may cause some conflicts.

Why doesn't the log facade suffice? Because ShardingSphere provides users with clustered logging configurations (such as changing the log level online), this requires dynamic construction of logger, which cannot be achieved with only the log facade.

Task

1. Design and implement logging SPI to support multiple logging frameworks (such as logback and log4j2)
2. Allow users to choose which logging framework to use through the logging rule

Relevant Skills

1. Master JAVA language

2. Basic knowledge of logback and log4j2

3. Maven

Mentor

Longtao Jiang, Committer of Apache ShardingSphere, jianglongtao@apache.org^{Image Added}

Trista Pan, PMC of Apache ShardingSphere, panjuan@apache.org^{Image Added}

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org
Github: https://github.com/apache/shardingsphere 

Background

ShardingSphere has designed its own metadata database to simulate metadata queries that support various databases.

More details:

https://github.com/apache/shardingsphere/issues/21268
https://github.com/apache/shardingsphere/issues/22052

Task

• Support PostgreSQL And openGauss `\d tableName`
• Support PostgreSQL And openGauss `\d+`
• Support PostgreSQL And openGauss `\d+ tableName`
• Support PostgreSQL And openGauss `l`
• Support query for MySQL metadata `TABLES`
• Support query for MySQL metadata `COLUMNS`
• Support query for MySQL metadata `schemata`
• Support query for MySQL metadata `ENGINES`
• Support query for MySQL metadata `FILES`
• Support query for MySQL metadata `VIEWS`

Notice, these issues can be a good example.

https://github.com/apache/shardingsphere/pull/22053
https://github.com/apache/shardingsphere/pull/22057/
https://github.com/apache/shardingsphere/pull/22166/
https://github.com/apache/shardingsphere/pull/22182

Relevant Skills

•  Master JAVA language
•  Have a basic understanding of Zookeeper
•  Be familiar with MySQL/Postgres SQLs 

Mentor

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org

Zhengqiang Duan, PMC of Apache ShardingSphere, duanzhengqiang@apache.org

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org
Github: https://github.com/apache/shardingsphere 

Background

The community just added CDC (change data capture) feature recently. Change feed will be published in created network connection after logging in, then it could be consumed.

Since Kafka is popular distributed event streaming platform, it's useful to import change feed into Kafka for later processing.

Task

1. Familiar with ShardingSphere CDC client usage, create publication and subscribe change feed.
2. Familiar with Kafka connector development, develop source connector, integrate with ShardingSphere CDC. Persist change feed to Kafka topics properly.
3. Add unit test and E2E integration test.

Relevant Skills

1. Java language
2. Basic knowledge of CDC and Kafka
3. Maven

References

• https://github.com/apache/shardingsphere/

pull

• issues/

14492

Relevant Skills

 
1. Master JAVA language

2. Have a basic understanding of Antlr g4 file

3. Be familiar with MySQL and Calcite SqlNode

Targets files

 
SQLNodeConverterEngineIT

• 22500
• https://kafka.apache.org/documentation/#connect_development
• https://github.com/apache/

• kafka/

• tree/

• trunk/

• connect/

• file/

• src
• https:/

• /

• github.com/

• confluentinc/kafka-connect-jdbc

Local Test Steps

1. Modify `conf/server.yaml`, uncomment `cdc-server-port: 33071` to enable CDC. (Refer to step 2)
2. Configure proxy, refer to `Prerequisites` and `Procedure` in build to configure proxy (Newer version could be used too, current stable version is 5.3.1).
3. Start proxy server, it'll start CDC server too.
4. Download ShardingSphere source code from https://github.com/apache/shardingsphere , modify and run `org.apache.shardingsphere.data.pipeline.cdc.client.example.Bootstrap`. It'll print `records:` by default in `Bootstrap`.
5. Execute some ISNERT/UPDATE/DELETE SQLs in proxy to generate change feed, and then check in `Bootstrap` console.

Mentor

Hongsheng Zhong, PMC of Apache ShardingSphere, zhonghongsheng@apache.org

Xinze Guo, Committer of Apache ShardingSphere, azexin@apache

Mentor

Zhengqiang Duan, PMC of Apache ShardingSphere, duanzhengqiang@apache.org

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org^{Image Removed}

Trista Pan, PMC of Apache ShardingSphere, panjuan@apache.org

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page:   https://shardingsphere.apache.org/
Github:  https://github.com/apache/shardingsphere 

Background

There is a proposal about new CRD Cluster and ComputeNode as belows:

• WIP: [New Feature] Introduce new CRD Cluster #167
• [Feat] Introduce new CRD as ComputeNode for better usability #166

shardingsphere 

Background

Currently we try to promote ComputeNode StorageNode as major CRD to represent a special ShardingSphere Proxy deployment. And plan to use Cluster indicating a special ShardingSphere Proxy cluster.

Task

This issue is to enhance ComputeNode reconciliation availability. The specific case list is as follows.

•  Add IT test case for Deployment spec volume
•  Add IT test case for Deployment spec template init containers
•  Add IT test case for Deployment spec template spec containers
•  Add IT test case for Deployment spec volume mounts
•  Add IT test case for Deployment spec container ports
•  Add IT test case for Deployment spec container image tag
•  Add IT test case for Service spec ports
•  Add IT test case for ConfigMap data serverconfig
•  Add IT test case for ConfigMap data logback
   
  Notice, these issues can be a good example.
• chore: add more Ginkgo tests for ComputeNode #203

Relevant Skills

set of storage units for ShardingSphere.

Task

The elementary task is that the storage node controller could manage the lifecycle of  a set of storage units, like PostgreSQL, in kubernetes. 

We don't hope to create another wheel like pg-operator. So consider using a predefined parameter group to generate the target CRD.

• [ ] Generate DistSQL according to the Golang struct `EncryptionRule`
• [ ] Generate DistSQL according to the Golang struct `ShardingRule`
• [ ] Generate DistSQL according to the Golang struct `ReadWriteSplittingRule`
• [ ] Generate DistSQL according to the Golang struct `MaskRule`
• [ ] Generate DistSQL according to the Golang struct `ShadowRule`

  Relevant Skills

1. Master Go language, Ginkgo test framework
2. Have a basic understanding of Apache ShardingSphere Concepts

and DistSQL

Targets files

ComputeNode IT DistSQL Converter -  https://github.com/apache/shardingsphere-on-cloud/blob/main/shardingsphere-operator/pkg/reconcile/computenode/compute_node_test.go

Mentor

Liyao Miao, Committer of Apache ShardingSphere,  miaoliyao@apache.org^{Image Removed}

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org^{Image Removed}

distsql/converter.go, etc.

Example

A struct defined as below:

```golang
type EncryptRule struct{}
func (t EncryptRule) ToDistSQL() string {}
```
While invoking ToDistSQL() it will generate a DistSQL regarding a EncryptRule like:

```SQL
CREATE ENCRYPT RULE t_encrypt (....
```

References:

• https://shardingsphere.apache.org/document/current/en/user-manual/shardingsphere-proxy/distsql/syntax/rdl/rule-definition/encrypt/create-encrypt-rule/

Mentor
Liyao Miao, Committer of Apache ShardingSphere, miaoliyao@apache.org

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org/
Github: https://github.com/apache/shardingsphere 

Background

ShardingSphere provides two adapters: ShardingSphere-JDBC and ShardingSphere-Proxy.

Now, ShardingSphere uses logback for logging, but consider the following situations:

• Users may need to switch the logging framework to meet special needs, such as log4j2 can provide better asynchronous performance;
• When using the JDBC adapter, the user application may not use logback, which may cause some conflicts.

Why doesn't the log facade suffice? Because ShardingSphere provides users with clustered logging configurations (such as changing the log level online), this requires dynamic construction of logger, which cannot be achieved with only the log facade.

Task

There is a proposal about new CRD Cluster and ComputeNode as belows:

• #167
• #166

Currently we try to promote StorageNode as major CRD to represent a set of storage units for ShardingSphere.

Task

The elementary task is that the storage node controller could manage the lifecycle of a set of storage units, like PostgreSQL, in kubernetes.

We don't hope to create another wheel like pg-operator. So consider using a predefined parameter group to generate the target CRD.

• [ ] Create a PostgreSQL cluster while a StorageNode with pg parameters is created
• [ ] Update the PostgreSQL cluster while updated StorageNode
• [ ] Delete the PostgreSQL cluster while deleted StorageNode. Notice this may need a deletion strategy.
• [ ] Reconciling StorageNode according to the status of PostgreSQL cluster.
• [ ] The status of StorageNode would be consumed by common storage units related DistSQLs

Relevant Skills

1. Master JAVA Go language, Ginkgo test framework
2. Basic knowledge of logback and log4j2

3. Maven

Mentor

Have a basic understanding of Apache ShardingSphere Concepts
3. Be familiar with Kubernetes Operator, kubebuilder framework

Targets files

StorageNode Controller - https://github.com/apache/shardingsphere-on-cloud/blob/main/shardingsphere-operator/pkg/controllers/storagenode_controller.go

Mentor

Liyao MiaoLongtao Jiang, Committer of Apache ShardingSphere, jianglongtao@apache miaoliyao@apache.org^{Image Modified}

Trista PanChuxin Chen, PMC Committer of Apache ShardingSphere, panjuan@apachetuichenchuxin@apache.org^{Image Modified}

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page:   https://shardingsphere.apache.org/
Github:  https://github.com/apache/shardingsphere 

Background

ShardingSphere has designed its own metadata database to simulate metadata queries that support various databases.

More details:

https://github.com/apache/shardingsphere/issues/21268
https://github.com/apache/shardingsphere/issues/22052

Task

• Support PostgreSQL And openGauss `\d tableName`
• Support PostgreSQL And openGauss `\d+`
• Support PostgreSQL And openGauss `\d+ tableName`
• Support PostgreSQL And openGauss `l`
• Support query for MySQL metadata `TABLES`
• Support query for MySQL metadata `COLUMNS`
• Support query for MySQL metadata `schemata`
• Support query for MySQL metadata `ENGINES`
• Support query for MySQL metadata `FILES`
• Support query for MySQL metadata `VIEWS`

Notice, these issues can be a good example.

There is a proposal about the background of ChaosEngineering as belows:

Introduce ChaosEngineering for ShardingSphere #32
And we also proposed a generic controller for ShardingSphereChaos as belows:

[GSoC 2023] Introduce New CRD ShardingSphereChaos #272
The ShardingSphereChaos controller is aiming at different chaos tests. This JVMChaos is an important one.

Task

Write several scripts to implement different JVMChaos for main features of ShardingSphere. The specific case list is as follows.

• Add scripts injecting chaos to DataSharding
• Add scripts injecting chaos to ReadWritingSplitting
• Add scripts injecting chaos to DatabaseDiscovery
• Add scripts injecting chaos to Encryption
• Add scripts injecting chaos to Mask
• Add scripts injecting chaos to Shadow
  Basically, these scripts will cause unexpected behaviour while executing the related. DistSQL.

Relevant Skills

• Master Go language, Ginkgo test framework
• Have a deep understanding of Apache ShardingSphere concepts and practices.
• JVM byte mechanisms like ByteMan, ByteBuddy.

Targets files

JVMChaos Scripts - https://github.com/apache/shardingsphere/pull/22053
https://github.com/apache/shardingsphere/pull/22057/
https://github.com/apache/shardingsphere-on-cloud/pullchaos/22166jvmchaos/
https://github.com/apache/shardingsphere/pull/22182

Relevant Skills

•  Master JAVA language
•  Have a basic understanding of Zookeeper
•  Be familiar with MySQL/Postgres SQLs 

Mentor

scripts/

Mentor
Liyao MiaoChuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apachemiaoliyao@apache.org
Zhengqiang DuanChuxin Chen, PMC Committer of Apache ShardingSphere, duanzhengqiang@apachetuichenchuxin@apache.org

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org/
Github: https://github.com/apache/shardingsphere 

Background

Currently we try to promote StorageNode as major CRD to represent a set of storage units for ShardingSphere.

Task

The elementary task is that the storage node controller could manage the lifecycle of  a set of storage units, like PostgreSQL, in kubernetes. 

We don't hope to create another wheel like pg-operator. So consider using a predefined parameter group to generate the target CRD.

There is a proposal about the background of ChaosEngineering as belows:

• Introduce ChaosEngineering for ShardingSphere · Issue #32 · apache/shardingsphere-on-cloud (github.com)

The ShardingSphereChaos controller is aiming at different chaos tests. 

Task

Propose a generic controller for ShardingSphereChaos, which reconcile CRD ShardingSphereChaos, prepare, execute and verify test.

• [ ] Support common ShardingSphere features, prepare test rules and dataset
• [ ] Generating chaos type according to the backend implementation
• [ ] Verify testing result with DistSQL or other tools
• [ ] Generate DistSQL according to the Golang struct `EncryptionRule`
• [ ] Generate DistSQL according to the Golang struct `ShardingRule`
• [ ] Generate DistSQL according to the Golang struct `ReadWriteSplittingRule`
• [ ] Generate DistSQL according to the Golang struct `MaskRule`
• [ ] Generate DistSQL according to the Golang struct `ShadowRule`

Relevant Skills

1. Master Go language, Ginkgo test framework
2. Have a basic deep understanding of Apache ShardingSphere Concepts and DistSQLconcepts and practices.
3. Kubernetes operator pattern, kube-builder 

Targets files

DistSQL Converter ShardingSphereChaos Controller - https://github.com/apache/shardingsphere-on-cloud/blob/main/shardingsphere-operator/pkg/distsqlcontrollers/converterchaos_controller.go, etc.

Example

A struct defined as below:

```golang
type EncryptRule struct{}
func (t EncryptRule) ToDistSQL() string {}
```
While invoking ToDistSQL() it will generate a DistSQL regarding a EncryptRule like:

```SQL
CREATE ENCRYPT RULE t_encrypt (....
```

References:

• https://shardingsphere.apache.org/document/current/en/user-manual/shardingsphere-proxy/distsql/syntax/rdl/rule-definition/encrypt/create-encrypt-rule/

Mentor
Liyao Miao, Committer of Apache ShardingSphere, miaoliyao@apache.org

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org

Mentor

Liyao Miao, Committer of Apache ShardingSphere,  miaoliyao@apache.org^{Image Added}

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org^{Image Added}

Apache ShardingSphere

Apache ShardingSphere

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org/
Github: https://github.com/apache/shardingsphere 

Background

There is a proposal about new CRD Cluster and ComputeNode as belows:

• #167
• #166

Currently we try to promote StorageNode as major CRD to represent a set of storage units for ShardingSphere.

Task

The elementary task is that the storage node controller could manage the lifecycle of a set of storage units, like PostgreSQL, in kubernetes.

We don't hope to create another wheel like pg-operator. So consider using a predefined parameter group to generate the target CRD.

• [ ] Create a PostgreSQL cluster while a StorageNode with pg parameters is created
• [ ] Update the PostgreSQL cluster while updated StorageNode
• [ ] Delete the PostgreSQL cluster while deleted StorageNode. Notice this may need a deletion strategy.
• [ ] Reconciling StorageNode according to the status of PostgreSQL cluster.
• [ ] The status of StorageNode would be consumed by common storage units related DistSQLs

Relevant Skills

1. Master Go language, Ginkgo test framework
2. Have a basic understanding of Apache ShardingSphere Concepts
3. Be familiar with Kubernetes Operator, kubebuilder framework

Targets files

StorageNode Controller - https://github.com/apache/shardingsphere-on-cloud/blob/main/shardingsphere-operator/pkg/controllers/storagenode_controller.go

Mentor

Liyao Miao, Committer of Apache ShardingSphere,  miaoliyao@apache.org^{Image Removed}

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org^{Image Removed}

The ShardingSphere SQL federation engine provides support for complex SQL statements, and it can well support cross-database join queries, subqueries, aggregation queries and other statements. An important part of SQL federation engine is to convert the SQL statement parsed by ShardingSphere into SqlNode, so that Calcite can be used to implement SQL optimization and federated query.

Task

This issue is to solve the MySQL exception that occurs during SQLNodeConverterEngine conversion. The specific case list is as follows.

• select_char
• select_extract
• select_from_dual
• select_from_with_table
• select_group_by_with_having_and_window
• select_not_between_with_single_table
• select_not_in_with_single_table
• select_substring
• select_trim
• select_weight_string
• select_where_with_bit_expr_with_ampersand
• select_where_with_bit_expr_with_caret
• select_where_with_bit_expr_with_div
• select_where_with_bit_expr_with_minus_interval
• select_where_with_bit_expr_with_mod
• select_where_with_bit_expr_with_mod_sign
• select_where_with_bit_expr_with_plus_interval
• select_where_with_bit_expr_with_signed_left_shift
• select_where_with_bit_expr_with_signed_right_shift
• select_where_with_bit_expr_with_vertical_bar
• select_where_with_boolean_primary_with_comparison_subquery
• select_where_with_boolean_primary_with_is
• select_where_with_boolean_primary_with_is_not
• select_where_with_boolean_primary_with_null_safe
• select_where_with_expr_with_and_sign
• select_where_with_expr_with_is
• select_where_with_expr_with_is_not
• select_where_with_expr_with_not
• select_where_with_expr_with_not_sign
• select_where_with_expr_with_or_sign
• select_where_with_expr_with_xor
• select_where_with_predicate_with_in_subquery
• select_where_with_predicate_with_regexp
• select_where_with_predicate_with_sounds_like
• select_where_with_simple_expr_with_collate
• select_where_with_simple_expr_with_match

You need to compare the difference between actual and expected, and then correct the logic in SQLNodeConverterEngine so that actual can be consistent with expected.

After you make changes, remember to add case to SUPPORTED_SQL_CASE_IDS to ensure it can be tested.

 
Notice, these issues can be a good example.
https://github.com/apache/shardingsphere/pull/14492

Relevant Skills

 
1. Master JAVA language

2. Have a basic understanding of Antlr g4 file

3. Be familiar with MySQL and Calcite SqlNode

Targets files

 
SQLNodeConverterEngineIT

Apache ShardingSphere

Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.

Page: https://shardingsphere.apache.org/
Github: https://github.com/apache/shardingsphere

Background

There is a proposal about the background of ChaosEngineering as belows:

Introduce ChaosEngineering for ShardingSphere #32
And we also proposed a generic controller for ShardingSphereChaos as belows:

[GSoC 2023] Introduce New CRD ShardingSphereChaos #272
The ShardingSphereChaos controller is aiming at different chaos tests. This JVMChaos is an important one.

Task

Write several scripts to implement different JVMChaos for main features of ShardingSphere. The specific case list is as follows.

• Add scripts injecting chaos to DataSharding
• Add scripts injecting chaos to ReadWritingSplitting
• Add scripts injecting chaos to DatabaseDiscovery
• Add scripts injecting chaos to Encryption
• Add scripts injecting chaos to Mask
• Add scripts injecting chaos to Shadow
  Basically, these scripts will cause unexpected behaviour while executing the related. DistSQL.

Relevant Skills

• Master Go language, Ginkgo test framework
• Have a deep understanding of Apache ShardingSphere concepts and practices.
• JVM byte mechanisms like ByteMan, ByteBuddy.

Targets files

JVMChaos Scripts - https://github.com/apache/shardingsphere-on-cloud/chaos/jvmchaos/scripts//blob/master/test/it/optimizer/src/test/java/org/apache/shardingsphere/test/it/optimize/SQLNodeConverterEngineIT.java 

Mentor

Zhengqiang Duan, PMC Mentor
Liyao Miao, Committer of Apache ShardingSphere, miaoliyao@apacheduanzhengqiang@apache.org

Chuxin Chen, Committer of Apache ShardingSphere, tuichenchuxin@apache.org^{Image Added}

Trista Pan, PMC of Apache ShardingSphere, panjuan@apache.org

Background

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance. Currently, ShenYu has good usability and performance in microservice scenarios. However, ShenYu's support for Kubernetes is still relatively weak.

Tasks

1. Support the registration of microservices deployed in K8s Pod to shenyu-admin and use K8s as the register center.
2. Discuss with mentors, and complete the requirements design and technical design of Shenyu K8s Register Center.
3. Complete the initial version of Shenyu K8s Register Center.
4. Complete the CI test of Shenyu K8s Register Center, verify the correctness of the code.
5. Write the necessary documentation, deployment guides, and instructions for users to connect microservices running inside the K8s Pod to ShenYu

Relevant Skills

1. Know the use of Apache ShenYu, especially the register center
2. Familiar with Java and Golang
3. Familiar with Kubernetes and can use Java or Golang to develop

Background

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance. Currently, ShenYu has good usability and performance in microservice scenarios. However, ShenYu's support for Kubernetes is still relatively weak.

Tasks

1. Discuss with mentors, and complete the requirements design and technical design of shenyu-ingress-controller.
2. Complete the initial version of shenyu-ingress-controller, implement the reconcile of k8s ingress api, and make ShenYu as the ingress gateway of k8s.
3. Complete the ci test of shenyu-ingress-controller, verify the correctness of the code.

Relevant Skills

1. Know the use of Apache ShenYu
2. Familiar with Java and Golang
3. Familiar with Kubernetes and can use java or golang to develop Kubernetes Controller

Description

Issues ： https://github.com/apache/shenyu/issues/4438
website ： https://shenyu.apache.org/

Apache SkyWalking is an application performance monitor tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures. This year we will proceed on log clustering implementation with a revised architecture and this task will require student to focus on algorithm optimiztion for the clustering technique.

Apache SkyWalking is an application performance monitor tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures. This task is about enhancing Python agent performance, the tracking issue can be seen here -< https://github.com/apache/skywalking/issues/10408

Apache SkyWalking is an application performance monitor tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures. This year we will proceed on log clustering implementation with a revised architecture and this task will require student to focus on Flink and its integration with SkyWalking OAP.

Background:

Shenyu is a native API gateway for service proxy, protocol translation and API governance. but Shenyu lack of End-To-End Tests.

Relevant skills：

1.Understand the architecture of ShenYu

2.Understand SpringCloud micro-service and ShenYu SpringCloud proxy plugin.

3.Understand ShenYu e2e framework and architecture.

How to coding

1.please refer to org.apache.shenyu.e2e.testcase.plugin.DividePluginCases

How to test

1.start shenyu admin in docker

2.start shenyu boostrap in docker

3.run test case org.apache.shenyu.e2e.testcase.plugin.PluginsTest#testDivide

Task List

1.develop e2e tests of the springcloud plug-in.

2.write shenyu e2e springcloud plugin documentation in shenyu-website.

3.refactor the existing plugin test cases.

Links:

website： https://shenyu.apache.org/

issues: https://github.com/apache/shenyu/issues/4474

Background

SkyWalking BanyanDB is an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data.

Objectives

1. Support EXPLAIN[1] for both measure query and stream query
2. Add self-observability including trace and metrics for query subsystem
3. Support EXPLAIN in the client SDK & CLI and add query plan visualization in the UI

[1]: EXPLAIN in MySQL

Recommended Skills

1. Familiar with Go
2. Have a basic understanding of database query engine
3. Have an experience of Apache SkyWalking or other APMs

Mentor

Mentor: Jiajing Lu, Apache SkyWalking PMC, lujiajing@apache.org^{Image Removed}

Mentor: Hongtao Gao, Apache SkyWalking PMC, Apache ShardingSphere PMC, hanahmily@apache.org^{Image Removed}

Background

SkyWalking BanyanDB is an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data.

Objectives

1. Fully unify/merge the query planner and executor for Measure and TopN

Recommended Skills

1. Familiar with Go
2. Have a basic understanding of database query engine
3. Have an experience of Apache SkyWalking

Mentor

• Mentor: Jiajing Lu, Apache SkyWalking PMC, lujiajing@apache.org^{Image Removed}
• Mentor: Hongtao Gao, Apache SkyWalking PMC, Apache ShardingSphere PMC, hanahmily@apache.org^{Image Removed}
• Mailing List: dev@skywalking.apache.org

Background:{}

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance. Currently, ShenYu has good scalability in the Java language. However, ShenYu's support for multiple languages is still relatively weak.

The wasm bytecode is designed to be encoded in a size- and load-time-efficient binary format. WebAssembly aims to execute at native speed by taking advantage of common hardware capabilities available on a wide range of platforms.

The goal of WasmPlugin is to be able to run wasm bytecode(wasmer-java is a good choice, if you find a better choice, please discuss with me), and other languages can write ShenYu plugins based on this language(such as Rust/golang/C++) as long as they can be compiled into wasm bytecode.

More documents on wasm and WASI are as follows:
https://github.com/WebAssembly/design
https://github.com/WebAssembly/WASI

Relevant Skills

Know the use of Apache ShenYu, especially the plugin
Familiar with Java and another language which can be compiled into wasm bytecode

Task List

1.develop shenyu-wasm-plugin.

2.write integrated test for shenyu-wasm-plugin.

3.write wasm plugin documentation in shenyu-website.

Links:

website： https://shenyu.apache.org/

issues:  https://github.com/apache/shenyu/issues/4492

Now the deployment methods for SkyWalking are limited, we only have Helm Chart for users to deploy in Kubernetes, other users that are not using Kubernetes have to do all the house keeping stuffs to set up SkyWalking on, for example, VM.

This issue aims to add a Terraform provider, so that users can conveniently  spin up a cluster for demonstration or testing, we should evolve the provider and allow users to customize as their need and finally users can use this in their production environment.

In this task, we will mainly focus on the support for AWS. In the Terraform provider, users need to provide their access key / secret key, and the provider does the rest stuffs: create VMs, create database/OpenSearch or RDS, download SkyWalking tars, configure the SkyWalking, and start the SkyWalking components (OAP/UI), create public IPs/domain name, etc.

Background

Apache ShenYu Shenyu is a Java native API Gateway gateway for service proxy, protocol conversion translation and API governance. Currently, ShenYu has good usability and performance in microservice scenarios. However, ShenYu's support for Kubernetes is still relatively weak.

Tasks

1. Support the registration of microservices deployed in K8s Pod to shenyu-admin and use K8s as the register center.
2. Discuss with mentors, and complete the requirements design and technical design of Shenyu K8s Register Center.
3. Complete the initial version of Shenyu K8s Register Center.
4. Complete the CI test of Shenyu K8s Register Center, verify the correctness of the code.
5. Write the necessary documentation, deployment guides, and instructions for users to connect microservices running inside the K8s Pod to ShenYu

Relevant Skills

1. Know the use of Apache ShenYu, especially the register center
2. Familiar with Java and Golang
3. Familiar with Kubernetes and can use Java or Golang to develop

Background

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance. Currently, ShenYu has good usability and performance in microservice scenarios. However, ShenYu's support for Kubernetes is still relatively weak.

Tasks

1. Discuss with mentors, and complete the requirements design and technical design of shenyu-ingress-controller.
2. Complete the initial version of shenyu-ingress-controller, implement the reconcile of k8s ingress api, and make ShenYu as the ingress gateway of k8s.
3. Complete the ci test of shenyu-ingress-controller, verify the correctness of the code.

Relevant Skills

1. Know the use of Apache ShenYu
2. Familiar with Java and Golang
3. Familiar with Kubernetes and can use java or golang to develop Kubernetes Controller

Description

Issues ： https://github.com/apache/shenyu/issues/4438
website ： https://shenyu.apache.org/

Background

At present, shenyu needs to manually check whether the license is correct one by one when releasing the version.

Tasks

1. Discuss with the tutor to complete the requirement design and technical design of the scanning license.
2. Finished scanning the initial version of the license.
3. Complete the corresponding test.

Relevant Skills

1. Familiar with Java.

It can manage and maintain the API through Shenyu-admin, and support internationalization in Chinese and English. Unfortunately, Shenyu-admin is only internationalized on the front end. The message prompt returned by the back-end interface is still in English. Therefore, we need to implement internationalization support for the back-end interface.This will lay a good foundation for shenyu to move towards more language support.

Relevant skills

• Related skills spring resources
• Spring Internationalization
• Front-end react framework

API reference

            java.util.Locale;
            org.springframework.context.MessageSource;
            org.springframework.context.support.ResourceBundleMessageSource; 

Interface effect example

            ## zh request example
            POST http://localhost:9095/plugin
            Content-Type: application/json
            Location: cn-zh
            X-Access-Token: xxx
            {
            "name": "test-create-plugin",
            "role": "test-create-plugin",
            "enabled": true,
            "sort": 100
            }
            Respone
            {
            "code": 600,
            "message": "未登录"
            }

Background

Shenyu is a native API gateway for service proxy, protocol translation and API governance. It can manage and maintain the API through Shenyu-admin, and support internationalization in Chinese and English. Unfortunately, Shenyu-admin is only internationalized on the front end. The message prompt returned by the back-end interface is still in English. Therefore, we need to implement internationalization support for the back-end interface.This will lay a good foundation for shenyu to move towards more language support.

Relevant skills

• Related skills spring resources
• Spring Internationalization
• Front-end react framework

API reference

            java.util.Locale;
            org.springframework.context.MessageSource;### en request example
            org.springframework.context.support.ResourceBundleMessageSource; 

Interface effect example

            ## zh request example
            POST POST http://localhost:9095/plugin
            Content-Type: application/json
            Location: cn-zhen
            X-Access-Token: xxx
            {
            "name": "test-create-plugin",
            "role": "test-create-plugin",
            "enabled": true,
            "sort": 100
            }
            Respone
            {
            "code": 600,
            "message": "未登录token is error"
            }
            
            ### en request example
            POST http://localhost:9095/plugin
            Content-Type: application/json
            Location: en
            X-Access-Token: xxx
            {
            "name": "test-create-plugin",
            "role": "test-create-plugin",
            "enabled": true,
            "sort": 100
            }
            Respone
            {
            "code": 600,
            "message": "token is error"
            } 

Task List

Task List

• The task discussed with the tutor how to achieve the internationalization of shenyu-admin background
• Some prompt message translation
• Get through the internationalization of front-end, obtain the client region information through http protocol, support the language of the corresponding region.
• Leave the extension of other multi-language internationalization support interface, so as to facilitate the localization transformation of subsequent users.

Background

At present, shenyu needs to manually check whether the license is correct one by one when releasing the version.

Tasks

1. Discuss with the tutor to complete the requirement design and technical design of the scanning license.
2. Finished scanning the initial version of the license.
3. Complete the corresponding test.

Relevant Skills

1. Familiar with Java.

Background
Apache Traffic Control is a Content Delivery Network (CDN) control plane for large scale content distribution.

Traffic Control currently requires Apache Traffic Server as the underlying cache. Help us expand the scope by integrating with the very popular Varnish Cache.

There are multiple aspects to this project:

• Configuration Generation: Write software to build Varnish configuration files (VCL). This code will be implemented in our Traffic Ops and cache client side utilities, both written in Go.

• Health Monitoring: Implement monitoring of the Varnish cache health and performance. This code will run both in the Traffic Monitor component and within Varnish. Traffic Monitor is written in Go and Varnish is written in C.

• Testing: Adding automated tests for new code

Skills:

• Proficiency in Go is required
• A basic knowledge of HTTP and caching is preferred, but not required for this project.

https://github.com/apache/trafficcontrol/issues/7076

Apache Doris
Apache Doris is a real-time analytical database based on MPP architecture. As a unified platform that supports multiple data processing scenarios, it ensures high performance for low-latency and high-throughput queries, allows for easy federated queries on data lakes, and supports various data ingestion methods.
Page: https://doris.apache.org

Github

Background:

Shenyu is a native API gateway for service proxy, protocol translation and API governance. but Shenyu lack of End-To-End Tests.

Relevant skills：

1.Understand the architecture of ShenYu

2.Understand SpringCloud micro-service and ShenYu SpringCloud proxy plugin.

3.Understand ShenYu e2e framework and architecture.

How to coding

1.please refer to org.apache.shenyu.e2e.testcase.plugin.DividePluginCases

How to test

1.start shenyu admin in docker

2.start shenyu boostrap in docker

3.run test case org.apache.shenyu.e2e.testcase.plugin.PluginsTest#testDivide

Task List

1.develop e2e tests of the springcloud plug-in.

2.write shenyu e2e springcloud plugin documentation in shenyu-website.

3.refactor the existing plugin test cases.

Links:

website： https://shenyu.apache.org/

issues: https://github.com/apache/shenyu/issues/4474

doris

Background

Apache Doris accelerates high-concurrency queries utilizing page cache, where the decompressed data is stored.
Currently, the page cache in Apache Doris uses a simple LRU algorithm, which reveals a few problems: 

• Hot data will be phased out in large queries
• The page cache configuration is immutable and does not support GC.

Task

• Phase One: Identify the impacts on queries when the decompressed data is stored in memory and SSD, respectively, and then determine whether full page cache is required.

• Phase Two: Improve the cache strategy for Apache Doris based on the results from Phase One.

Learning Material

Page: https://doris.apache.org
Github:

Background:{}

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance. Currently, ShenYu has good scalability in the Java language. However, ShenYu's support for multiple languages is still relatively weak.

The wasm bytecode is designed to be encoded in a size- and load-time-efficient binary format. WebAssembly aims to execute at native speed by taking advantage of common hardware capabilities available on a wide range of platforms.

The goal of WasmPlugin is to be able to run wasm bytecode(wasmer-java is a good choice, if you find a better choice, please discuss with me), and other languages can write ShenYu plugins based on this language(such as Rust/golang/C++) as long as they can be compiled into wasm bytecode.

More documents on wasm and WASI are as follows:
https://github.com/WebAssembly/design
https://github.com/WebAssemblyapache/WASIdoris

Relevant Skills

Know the use of Apache ShenYu, especially the plugin
Familiar with Java and another language which can be compiled into wasm bytecode

Task List

1.develop shenyu-wasm-plugin.

2.write integrated test for shenyu-wasm-plugin.

3.write wasm plugin documentation in shenyu-website.

Links:

website： https://shenyu.apache.org/

issues:  https://github.com/apache/shenyu/issues/4492

Mentor

• Mentor: Yongqiang Yang, Apache Doris PMC member & Committer, yangyongqiang@apache.org ^{Image Added}
• Mentor: Haopeng Li, Apache Doris PMC member & Committer, lihaopeng@apache.org^{Image Added}  
• Mailing List: dev@doris.apache.org

Background
Apache Traffic Control is a Content Delivery Network (CDN) control plane for large scale content distribution.

Traffic Control currently requires Apache Traffic Server as the underlying cache. Help us expand the scope by integrating with the very popular Varnish Cache.

There are multiple aspects to this project:

• Configuration Generation: Write software to build Varnish configuration files (VCL). This code will be implemented in our Traffic Ops and cache client side utilities, both written in Go.

• Health Monitoring: Implement monitoring of the Varnish cache health and performance. This code will run both in the Traffic Monitor component and within Varnish. Traffic Monitor is written in Go and Varnish is written in C.

• Testing: Adding automated tests for new code

Skills:

• Proficiency in Go is required
• A basic knowledge of HTTP and caching is preferred, but not required for this project.

Apache Doris
Apache Doris is a real-time analytical database based on MPP architecture. As a unified platform that supports multiple data processing scenarios, it ensures high performance for low-latency and high-throughput queries, allows for easy federated queries on data lakes, and supports various data ingestion methods.
Page: https://doris.apache.org
Github: https://github.com/apache/doris

Background

Apache Doris supports acceleration of queries on external data sources to meet users' needs for federated queries and analysis.
Currently, Apache Doris supports multiple external catalogs including those from Hive, Iceberg, Hudi, and JDBC. Developers can connect more data sources to Apache Doris based on a unified framework.

Objective

• Enable Apache Doris to access one or more of these data sources via the Multi-Catalog feature: BigQuery/Kudu/Cassandra/Druid;
• Compile relevant documentation. See an example here: https://doris.apache.org/docs/dev/lakehouse/multi-catalog/hive

Task
Phase One:

• Get familiar with the Multi-Catalog structure of Apache Doris, including the metadata synchronization mechanism in FE and the data reading mechanism of BE.
• Investigate how metadata should be acquired and how data access works regarding the picked data source(s); produce the corresponding design documentation.

Phase Two:

• Develop connections to the picked data source(s) and implement access to metadata and data.

Learning Material

Page: https://doris.apache.org
Github: https://github.com/apache/doris

Mentor

• Mentor: Mingyu Chen, Apache Doris PMC Member & Committer, morningman@apache.org ^{Image Added}
• Mentor: Calvin Kirs, Apache Geode PMC & Committer, Kirs@apache.org^{Image Added}
• Mailing List: dev@doris.apache.org

https://github.com/apache/trafficcontrol/issues/7076

Apache Doris
Apache Doris is a real-time analytical database based on MPP architecture. As a unified platform that supports multiple data processing scenarios, it ensures high performance for low-latency and high-throughput queries, allows for easy federated queries on data lakes, and supports various data ingestion methods.
Page: https://doris.apache.org

Github: https://github.com/apache/doris

Background

Apache Doris accelerates high-concurrency queries utilizing page cache, where the decompressed data is stored.
Currently, the page cache in Apache Doris uses a simple LRU algorithm, which reveals a few problems: 

In Apache Doris, dictionary encoding is performed during data writing and compaction. Dictionary encoding will be implemented on string data types by default. The dictionary size of a column for one segment is 1M at most. The dictionary encoding technology accelerates strings during queries, converting them into INT, for example.
 

Task

• Phase One: Identify the impacts on queries when the decompressed data is stored in memory and SSD, respectively, and then determine whether full page cache is required.Phase Two: Improve the cache strategy for Apache Doris based on the results from Phase One.Get familiar with the implementation of Apache Doris dictionary encoding; learning how Apache Doris dictionary encoding accelerates queries.
•  Phase Two: Evaluate the effectiveness of full dictionary encoding and figure out how to optimize memory in such a case.

Learning Material

Page: https://doris.apache.org
Github: https://github.com/apache/doris

Mentor

• Mentor: Yongqiang YangChen Zhang, Apache Doris PMC member & Committer, yangyongqiang@apachezhangchen@apache.org 
• Mentor: Haopeng LiZhijing Lu, Apache Doris PMC member & Committer, lihaopeng@apacheluzhijing@apache.org  
• Mailing List: dev@doris.apache.org

Background

SkyWalking BanyanDB is an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data.

Objectives

1. Support EXPLAIN[1] for both measure query and stream query
2. Add self-observability including trace and metrics for query subsystem
3. Support EXPLAIN in the client SDK & CLI and add query plan visualization in the UI

[1]: EXPLAIN in MySQL

Recommended Skills

1. Familiar with Go
2. Have a basic understanding of database query engine
3. Have an experience of Apache SkyWalking or other APMs

Mentor

• Mentor: Jiajing Lu, Apache SkyWalking PMC, lujiajing@apache.org^{Image Added}
• Mentor: Hongtao Gao, Apache SkyWalking PMC, Apache ShardingSphere PMC, hanahmily@apache.org^{Image Added}
• Mailing List: dev@skywalking.apache.org

Apache Doris
Apache Doris is a real-time analytical database based on MPP architecture. As a unified platform that supports multiple data processing scenarios, it ensures high performance for low-latency and high-throughput queries, allows for easy federated queries on data lakes, and supports various data ingestion methods.
Page: https://doris.apache.org
Github: https://github.com/apache/doris

Background

Apache Doris supports acceleration of queries on external data sources to meet users' needs for federated queries and analysis.
Currently, Apache Doris supports multiple external catalogs including those from Hive, Iceberg, Hudi, and JDBC. Developers can connect more data sources to Apache Doris based on a unified framework.

Objective

• Enable Apache Doris to access one or more of these data sources via the Multi-Catalog feature: BigQuery/Kudu/Cassandra/Druid;
• Compile relevant documentation. See an example here: https://doris.apache.org/docs/dev/lakehouse/multi-catalog/hive

Task
Phase One:

• Get familiar with the Multi-Catalog structure of Apache Doris, including the metadata synchronization mechanism in FE and the data reading mechanism of BE.
• Investigate how metadata should be acquired and how data access works regarding the picked data source(s); produce the corresponding design documentation.

Phase Two:

• Develop connections to the picked data source(s) and implement access to metadata and data.

Learning Material

Page: https://doris.apache.org
Github: https://github.com/apache/doris

Mentor

• Mentor: Mingyu Chen, Apache Doris PMC Member & Committer, morningman@apache.org ^{Image Removed}
• Mentor: Calvin Kirs, Apache Geode PMC & Committer, Kirs@apache.org^{Image Removed}
• Mailing List: dev@doris.apache.org

Apache Doris
Apache Doris is a real-time analytical database based on MPP architecture. As a unified platform that supports multiple data processing scenarios, it ensures high performance for low-latency and high-throughput queries, allows for easy federated queries on data lakes, and supports various data ingestion methods.
Page: https://doris.apache.org

Github: https://github.com/apache/doris

Background

In Apache Doris, dictionary encoding is performed during data writing and compaction. Dictionary encoding will be implemented on string data types by default. The dictionary size of a column for one segment is 1M at most. The dictionary encoding technology accelerates strings during queries, converting them into INT, for example.
 

Task

• Phase One: Get familiar with the implementation of Apache Doris dictionary encoding; learning how Apache Doris dictionary encoding accelerates queries.
•  Phase Two: Evaluate the effectiveness of full dictionary encoding and figure out how to optimize memory in such a case.

Learning Material

Page: https://doris.apache.org
Github: https://github.com/apache/doris

Mentor

• Mentor: Chen Zhang, Apache Doris Committer, zhangchen@apache.org ^{Image Removed}
• Mentor: Zhijing Lu, Apache Doris Committer, luzhijing@apache.org^{Image Removed}  
• Mailing List: dev@doris.apache.org

Background

SkyWalking BanyanDB is an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data.

Objectives

1. Fully unify/merge the query planner and executor for Measure and TopN

Recommended Skills

1. Familiar with Go
2. Have a basic understanding of database query engine
3. Have an experience of Apache SkyWalking

Mentor

• Mentor: Jiajing Lu, Apache SkyWalking PMC, lujiajing@apache.org^{Image Added}
• Mentor: Hongtao Gao, Apache SkyWalking PMC, Apache ShardingSphere PMC, hanahmily@apache.org^{Image Added}
• Mailing List: dev@skywalking.apache.org

Now the deployment methods for SkyWalking are limited, we only have Helm Chart for users to deploy in Kubernetes, other users that are not using Kubernetes have to do all the house keeping stuffs to set up SkyWalking on, for example, VM.

This issue aims to add a Terraform provider, so that users can conveniently  spin up a cluster for demonstration or testing, we should evolve the provider and allow users to customize as their need and finally users can use this in their production environment.

In this task, we will mainly focus on the support for AWS. In the Terraform provider, users need to provide their access key / secret key, and the provider does the rest stuffs: create VMs, create database/OpenSearch or RDS, download SkyWalking tars, configure the SkyWalking, and start the SkyWalking components (OAP/UI), create public IPs/domain name, etc.

Currently skywalking OAP is bundled as a tar ball when releasing, and the start time is long, we are looking for a way to distribute the binary executable in a more convenient way and speed up the bootstrap time. Now we found that GraalVM is a good fit not only it can solve the two aforementioned points but also it will bring benefits that, we can rewrite our LAL or even MAL system in the future with a more secure and isolated method, wasm, which is supported GraalVM too!

so this task is to adjust OAP, build it into GraalVM and make all tests in OAP passed. 

Today, you can do all sorts of Machine Learning using Apache Beam (https://beam.apache.org/documentation/ml/overview/).
 
Many of our users, however, have a hard time getting started with ML and understanding how Beam can be applied to their day to day work. The goal of this project is to build out a series of Beam pipelines as Jupyter Notebooks demonstrating real world ML use cases, from NLP to image recognition to using large language models. As you go, there may be bugs or friction points as well which will provide opportunities to contribute back to Beam's core ML libraries.

Mentor for this will be Danny McCormick

Background

SkyWalking BanyanDB is an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data.

The BanyanDB UI is a web interface provided BanyanDB server. It's developed with Vue3 and Vite3

Objectives

The UI should have a user-friendly Overview page.
The Overview page must display a list of nodes running in a cluster.
For each node in the list, the following information must be shown:

• Node ID or name
• Uptime
• CPU usage (percentage)
• Memory usage (percentage)
• Disk usage (percentage)
• Ports(gRPC and HTTP)

The web app must automatically refresh the node data at a configurable interval to show the most recent information.

Recommended Skills

1. Familiar with Vue and Vite
2. Have a basic understanding of RESTFul
3. Have an experience of Apache SkyWalking

There is a community effort to build a Beam runner to run Beam pipelines on top of Ray: https://github.com/ray-project/ray_beam_runner/

This involves pushing that project forward. It will require writing lots of Python code, and specifically going through the list of issues (https://github.com/ray-project/ray_beam_runner/issues) and solving as many of them as possible to make sure the runner is compliant.

Good resource docs:

• https://docs.google.com/document/d/1vt78s48Q0aBhaUCHrVrTUsProJSP8-EBqDDRGTPEr0Y/edit#heading=h.bhgd35otqjly
• https://docs.google.com/document/d/1k5JpMzIP-l7BssLpp6-gSNR-796ttZcACaY1p2CEFho/edit#heading=h.f5fgmvcor6lo 

Apache SkyWalking is an application performance monitor tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures. This year we will proceed on log clustering implementation with a revised architecture and this task will require student to focus on algorithm optimiztion for the clustering technique.

Mentor

• Mentor: Sheng Wu, Apache SkyWalking PMC, wusheng@apache.org^{Image Added}
• Co-mentor: Yihao Chen (Superskyyy), Apache SkyWalking PMC, yihaochen@apache.org^{Image Added}
• Mailing List: dev@skywalking.apache.org

Beam has an experimental, ongoing implementation for a Rust SDK.

This project involves advancing that implementation and making sure it's compiant with Beam standards.

Good resource materials:

• https://lists.apache.org/thread/xg9xq0btp8k1wh2v1gpqyfhwpsyxq4ds

Apache SkyWalking is an application performance monitor tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures. This year we will proceed on log clustering implementation with a revised architecture and this task will require student to focus on Flink and its integration with SkyWalking OAP.

Mentor

• Mentor: Yanlong He, Apache SkyWalking PMC, heyanlong@apache.org^{Image Added}
• Co-mentor: Yihao Chen (Superskyyy), Apache SkyWalking PMC, yihaochen@apache.org^{Image Added}
• Mailing List: dev@skywalking.apache.org

This is a project to implement a tool for PMC task automation.

This is a large project.