Versions Compared

Old Version 10

changes.mady.by.user Unknown User (ztang)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (cdmikechen)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Planned releases

Release 0.

...

9.0(Planned)

Target Date: 

Code Freeze Date: 

Release Manager: 

Target Features: 

JIRA
Feature
CategoryDescription
Status
Assignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

164

Mini-Submarine

...

1306

ExperimentUpgrade database support (Hibernate, Postgresql ...)Need assignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues

...

10
jqlQueryproject = SUBMARINE AND

...

(

...

"

...

Target Version" = 0.9.0 OR fixVersion = 0.9.0 )
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.8.1(Planned)

Target Date: 

Code Freeze Date: 

Release Manager: 

Target Features: 

JIRACategoryDescriptionAssignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1345

ExperimentAdd XGBoost support.Need assignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1385

Python SDKUpdate TensorFlow/Pytorch/Mlflow/Tensorboard/Conda/Minio versions to the latest and deal with related version dependency issues encounteredNeed assignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues10
jqlQueryproject = SUBMARINE AND ("Target Version" = 0.

...

8.1 OR fixVersion = 0.8.1 )
serverId5aa69414-a9e9-3523-82ec-879b028fb15b


Completed releases (Following releases are all completed)

Release 0.

...

8.0(Released)

Target Date: 9/23/2023

Code Freeze Date: 

Release Manager: cdmikechen

Target Features: 


JIRACategory
Feature
Description
Design DocStatus
Assignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1152

Workbench

Making submarine support i18n is a meaningful work, which can make it more convenient for users using different languages.

cdmikechen

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

165

Submarine Workbench

https://docs.google.com/document/d/1LJc5hRTWfIs6K5f7vMgGC_FUizeQf-qqiWICqO8SoR8/edit

1272

Cloud-native DeploymentWe want to build a more robust and easy develop operator with operator-sdk (submarine-cloud-v3) and update supported k8s version to latest(1.24/1.25).cdmikechen

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1138

Security

Use pac4j to support OIDC and other SSO.

cdmikechen
in development

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

180

1320

Python SDK

Python2.7 has reached EOL since January 1, 2020. Python3.6 also has reached EOL since Dec 23, 2021. Popular ML/DL packages, such as Tensorflow and Pytorch, have not support Python 3.6. The support for Python3.6 will affect the dependency maintenance in the future.

Therefore, we should upgrade syntax for newer versions of Python.

Yao-Ching Huang

Submarine E2E Test

in development


Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1327

Cloud-native Deployment

We are still only up to 1.21 on the master branch and we need to consider how to support the latest releases of k8s and make the relevant adjustments.
Currently planned support for k8s versions: 1.22, 1.23, 1.24. And 1.25 will be supported when it is relatively stable.

cdmikechen


Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues10
jqlQueryproject = SUBMARINE AND ("Target Version" = 0.8.0 OR fixVersion = 0.8.0 )
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.7.0(Released)

Target Date: 4/25/2022

Code Freeze Date: 

Release Manager: Kevin Su

Target Features: 


Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues10
jqlQueryproject = SUBMARINE AND fixVersion = 0.7.0
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.6.0(Released)

Target Date: 10/21/2021

Code Freeze Date: 

Release Manager: Kevin Su

Target Features: 


JIRACategoryDescriptionAssignee

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-1024

Experiment Template

Fix the error when building the jupyter image

Kevin Su

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,Priority,status,resolution
maximumIssues10
jqlQueryproject = SUBMARINE AND "Target Version" = 0.6.0
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.5.0(Released)

Target Date: 12/24/2020

Code Freeze Date: 

Release Manager: Kevin Su

Target Features: 


JIRACategoryDescriptionAssignee
-148

Submarine SDK

https://docs.google.com/document/d/1aARS2XWsuyiyuUGho9N0cwKXSoqJcfTwgYbjjij-HJo/editin development

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

230

Python/Spark Interpreter

https://docs.google.com/document/d/1LJc5hRTWfIs6K5f7vMgGC_FUizeQf-qqiWICqO8SoR8/edit

559

Predefined Experiment

Predefined Template:

  • [API] Define Experiment API for pre-defined template
  • [SDK] Add Python SDK to support pre-defined experiment
  • [Sever] Support submit pre-defined template, and translate it to an actual job
John Ting

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-569

Experiment UI 
  • [UI]: Allow Run Adhoc Python/TF/PyTorch Experiment via UI
  • [UI] Allow Run pre-defined experiment
Various
in development

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

154

570

CommonSupport run experiment/notebook with defined code sync (like Git repo)Need assignee

Submarine On K8s

in development

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

224

571

Submarine Server

https://docs.google.com/document/d/1vCjt2z3DoagfeaVQV0nfyWqSYLymoP53rDcPcz-YcSc/edit
Environment UIUI for environment management (add/remove/update)Need assignee

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-507

Environment
  • [API] Define swagger API for environment management (add/remove/update/list)
  • [Server] Support get an environment definition, and generate experiment/notebook submission spec
  • [Dockerfile] Prepare sample Dockerfile for Environment
Mani
in development

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

278

477

Submarine MetaStore

https://docs.google.com/document/d/1C2zaJJ0DuTn0amS83tNqJTEmzsQdIzgLSrJ5LQEBsSQ/edit#in development
Notebook
  • [API] Define Swagger API for notebook managmeent (Launch/Delete/Get)
  • [Server] Implement logic to submit Jupyter notebook to K8s (with Environment/code sync)
Ryan Lo

Jira
serverASF JIRA
columnstype,key,summary,priority,status,created,updated,assignee,reporter
maximumIssues10
jqlQueryproject in ("Apache Submarine") AND "Target Version" = 0.5.0
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.4.0

Target Date: 30/06/2020

Code Freeze Date: 25/06/2020

Release Manager: Wanqiang Ji

Target Features

JIRAFeature DescriptionDesign DocStatus

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-499

Submarine Experiment


Done

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-462

Submarine Helm Charts


Done

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-376

PySubmarine
Done

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

245

224

Submarine

Commons

Server

https://docs.google.com/document/d/
1C2zaJJ0DuTn0amS83tNqJTEmzsQdIzgLSrJ5LQEBsSQ/edit#
1vCjt2z3DoagfeaVQV0nfyWqSYLymoP53rDcPcz-YcSc/edit

Done

Jira
serverASF JIRA
columnstype,key,summary,priority,status,created,updated,assignee,reporter
maximumIssues10
jqlQueryproject in ("Apache Submarine") AND "Target Version" = 0.4.0
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Release 0.3.0 (Released)

Release date: 02/01/2020

Release Manager:  Zhankun Tang

Target Features

JIRAFeature DescriptionStatus
in development

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-

254

164

Mini-Submarine

Done

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-224

Basic submarine TF job submittion to K8sDone

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keySUBMARINE-303

Cluster
https://docs.google.com/document/d/1Ax6FQ5CAP-jowm2_Mp2r5kc9r7s1bkFLRfzvvbm5Wzc/edit#heading=h.s41ckl271z8sDone
mode job submittion on YARNDone

Pending JIRAs:

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQueryproject = SUBMARINE AND status in (Open, "In Progress", Reopened, "Patch Available") AND "Target Version" = 0.3.0 AND (component = "Mini Submarine" OR component = "Backend Server" OR component = Commons OR component = Doc)
serverId5aa69414-a9e9-3523-82ec-879b028fb15b


Legacy issues

Release 0.6.1

Some issues (like hadoop related issues) are no longer considered for maintenance, so we create a new release to temporarily categorize these issues.

Target Features

Jira
serverASF JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues10
jqlQueryproject = SUBMARINE AND ("Target Version" = 0.6.1 OR fixVersion = 0.6.1 )
serverId5aa69414-a9e9-3523-82ec-879b028fb15b