DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Summary
Important highlights from this release
All updates for this release
Epic
- SYSTEMDS-3572 OPEN This epic contains all the task associated with morphing of columngroups.
- SYSTEMDS-3019 RESOLVED JUnit test suite for GPU
- SYSTEMDS-2840 RESOLVED Add Builtin Functions
- SYSTEMDS-493 RESOLVED Modularize Existing DML Algorithms
New Feature
- SYSTEMDS-3640 RESOLVED Hash Column in Frames
- SYSTEMDS-3635 RESOLVED Cast as frame with column names
- SYSTEMDS-3602 RESOLVED Settings warnings (Memory)
- SYSTEMDS-3601 RESOLVED Performance Jar
- SYSTEMDS-3592 RESOLVED Compress Frame
- SYSTEMDS-3590 RESOLVED Frame DDC Encoding
- SYSTEMDS-3578 RESOLVED ColGroup Info for Morphing individual groups.
- SYSTEMDS-3576 RESOLVED Morph ColumnGroup instruction
- SYSTEMDS-3575 RESOLVED Column Group get compression Info
- SYSTEMDS-3574 RESOLVED Recompress Compressed Matrix
- SYSTEMDS-3573 RESOLVED CLALibCombineGroups
- SYSTEMDS-3570 RESOLVED Error statistics builtins
- SYSTEMDS-3495 RESOLVED Parallel Compressed Encode
- SYSTEMDS-3405 OPEN Federated Write at site
- SYSTEMDS-3317 CLOSED Lineage-based pruning of ML tasks
- SYSTEMDS-2879 RESOLVED Cspline Predict function
- SYSTEMDS-1027 CLOSED Compile to Julia
Improvement
- SYSTEMDS-3663 RESOLVED Low overhead join indexes
- SYSTEMDS-3662 RESOLVED Parfor Merge Sparse
- SYSTEMDS-3652 OPEN Add row direction for unique() builtin
- SYSTEMDS-3637 RESOLVED Manifest for SystemDS
- SYSTEMDS-3619 RESOLVED IO save Encoded
- SYSTEMDS-3618 RESOLVED Compresseed Schemas
- SYSTEMDS-3610 RESOLVED BuiltinSwitch Aggregate Skip
- SYSTEMDS-3609 RESOLVED Common ThreadPool Reuse ThreadLocal Pools
- SYSTEMDS-3608 RESOLVED Cocode shortcut
- SYSTEMDS-3607 RESOLVED Split Balanced Internal
- SYSTEMDS-3532 RESOLVED CLA Compressed Parallel Writing
- SYSTEMDS-3531 RESOLVED Unify Application Function and Component Test GitHub Action
- SYSTEMDS-3528 RESOLVED Refine allocation of GitHub Artifacts
- SYSTEMDS-3527 RESOLVED CLA DeltaOffset Skip list
- SYSTEMDS-3526 RESOLVED Transpose Sparse to Dense
- SYSTEMDS-3525 RESOLVED Binary Inplace Operations
- SYSTEMDS-3508 RESOLVED Remove Spark Checkpoint if CP
- SYSTEMDS-3491 RESOLVED CLA Specialized Column Indexes
- SYSTEMDS-3490 RESOLVED Compressed Transform Encode
- SYSTEMDS-2985 RESOLVED List nested access
- SYSTEMDS-2753 RESOLVED Approximation and information loss information for LCLA
- SYSTEMDS-2102 OPEN Vectorize gradients for Factorization Machines function
- SYSTEMDS-496 RESOLVED Running dml on hadoop writes files to dir that requires root permission
- SYSTEMDS-339 RESOLVED DML-Improvement. Unneccessary warn in If else
Bug
- SYSTEMDS-3673 RESOLVED SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"
- SYSTEMDS-3672 CLOSED GIO Test Cases false positive
- SYSTEMDS-3667 CLOSED Erroneous cleanup of List objects after function return
- SYSTEMDS-3659 RESOLVED Federated GitHub Actions Fail
- SYSTEMDS-3657 OPEN smaller adjustments around the word embedding encoder
- SYSTEMDS-3654 OPEN FederatedCentralMomentTest infinite cloud runtime
- SYSTEMDS-3642 RESOLVED CLA NaN in Dictionaries replace
- SYSTEMDS-3639 RESOLVED logLoss Nan
- SYSTEMDS-3622 CLOSED Single-threaded libsvm writer hangs forever
- SYSTEMDS-3617 RESOLVED Offsets Separated
- SYSTEMDS-3612 CLOSED Persistence is not respected for RDD
- SYSTEMDS-3606 CLOSED pnmf with ultra-sparse data never finishes
- SYSTEMDS-3600 RESOLVED POM Shader bug
- SYSTEMDS-3598 RESOLVED Federated Compression Redundant Release
- SYSTEMDS-3596 CLOSED Incorrect heavy hitter statistics
- SYSTEMDS-3593 RESOLVED Fix test coverage badges
- SYSTEMDS-3591 CLOSED Recompilation Issue XGBoost
- SYSTEMDS-3588 RESOLVED Federated Remote Monitoring
- SYSTEMDS-3587 CLOSED Federated missing variable on toString
- SYSTEMDS-3586 CLOSED Federated missing release variable
- SYSTEMDS-3584 CLOSED OOM while reading matrix from HDFS
- SYSTEMDS-3577 CLOSED Fix default storage memory fractions in Spark
- SYSTEMDS-3520 RESOLVED Error if multiple encoders applied on one column
- SYSTEMDS-3514 CLOSED Parfor optimizer adds redundant checkpointing statement blocks
- SYSTEMDS-3512 CLOSED Bug in Decision Trees
- SYSTEMDS-3509 CLOSED Function call with named arguments misassigns variables
- SYSTEMDS-3504 OPEN GPU Junit tests
- SYSTEMDS-3502 CLOSED Perftest: lmDS on 1M_1k_dense with unnecessary read from checkpoint
- SYSTEMDS-3441 RESOLVED Monitoring figues lies enter bottom.
- SYSTEMDS-3423 RESOLVED Debug runtime of Animal dataset
- SYSTEMDS-3391 IN PROGRESS Correct the release artifact generation date
- SYSTEMDS-3372 OPEN Regression in Federated SSL Tests
- SYSTEMDS-3315 CLOSED Fix lineage exploitation in the buffer pool
- SYSTEMDS-3292 CLOSED Decision tree training built-in does not stop when ran with data generated for logistic regression
- SYSTEMDS-3251 RESOLVED Unstable Tests: Async PrefetchRDD
- SYSTEMDS-3200 RESOLVED Federated Lineage full reuse bug
- SYSTEMDS-3110 RESOLVED Codegen CNodeBinary - No enum constant RBIND
- SYSTEMDS-3077 RESOLVED Codegen failing w/ unsupported operation
- SYSTEMDS-3071 RESOLVED Builtin MultiLogReg Fails if forced spark mode.
- SYSTEMDS-3070 RESOLVED Spark aggregation on 0 row or col matrix
- SYSTEMDS-3038 RESOLVED CLA Spark Left Multiplication No Unification of matrix
- SYSTEMDS-3029 CLOSED Missing estimate of intermediate size in codegen hop SpoofFusedOp
- SYSTEMDS-2672 CLOSED Range Indexing (all rows, columns) recompile
- SYSTEMDS-2541 RESOLVED Runtime exception in frame cbind operation in Spark context
- SYSTEMDS-2363 CLOSED Inflexible write format parameter requires constant propagation
- SYSTEMDS-2341 CLOSED Improve right indexing recompilation
- SYSTEMDS-2316 CLOSED Data type propagation issue with mixed type accumulate
- SYSTEMDS-2080 RESOLVED Lenet over mnist shows accuracy issues on CPU backend
- SYSTEMDS-1965 CLOSED Refactor nn layers to move the computation in forward/backward function known at compile time to init function
- SYSTEMDS-1597 CLOSED Random Forest With Categorical Variables
- SYSTEMDS-1371 CLOSED The "sparsity" parameter for the "rand" function is confusingly named
- SYSTEMDS-624 RESOLVED writeMM and write.csv code currently not called
- SYSTEMDS-614 CLOSED Single variable DML script reports wrong line number
Task
- SYSTEMDS-3661 CLOSED Native builtin for contains row vector operations
- SYSTEMDS-3636 RESOLVED Sparse Transpose-Self MatMult w/ Sparse Output
- SYSTEMDS-3615 CLOSED Array-based lookups in recode transformdecode
- SYSTEMDS-3613 CLOSED Missing Size Propagation transformapply/transformdecode
- SYSTEMDS-3595 CLOSED Extended lineage exploitation in the buffer pool
- SYSTEMDS-3542 RESOLVED Compressed transfer to/from federated workers
- SYSTEMDS-3329 CLOSED Builtin for pageRank (multiple methods)
- SYSTEMDS-3153 RESOLVED Missing value imputation using KNN
- SYSTEMDS-3149 CLOSED Decision Tree cleanups and Builtin
- SYSTEMDS-3135 RESOLVED Reader Generation for Custom Text Formats
- SYSTEMDS-346 RESOLVED Document SystemML best practices
- SYSTEMDS-229 RESOLVED New result merge function 'append' (row and column append)
Sub-task
- SYSTEMDS-3646 CLOSED Extend max_parallelize operator ordering to support GPU OPs
- SYSTEMDS-3582 CLOSED Optional Data Compaction in decisionTree/randomForest
- SYSTEMDS-3567 CLOSED Cost-based eviction from GPU lineage cache
- SYSTEMDS-3564 REOPENED Cost-based operator ordering
- SYSTEMDS-3562 CLOSED Compile-time chkpoint placement for Spark operators in loop
- SYSTEMDS-3555 CLOSED Add unit tests for word embedding
- SYSTEMDS-3522 CLOSED Add missing binning decoder to transformdecode
- SYSTEMDS-3513 CLOSED Multi-level reuse for GPU intermediates
- SYSTEMDS-3510 CLOSED Recycling intermediates from GPU lineage cache
- SYSTEMDS-3506 CLOSED Additional decisionTree/randomForest inference methods
- SYSTEMDS-3505 CLOSED Cleanup randomForest builtin
- SYSTEMDS-3483 CLOSED Compile-time chkpoint placement for shared Spark operators
- SYSTEMDS-3277 CLOSED Cleanup decisionTree builtin
- SYSTEMDS-3140 RESOLVED Structured data formats (JSON, XML)
- SYSTEMDS-3139 RESOLVED Performance improvements readers
- SYSTEMDS-3138 RESOLVED Performance improvements analysis
- SYSTEMDS-3137 RESOLVED Matrix and Frame Reader Generation
- SYSTEMDS-3136 RESOLVED Analysis: structure Identification
- SYSTEMDS-2947 CLOSED Lineage cache eviction in GPU
- SYSTEMDS-1320 RESOLVED Map writeMM and write.csv to write
Dependency
- SYSTEMDS-3656 RESOLVED Update Hadoop 3.3.6
- SYSTEMDS-3655 RESOLVED Update Spark Version 3.5.0
- SYSTEMDS-3375 OPEN CUDA11 / CUDNN8 support
Dependency upgrade
- SYSTEMDS-3515 CLOSED node-sass and browser-sync upgrade
Story
- SYSTEMDS-3087 OPEN Memory management and lazy evaluation in dynamic environments
Wish
- SYSTEMDS-3428 RESOLVED Code coverage in GitHub Actions (Java)
- SYSTEMDS-3427 OPEN Code coverage in Github Actions (Python)