THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Release 4.0.0 - Unreleased (as of 2023-12-12)
IMPROVEMENTS:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| HIVE-26666 | Filter out compactions by id to minimise expense of db operations | Major | . | KIRTI RUGE | KIRTI RUGE |
| HIVE-27563 | Add typeof UDF | Minor | UDF | John Sherman | John Sherman |
| HIVE-27575 | MASK_HASH UDF generate a log per row | Major | UDF | okumin | okumin |
| HIVE-27396 | Use -strict argument for Thrift code generation to prevent compatibility issues | Major | Build Infrastructure | Joe McDonnell | Joe McDonnell |
| HIVE-27554 | Validate URL used by SSO workflow for JDBC connection | Minor | JDBC | Henri Biestro | Henri Biestro |
| HIVE-27516 | Add programatically added DAG scoped properties to DAG Configuration | Major | . | László Bodor | László Bodor |
| HIVE-27586 | Parse dates from strings ignoring trailing (potentialy) invalid chars | Major | HiveServer2 | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27578 | Refactor genJoinRelNode to use genAllRexNode instead of genAllExprNodeDesc | Major | . | Soumyakanti Das | Soumyakanti Das |
| HIVE-27600 | Reduce filesystem calls in OrcFileMergeOperator | Minor | Hive | Yi Zhang | Yi Zhang |
| HIVE-25576 | Configurable datetime formatter for unix_timestamp, from_unixtime | Major | . | Ashish Sharma | Stamatis Zampetakis |
| HIVE-27585 | Upgrade kryo serialization lib to latest version | Minor | Serializers/Deserializers | Suprith Chandrashekharachar | Suprith Chandrashekharachar |
| HIVE-27595 | Improve efficiency in the filtering hooks | Minor | HiveServer2 | Naveen Gangam | Henri Biestro |
| HIVE-27645 | Clean test cases by refactoring assertFalse(equals()) using assertNotEquals & @Test(excepted) using assertThrows | Minor | . | Taher Ghaleb | Taher Ghaleb |
| HIVE-27599 | ConvertAstToSearchArg improvement with output and typo in comments | Major | Server Infrastructure | xy | xy |
| HIVE-27627 | Iceberg: Insert into/overwrite partition support | Major | . | Sourabh Badhya | Sourabh Badhya |
| HIVE-27525 | Ease the write permissions on external table during create table operation | Major | Standalone Metastore | Sai Hemanth Gantasala | Riju Trivedi |
| HIVE-27665 | Change Filter Parser on HMS to allow backticks | Major | Standalone Metastore | Steve Carlin | Steve Carlin |
| HIVE-27461 | HiveMetaStoreAuthorizer should log the root cause of checkPrivileges | Major | Hive | Wechar | Wechar |
| HIVE-27661 | Auth mode inferred from the Authorization header | Major | HiveServer2 | Gergely Farkas | Gergely Farkas |
| HIVE-27309 | Large number of partitions and small files causes OOM in query coordinator | Major | Iceberg integration | Rajesh Balamohan | Dmitriy Fingerman |
| HIVE-27656 | Upgrade jansi.version to 2.4.0 | Major | . | Simhadri Govindappa | Simhadri Govindappa |
| HIVE-27673 | Configurable datetime formatter for date_format | Major | HiveServer2 | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27664 | AlterTableSetLocationAnalyzer threw a confusing exception "Cannot connect to namenode" | Major | . | xiongyinke | xiongyinke |
| HIVE-27564 | Add log for ZooKeeperTokenStore | Major | . | lvyanquan | lvyanquan |
| HIVE-27694 | Include HiveIcebergSerDe in default list of serdes using HMS | Minor | Standalone Metastore | Naveen Gangam | Naveen Gangam |
| HIVE-27696 | Docker build from source should include iceberg profile | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-21100 | Allow flattening of table subdirectories resulted when using TEZ engine and UNION clause | Major | . | George Pachitariu | Ayush Saxena |
| HIVE-27646 | Iceberg: Retry query when concurrent write queries fail due to conflicting writes | Major | . | Simhadri Govindappa | Simhadri Govindappa |
| HIVE-27672 | Iceberg: Truncate partition support | Major | . | Sourabh Badhya | Sourabh Badhya |
| HIVE-27729 | Iceberg: Check Iceberg type in AlterTableExecuteAnalyzer | Major | Iceberg integration | Butao Zhang | Butao Zhang |
| HIVE-27406 | CompactionTxnHandler cleanup | Major | Hive | László Végh | László Végh |
| HIVE-27752 | Remove DagUtils duplicate class | Minor | . | László Bodor | Akshat Mathur |
| HIVE-27757 | Upgrade hadoop to 3.3.6 | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27723 | Prevent localizing the same original file more than once if symlinks are present | Major | . | László Bodor | László Bodor |
| HIVE-27771 | Iceberg: Allow expire snapshot by time range | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27755 | Quote identifiers in SQL emitted by SchemaTool for MySQL | Major | Standalone Metastore | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27793 | Iceberg: Support setting current snapshot with SnapshotRef | Major | Iceberg integration | Butao Zhang | Butao Zhang |
| HIVE-27799 | Bump org.apache.zookeeper:zookeeper from 3.7.1 to 3.7.2 | Major | . | Ayush Saxena | |
| HIVE-27182 | tez_union_with_udf.q with TestMiniTezCliDriver is flaky | Major | . | Ayush Saxena | Stamatis Zampetakis |
| HIVE-27686 | Use ORC 1.8.5. | Major | . | Zoltán Rátkai | Zoltán Rátkai |
| HIVE-27802 | Simplify TestTezSessionState.testSymlinkedLocalFilesAreLocalizedOnce | Major | . | László Bodor | László Bodor |
| HIVE-27786 | Iceberg: Eliminate engine.hive.enabled table property | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27346 | Getting exception for wildcard (*) search for database and table name | Major | . | Hongdan Zhu | Hongdan Zhu |
| HIVE-27731 | Perform metadata delete when only static filters are present | Major | . | Sourabh Badhya | Sourabh Badhya |
| HIVE-27826 | Upgrade to Parquet 1.13.1 | Major | Parquet | Butao Zhang | Butao Zhang |
| HIVE-27780 | Implement direct SQL for get_all_functions | Major | Standalone Metastore | Butao Zhang | Butao Zhang |
| HIVE-27825 | Better error message for an empty quoted identifier | Minor | Parser | okumin | okumin |
| HIVE-24815 | Remove "IDXS" Table from Metastore Schema | Major | Metastore, Standalone Metastore | Hunter Logan | Riju Trivedi |
| HIVE-27779 | Iceberg: Drop partition support | Major | . | Sourabh Badhya | Sourabh Badhya |
| HIVE-27789 | Iceberg: Add a way to expire snapshots with retain last | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27593 | Iceberg: Keep iceberg properties in sync with hms properties | Major | Iceberg integration | Butao Zhang | Butao Zhang |
| HIVE-27864 | Update plugin for SBOM generation to 2.7.10 | Major | . | Vinod Anandan | Vinod Anandan |
| HIVE-27762 | Don't fall back to jdo query in ObjectStore if direct sql throws unrecoverable exception | Major | Hive | Wechar | Wechar |
| HIVE-27819 | Iceberg: Upgrade iceberg version to 1.4.2 | Major | Iceberg integration | Butao Zhang | Butao Zhang |
| HIVE-27869 | Iceberg: Select on HadoopTable fails at HiveIcebergStorageHandler#canProvideColStats | Major | Iceberg integration | Butao Zhang | Butao Zhang |
| HIVE-27828 | Iceberg integration: enable copy on write update when split update is on | Major | Iceberg integration | Krisztian Kasa | Krisztian Kasa |
| HIVE-27877 | Bump org.apache.avro:avro from 1.11.1 to 1.11.3 | Major | . | Ayush Saxena | |
| HIVE-27889 | Bump org.apache.santuario:xmlsec from 2.3.0 to 2.3.4 | Major | . | Ayush Saxena | |
| HIVE-27871 | Fix some formatting problems is YarnQueueHelper | Major | . | László Bodor | Mahesh Raju Somalaraju |
| HIVE-27896 | Remove common-lang usage | Major | . | Cheng Pan | Cheng Pan |
| HIVE-27906 | Iceberg: Implement Delete Orphan Files | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27912 | Include Iceberg module in nightly builds | Major | . | Butao Zhang | Butao Zhang |
| HIVE-27903 | Iceberg: Implement Expire Snapshot with default table properties | Major | Hive | JK Pasimuthu | Ayush Saxena |
| HIVE-27633 | HMS: MTable to Table process reduces view related SQL | Minor | Metastore | dzcxzl | dzcxzl |
| HIVE-26630 | Introduce copy-on-write DML implementations for Iceberg tables | Major | . | Ádám Szita | Denys Kuzmenko |
| HIVE-27856 | Change the default value of hive.optimize.cte.materialize.threshold to -1 | Major | . | Seonggon Namgung | Seonggon Namgung |
| HIVE-27186 | A persistent property store | Major | Metastore | Henri Biestro | Henri Biestro |
BUG FIXES:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| HIVE-27562 | Iceberg: Fetching virtual columns failing | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27487 | NPE in Hive JDBC storage handler | Major | JDBC storage handler | Zhihua Deng | Zhihua Deng |
| HIVE-27582 | Do not cache HBase table input format in FetchOperator | Major | . | Ganesha Shreedhara | Ganesha Shreedhara |
| HIVE-27304 | Exclude CTAS condition while forming storage handler url permissions in HS2 authorizer. | Major | Hive | Sai Hemanth Gantasala | Sai Hemanth Gantasala |
| HIVE-24771 | Fix hang of TransactionalKafkaWriterTest | Major | . | Zoltan Haindrich | Kokila N |
| HIVE-27631 | Fix CCE when set fs.hdfs.impl other than DistributedFileSystem | Major | Hive | Baolong Mao | Baolong Mao |
| HIVE-22961 | Drop function in Hive should not send request for drop database to Ranger plugin. | Major | Hive | Sam An | Riju Trivedi |
| HIVE-27463 | Non-daemon thread prevents HMS from exiting when failed to start thrift server | Minor | . | Zhihua Deng | Zhihua Deng |
| HIVE-27536 | Merge task must be invoked after optimisation for external CTAS queries | Major | . | Sourabh Badhya | Sourabh Badhya |
| HIVE-27566 | Fix some yarn cluster options for tests | Major | . | László Bodor | László Bodor |
| HIVE-27632 | ClassCast Exception in Vectorization converting decimal64 to decimal | Major | Vectorization | Riju Trivedi | Stephen Carlin |
| HIVE-27539 | Drop renamed external table fails when hive.metastore.try.direct.sql.ddl is disabled | Major | . | Venugopal Reddy K | Venugopal Reddy K |
| HIVE-21213 | Acid table bootstrap replication needs to handle directory created by compaction with txn id | Major | Hive, HiveServer2, repl | mahesh kumar behera | mahesh kumar behera |
| HIVE-27667 | Fix get partitions with max_parts | Major | Metastore | Yuming Wang | Yuming Wang |
| HIVE-27657 | Change hive.fetch.task.conversion.threshold default value | Major | . | Mayank Kunwar | Mayank Kunwar |
| HIVE-26961 | Fix improper replication metric count when hive.repl.filter.transactions is set to true. | Major | . | Rakshith C | Rakshith C |
| HIVE-27642 | StartMiniHS2Cluster fails to run due to missing JDBC driver with Postgres | Major | HiveServer2 | Zoltán Rátkai | Zoltán Rátkai |
| HIVE-27643 | Exclude compaction queries from ranger policies | Critical | . | László Végh | László Végh |
| HIVE-17350 | metrics errors when retrying HS2 startup | Major | . | Sergey Shelukhin | Mayank Kunwar |
| HIVE-27648 | CREATE TABLE with CHECK constraint fails with SemanticException | Major | Hive | Soumyakanti Das | Krisztian Kasa |
| HIVE-24606 | Multi-stage materialized CTEs can lose intermediate data | Major | Query Planning | okumin | okumin |
| HIVE-27138 | MapJoinOperator throws NPE when computing OuterJoin with filter expressions on small table | Blocker | . | Seonggon Namgung | Seonggon Namgung |
| HIVE-27675 | Support keystore/truststore types for hive to zookeeper integration points | Major | HiveServer2, JDBC, Standalone Metastore | Naveen Gangam | Naveen Gangam |
| HIVE-27730 | Bump org.xerial.snappy:snappy-java from 1.1.10.1 to 1.1.10.4 | Major | . | Ayush Saxena | |
| HIVE-27738 | SchemaTool picks incorrect schema script after 4.0.0-beta-1 release | Major | . | KIRTI RUGE | KIRTI RUGE |
| HIVE-27649 | Support ORDER BY clause in subqueries with set operators | Major | Parser | Nicolas Richard | Nicolas Richard |
| HIVE-27760 | WHERE condition on DATE type partitioning column leads to wrong results | Major | HiveServer2 | Dayakar M | Dayakar M |
| HIVE-27728 | Changed behavior for alter table rename partition from legacy tables | Major | . | Naveen Gangam | Zhihua Deng |
| HIVE-27733 | Intermittent ConcurrentModificationException in HiveServer2 | Major | HiveServer2 | Henri Biestro | Henri Biestro |
| HIVE-27764 | Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing | Major | HiveServer2 | Gergely Farkas | Gergely Farkas |
| HIVE-27695 | Intermittent OOM when running TestMiniTezCliDriver | Major | Test | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-26828 | Fix OOM for hybridgrace_hashjoin_2.q | Major | Test, Tez | Alessandro Solimando | Stamatis Zampetakis |
| HIVE-27798 | Correct configuration item in hive-site.xml in docker. | Major | . | 易霖威 | 易霖威 |
| HIVE-27772 | UNIX_TIMESTAMP should return NULL when date fields are out of bounds | Major | . | Simhadri Govindappa | Simhadri Govindappa |
| HIVE-27682 | AlterTableAlterPartitionOperation cannot change the type if the column has default partition | Minor | HiveServer2 | Zhihua Deng | Zhihua Deng |
| HIVE-27777 | CBO fails on multi insert overwrites with common group expression | Major | HiveServer2 | Steve Carlin | Steve Carlin |
| HIVE-27651 | Upgrade hbase version | Major | . | Ayush Saxena | Butao Zhang |
| HIVE-27324 | Hive query with NOT IN condition is giving incorrect results when the sub query table contains the null value. | Major | Hive | Shobika Selvaraj | Diksha |
| HIVE-27114 | Provide a configurable filter for removing useless properties in Partition objects from listPartitions HMS Calls | Major | . | Naresh P R | Zhihua Deng |
| HIVE-27113 | Increasing default for hive.thrift.client.max.message.size to 2 GB | Major | Hive | Riju Trivedi | Riju Trivedi |
| HIVE-27846 | Tests under hive-unit module are not running | Major | Testing Infrastructure | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27849 | Replication tests using ivy fail after the upgrade to 2.5.2 | Major | . | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27866 | JDBC: HttpRequestInterceptorBase should not add an empty "Cookie:" header to the request if no custom cookies have been specified | Major | JDBC | Gergely Farkas | Gergely Farkas |
| HIVE-27862 | Map propertyContent to a wrong column in package.jdo | Major | . | Zhihua Deng | Zhihua Deng |
| HIVE-27679 | Ranger Yarn Queue policies are not applying correctly, rework done for HIVE-26352 | Major | . | Mahesh Raju Somalaraju | Mahesh Raju Somalaraju |
| HIVE-27865 | HMS in http mode shuts down silently with no errors | Major | . | Zhihua Deng | Zhihua Deng |
| HIVE-27885 | Cast decimal from string with space without digits before dot returns NULL | Major | . | Naresh P R | Naresh P R |
| HIVE-27867 | Incremental materialized view throws NPE whew Iceberg source table is empty | Major | . | Krisztian Kasa | Krisztian Kasa |
| HIVE-27662 | Incorrect parsing of nested complex types containing map during vectorized text processing | Major | Vectorization | Raghav Aggarwal | Raghav Aggarwal |
| HIVE-27713 | Iceberg: metadata location overrides can cause data breach | Major | Authorization, Iceberg integration | Janos Kovacs | Ayush Saxena |
| HIVE-27797 | Transactions that got timed out are not getting logged as 'ABORTED' in NOTIFICATION_LOG | Major | repl, Transactions | Taraka Rama Rao Lethavadla | Taraka Rama Rao Lethavadla |
| HIVE-27093 | Fix NPE in initialize() of Partition class | Critical | . | Wechar | Wechar |
| HIVE-27240 | NPE on Hive Hook Proto Log Writer | Critical | Hive | Shubham Sharma | Shubham Sharma |
| HIVE-27555 | Upgrade issues with Kudu table on backend db | Critical | . | Zhihua Deng | Zhihua Deng |
| HIVE-24730 | Shims classes override values from hive-site.xml and tez-site.xml silently | Major | . | László Bodor | László Bodor |
TESTS:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| HIVE-27431 | Clean invalid properties in test module | Minor | Test | Butao Zhang | Butao Zhang |
| HIVE-27747 | Generalize TestSchemaToolForMetastore to run on every supported DBMS | Major | Tests | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-27745 | Add unit test to ensure short version is inline with full version | Major | . | KIRTI RUGE | KIRTI RUGE |
SUB-TASKS:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| HIVE-27031 | Iceberg: Implement Copy-On-Write for Delete queries | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27523 | Implement array_union UDF in Hive | Major | . | Taraka Rama Rao Lethavadla | Taraka Rama Rao Lethavadla |
| HIVE-27277 | Set up github actions workflow to build and push docker image to docker hub | Major | . | Simhadri Govindappa | Simhadri Govindappa |
| HIVE-27630 | Iceberg: Fast forward branch | Major | . | Denys Kuzmenko | Ayush Saxena |
| HIVE-27654 | Iceberg: Cherry-Pick commit to a branch | Major | . | Ayush Saxena | Ayush Saxena |
| HIVE-27670 | Failed to build the image locally on Apple silicon | Major | . | Zhihua Deng | Zhihua Deng |
| HIVE-27322 | Iceberg: metadata location overrides can cause data breach - custom location to AuthZ | Blocker | Iceberg integration | Janos Kovacs | Ayush Saxena |
| HIVE-27716 | Precommit: Save log files for first 10 failures | Major | . | László Bodor | László Bodor |
| HIVE-27711 | Allow creating a branch from tag name | Major | . | Ayush Saxena | Butao Zhang |
| HIVE-27702 | Remove PowerMock from beeline and upgrade mockito to 4.11 | Major | HiveServer2 | Zsolt Miskolczi | Mayank Kunwar |
| HIVE-27736 | Remove PowerMock from itests-jmh and upgrade mockito | Major | . | Ayush Saxena | Zsolt Miskolczi |
| HIVE-27705 | Remove PowerMock from service (hive-service) and upgrade mockito to 4.11 | Major | HiveServer2 | Zsolt Miskolczi | KIRTI RUGE |
| HIVE-27701 | Remove PowerMock from llap-client and upgrade mockito to 4.11 | Major | HiveServer2 | Zsolt Miskolczi | Zsolt Miskolczi |
| HIVE-27704 | Remove PowerMock from jdbc-handler and upgrade mockito to 4.11 | Major | HiveServer2 | Zsolt Miskolczi | KIRTI RUGE |
| HIVE-26455 | Remove PowerMockito from hive-exec | Minor | Hive | Zsolt Miskolczi | Zsolt Miskolczi |
| HIVE-27399 | Add lateral view support for CBO | Major | HiveServer2 | Steve Carlin | Steve Carlin |
| HIVE-27597 | Implement JDBC Connector for HiveServer | Major | Hive | Naveen Gangam | Naveen Gangam |
| HIVE-27783 | Iceberg: Implement Copy-On-Write for Update queries | Major | . | Denys Kuzmenko | Denys Kuzmenko |
| HIVE-27006 | ParallelEdgeFixer inserts misconfigured operator and does not connect it in Tez DAG | Major | . | Seonggon Namgung | Seonggon Namgung |
| HIVE-26621 | TPC-DS query 2 fails with java.lang.RuntimeException: cannot find field _col0 from [] | Major | . | Sungwoo Park | Seonggon Namgung |
| HIVE-27794 | Iceberg: Implement Copy-On-Write for Merge queries | Major | . | Denys Kuzmenko | Denys Kuzmenko |
| HIVE-27269 | VectorizedMapJoin returns wrong result for TPC-DS query 97 | Critical | . | Seonggon Namgung | Seonggon Namgung |
| HIVE-27714 | Iceberg: metadata location overrides can cause data breach - handling default locations | Critical | Authorization, Iceberg integration | Janos Kovacs | Ayush Saxena |
| HIVE-27926 | Iceberg: Allow restricting Iceberg data file reads to table location | Blocker | Iceberg integration | Janos Kovacs | Ayush Saxena |
OTHER:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| HIVE-27589 | Iceberg: Branches of Merge/Update statements should be committed atomically | Major | . | Denys Kuzmenko | Simhadri Govindappa |
| HIVE-27638 | Preparing for 4.0.0-beta-2 development | Major | . | Stamatis Zampetakis | Stamatis Zampetakis |
| HIVE-22618 | Fix checkstyle violations for ParseUtils | Minor | Query Processor | okumin | okumin |
| HIVE-27558 | HBase table query does not push BETWEEN predicate to storage layer | Major | . | Denys Kuzmenko | Dayakar M |
| HIVE-27526 | Cleaner should honor compaction writeIdHwm | Major | . | Denys Kuzmenko | Denys Kuzmenko |
| HIVE-27687 | Logger variable should be static final as its creation takes more time in query compilation | Major | Hive | Ramesh Kumar Thangarajan | Ramesh Kumar Thangarajan |
| HIVE-27843 | Add QueryOperation to Hive proto logger for post execution hook information | Major | . | Ramesh Kumar Thangarajan | Ramesh Kumar Thangarajan |
| HIVE-27907 | Upgrade aws-java-sdk version in HIVE | Minor | . | Devaspati Krishnatri | Devaspati Krishnatri |