THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
March 21, 2021, 2021, Sunday.
Time: 2021-03-21 9:00 PM Beijing Time
WeMeet url: https://meeting.tencent.com/s/xoV6xnkBw5oT
Host: Yuyuan Kang
Status Updates
1. Announce
Apache IoTDB 0.11.13 to release
2. Bug Fixes
- ISSUE-2505 ignore PathNotExistException in recover and change recover error to warn
- IOTDB-1119 Fix C++ SessionDataSet bug when reading value buffer
- Fix SessionPool does not recycle session and can not offer new Session due to RunTimeException
- ISSUE-2588 Fix dead lock between deleting data and querying in parallel
- ISSUE-2546 Fix first chunkmetadata should be consumed first
- IOTDB-1126 Fix unseq tsfile is deleted due to compaction
- IOTDB-1137 MNode.getLeafCount error when existing sub-device
- ISSUE-2624 ISSUE-2625 Avoid OOM if user don't close Statement and Session manually
- ISSUE-2639 Fix possible NPE during end query process
- Alter IT for An error is reported and the system is suspended occasionally
- IOTDB-1149 print error for -e param when set maxPRC<=0
- IOTDB-2648 Last query not right when having multiple devices
- Delete mods files after compaction
- ISSUE-2687 fix insert NaN bug
- ISSUE-2598 Throw explicit exception when time series is unknown in where clause
- Fix timeseriesMetadata cache is not cleared after the TsFile is deleted by a compaction
- ISSUE-2611 An unsequence file that covers too many sequence file causes OOM query
- IOTDB-1135 Fix count timeseries bug when the paths are nested
- ISSUE-2709 IOTDB-1178 Fix cache is not cleared after compaction
- ISSUE-2746 Fix data overlapped bug after the elimination unseq compaction process
- Fix getObject method in JDBC should return an Object
- IOTDB-1188 Fix IoTDB 0.11 unable to delete data bug
- Fix when covering a tsfile resource with HistoricalVersion = null, it鈥檒l throw a NPE
- fix the elimination unseq compaction may loss data bug after a delete operation is executed
3. New Features
- Add explain sql support
4. Improvements
- IOTDB-1140 optimize regular data encoding
- Add more log for better tracing
- Add backgroup exec for cli -e function
- Add max direct memory size parameter to env.sh
5. Closed development
1) Meta data:
- Implement device template (a group of measurements)
- Eliminate redundant definition of the leaf node
2) Data in Memory:
- New data structure for multiple columns sharing a same timestamp
- Eliminate redundant timestamp
- Reduce memory consumption
3) Data on disk:
- Designed new TsFile format
- Support data of multiple columns sharing a same timestamp
- Reduce disk consumption
4) Result:
One of the scenarios:
...
Write efficiency (row/sec)
...
Single-state, old strategy
...
190,624
...
Single-state, new strategy
...
487,656
...
Distributed, 3 nodes 1 replica, new strategy
...
1,095,331
6. Open Floor
new ideas, feedback, suggestions.
1) [Discussion] Monthly Contributor Award
February‘s “Contributors of The Month”
- sunjincheng121(Jincheng Sun), with 132571 lines changed and 8 mails sent;
- HTHou(Haonan Hou), with 2116 lines changed and 4 mails sent;
- neuyilan(Houliang Qi), with 2692 lines changed and 3 mails sent.
There are also 22 other contributors who contributed to IoTDB this month.
7 New contributors submitted their first PR in IoTDB
- wuzhaojie(Zhaojie Wu),
- WilliamSong11(Yuxiang Song),
- GLBB,
- THUMarkLau(Xuxin Liu),
- chenjun40,
- 543202718(Haoyu Wang)
- jxlgzwh(Wenhao Zhong).
2) [Proposal] Hackthon (April)
Mar 7st, 2021, Sunday.
Time:
Host:
...
https://github.com/apache/iotdb/blob/0f48ba2de83d2b1c0da00bba8485baa1c60abe28/docs/zh/UserGuide/Operation%20Manual/Triggers.md
...
[I24] so cl ndes a
...
...
...
Future work：
new ideas, feedback, suggestions.
Goal:
Attention:
Preliminary measure:
...
The final equation would be S=0.8Z(code)+0.1Z(doc)+0.1Z(mail)
- Q & A docs
- start up docs
- Sign up : 45 people
https://github.com/apache/iotdb/issues/2715
<= 5
T-shirt, backpack, bottle with IoTDB logo
agenda:
April: get familiar with each other and communicate with mentor
May: coding
Time:
Host
Feb 21st, 2021, Sunday.
Host: Yuan Tian
Online Meeting (Tencent Meeting):
...
meeting room ID:
password:
Time:
Host:
...
nohup ./start-cli.sh -eNotice:
new ideas, feedback, suggestions.
Goal:
Attention:
Preliminary measure:
...
The final equation would be S=0.9Z(code)+0.1Z(mail)
两个方向上的称号：
代码贡献+代码review贡献
- Q & A docs
- start up docs
细分设计
模块
https://github.com/apache/iotdb/issues/2715
Time:
Host
Meeting bi-weekly Sunday at 9:00 pm Beijing Time.
Language:
- Meeting Minutes: English
- Speaking: Chinese (if all the attendees are Chinese), English (if there is at least one non-chinese speaker )
Online Meeting (Tencent Meeting or VooV Meeting):
meeting room ID：469 7527 6512
password：2020
January 31st, 2021, Sunday.
Time: 2021-01-31 9:00 PM Beijing Time
Host: Zesong Sun
https://shimo.im/docs/dt8C9cQtKHGgkH89
Status Updates
Announce
Apache IoTDB 0.11.2 released (01-23)
Notice: Prepare to release 0.11.3, and keep maintaining 0.11.x version for a while until v0.12.x is stable.
Currently, the main branch is serving for 0.12 and the version changes the TsFile format once again.
Though we will provide the upgrade tool, it is still a big decision for users to upgrade the version (as they need to upgrade existing data files).
Besides, even 0.12.0 will be released, we may need 1~2 versions (or more) to guarantee the version stable.
Vote
change release tags from release/{full-version} to v{full-version}
eg. release/0.11.2 to v0.11.2
Currently, we use release/{major}.{minor}.{patch} as our tag stale, which may bring some troubles for some languages like GoLang.
New Features
...
Currently out concurrency unit is storage group which can't make full use of cpu cores when user only set 1 or 2 storage group. We need virtual storage group to improve concurrent performance. This PR finish first part of the design
#2488 Add DirectByteBufferPool for WAL
There are two problems in the process of current wal flushing.
Because wal uses FileChannel while flushing, it involves one more memory copy. FileChannel will copy the contents of HeapByteBuffer to DirectByteBuffer before calling native write method, and then call native write with DirectByteBuffer as a parameter, which will cause An extra memory copy operation.
Although 0.12 uses the thread pool to complete asynchronous flushing, to a certain extent, solves the problem of the surge of non-heap memory caused by the wal of 0.11, but when the non-heap memory is released is still depends on the jvm (depends on when gc is performed), jvm cannot know whether non-heap memory should be released in time, so it may also cause non-heap memory not be released in time, but we are clear about that.
So, I want to use DirectByteBufferPool to solve the above two questions.
#2184 [IOTDB-1081] New TsFile Format
The TsFile format version will update to V3.
It will be applied in the next major version (i.e., 0.12).
We find that the former tsfile format waste some disk space, some information was stored duplicately and some legacy fields are no longer useful.
#2603 Accelearate CI by split a CI into several parts
split the main-ci into 3 parts: server, cluster, and others
add two parameters to just test the server or the cluster, e.g.,
-DskipTests -Diotdb.test.only=true
-DskipTests -Dcluster.test.only=true
Split the main-ci into two github action groups to save resource when re-run some actions.
disable enforcer by default, to enable it, use -P enforce
disable generating jar-with-dependencies, to enable it, use -P get-jar-with-dependencies
Open Floor
new ideas, feedback, suggestions.
1. [Discussion] Monthly Contributor Award
Goal: Keep community more vibrant and attract new contributors
Attention: Not really an award, just a title that we can recognize with an announcement on the official wechat account
Preliminary measure:
Amount of PR and reviews on GitHub (codes amount)
Activity on mailing lists
Contributions(publicity, tutorial, etc.) on social media (e.g. WeChat official account, CSDN, Stack Overflow)
Being a conference host or event organizer will also count.
2. [Discussion] Community Mentors
Call for mentors and plans
Q & A docs
start up docs
Wechat group
Sign up : 45 people
To be as commiter : 17
Users needing professional help : 5
Both : 23
3. [Experiment] Compares to influxDB in terms of performance
Need to update to the website:
https://iotdb.apache.org/UserGuide/Master/Comparison/TSDB-Comparison.html
4. [Proposal] Hackthon (April)
1 committer + 3 non-committer
January 3rd, 2021, Sunday.
Time: 9:00 PM Beijing Time
Host：Xinyu Tan
Attendees：
Agenda: (put what you want to discuss in the meeting here before the meeting starts)
...
...
...
...
- Houliang Qi
- Xinyu Tan
...
- code
- PRs(xinyu)
- a clear impression of someone
- doc or others
- pingcap : https://pingcap.com/blog-cn/tidb-community-upgrade/
...
- avoid pushing to the repository directly
- provide relatively detailed commit message.
- Do not force-update a commit if someone has reviewed your code
- use squash merge(whether need to disable directly-merge?)(xinyu Tan)
- list the reproduction procedure when reporting bugs
- detailed pr/issue title(is Chinese needed?)
...
Minutes:
- discuss1 (proposer): content
- discuss2 (): ...
December 20th, 2020, Sunday.
Time: 9:00 PM Beijing Time
Host：Lingzhe Zhang
Attendees：
Agenda: (put what you want to discuss in the meeting here before the meeting starts.)
- liit
- contib
- New committer introduction - qiaojialin
- Houliang Qi
- Xinyu Tan
- cont
- cont
- code
- PRs(xinyu)
- a clear impression of someone
- doc or others
- code
- ( presenter)
Minutes:
- discuss1 (proposer): content
- discuss2 (): ...
December 6th, 2020, Sunday.
Time: 9:00 PM Beijing Time
Join Tencent Meeting or VooV Meeting:
https://meeting.tencent.com/s/U17cHJmg5T2n
Meeting ID: 335854726
Host：Xiangwei Wei
Attendees：
Minutes:
...
...
All records before Dec 6th, 2020, can be accessed here : https://shimo.im/docs/rqxJrjHwx6r8tJwd (in Chinese)