Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

c. 上传你的key到公共服务器



$ gpg2 --keyserver pgpkeys.mitopenpgp.eduorg --send-keys <key id>


这个 <key id> 就是 ~/.gnupg/openpgp-revocs.d/下的一个.rev文件的文件名,打开可以看到以下信息

...

接下来在下边的链接通过邮箱查询上传成功没,大概需要一分钟才能查到

httphttps://keys.gnupgopenpgp.netorg/


查询时候把 advance 下边的 show full-key hashes 勾上

...

Code Block
-----BEGIN PGP PUBLIC KEY BLOCK-----

mQINBF2LLTABEACRM0lzt9BMSsIg9cCfcEBRH6Fc2etS7vrOt8NPq2VTuHYHUMzA
...
-----END PGP PUBLIC KEY BLOCK-----


接下来手动上传上述内容,上传链接  http接下来手动上传上述内容,上传链接 https://pgp.mit.edu/ https://keys.gnupgopenpgp.netorg/


到现在应该能查到了。。。


c.2 寻找你认识的Apache社区的Committer来给你的公钥认证 寻找你认识的Apache社区的Committer来给你的公钥认证 (这一步不是必须的,可以先往后走)

参考链接 https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84 主要都是对方有事情要做,所以就不详细解释了。

可以找黄向东,刘大伟,乔佳林帮你签名可以找黄向东,刘大伟,乔嘉林帮你签名


d. 将你的gpg 公钥加入KEYS文件中

这个步骤需要使用SVN。MacOS已经默认安装了SVN。

...

接下来会输用户名和密码,就用 apache的用户名和密码。


d.2 在release分支中添加公钥到KEYS,用于发布正式版本在release分支中添加公钥到KEYS,用于发布正式版本(只有PMC有release分支的写权限)

注意,以下命令又会建个 iotdb-dist-release的文件夹,最好 cd .. 到 iotdb-dist-dev 的上级目录

...

$ (gpg2 --list-sigs YOUR_NAME@apache.org && gpg2 --export --armor YOUR_NAME@apache.org) >> KEYS

$ svn $ svn ci -m "add gpg key for YOUR_NAME"(如果不是PMC,可以将KEYS文件发给PMC,让PMC帮忙执行这一步)

这里注意检查一下你追加到 KEYS 里的东西千万不要有中文,出现下图这种文字的时候:

...

Code Block
languagexml
title~/.m2/settings.xml文件
<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <servers>
    <!-- Apache Repo Settings -->
    <server>
        <id>apache.snapshots.https</id>
        <username>{user-id}</username>
        <password>{user-pass}</password>
    </server>
    <server>
        <id>apache.releases.https</id>
        <username>{user-id}</username>
        <password>{user-pass}</password>
    </server>
  </servers>
</settings>
  <profiles>
    <profile>
      <id>apache-release</id>
      <properties>
        <gpg.keyname>你的KEYID</gpg.keyname><!-- Your GPG Keyname here -->
        <!-- Use an agent: Prevents being asked for the password during the build -->
        <gpg.useagent>true</gpg.useagent>
        <gpg.passphrase>你的私钥的密码</gpg.passphrase>
      </properties>
    </profile>
  </profiles> 
</settings>


f. 上传GPG公钥到Github账户s

!!重要!! 许多教程中都没提及这一步,而如果不做这一步的话,你在做一些步骤的时候,maven会告诉你没有权限。。

...

a. 编译并生成第三方license集合:

mvn clean package  license:add-third-party license:addaggregate-third-party-report -Dlicense.useMissingFile -DskipTests

b. 检查所有catalogX的license:

mac/linux下运行:

find . -name THIRD-PARTY.txt| xargs cat | grep -E 'BCL|Intel|JSR-275|Microsoft Limited Public License|ASL|Java SDK for Satori RTM| Redis Source Available License|RSAL|Booz Allen Public License|Creative Commons|Sun Community|GPL|Affero|LGPL|QPL|Sleepycat|Server Side|SSPL|Code Project|CPOL|BSD-4|Facebook BSD+|Solipsistic Eclipse Public|Be A Dick|JSON License' | grep -v CDDL |grep -v Apache | grep -v MIT | grep -v "Creative Commons CC0"


2.4 手动修改client-py的版本(新增内容)

手动修改 client-py/pypi/setup.py 文件中的版本号为当前发布版。


2.5 手动修改 jdbc feature.xml 的版本

手动修改 jdbc/src/main/feature/feature.xml 文件中的版本号为当前发布版。


2.6 在仓库中新增当前发布版本的 Dockerfile

Image Added


然后开始正式的发版。

3.发布版本

3.1 自动在github上打tag,并生成各种签名文件

...

mvn release:prepare -DautoVersionSubmodules=true -P site -P compile-cpp -P code-coverage -P get-jar-with-dependencies

执行时,会让你输入三个内容:

a. 希望发布的版本号,默认值为pom中的版本号去掉“-SNAPSHOT”,一般可以不修改


注:0.13版本中的编译命令:

mvn release:prepare -DautoVersionSubmodules=true -P site -P compile-grafana-plugin -P compile-cpp -P code-coverage -P get-jar-with-dependencies

注2,若自身电脑没有docker,建议将testcontainer模块中的版本号进行手动修改。



如果出现sign失败,可以尝试以下命令
export GPG_TTY=$(tty)


执行时,会让你输入三个内容:

a. 希望发布的版本号,默认值为pom中的版本号去掉“-SNAPSHOT”,一般可以不修改

b. b. 希望在github打的tag名字是什么,注意我们目前使用的命名风格是: v0.12.0

...

最后会提示 success,并且除了第一个parent 的pom,其他的都 skippted 了。


如果成功了,根据PLC4X的文档,还应该去apache的仓库看一看是否有最新的记录。地址是 如果成功了,根据PLC4X的文档,还应该去apache的仓库看一看是否有最新的记录。地址是 https://gitbox.apache.org/repos/asf?p=incubator-iotdb.git;a=shortlog;h=refs/heads/rel/0.8 注意分支名。12 注意分支名。

应该能看到刚才mvn release:prepare自动做的git操作,commit日志是:[maven-release-plugin] prepare release release/0.11.1 和 [maven-release-plugin] prepare for next development iteration

...

然而这个rollback并不会删除tag,还需要手动删除。

删本地 tag:$git tag -d release/0v0.1112.x

删远端 tag:git push origin :refs/tags/release/0v0.1112.x

我的话,没有用rollback这个指令,而是直接覆盖重新再来:

...

mvn release:perform  -P site -P compile-cpp -P code-coverage -P get-jar-with-dependencies

就开始真正的发布并上传到apache的服务器了。

。。。

。。。

漫长的等待

0.13版本开始,命令改为:

mvn release:perform  -P site -P compile-grafana-plugin -P compile-cpp -P code-coverage -P get-jar-with-dependencies


就开始真正的发布并上传到apache的服务器了。

。。。

。。。

漫长的等待

。。。。。。

。。。

我使用教育网花费了46分钟。。

(注:建议在北京时间的早晨执行该步,网速比晚上快很多)

...

然后从iotdb源码的根目录下的target/checkout/将README.md,README_ZH.mdRELEASE_NOTES.md拷贝进来。

再从iotdb源码再从iotdb源码/targettarget/checkout/target/apache-iotdb-0.11.1-source-release.zip* (zip, zip.asc, zip.sha512三个文件)拷贝进来;

再从Iotdb源码/target/checkout/distribution/target/apache-iotdb-0.11.1-bin.zip* (zip, zip.asc, zip.sha512三个文件)拷贝进来;

(注:0.12版本的二进制文件更多12版本的二进制文件更多,源码包要注意checkout的路径,“target/checkout/target”)


然后用svn命令提交:

$ svn add 0.11.1

...

$ svn commit -m "perparing RC1 of 0.11.1 of IoTDB release"


在孵化期阶段,release那个svn,最多保留5个版本的下载!如果过多了,请记得删除之前的版本!release那个svn,最多保留5个版本的下载!如果过多了,请记得删除之前的版本!



3.4 标记github上的release为pre-release

...

Apache IoTDB 0.8.1 has been staged under [2] and it’s time to vote
on accepting it for release.  All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.

Release tag: release/v0.11.1
Hash for the release tag: 37b213b6ab6fc8ca13ab60f21dff099042a0e295

...

其中tag的hash值可以通过如下命令拿到:

$ git show release/0v0.11.1 --name-only

4 投票失败了怎么办

...

(在这个文档中,一定不要把下面的分支名字换成一个真实存在的名字,防止有人直接复制粘贴)

删本地 tag:$git tag -d v0.12.x

删远端 tag:git push origin :refs/tags/v0.12.x

删除远端 tag:git push origin :refs/tags/release/0.11.x

...

Panel

主题:[RESULT][VOTE] Release Apache IoTDB 0.9.0


Hi all,

The vote to release Apache IoTDB (incubating) 0.8.2 has passed with 3 +1 binding votes and 1 +1 non-binding vote.

binding votes:

xxx

xxx

nondonon-binding vote:
xxx

Vote thread:
https://lists.apache.org/thread.html/9653646c533df93e286a512b2b5c544b8d7bc9a565183806feeec642%40%3Cgeneral.incubator.apache.org%3E


感谢的话

你的名字

...

$svn commit -m "upload 0.11.1 release files"(只有PMC有写权限,可以把文件夹打个包发给PMC,让PMC帮忙执行这一步)


如果你的keys也修改了,可能还要在更新下keys文件。

...

就能在本地测试网页了。(也可以在第13节的下载链接修改完成后但是还没部署前做测试)


13. 更新官网的下载链接

修改master分支docs/Download/README.md 和 zh/Download/README.md,注意表格中只保留每个大版本的最新版本。

...

Code Block
- [In progress version](https://iotdb.apache.org/UserGuide/Master/Get%20Started/QuickStart.html) 
- [Version 0.8.x](/zh/UserGuide/V0.8.x/0-Get%20Started/1-QuickStart.html) - [Version 0.10.x](https://iotdb.apache.org/UserGuide/V0.10.x/Get%20Started/QuickStart.html)
- [Version 0.9.x](/zh/UserGuide/V0.9.x/0-Get%20Started/1-QuickStart.html) - [Version 0.9.x](https://iotdb.apache.org/UserGuide/V0.9.x/0-Get%20Started/1-QuickStart.html)
- [Version 0.8.x](https://iotdb.apache.org/UserGuide/V0.8.x/0-Get%20Started/1-QuickStart.html)



之后需要手动部署网页,等待部署成功、官网更新后,再昭告天下:往master分支提个pr,pr合进master后网页会自动部署,等待部署成功、官网更新后,再昭告天下:

14. 昭告天下!

当你确认多数镜像都同步好了之后,可以昭告天下发布新版本的信息了:

发邮件的邮箱:务必是apache.org的邮箱,否则会被拒收。邮件必须是纯文本,否则,也会被拒收。

发送邮件到:announce@apache.org

...

消息主题:[ANNOUNCE] Apache IoTDB (incubating) 0.1012.0 6 released

消息内容示例:


Panel

The   Apache   IoTDB   team is pleased to   announce   the release of   Apache  IoTDB (incubating) 0.10.0IoTDB 0.12.6.



Apache   IoTDB   (incubating) (Database for Internet of Things) is an integrated data management engine designed for timeseries data. It provides users with services for data collection, storage and analysis. 

The current release is the 3rd major version of IoTDB, which provides many new features and improvements. A part of features are as follows:

* IOTDB-217 A new GROUPBY syntax, e.g., select avg(s1) from root.sg.d1.s1 GROUP BY ([1, 50), 5ms)
* IOTDB-220 Add hot-load configuration function
* IOTDB-292 Add load external tsfile feature
* IOTDB-298 Support new Last point query. e.g, select last * from root
* IOTDB-305 Add value filter function while executing align by device
* IOTDB-396 Support new query clause: disable align, e.g., select * from root disable align
* IOTDB-447 Support querying non-existing measurement and constant measurement
* IOTDB-448 Add IN operation, e.g., where time in (1,2,3)
* IOTDB-456 Support GroupByFill Query, e.g., select last_value(s1) from root.sg.d1 GROUP BY ([1, 10), 2ms) FILL(int32[previousUntilLast])
* IOTDB-497 Support Apache Flink Connector with IoTDB
* IOTDB-565 MQTT Protocol Support, disabled by default, open in iotdb-engine.properties
* IOTDB-588 Add tags and attributes management
* Online upgrade from 0.9.x
* Make JDBC OSGi usable and added a feature file
* Allow count timeseries group by level=x using default path
* IOTDB-700 Add OpenID Connect based JWT Access as alternative to Username / Password
* IOTDB-715 Support previous time range in previousuntillast
* IOTDB-719 add avg_series_point_number_threshold in config
* IOTDB-734 Add Support for NaN in Double / Floats in SQL Syntax.

The full release note is available at: 

https://raw.githubusercontent.com/apache/incubator-iotdb/release/0.10.0/RELEASE_NOTES.md

The release is available for download at: 

http://iotdb.apache.org/Download

Maven artifacts for JDBC driver, session SDK, TsFile SDK, Spark-connector, Hadoop-connector, Hive-connector and Flink-connector can be found at:

https://search.maven.org/search?q=g:org.apache.iotdb

Docker image of IoTDB server can be found at:

https://hub.docker.com/r/apache/iotdb

Python API package can be found at:

https://pypi.org/project/apache-iotdb/

Regards,
The Apache IoTDB team

=====

Disclaimer 

Apache IoTDB (incubating) is an effort undergoing incubation at The
Apache Software Foundation (ASF), sponsored by the Apache Incubator.
Incubation is required of all newly accepted projects until a further
review indicates that the infrastructure, communications, and decision
making process have stabilized in a manner consistent with other
successful ASF projects. While incubation status is not necessarily a
reflection of the completeness or stability of the code, it does
indicate that the project has yet to be fully endorsed by the ASF.

更多参考例子:https://lists.apache.org/list.html?dev@iotdb.apache.org:gte=1d:announce

发布Python包到Pypi

首先,你要有一个账号,在这里注册:https://pypi.org/account/register/

其次,最好在注册一个测试仓库的账号,在这里https://test.pypi.org/account/register/

注意:在正式发布之前要先在测试仓库确保没有问题。因为正式发布之后PyPI does not allow for a filename to be reused, even once a project has been deleted and recreated.

即正式发布之后再发现有问题,即便删除了那个包,已经用过的名字(比如apache-iotdb 0.11.1)也不可再用于重新上传了。

自己电脑上需要安装python3、pip3;安装后,还需要安装setuptools和wheel和twine

python3 -m pip install --user --upgrade setuptools wheel

...

IoT native database with high performance for data management and analysis, deployable on the edge and the cloud.

This is a bug-fixed version of 0.12.5, which includes a number of improvements:
##Improvements
*[IOTDB-3515] Make the message clear when loading empty TsFile

##Bug Fixes
*[IOTDB-2773] Fix overlapped data should be consumed first bug
*[IOTDB-3135] Fix parameter max_select_unseq_file_num_in_each_unseq_compaction doesn't work
*[IOTDB-3292] Fix chunk size overflow in level compaction
*[IOTDB-3392] Fix File doesn't exist when moving TsFile when virtual_storage_group > 1
*[IOTDB-3398] Remove the exception when query meet a device is not in TsFile, which is a normal case
*[IOTDB-3602] Skip empty TsFile when loading directory
*[ISSUE-4399] When non-root user get an empty query result, return no permission error message
*[ISSUE-5809] Fix bug: sum aggregation result return 0 when no data
* Fix data in seq files overlapped


The full release note is available at:
https://raw.githubusercontent.com/apache/iotdb/v0.12.6/RELEASE_NOTES.md

The release is available for download at:
http://iotdb.apache.org/Download

Maven artifacts for JDBC driver, session SDK, TsFile SDK, Spark-connector, Hadoop-connector, Hive-connector and Flink-connector can be found at:
https://search.maven.org/search?q=g:org.apache.iotdb


Docker image of IoTDB server can be found at:
https://hub.docker.com/r/apache/iotdb

Python API package can be found at:
https://pypi.org/project/apache-iotdb/


Regards,
The Apache IoTDB team

更多参考例子:https://lists.apache.org/list.html?dev@iotdb.apache.org:gte=1d:announce


发布Python包到Pypi

首先,你要有一个账号,在这里注册:https://pypi.org/account/register/

其次,最好在注册一个测试仓库的账号,在这里https://test.pypi.org/account/register/


注意:在正式发布之前要先在测试仓库确保没有问题。因为正式发布之后PyPI does not allow for a filename to be reused, even once a project has been deleted and recreated.

即正式发布之后再发现有问题,即便删除了那个包,已经用过的名字(比如apache-iotdb 0.11.1)也不可再用于重新上传了。


自己电脑上需要安装python3、pip3;安装后,还需要安装setuptools和wheel和twine

python3 -m pip install --user --upgrade setuptools wheel
python3 -m pip install --user --upgrade twine

我执行上面这个指令时候,报错: creating build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c build/temp.linux-x86_64-3.6/_openssl.c -o build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6/_openssl.o -Wconversion -Wno-error=sign-conversion
build/temp.linux-x86_64-3.6/_openssl.c:22:24: fatal error: pyconfig.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
解决办法:sudo apt install python3.6-dev (我的python是3.6版本)


然后,在整个项目编译后,进入client-py/文件夹。


注意一定要在进行下一步前编译项目,否则发布出来的版本会缺少thrift 相关的依赖。。。。(血的教训)


(注意:iotdb/client-py/src/pypi/setup.py和iotdb/client-py/src/pypi/README.md中的版本号、链接等,应该在上述第2步“准备发布版本”就已完成更新。)


如果是第一次发布,可以在测试仓库进行实验:

python3 setup.py sdist bdist_wheel

python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*


注意:上传测试版本时,python3 setup.py sdist bdist_wheel使用的setup.py里的name要修改,可以在后面后缀上自己的用户名(像这样:name="example-pkg-YOUR-USERNAME-HERE"),否则无法会报错:The credential associated with user 'xxx' isn't allowed to upload to project 'apache-iotdb'。正式上传之前,记得把setup.py里的name改回来,删掉egg-info,dist,build这三个文件夹,之后重新生成。


完成后的输出示例:

Panel

MacMini-FIT:generated-sources-python hxd$ python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*
Uploading distributions to https://test.pypi.org/legacy/
Enter your username:

 
Enter your password:
Uploading apache_iotdb-0.9.0rc5-py3-none-any.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33.5k/33.5k [00:04<00:00, 7.14kB/s]
Uploading apache-iotdb-0.9.0rc5.tar.gz
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26.0k/26.0k [00:01<00:00, 16.0kB/s]

View at:
https://test.pypi.org/project/apache-iotdb/0.9.0rc5/

网页的效果:https://test.pypi.org/project/apache-iotdb/

然后本地可以下载下来试用试用:

Panel

(官方教程建议使用virtualevn,但是我在virtualevn中pip install的包不生效 很奇怪):

python3 -m pip install --index-url https://test.pypi.org/simple/ apache-iotdb==0.9.0-rc5 (假设版本是0.9.0-rc5)

注意,如果上述命令失败,那说明这个test pypi下没有thrift包,此时可以使用--no-deps参数跳过thrift安装:

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps apache-iotdb==0.9.0-rc5 (假设版本是0.9.0-rc5)

不过并没有什么用。。因为没有thrift就没法运行。。

所以准确流程是:

python3 -m pip install thrift==0.13.0

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps apache-iotdb==0.9.0-rc5


然后跑一下client-py下的client-example.py看是否正确。


如果不是在virtualevn环境下,记得用完后删除这个临时包:python3 -m pip uninstall apache-iotdb



一切OK,就可以正式发布了:(注:如果之前为了测试修改过setup.py,这里记得把setup.py里的name改回来重新生成dist文件夹)

控制台运行:python3 setup.py sdist bdist_wheel

python3 -m twine upload  dist/*

注:如果没有上传权限找项目管理员。


这部分内容参考了

https://packaging.python.org/tutorials/packaging-projects/


发布Docker 镜像包到DockerHub

首先,你需要有dockerhub网站下的用户名。

其次,需要去JIRA的INFRA项目中申请(在模仿

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyINFRA-22636
,创建一个ISSUE),给自己的用户名增加dockerhub中apache/iotdb项目的权限。如果不是PMC,还需要PMC去那个ISSUE下面留个言,approve一下。

注意,每个项目的 dockerhub 只允许 5 个名额,如果需要新增用户进 dockerhub 的 iotdb/committer,则需要在 Jira 中说明被替换走的用户是谁。

以上准备工作就做好了。

然后编写Dockerfile脚本,0.9.1的示例如下(该脚本需要根据每个版本的实际部署方式不同进行修改):

No Format
FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  #&& wget https://www-us.apache.org/dist/incubator/iotdb/0.9.1-incubating/apache-iotdb-0.9.1-incubating-bin.zip \
  && wget http://mirrors.tuna.tsinghua.edu.cn/apache/incubator/iotdb/0.9.1-incubating/apache-iotdb-0.9.1-incubating-bin.zip \
  && unzip apache-iotdb-0.9.1-incubating-bin.zip \
  && rm apache-iotdb-0.9.1-incubating-bin.zip \
  && mv apache-iotdb-0.9.1-incubating /iotdb \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
EXPOSE 6667
EXPOSE 31999
EXPOSE 5555
EXPOSE 8181
VOLUME /iotdb/data
VOLUME /iotdb/logs
ENV PATH="/iotdb/sbin/:/iotdb/tools/:${PATH}"
ENTRYPOINT ["/iotdb/sbin/start-server.sh"]


Dockerfile-0.12.4-node的示例

No Format
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  && wget https://downloads.apache.org/iotdb/0.12.4/apache-iotdb-0.12.4-server-bin.zip \
  # if you are in China, use the following URL
  #&& wget https://mirrors.tuna.tsinghua.edu.cn/apache/iotdb/0.12.4/apache-iotdb-0.12.4-server-bin.zip \
  && unzip apache-iotdb-0.12.4-server-bin.zip \
  && rm apache-iotdb-0.12.4-server-bin.zip \
  && mv apache-iotdb-0.12.4-server-bin /iotdb \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
# rpc port
EXPOSE 6667
# JMX port
EXPOSE 31999
# sync port
EXPOSE 5555
# monitor port
EXPOSE 8181
VOLUME /iotdb/data
VOLUME /iotdb/logs
ENV PATH="/iotdb/sbin/:/iotdb/tools/:${PATH}"
ENTRYPOINT ["/iotdb/sbin/start-server.sh"]


Dockerfile-0.12.4-cluster的示例

No Format
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  && wget https://downloads.apache.org/iotdb/0.12.4/apache-iotdb-0.12.4-cluster-bin.zip \
  # if you are in China, use the following URL
  #&& wget https://mirrors.tuna.tsinghua.edu.cn/apache/iotdb/0.12.4/apache-iotdb-0.12.4-cluster-bin.zip \
  && unzip apache-iotdb-0.12.4-cluster-bin.zip \
  && rm apache-iotdb-0.12.4-cluster-bin.zip \
  && mv apache-iotdb-0.12.4-cluster-bin /iotdb \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y \
  # modify the seeds in configuration file
  && sed -i '/^seed_nodes/cseed_nodes=127.0.0.1:9003' /iotdb/conf/iotdb-cluster.properties \
  && sed -i '/^default_replica_num/cdefault_replica_num=1' /iotdb/conf/iotdb-cluster.properties

# rpc port
EXPOSE 6667
# JMX port
EXPOSE 31999
# sync port
EXPOSE 5555
# monitor port
EXPOSE 8181
# internal meta port
EXPOSE 9003
# internal data port
EXPOSE 40010
VOLUME /iotdb/data
VOLUME /iotdb/logs
ENV PATH="/iotdb/sbin/:/iotdb/tools/:${PATH}"
ENTRYPOINT ["/iotdb/sbin/start-node.sh"]

Dockerfile-0.12.4-grafana的示例

No Format
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  && wget https://downloads.apache.org/iotdb/0.12.4/apache-iotdb-0.12.4-grafana-bin.zip \
  # if you are in China, use the following URL
  #&& wget https://mirrors.tuna.tsinghua.edu.cn/apache/iotdb/0.12.4/apache-iotdb-0.12.4-grafana-bin.zip \
  && unzip apache-iotdb-0.12.4-grafana-bin.zip \
  && rm apache-iotdb-0.12.4-grafana-bin.zip \
  && mv apache-iotdb-0.12.4-grafana-bin /iotdb-grafana \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
# rpc port
EXPOSE 8888
VOLUME /iotdb-grafana/config
RUN echo "#!/bin/bash" > /iotdb-grafana/runboot.sh
RUN echo "java -Djava.security.egd=file:/dev/./urandom -jar /iotdb-grafana/iotdb-grafana.war" >> /iotdb-grafana/runboot.sh
RUN chmod a+x /iotdb-grafana/runboot.sh
WORKDIR /iotdb-grafana
ENTRYPOINT ["./runboot.sh"]

Dockerfile-0.13.0-grafana-connector的示例

No Format
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  && wget https://downloads.apache.org/iotdb/0.13.0/apache-iotdb-0.13.0-grafana-connector-bin.zip \
  # if you are in China, use the following URL
  #&& wget https://mirrors.tuna.tsinghua.edu.cn/apache/iotdb/0.13.0/apache-iotdb-0.13.0-grafana-connector-bin.zip \
  && unzip apache-iotdb-0.13.0-grafana-connector-bin.zip \
  && rm apache-iotdb-0.13.0-grafana-connector-bin.zip \
  && mv apache-iotdb-0.13.0-grafana-connector-bin /iotdb-grafana-connector \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
# rpc port
EXPOSE 8888
VOLUME /iotdb-grafana-connector/config
RUN echo "#!/bin/bash" > /iotdb-grafana-connector/runboot.sh
RUN echo "java -Djava.security.egd=file:/dev/./urandom -jar /iotdb-grafana-connector/iotdb-grafana-connector.war" >> /iotdb-grafana-connector/runboot.sh
RUN chmod a+x /iotdb-grafana-connector/runboot.sh
WORKDIR /iotdb-grafana-connector
ENTRYPOINT ["./runboot.sh"]


注意该脚本可以存储到docker/src/main下。


然后本地构造image (注意运行的时候 要确保Dockerfile文件夹内没有别的文件):

docker build -t apache/iotdb:0.9.1 .

0.12.X版本可以用下面的命令

docker build -t apache/iotdb:0.12.4-node . -f Dockerfile-0.12.4-node

docker build -t apache/iotdb:0.12.4-cluster . -f Dockerfile-0.12.4-cluster

docker build -t apache/iotdb:0.12.4-grafana . -f Dockerfile-0.12.4-grafana

0.13.X版本可以用下面的命令

docker build -t apache/iotdb:0.13.0-node . -f Dockerfile-0.13.0-node

docker build -t apache/iotdb:0.13.0-cluster . -f Dockerfile-0.13.0-cluster

docker build -t apache/iotdb:0.13.0-grafana-connector . -f Dockerfile-0.13.0-grafana-connector


然后本地尝试运行:

docker run -d -p 6667:6667 -p 31999:31999 -p 8181:8181 apache/iotdb:0.12.4-node


如果需要杀死该镜像,使用docker ps 查看容器id,然后使用docker kill 容器id即可。


进行测试,启动iotdb-client进行连接和增删改查操作:

docker exec -it a4434aff6123(需要替换成上一步自己运行的的CONTAINER ID) /bin/bash


都没问题的话,可以上传镜像了:

首先需要登录docker:

docker login --username=yourhubusername 

使用docker images再确认一遍容器,以及容器的tag:

Image Added

以上图为例,其中apache/iotdb 0.9.0和0.9.1拥有相同的Image ID的原因是,我在构建0.9.1的时候,错误地将版本号指定成了0.9.0。

因此构建完成镜像后一定要使用客户端连接iotdb进行测试。


如果构建镜像的时候,名字不是apache/iotdb这种类型,或者版本号不正确,还有机会修改:

docker tag 6a6365cd99d0  apache/iotdb:0.12.4-node

上传吧:

docker push apache/iotdb:0.12.4-node

如果是最新版本(比如还没有0.10,那么0.9.x就是最新版本。但是假设0.10.0已经发布,那么再上传0.9.5就不算最新版),还要更新下latest标签:

docker tag  apache/iotdb:0.12.4-node  apache/iotdb:latest

docker push apache/iotdb:latest


(注意目前有三个镜像要打:node,cluster,grafana)。

完成。


docker push apache/iotdb:0.12.4-node

docker push apache/iotdb:0.12.4-cluster 

docker push apache/iotdb:0.12.4-grafana 


0.13.x之后

docker push apache/iotdb:0.13.0-node

docker push apache/iotdb:0.13.0-cluster 

docker push apache/iotdb:0.13.0-grafana-connector


发布 跨平台的docker镜像

以0.13.1为例,在 Docker 19.03+ 版本中可以使用 docker buildx build 命令使用 BuildKit 构建镜像。该命令支持 --platform 参数可以同时构建支持多种系统架构的 Docker 镜像。(例如linux/amd64,linux/arm64)

dockerfile如下:

只修改了第一行,将jdk 替换成了eclipse-temurin:11-jre-focal,因为这个JDK支持amd64、arm/v7、arm64/v8三种版本,并且是ubuntu-focal,可以正常使用apt-get

No Format
FROM --platform=$TARGETPLATFORM eclipse-temurin:11-jre-focal

然后,在整个项目编译后,进入client-py/target/pypi文件夹。

(注意:iotdb/client-py/src/pypi/setup.py和iotdb/client-py/src/pypi/README.md中的版本号、链接等,应该在上述第2步“准备发布版本”就已完成更新。)

如果是第一次发布,可以在测试仓库进行实验:

python3 setup.py sdist bdist_wheel

python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*

注意:上传测试版本时,python3 setup.py sdist bdist_wheel使用的setup.py里的name要修改,可以在后面后缀上自己的用户名(像这样:name="example-pkg-YOUR-USERNAME-HERE"),否则无法会报错:The credential associated with user 'xxx' isn't allowed to upload to project 'apache-iotdb'。正式上传之前,记得把setup.py里的name改回来重新生成。

完成后的输出示例:

Panel

MacMini-FIT:generated-sources-python hxd$ python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*
Uploading distributions to https://test.pypi.org/legacy/
Enter your username:

 
Enter your password:
Uploading apache_iotdb-0.9.0rc5-py3-none-any.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33.5k/33.5k [00:04<00:00, 7.14kB/s]
Uploading apache-iotdb-0.9.0rc5.tar.gz
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26.0k/26.0k [00:01<00:00, 16.0kB/s]

View at:
https://test.pypi.org/project/apache-iotdb/0.9.0rc5/

网页的效果:https://test.pypi.org/project/apache-iotdb/

然后本地可以下载下来试用试用:

...

(官方教程建议使用virtualevn,但是我在virtualevn中pip install的包不生效 很奇怪):

...

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps apache-iotdb==0.9.0-rc5 (假设版本是0.9.0-rc5)

不过并没有什么用。。因为没有thrift就没法运行。。

所以准确流程是:

python3 -m pip install thrift==0.13.0

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps apache-iotdb==0.9.0-rc5

然后跑一下client-py下的client-example.py看是否正确。

如果不是在virtualevn环境下,记得用完后删除这个临时包:python3 -m pip uninstall apache-iotdb

一切OK,就可以正式发布了:(注:如果之前为了测试修改过setup.py,这里记得把setup.py里的name改回来重新生成dist文件夹)

控制台运行:python3 setup.py sdist bdist_wheel

python3 -m twine upload  dist/*

注:如果没有上传权限找项目管理员。

这部分内容参考了

https://packaging.python.org/tutorials/packaging-projects/

发布Docker 镜像包到DockerHub

首先,你需要有dockerhub网站下的用户名。

其次,需要去JIRA的INFRA项目中申请,给自己的用户名增加dockerhub中apache/iotdb项目的权限。

以上准备工作就做好了。

然后编写Dockerfile脚本,0.9.1的示例如下(该脚本需要根据每个版本的实际部署方式不同进行修改):

No Format
FROM openjdk:11-jre-slim
RUN apt update \
  # procps is for `free` command
  && apt install wget unzip lsof procps -y \
  #&& wget https://www-usdownloads.apache.org/dist/incubator/iotdb/0.913.1-incubating/apache-iotdb-0.913.1-incubatingserver-bin.zip \
  # if you are in China, use the following URL
  #&& wget httphttps://mirrors.tuna.tsinghua.edu.cn/apache/incubator/iotdb/0.9.1-incubating//iotdb/0.13.1/apache-iotdb-0.13.1-server-bin.zip \
  && unzip apache-iotdb-0.13.1-server-bin.zip \
  && rm apache-iotdb-0.913.1-incubatingserver-bin.zip \
  && unzipmv apache-iotdb-0.913.1-incubating-bin.zip \
  && rm apache-iotdb-0.9.1-incubating-bin.zip \
  && mv apache-iotdb-0.9.1-incubating /iotdb \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
EXPOSE 6667
EXPOSE 31999
EXPOSE 5555
EXPOSE 8181
VOLUME /iotdb/data
VOLUME /iotdb/logs
ENV PATH="/iotdb/sbin/:/iotdb/tools/:${PATH}"
ENTRYPOINT ["/iotdb/sbin/start-server.sh"]

注意该脚本可以存储到docker/src/main下。

然后本地构造image (注意运行的时候 要确保Dockerfile文件夹内没有别的文件):

...

server-bin /iotdb \
  && apt remove wget unzip -y \
  && apt autoremove -y \
  && apt purge --auto-remove -y \
  && apt clean -y
# rpc port
EXPOSE 6667
# JMX port
EXPOSE 31999
# sync port
EXPOSE 5555
# monitor port
EXPOSE 8181
VOLUME /iotdb/data
VOLUME /iotdb/logs
ENV PATH="/iotdb/sbin/:/iotdb/tools/:${PATH}"
ENTRYPOINT ["/iotdb/sbin/start-server.sh"]


安装buildx:

# 确定可以使用buildx,否则要升级docker

docker buildx version

# 指定buildx使用docker-container

docker buildx create --name mybuilder --driver docker-container

docker buildx use mybuilder

# 开启用于多平台镜像构建的镜像

docker run --rm --privileged tonistiigi/binfmt:latest --install all 


构建并上传

## apache/iotdb:latest

docker buildx build --platform linux/amd64,linux/arm64/v8,linux/arm/v7 -t apache/iotdb:latest -f Dockerfile-0.13.1-node . --push

## apache/iotdb:0.13.1-node

docker buildx build --platform linux/amd64,linux/arm64/v8,linux/arm/v7 -t apache/iotdb:0.

...

13.1

...

-node -f Dockerfile-0.13.1-node . --push


发布 Go client (如果有需要发布)


本地clone go client 代码

git clone https://github.com/apache/iotdb-client-go

checkout 到相应的 rel/xx 分支,检查是否有遗漏的commit,如果有,可以从main分支cherry-pick过来。

打tag 并push到 github 仓库

git tag v0.xx.x

git push origin v0.xx.x


进入下面的链接,点 draft a new release,选刚才打的那个tag 之后 release 就行。

https://github.com/apache/iotdb-client-go/releases

然后本地尝试运行:

docker run -d -p 6667:6667 -p 31999:31999 -p 8181:8181 apache/iotdb:0.9.1

如果需要杀死该镜像,使用docker ps 查看容器id,然后使用docker kill 容器id即可。

进行测试,启动iotdb-client进行连接和增删改查操作:

docker exec -it a4434aff6123 /bin/bash

都没问题的话,可以上传镜像了:

首先需要登录docker:

docker login --username=yourhubusername 

使用docker images再确认一遍容器,以及容器的tag:

Image Removed

以上图为例,其中apache/iotdb 0.9.0和0.9.1拥有相同的Image ID的原因是,我在构建0.9.1的时候,错误地将版本号指定成了0.9.0。

因此构建完成镜像后一定要使用客户端连接iotdb进行测试。

如果构建镜像的时候,名字不是apache/iotdb这种类型,或者版本号不正确,还有机会修改:

docker tag 6a6365cd99d0  apache/iotdb:0.9.1

上传吧:

docker push apache/iotdb:0.9.1

如果是最新版本(比如还没有0.10,那么0.9.x就是最新版本。但是假设0.10.0已经发布,那么再上传0.9.5就不算最新版),还要更新下latest标签:

docker tag  apache/iotdb:0.9.1  apache/iotdb:latest

docker push apache/iotdb:latest

(注意目前有三个镜像要打:node,cluster,grafana)。

完成。




That is all.