How to Release Version
This tutorial describes in detail how to release Apache StreamPark, take the release of version 2.1.0 as an example.
1. Environmental requirements
This release process is operated in the Ubuntu OS, and the following tools are required:
- JDK 1.8+
- Apache Maven 3.x (this process uses 3.8.7)
- GnuPG 2.x
- Git
- SVN (apache uses svn to host project releases)
Pay attention to setting environment variables
export GPG_TTY=$(tty)
2. Preparing for release
First summarize the account information to better understand the operation process, will be used many times later.
- apache id:
muchunjin (APACHE LDAP UserName)
- apache passphrase:
APACHE LDAP Passphrase
- apache email:
muchunjin@apache.org
- gpg real name:
muchunjin (Any name can be used, here I set it to the same name as the apache id)
- gpg key passphrase:
The password set when creating the gpg key, you need to remember this password
2.1 Key generation
$ gpg --full-gen-key
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
(14) Existing key from card
Your selection? 1 # Please enter 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (3072) 4096 # Please enter 4096 here
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0 # Please enter 0
Key does not expire at all
Is this correct? (y/N) y # Please enter y here
GnuPG needs to construct a user ID to identify your key.
Real name: muchunjin # Please enter 'gpg real name'
Email address: muchunjin@apache.org # Please enter your apache email address here
Comment: for apache StreamPark release create at 20230501 # Please enter some comments here
You selected this USER-ID:
"muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O # Please enter O here
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
# At this time, a dialog box will pop up, asking you to enter the key for this gpg.
# you need to remember that it will be used in subsequent steps.
┌─────────────────────────────────────────────────────┐
│ Please enter this passphrase to │
│ protect your new key │
│ │
│ Passphrase: _______________________________________ │
│ │
│ <OK> <Cancel> │
└─────────────────────────────────────────────────────┘
# Here you need to re-enter the password in the previous step.
┌─────────────────────────────────────────────────────┐
│ Please re-enter this passphrase │
│ │
│ Passphrase: _______________________________________ │
│ │
│ <OK> <Cancel> │
└─────────────────────────────────────────────────────┘
gpg: key ACFB69E705016886 marked as ultimately trusted
gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/DC12398CCC33A5349EB9663DF9D970AB18C9EDF6.rev'
public and secret key created and signed.
pub rsa4096 2023-05-01 [SC]
85778A4CE4DD04B7E07813ABACFB69E705016886
uid muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>
sub rsa4096 2023-05-01 [E]
Keys can be viewed through the gpg --list-signatures
command
2.2 Upload the generated key to the public server
$ gpg --keyid-format SHORT --list-keys
/root/.gnupg/pubring.kbx
------------------------
pub rsa4096/05016886 2023-05-01 [SC]
85778A4CE4DD04B7E07813ABACFB69E705016886
uid [ultimate] muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>
sub rsa4096/0C5A4E1C 2023-05-01 [E]
# Send public key to keyserver via key id
$ gpg --keyserver keyserver.ubuntu.com --send-key 05016886
# Among them, keyserver.ubuntu.com is the selected keyserver, it is recommended to use this, because the Apache Nexus verification uses this keyserver
Configure gnupg to use standard DNS resolution:
# resolves common "gpg: keyserver receive failed: Network is unreachable" and
# "gpg: keyserver receive failed: No keyserver available" errors
echo "standard-resolver" > ~/.gnupg/dirmngr.conf
sudo pkill dirmngr
2.3 Check if the key is created successfully
Verify whether it is synchronized to the public network, it will take about a minute to find out the answer, if not successful, you can upload and retry multiple times.
$ gpg --keyserver keyserver.ubuntu.com --recv-keys 05016886 # If the following content appears, it means success
gpg: key ACFB69E705016886: "muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>" not changed
gpg: Total number processed: 1
gpg: unchanged: 1
Or enter https://keyserver.ubuntu.com/ address in the browser, enter the name of the key and click 'Search key'
If the query results are as follows, it means that the key is successfully created.
2.4 Add the gpg public key to the KEYS file of the Apache SVN project warehouse
- Apache StreamPark Branch Dev https://dist.apache.org/repos/dist/dev/incubator/streampark
- Apache StreamPark Branch Release https://dist.apache.org/repos/dist/release/incubator/streampark/
2.4.1 Add public key to KEYS in dev branch
$ mkdir -p streampark_svn/dev
$ cd streampark_svn/dev
$ svn co https://dist.apache.org/repos/dist/dev/incubator/streampark
$ cd streampark_svn/dev/streampark
# Append the KEY you generated to the file KEYS, and check if it is added correctly
$ (gpg --list-sigs muchunjin@apache.org && gpg --export --armor muchunjin@apache.org) >> KEYS
$ svn ci -m "add gpg key for muchunjin"
2.4.2 Add public key to KEYS in release branch
$ mkdir -p streampark_svn/release
$ cd streampark_svn/release
$ svn co https://dist.apache.org/repos/dist/release/incubator/streampark/
$ cd streampark_svn/release/streampark
# Append the KEY you generated to the file KEYS, and check if it is added correctly
$ (gpg --list-sigs muchunjin@apache.org && gpg --export --armor muchunjin@apache.org) >> KEYS
$ svn ci -m "add gpg key for muchunjin"
3. Prepare material package
3.1 Based on dev or dev-2.1. x branch depending on the situation, create a release-${release_version}-rcx
branch, such as release-2.1.0-rc1, And create a tag named v2.1.0-rc1 based on the release-2.1.0-rc1 branch, and set this tag as pre-release.
3.2 clone release branch to local
git clone -b release-2.1.0-rc1 -c core.autocrlf=false git@github.com:apache/incubator-streampark.git
3.3 Compile the binary package
Scala 2.11 compilation and packaging
> ./build.sh # choose "mixed mode" and "scala 2.11"
Scala 2.12 compilation and packaging
> ./build.sh # choose "mixed mode" and "scala 2.12"
Package the project source code
git archive \
--format=tar.gz \
--output="dist/apache-streampark-2.1.0-incubating-src.tar.gz" \
--prefix=apache-streampark-2.1.0-incubating-src/ \
release-2.1.0-rc1
The following 3 files will be generated
apache-streampark-2.1.0-incubating-src.tar.gz
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz
3.4 Sign binary and source packages
cd dist
# sign
for i in *.tar.gz; do echo $i; gpg --armor --output $i.asc --detach-sig $i ; done
# SHA512
for i in *.tar.gz; do echo $i; sha512sum $i > $i.sha512 ; done
The final file list is as follows
apache-streampark-2.1.0-incubating-src.tar.gz
apache-streampark-2.1.0-incubating-src.tar.gz.asc
apache-streampark-2.1.0-incubating-src.tar.gz.sha512
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz.asc
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz.sha512
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz.asc
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz.sha512
3.5 Verify signature
$ cd dist
# Verify signature
$ for i in *.tar.gz; do echo $i; gpg --verify $i.asc $i ; done
apache-streampark-2.1.0-incubating-src.tar.gz
gpg: Signature made Tue May 2 12:16:35 2023 CST
gpg: using RSA key 85778A4CE4DD04B7E07813ABACFB69E705016886
gpg: Good signature from "muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>" [ultimate]
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz
gpg: Signature made Tue May 2 12:16:36 2023 CST
gpg: using RSA key 85778A4CE4DD04B7E07813ABACFB69E705016886
gpg: Good signature from "muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>" [ultimate]
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz
gpg: Signature made Tue May 2 12:16:37 2023 CST
gpg: using RSA key 85778A4CE4DD04B7E07813ABACFB69E705016886
gpg: BAD signature from "muchunjin (for apache StreamPark release create at 20230501) <muchunjin@apache.org>" [ultimate]
# Verify SHA512
$ for i in *.tar.gz; do echo $i; sha512sum --check $i.sha512; done
apache-streampark-2.1.0-incubating-src.tar.gz
apache-streampark-2.1.0-incubating-src.tar.gz: OK
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz
apache-streampark_2.11-2.1.0-incubating-bin.tar.gz: OK
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz
apache-streampark_2.12-2.1.0-incubating-bin.tar.gz: OK
3.6 Publish the dev directory of the Apache SVN warehouse of the material package
# Check out the dev directory of the Apache SVN warehouse to the streampark_svn_dev directory under dist in the root directory of the Apache StreamPark project
# svn co https://dist.apache.org/repos/dist/dev/incubator/streampark dist/streampark_svn_dev
svn co --depth empty https://dist.apache.org/repos/dist/dev/incubator/streampark dist/streampark_svn_dev
Create a version number directory and name it in the form of ${release_version}-${RC_version}
. RC_version starts from 1, that is, the candidate version starts from RC1. During the release process, there is a problem that causes the vote to fail. If it needs to be corrected, it needs to iterate the RC version , the RC version number needs to be +1. For example: Vote for version 2.1.0-RC1. If the vote passes without any problems, the RC1 version material will be released as the final version material. If there is a problem (when the streampark/incubator community votes, the voters will strictly check various release requirements and compliance issues) and need to be corrected, then re-initiate the vote after the correction, and the candidate version for the next vote is 2.1.0- RC2.
mkdir -p dist/streampark_svn_dev/2.1.0-RC1
cp -f dist/* dist/streampark_svn_dev/2.1.0-RC1
Commit to SVN
cd dist/streampark_svn_dev/
# 1.check svn status
svn status
# 2. add to svn
svn add 2.0.0-RC1
svn status
# 3. Submit to svn remote server
svn commit -m "release for StreamPark 2.1.0"
3.7 Check Apache SVN Commit Results
Visit the address https://dist.apache.org/repos/dist/dev/incubator/streampark/2.1.0-RC1/ in the browser
4. Release Apache Nexus
4.1 Configure apache maven address and user password settings
- Generate master password
$ mvn --encrypt-master-password <apache password>
{EM+4/TYVDXYHRbkwjjAS3mE1RhRJXJUSG8aIO5RSxuHU26rKCjuS2vG+/wMjz9te}
- Create the file
${user.home}/.m2/settings-security.xml
and configure the password created in the previous step
<settingsSecurity>
<master>{EM+4/TYVDXYHRbkwjjAS3mE1RhRJXJUSG8aIO5RSxuHU26rKCjuS2vG+/wMjz9te}</master>
</settingsSecurity>
- Generate the final encrypted password and add it to the ~/.m2/settings.xml file
$ mvn --encrypt-password <apache passphrase>
{/ZLaH78TWboH5IRqNv9pgU4uamuqm9fCIbw0gRWT01c=}
In the maven configuration file ~/.m2/settings.xml, add the following
server
item
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>/path/to/local/repo</localRepository>
<servers>
<server>
<id>apache.snapshots.https</id>
<!-- APACHE LDAP UserName -->
<username>muchunjin</username>
<!-- APACHE LDAP password (Fill in the password you just created with the command `mvn --encrypt-password <apache passphrase>`) -->
<password>{/ZLaH78TWboH5IRqNv9pgU4uamuqm9fCIbw0gRWT01c=}</password>
</server>
<server>
<id>apache.releases.https</id>
<!-- APACHE LDAP UserName -->
<username>muchunjin</username>
<!-- APACHE LDAP password (Fill in the password you just created with the command `mvn --encrypt-password <apache passphrase>`) -->
<password>{/ZLaH78TWboH5IRqNv9pgU4uamuqm9fCIbw0gRWT01c=}</password>
</server>
</servers>
<profiles>
<profile>
<id>apache-release</id>
<properties>
<gpg.keyname>05016886</gpg.keyname>
<!-- Use an agent: Prevents being asked for the password during the build -->
<gpg.useagent>true</gpg.useagent>
<gpg.passphrase>passphrase for your gpg key</gpg.passphrase>
</properties>
</profile>
</profiles>
4.2 Release scala 2.11 to the Apache Nexus repository
mvn clean install \
-Pscala-2.11 \
-DskipTests \
-Dcheckstyle.skip=true \
-Dmaven.javadoc.skip=true \
-pl 'streampark-common,streampark-flink' \
-pl '!streampark-console/streampark-console-service' \
-amd
mvn deploy \
-Pscala-2.11,apache-release \
-DskipTests \
-Dmaven.javadoc.skip=true \
-DretryFailedDeploymentCount=3
4.3 Release scala 2.12 to the Apache Nexus repository
mvn clean install \
-Pscala-2.12 \
-DskipTests \
-Dcheckstyle.skip=true \
-Dmaven.javadoc.skip=true \
-pl 'streampark-common,streampark-flink' \
-pl '!streampark-console/streampark-console-service' \
-amd
mvn deploy \
-Pscala-2.12,apache-release \
-DskipTests \
-Dmaven.javadoc.skip=true \
-DretryFailedDeploymentCount=3
mvn deploy \
-Papache-release \
-DskipTests \
-Dmaven.javadoc.skip=true \
-DretryFailedDeploymentCount=3
4.4 Check for successful publishing to the Apache Nexus repository
Visit https://repository.apache.org/ and log in, if there are scala 2.11, scala 2.12, it means success.
5. Enter the community voting stage
5.1 Send a Community Vote Email
Send a voting email in the community requires at least three +1
and no -1
.
Send to
: dev@streampark.apache.org
Title
: [VOTE] Release Apache StreamPark (Incubating) 2.1.0 rc1
Body
:
Hello StreamPark Community:
This is a call for vote to release Apache StreamPark(Incubating) version release-2.1.0-RC1.
Apache Streamark - Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
Release notes:
https://streampark.apache.org/download/release-note/2.1.0/
The release candidates:
https://dist.apache.org/repos/dist/dev/incubator/streampark/2.1.0-RC1/
Maven artifacts are available in a staging repository at:
https://repository.apache.org/content/repositories/orgapachestreampark-1012/
Git tag for the release:
https://github.com/apache/incubator-streampark/releases/tag/v2.1.0-rc1
The artifacts signed with PGP key [05016886], corresponding to [muchunjin@apache.org], that can be found in keys file:
https://downloads.apache.org/incubator/streampark/KEYS
The vote will be open for at least 72 hours or until the necessary number of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason
*Valid check is a requirement for a vote. *Checklist for reference:
[ ] Download StreamPark are valid.
[ ] Checksums and PGP signatures are valid.
[ ] Source code distributions have correct names matching the current
release.
[ ] LICENSE and NOTICE files are correct for each StreamPark repo.
[ ] All files have license headers if necessary.
[ ] No compiled archives bundled in source archive.
[ ] Can compile from source.
More detail checklist please refer:
https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist
Steps to validate the release,Please refer to:
https://streampark.apache.org/community/release/how-to-verify
How to Build:
1.cd incubator-streampark
2.sh ./build.sh
Thanks!
After 72 hours, the voting results will be counted, and the voting result email will be sent, as follows.
Send to
: dev@streampark.apache.org
Title
: [RESULT][VOTE] Release Apache StreamPark (Incubating) 2.1.0-rc1
Body
:
Dear StreamPark community,
Thanks for your review and vote for "Release Apache StreamPark (Incubating) 2.1.0-rc1"
I'm happy to announce the vote has passed:
4 binding +1, from:
- Zhuoyu Chen
- Huajie Wang
- Linying Yu
- Shaokang Lv
1 non-binding +1, from:
- Roc Marshal
no 0 or -1 votes.
Vote thread:
https://lists.apache.org/thread/t01b2lbtqzyt7j4dsbdp5qjc3gngjsdq
Thank you to everyone who helped us to verify and vote for this release. We will move to the ASF Incubator voting shortly.
Best,
ChunJin Mu
One item of the email content is Vote thread
, and the link is obtained as follows:
Visit this address https://lists.apache.org/list.html?dev@streampark.apache.org, and find the mail title and click to display the voting content
Right-click the title and click Copy Link Address to get the link
5.2 Send Incubator Community voting mail
Send a voting email in the incubator community requires at least three +1
and no -1
.
Send to
: general@incubator.apache.org
cc
: dev@streampark.apache.org、tison@apache.org、willem.jiang@gmail.com
Title
: [VOTE] Release Apache StreamPark(Incubating) 2.1.0-rc1
Body
:
Hello Incubator Community:
This is a call for a vote to release Apache StreamPark(Incubating) version 2.1.0-RC1.
The Apache StreamPark community has voted on and approved a proposal to release Apache StreamPark(Incubating) version 2.1.0-RC1.
We now kindly request the Incubator PMC members review and vote on this incubator release.
Apache StreamPark, Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
StreamPark community vote thread:
https://lists.apache.org/thread/t01b2lbtqzyt7j4dsbdp5qjc3gngjsdq
Vote result thread:
https://lists.apache.org/thread/t5z58mvrs1drgzfyc48c9lhmd8skswn7
The release candidate:
https://dist.apache.org/repos/dist/dev/incubator/streampark/2.1.0-RC1/
Git tag for the release:
https://github.com/apache/incubator-streampark/releases/tag/v2.1.0-rc1
Maven artifacts are available in a staging repository at:
https://repository.apache.org/content/repositories/orgapachestreampark-1012/
The artifacts signed with PGP key [05016886], corresponding to [muchunjin@apache.org], that can be found in keys file:
https://downloads.apache.org/incubator/streampark/KEYS
The vote will be open for at least 72 hours or until the necessary number of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason
More detailed checklist please refer:
• https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist
Steps to validate the release, Please refer to:
• https://www.apache.org/info/verification.html
• https://streampark.apache.org/community/release/how_to_verify_release
How to Build:
1) clone source code:
> git clone -b v2.1.0-rc1 git@github.com:apache/incubator-streampark.git
2) build project:
> cd incubator-streampark && sh ./build.sh
Thanks,
On behalf of Apache StreamPark(Incubating) community
Best,
ChunJin Mu
If there is no -1 after 72 hours, reply to the email as follows
Send to
: general@incubator.apache.org
cc
: dev@streampark.apache.org
Body
:
Thanks everyone for review and vote, 72H passed. I'll announce the vote result soon.
Best,
Chunjin Mu
Then the voting results will be counted, and the voting result email will be sent, as follows.
Send to
: general@incubator.apache.org
cc
: dev@streampark.apache.org、tison@apache.org、willem.jiang@gmail.com
Title
: [RESULT][VOTE] Release Apache StreamPark(Incubating) 2.1.0-rc1
Body
:
Hi all,
Thanks for your review and vote for "Release Apache StreamPark (Incubating) 2.1.0-rc1"
I'm happy to announce the vote has passed:
3 binding votes, no +0 or -1 votes.
Thanks for reviewing and voting.
+3 (binding) +1, from:
- tison
- Gang Li
- Willem Jiang
no 0 or -1 votes.
Vote thread:
https://lists.apache.org/thread/k3cvcbzxqs6qy62d1o6r9pqpykcgvvhm
Thanks everyone for your feedback and help with StreamPark apache release. The StreamPark team will take the steps to complete this release and will announce it soon.
Best,
ChunJin Mu
One item of the email content is Vote thread
, and the link is obtained as follows:
Visit this address https://lists.apache.org/list.html?general@incubator.apache.org, and find the mail title and click to display the voting content
Then right-click the title and click Copy Link Address to get the link.
Wait a day to see if the tutor has any other comments, if not, send the following announcement email
6. Complete the final publishing steps
6.1 Migrating source and binary packages
svn mv https://dist.apache.org/repos/dist/dev/incubator/streampark/2.1.0-RC1 https://dist.apache.org/repos/dist/release/incubator/streampark/2.1.0 -m "transfer packages for 2.1.0-RC1"
6.2 Publish releases in the Apache Staging repository
- Log in to http://repository.apache.org , log in with your Apache account
- Click Staging repositories on the left
- Select your most recently uploaded warehouse, the warehouse specified in the voting email
- Click the Release button above, this process will perform a series of checks
It usually takes 24 hours for the warehouse to synchronize to other data sources
6.3 Add the new version download address to the official website
Add the following to the src/pages/download/data.json file on the official website
{
"version": "2.1.0",
"date": "2023-05-11"
}
The final file content is as follows
[
{
"version": "2.0.0",
"date": "2023-02-21"
},
{
"version": "2.1.0",
"date": "2023-05-11"
}
]
Open the official website address https://streampark.apache.org/download/ to see if there is a new version of the download
It should be noted that the download link may take effect after an hour, so please pay attention to it.
6.4 Generate a release on github
Create a tag named v2.1.0 based on the release-2.1.0-rc1 branch, and set this tag to latest release.
Then enter Release Title and Describe
- Release Title:
v2.1.0-incubating
- Describe:
Release-2.1.0-incubating
release note: https://streampark.apache.org/download/release-note/2.1.0
Then click the Publish release
button.
The rename the release-2.1.0-rc1 branch to release-2.1.0.
6.5 Send new version announcement email
Send to
: general@incubator.apache.org
cc
: dev@streampark.apache.org
Title
: [ANNOUNCE] Release Apache StreamPark (Incubating) 2.1.0
Body
:
Hi all,
We are glad to announce the release of Apache StreamPark(incubating) 2.1.0.
Once again I would like to express my thanks to your help.
Apache StreamPark(https://streampark.apache.org/) - Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
Download Links: https://streampark.apache.org/download/
Release Notes: https://streampark.apache.org/download/release-note/2.1.0
StreamPark Resources:
- Issue: https://github.com/apache/incubator-streampark/issues
- Mailing list: dev@streampark.apache.org
- Apache StreamPark Team
Best,
ChunJin Mu
This version release is over.