第1章 第1节 准备工作

1.1 开发工具

（1）代码集成环境：IntelliJ IDEA 2023.2 (Community Edition)

下载链接：https://download-cdn.jetbrains.com.cn/idea/ideaIC-2023.3.4.exe

（2）编译工具：apache-maven-3.9.6

下载链接：https://dlcdn.apache.org/maven/maven-3/3.9.6/binaries/apache-maven-3.9.6-bin.tar.gz

（3）代码管理平台：github

访问地址：https://github.com/

（4）访问github客户端：Github Desktop

下载链接：https://desktop.githubusercontent.com/github-desktop/releases/3.3.8-48bb7059/GitHubDesktopSetup-x64.exe

（5）ssh 工具：mobaxterm

下载链接：https://ma.jsyidian.top/MobaXterm_Portable_v23.6.zip

（6）虚拟机客户端：Vmware

下载地址：https://download3.vmware.com/software/WKST-1750-WIN/VMware-workstation-full-17.5.0-22583795.exe

（7）Linux镜像：centos8

下载地址：https://mirrors.aliyun.com/centos/8/isos/x86_64/CentOS-8.5.2111-x86_64-dvd1.iso

（8）Java JDK:  openjdk 8

下载地址：https://builds.openlogic.com/downloadJDK/openlogic-openjdk/8u402-b06/openlogic-openjdk-8u402-b06-linux-x64.tar.gz

（9）数据库链接客户端：dbeaver

下载链接：https://download.dbeaver.com/community/23.3.5/dbeaver-ce-23.3.5-x86_64-setup.exe

（10）文本编辑工具：notepad++

下载地址：http://xz.kksoft.net/cxb/kktnn31/Notepad%2B%2B%20%E5%AE%89%E8%A3%85%E7%A8%8B%E5%BA%8F.exe

1.2 大数据组件安装包

（1）协调服务：zookeeper

下载地址：https://dlcdn.apache.org/zookeeper/zookeeper-3.9.1/apache-zookeeper-3.9.1-bin.tar.gz

（2）消息队列：Kafka

下载地址：https://downloads.apache.org/kafka/3.6.1/kafka_2.12-3.6.1.tgz

（3）数据同步：seatunnel

下载地址：https://dlcdn.apache.org/seatunnel/2.3.3/apache-seatunnel-2.3.3-bin.tar.gz

（4）seatunnel依赖包：

https://repo.maven.apache.org/maven2/org/apache/seatunnel/seatunnel-hadoop3-3.1.4-uber/2.3.3/seatunnel-hadoop3-3.1.4-uber-2.3.3-optional.jar

（5）任务调度：dolphinscheduler

下载地址：https://dlcdn.apache.org/dolphinscheduler/3.2.0/apache-dolphinscheduler-3.2.0-bin.tar.gz

（6）数仓软件：doris

https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-2.0.4-bin-x64.tar.gz

（7）计算引擎：flink

下载链接：https://dlcdn.apache.org/flink/flink-1.18.1/flink-1.18.1-bin-scala_2.12.tgz

Flink Bundled Jar

https://repo1.maven.org/maven2/com/ververica/flink-sql-connector-mysql-cdc/2.4.2/flink-sql-connector-mysql-cdc-2.4.2.jar

https://repo1.maven.org/maven2/org/apache/flink/flink-sql-connector-kafka/3.1.0-1.18/flink-sql-connector-kafka-3.1.0-1.18.jar

https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-hive-3.1.3_2.12/1.19.0/flink-sql-connector-hive-3.1.3_2.12-1.19.0.jar

https://repo.maven.apache.org/maven2/org/apache/iceberg/iceberg-flink-runtime-1.18/1.5.2/iceberg-flink-runtime-1.18-1.5.2.jar

https://repo.maven.apache.org/maven2/org/apache/hudi/hudi-flink1.18-bundle/0.15.0/hudi-flink1.18-bundle-0.15.0.jar

（8）实时数仓：paimon

Paimon Bundled Jar

下载地址：https://repository.apache.org/content/groups/snapshots/org/apache/paimon/paimon-flink-1.18/0.8-SNAPSHOT/paimon-flink-1.18-0.8-20240301.002155-30.jar

Hadoop Bundled Jar

https://repo1.maven.org/maven2/org/apache/flink/flink-shaded-hadoop-2-uber/2.7.5-9.0/flink-shaded-hadoop-2-uber-2.7.5-9.0.jar

（9）FlinkSQL开发平台：dinky

下载地址：https://github.com/DataLinkDC/dinky/releases/download/v1.0.0-rc4/dinky-release-1.18-1.0.0-rc4.tar.gz

（10）可视化工具：superset

（11）Anaconda

下载地址：

https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2023.07-1-Linux-x86_64.sh

（12）hadoop

https://archive.apache.org/dist/hadoop/common/hadoop-3.1.3/hadoop-3.1.3.tar.gz

（13）hive

https://mirrors.aliyun.com/apache/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz?spm=a2c6h.25603864.0.0.19d3158fHTtPFv

hive Bundled Jar

https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-hive-connector-3.1/0.7.0-incubating/paimon-hive-connector-3.1-0.7.0-incubating.jar

https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-hive-runtime/1.5.2/iceberg-hive-runtime-1.5.2.jar

https://repo.maven.apache.org/maven2/org/apache/hudi/hudi-hadoop-mr-bundle/0.15.0/hudi-hadoop-mr-bundle-0.15.0.jar

mysql-connector-java

https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar