Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Installing Flink

  1. Go to the /usr/local directory on the physical machine and download the Flink software package.
    1
2
    cd /usr/local
wget  --no-check-certificate https://archive.apache.org/dist/flink/flink-1.16.3/flink-1.16.3-bin-scala_2.12.tgz

    If the Flink installation package name is not flink-1.16.3-bin-scala_2.12.tgz, replace it with the actual one.

  2. Decompress the flink-1.16.3-bin-scala_2.12.tgz package in the /usr/local path and create a soft link for later version replacement.
    1
2
3
    tar -zxvf flink-1.16.3-bin-scala_2.12.tgz
chown -R root:root flink-1.16.3
ln -s flink-1.16.3 flink

    If the Flink software directory name is not flink-1.16.3, replace it with the actual one.

  3. Configure the masters and workers files of Flink.
    1. Open the /usr/local/flink/conf/masters file.
      1
      vi /usr/local/flink/conf/masters
    2. Press i to enter the insert mode and change the content of masters to the flink_jm_8c32g container ID 8081. For example:
      1
      4a376b30106b:8081
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Open the /usr/local/flink/conf/workers file.
      1
      vi /usr/local/flink/conf/workers
    5. Press i to enter the insert mode, and change the content of workers to the IDs of the flink_tm1_8c32g and flink_tm2_8c32g containers. For example, configure four TaskManagers for each of two Docker containers.
      1
2
3
4
5
6
7
8
      c3ddf10d0353
c3ddf10d0353
c3ddf10d0353
c3ddf10d0353
c3bbdbcc1ae1
c3bbdbcc1ae1
c3bbdbcc1ae1
c3bbdbcc1ae1
    6. Press Esc, type :wq!, and press Enter to save the file and exit.
    7. Open the /usr/local/flink/conf/workers file.
      1
      vi /usr/local/flink/conf/flink-conf.yaml
    8. Press i to enter the insert mode, replace the configuration with the following, and change the value of jobmanager.rpc.address to the ID of the flink_jm_8c32g container ID. It is recommended that the total number of slots be greater than the degree of parallelism. For example:
      taskmanager.memory.process.size: 8G
jobmanager.rpc.address: 4a376b30106b
jobmanager.rpc.port: 6123
jobmanager.memory.process.size: 8G
taskmanager.memory.process.size: 8G
taskmanager.numberOfTaskSlots: 2
parallelism.default: 16
rest.port: 8081
rest.bind-port: 8081
io.tmp.dirs: /tmp
#pipeline.operator-chaining: true
#execution:runtime-mode: STREAMING
#jobmanager.execution.failover-strategy: region
#table.optimizer.agg-phase-strategy: ONE_PHASE
#pipeline.object-reuse: true
#==============================================================================
# JVM
#==============================================================================
# JVM options for GC
#env.java.opts: -verbose:gc -XX:NewRatio=3 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:ParallelGCThreads=4
    9. Press Esc, type :wq!, and press Enter to save the file and exit.