[{"content":"Introduction This experiment implements a Fully Distributed Mode cluster deployment of Hadoop HA, HBase HA, and YARN HA using virtual machines.\nThe list of devices fully suitable for this experiment:\n2 computers with Oracle VM VirtualBox installed and a virtual machine with Ubuntu Server Linux installed on 1 of them. 1 Raspberry Pi 3 Model B. 1 router. 3 network cables to connect each of the 3 devices to the router. The equipments you choose do not need to be identical, you can refer to this article according to your own environment, for example:\nIf the performance of 1 computer is sufficient for enough VMs to work properly (each of the 2 computers here runs 3 VMs each), 2 and more are not required. The introduction of a Raspberry Pi into the cluster is non-essential. VirtualBox is not the only optional virtual machine software. A wireless router can also be used to build an network environment without network cables. \u0026hellip;\u0026hellip; How to determine the devices you really need:\nTake an inventory of the number of devices on hand, estimate their performance (to determine how many VMs they can make work). Design the cluster rationally to determine ZooKeeper, Hadoop, HBase and YARN deployed on which one of the \u0026ldquo;machines\u0026rdquo; (bare or virtual). Confirm the number of virtual machines required and assign virtual machines to the host according to the performance of each device. 💡 It is important to note that for more than 1 device, all devices need to be deployed in the same LAN.\nThe following is a diagram of the services deployed on each machine in this distributed clustering system.\ngraph TD subgraph fa:fa-desktop PC_1 H1[fa:fa-server H1NameNodezkfc] H2[fa:fa-server H2DataNodeHMasterResourceManager] H6[fa:fa-server H6DataNodeHRegionServerQuorumPeerMain] end\rgraph TD subgraph fa:fa-desktop PC_2 H4[fa:fa-server H4NameNodezkfc] H5[fa:fa-server H5DataNodeHMasterResourceManager] H7[fa:fa-server H7DataNodeHRegionServerQuorumPeerMain] end\rgraph TD subgraph fa:fa-desktop Raspberry_Pi H3[fa:fa-server H3DataNodeHRegionServerQuorumPeerMain] end\rPreparation Install apt-transport-https and ca-certificates to enable https sources.\n1 sudo apt install apt-transport-https ca-certificates Backup sources.list.\n1 sudo mv /etc/apt/sources.list /etc/apt/sources.list.bak Modify sources.list.\n1 sudo nano /etc/apt/sources.list Update the software source.\n1 2 sudo apt update sudo apt upgrade Install openjdk-8-jdk.\n1 sudo apt install openjdk-8-jdk Download Hadoop binary package, and do the same to HBase and ZooKeeper.\n1 wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz Unzip to the current directory, and do the same to HBase and ZooKeeper.\n1 tar xzf hadoop-3.3.1.tar.gz Edit .bashrc.\n1 sudo nano .bashrc Add relevant environment variables.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 #Hadoop Related Options export HADOOP_HOME=/home/hadoop/hadoop-3.3.1 export HADOOP_INSTALL=$HADOOP_HOME export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS=\u0026#34;-Djava.library.path=$HADOOP_HOME/lib/native -Djava.net.preferIPv4Stack=true\u0026#34; export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin #HBase Related Options export HBASE_HOME=/home/hadoop/hbase-2.4.6 export PATH=$PATH:$HBASE_HOME/sbin:$HBASE_HOME/bin #ZooKeeper Related Options export ZOOKEEPER_HOME=/home/hadoop/apache-zookeeper-3.7.0-bin export PATH=$PATH:$ZOOKEEPER_HOME/bin export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 export PATH=$PATH:$JAVA_HOME/bin 💡 hadoop in the example is the username, please change it as appropriate, same below.\nMake changes to .bashrc take effect.\n1 source ~/.bashrc Modify the sshd_config settings.\n1 sudo nano /etc/ssh/sshd_config 1 2 3 4 5 Port 22 ListenAddress 0.0.0.0 PermitRootLogin yes PasswordAuthentication yes X11Forwarding no Restart the ssh service to take effect.\n1 sudo service ssh restart Clone the virtual machine Name the previously configured virtual machine H1, clone H2 to H7 (select Full Clone in VirtualBox and select Reinitialize the MAC address of all network cards), and modify the corresponding /etc/hostname and /etc/hosts.\nConfigure the network Configure the virtual machine to get a static IP.\n1 sudo nano /etc/netplan/00-installer-config.yaml 1 2 3 4 5 6 7 8 9 10 11 network: renderer: networkd ethernets: enp0s3: dhcp4: false addresses: [192.168.0.201/24] gateway4: 192.168.0.1 nameservers: addresses: [8.8.8.8] optional: true version: 2 💡 enp0s3 is the physical network card name, please modify it according to the actual situation, which can be obtained through ip addr; gateway4 refers to the default gateway, which should be consistent with the default gateway setting of the host.\nIn this cluster, a total of 7 virtual machines from H1 to H7 are configured with static IPs 192.168.0.201 to 192.168.0.207, so the above changes need to be made 7 times.\nApply the changes to take effect.\n1 sudo netplan apply Check if it is in effect.\n1 ip addr The result seems good.\n1 2 3 4 5 6 7 8 9 10 11 12 1: lo: \u0026lt;LOOPBACK,UP,LOWER_UP\u0026gt; mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: **enp0s3**: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 08:00:27:9a:bf:e9 brd ff:ff:ff:ff:ff:ff inet **192.168.0.201**/24 brd 192.168.0.255 scope global enp0s3 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe9a:bfe9/64 scope link valid_lft forever preferred_lft forever Modify /etc/hostname.\n1 sudo nano /etc/hostname The content is as follows, which is an example of H1.\n1 h1 Modify /etc/hosts.\n1 sudo nano /etc/hosts The following is an example of H1, commenting out the line that corresponds to the original host name, the place that needs to be changed for different virtual machines is localhost.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 127.0.0.1 localhost **## 127.0.1.1 h1** 192.168.0.201 h1 192.168.0.202 h2 192.168.0.203 h3 192.168.0.204 h4 192.168.0.205 h5 192.168.0.206 h6 192.168.0.207 h7 ## The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters Configure SSH Generate the ssh key.\n1 ssh-keygen -t rsa -P \u0026#34;\u0026#34; Copy the public key to ~/.ssh/authorized_keys.\n1 cat ~/.ssh/id_rsa.pub \u0026gt;\u0026gt; ~/.ssh/authorized_keys All virtual machines in the cluster copy ssh keys to each other\u0026rsquo;s virtual machines.\n1 2 3 4 5 6 7 ssh-copy-id hadoop@h1 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h2 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h3 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h4 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h5 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h6 \\ \u0026amp;\u0026amp; ssh-copy-id hadoop@h7 Below is a partial output for copying H1 key.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 hadoop@h1:~$ ssh-copy-id hadoop@h1 \u0026amp;\u0026amp; ssh-copy-id hadoop@h2 \u0026amp;\u0026amp;ssh-copy-id hadoop@h3 \u0026amp;\u0026amp; ssh-copy-id hadoop@h4 \u0026amp;\u0026amp; ssh-copy-id hadoop@h5 \u0026amp;\u0026amp; ssh-copy-id hadoop@h6 \u0026amp;\u0026amp; ssh-copy-id hadoop@h7 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: \u0026#34;/home/hadoop/.ssh/id_rsa.pub\u0026#34; /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system. (if you think this is a mistake, you may want to use -f option) /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: \u0026#34;/home/hadoop/.ssh/id_rsa.pub\u0026#34; The authenticity of host \u0026#39;h2 (192.168.0.202)\u0026#39; can\u0026#39;t be established. ECDSA key fingerprint is SHA256:C6ydAa+dfI5lcMJkMUucz60WE7p3eFLIs7fWZrTYfDE. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys hadoop@h2\u0026#39;s password: Number of key(s) added: 1 Now try logging into the machine, with: \u0026#34;ssh \u0026#39;hadoop@h2\u0026#39;\u0026#34; and check to make sure that only the key(s) you wanted were added. /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: \u0026#34;/home/hadoop/.ssh/id_rsa.pub\u0026#34; The authenticity of host \u0026#39;h3 (192.168.0.203)\u0026#39; can\u0026#39;t be established. ECDSA key fingerprint is SHA256:OVEZc5ls6hhBFNgqmZxT/EjubDKr8oyoqwE4Wtvsk+k. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys hadoop@h3\u0026#39;s password: Number of key(s) added: 1 Now try logging into the machine, with: \u0026#34;ssh \u0026#39;hadoop@h3\u0026#39;\u0026#34; and check to make sure that only the key(s) you wanted were added. Build a ZooKeeper cluster Switch to H3.\nCopy the ZooKeeper template configuration file and modify the configuration.\n1 cp $ZOOKEEPER_HOME/conf/zoo_sample.cfg $ZOOKEEPER_HOME/conf/zoo.cfg \u0026amp;\u0026amp; nano $ZOOKEEPER_HOME/conf/zoo.cfg 1 2 3 4 5 dataDir=/home/hadoop/tmp/zookeeper server.3=h3:2888:3888 server.6=h6:2888:3888 server.7=h7:2888:3888 💡 The cluster information format is server.\u0026lt;id\u0026gt;=\u0026lt;hostname\u0026gt;:2888:3888.\nwhere id is a unique number for each machine and hostname is the corresponding hostname for each machine. For :2888:3888, the former indicates the port on which Follower communicates with the Leader, i.e. the port on which the server communicates internally (default 2888); the latter is the election port (default is 3888).\nCreate a new dataDir for ZooKeeper on each machine, create a new myid in the directory and fill in the ids, the following is an example of the operation on H3, H6 and H7 respectively.\n1 mkdir -p /home/hadoop/tmp/zookeeper \u0026amp;\u0026amp; echo 3 \u0026gt; /home/hadoop/tmp/zookeeper/myid 1 mkdir -p /home/hadoop/tmp/zookeeper \u0026amp;\u0026amp; echo 6 \u0026gt; /home/hadoop/tmp/zookeeper/myid 1 mkdir -p /home/hadoop/tmp/zookeeper \u0026amp;\u0026amp; echo 7 \u0026gt; /home/hadoop/tmp/zookeeper/myid Copy the ZooKeeper from H3, which you just configured, to H6 and H7.\n1 scp -r $ZOOKEEPER_HOME/conf/* h6:$ZOOKEEPER_HOME/conf \u0026amp;\u0026amp; scp -r $ZOOKEEPER_HOME/conf/* h7:$ZOOKEEPER_HOME/conf Start the ZooKeeper service on the 3 machines, i.e., in H3, H6, and H7 and execute the following command.\n1 zkServer.sh start Check the cluster running status on 3 machines, because the cluster must start at least more than half of the nodes to work properly, so the above command should be executed on 3 machines before you can see the status feedback that the cluster started properly.\n1 zkServer.sh status The output message means start successfully.\n1 2 3 4 5 hadoop@h3:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: follower 1 2 3 4 5 hadoop@h6:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: leader 1 2 3 4 5 hadoop@h7:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: follower 💡 You can see that H6 is currently elected as the Leader.\nGet the relevant services that are currently running by executing jps.\n1 2 3 hadoop@h3:~$ jps 2499 Jps 2378 QuorumPeerMain 1 2 3 hadoop@h6:~$ jps 3364 Jps 3279 QuorumPeerMain 1 2 3 hadoop@h7:~$ jps 3511 QuorumPeerMain 3599 Jps Stop the ZooKeeper service on 3 machines, i.e., execute the following command on H3, H6, and H7.\n1 zkServer.sh stop Build a Hadoop cluster Switch to H1.\nModify hadoop-env.sh\n1 nano $HADOOP_HOME/etc/hadoop/hadoop-env.sh set JAVA_HOME.\n1 export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 💡 If there are ARM architecture devices such as Raspberry Pi in the cluster, you should install the corresponding version of JDK and modify the corresponding version of JAVA_HOME, for example.\n1 export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64 Modify hdfs-site.xml.\n1 nano $HADOOP_HOME/etc/hadoop/hdfs-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 \u0026lt;configuration\u0026gt; \u0026lt;!-- Specify the number of replications to be generated --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.replication\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;2\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Turn off permission restrictions to facilitate development debugging --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.permissions\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;false\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Name of NameService, consistent with core-site.xml --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.nameservices\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;ns1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify 2 NameNodes --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.ha.namenodes.ns1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;nn1,nn2\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- nn1 RPC address --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.rpc-address.ns1.nn1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h1:9000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- nn1 HTTP address --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.http-address.ns1.nn1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h1:50070\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- nn2 RPC address --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.rpc-address.ns1.nn2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h4:9000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- nn2 HTTP address --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.http-address.ns1.nn2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h4:50070\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- JournalNode depends on ZooKeeper and is deployed on the ZooKeeper machine --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.shared.edits.dir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;qjournal://h3:8485;h6:8485;h7:8485/ns1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify where the JournalNode stores edits data on local disk --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.journalnode.edits.dir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;/home/hadoop/tmp/journaldata\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Enable automatic failover for NameNode --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.ha.automatic-failover.enabled\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;true\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Configurate failover proxy provider --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.client.failover.proxy.provider.ns1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Configurate fencing methods --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.ha.fencing.methods\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt; sshfence shell(/bin/true) \u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- To use the sshfence isolation mechanism, you need to be ssh login free --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.ha.fencing.ssh.private-key-files\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;/home/hadoop/.ssh/id_rsa\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Configure the sshfence isolation mechanism timeout in ms --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.ha.fencing.ssh.connect-timeout\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;30000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- The directory where the FsImage image is saved --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.namenode.name.dir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;/home/hadoop/data/namenode\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- The directory where the HDFS file system data files are stored --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.datanode.data.dir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;/home/hadoop/data/datanode\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;heartbeat.recheck.interval\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;2000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;dfs.heartbeat.interval\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;/configuration\u0026gt; 💡 Split-Brain: 2 namenodes are serving at the same time, there are 2 isolation methods available for this.\nsshfence: remote login to kill the namenode in question (if the port number of the remote ssh login is not 22, you need to configure sshfence(username:port number) at dfs.ha.fencing.methods. shell: the subsequent solution for unresponsive remote login timeout, you can fill in the custom script in parentheses, such as shell(/bin/true), then return true to switch directly. Modify core-site.xml.\n1 nano $HADOOP_HOME/etc/hadoop/core-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 \u0026lt;configuration\u0026gt; \u0026lt;!-- Specify the NameService for HDFS as ns1 --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;fs.defaultFS\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;hdfs://ns1/\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the Hadoop tmp directory --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hadoop.tmp.dir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;/home/hadoop/tmp/hadoop\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the ZooKeeper addresses --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;ha.zookeeper.quorum\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h3:2181,h6:2181,h7:2181\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Number of client connection retries --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;ipc.client.connect.max.retries\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;100\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- The time interval between 2 connection re-establishments in ms --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;ipc.client.connect.retry.interval\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;5000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;/configuration\u0026gt; Modify mapred-site.xml.\n1 nano $HADOOP_HOME/etc/hadoop/mapred-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 \u0026lt;configuration\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;mapreduce.framework.name\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;yarn\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.app.mapreduce.am.env\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;HADOOP_MAPRED_HOME=/home/hadoop/hadoop-3.3.1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;mapreduce.map.env\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;HADOOP_MAPRED_HOME=/home/hadoop/hadoop-3.3.1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;mapreduce.reduce.env\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;HADOOP_MAPRED_HOME=/home/hadoop/hadoop-3.3.1\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;/configuration\u0026gt; Modify yarn-site.xml.\n1 nano $HADOOP_HOME/etc/hadoop/yarn-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 \u0026lt;configuration\u0026gt; \u0026lt;!-- Enable ResourceManager High Availability --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.ha.enabled\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;true\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify ResourceManager cluster id --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.cluster-id\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;yrc\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the name of the ResourceManager under the group --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.ha.rm-ids\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;rm1,rm2\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Configure RM1 --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.hostname.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:8032\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.scheduler.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:8030\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.webapp.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:8088\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.resource-tracker.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:8031\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.admin.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:8033\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.ha.admin.address.rm1\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h2:23142\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Configure RM2 --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.hostname.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:8032\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.scheduler.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:8030\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.webapp.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:8088\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.resource-tracker.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:8031\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.admin.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:8033\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.ha.admin.address.rm2\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h5:23142\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the ZooKeeper cluster address --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.resourcemanager.zk-address\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h3:2181,h6:2181,h7:2181\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- The reducer fetches data as mapreduce_shuffle --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;yarn.nodemanager.aux-services\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;mapreduce_shuffle\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;/configuration\u0026gt; Configure workers.\n1 nano $HADOOP_HOME/etc/hadoop/workers 1 2 3 4 5 h2 h3 h5 h6 h7 Create the directory needed in the configuration file above.\n1 mkdir -p /home/hadoop/tmp/journaldata /home/hadoop/tmp/hadoop Copy the Hadoop from H1, which you just configured, to the other 6 machines.\n1 2 3 4 5 6 scp -r $HADOOP_HOME/etc/hadoop/* h2:$HADOOP_HOME/etc/hadoop \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/* h3:$HADOOP_HOME/etc/hadoop \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/* h4:$HADOOP_HOME/etc/hadoop \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/* h5:$HADOOP_HOME/etc/hadoop \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/* h6:$HADOOP_HOME/etc/hadoop \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/* h7:$HADOOP_HOME/etc/hadoop Switch to H4 to create a new directory for NameNode to save the FsImage image.\n1 mkdir -p /home/hadoop/data/namenode Start HDFS HA Start the ZooKeeper service for H3, H6, and H7, i.e., execute the following commands in H3, H6, and H7.\n1 zkServer.sh start Check the status of clusters on H3, H6, and H7.\n1 zkServer.sh status The output message is good.\n1 2 3 4 5 hadoop@h3:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: follower 1 2 3 4 5 hadoop@h6:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: leader 1 2 3 4 5 hadoop@h7:~$ zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/hadoop/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Client SSL: false. Mode: follower 💡 You can see that H6 is still elected as the Leader.\nStart the JournalNode service of H2, H3, H5, H6, and H7, i.e., execute the following command in H2, H3, H5, H6, and H7.\n1 hdfs --daemon start journalnode 💡 You need to start the JournalNode manually when the Hadoop cluster starts for the first time, the above steps are not required later.\nGet the relevant services that are currently running by executing jps.\n1 2 3 hadoop@h2:~$ jps 4631 Jps 4588 JournalNode 1 2 3 4 hadoop@h3:~$ jps 2864 JournalNode 2729 QuorumPeerMain 2938 Jps 1 2 3 hadoop@h5:~$ jps 4371 Jps 4325 JournalNode 1 2 3 4 hadoop@h6:~$ jps 3474 QuorumPeerMain 3594 JournalNode 3644 Jps 1 2 3 4 hadoop@h7:~$ jps 3890 Jps 3848 JournalNode 3736 QuorumPeerMain Switch to H1 to format HDFS.\n1 hdfs namenode -format Copy the FSImage from the H1 NameNode to the H4 NameNode, making sure they are identical.\n1 scp -r /home/hadoop/data/namenode/* h4:/home/hadoop/data/namenode Format ZKFC.\n1 hdfs zkfc -formatZK Start HDFS.\n1 start-dfs.sh Get the relevant services that are currently running by executing jps.\n1 2 3 4 hadoop@h1:~$ jps 22368 Jps 21960 NameNode 22317 DFSZKFailoverController A review of the report shows that the status of the 5 DataNodes is alive.\n1 hdfs dfsadmin -report 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 Configured Capacity: 101142011904 (94.20 GB) Present Capacity: 59280982016 (55.21 GB) DFS Remaining: 59280859136 (55.21 GB) DFS Used: 122880 (120 KB) DFS Used%: 0.00% Replicated Blocks: Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 Low redundancy blocks with highest priority to recover: 0 Pending deletion blocks: 0 Erasure Coded Block Groups: Low redundancy block groups: 0 Block groups with corrupt internal blocks: 0 Missing block groups: 0 Low redundancy blocks with highest priority to recover: 0 Pending deletion blocks: 0 ------------------------------------------------- Live datanodes (5): Name: 192.168.0.202:9866 (h2) Hostname: ip6-localhost Decommission Status : Normal Configured Capacity: 9971224576 (9.29 GB) DFS Used: 24576 (24 KB) Non DFS Used: 7673147392 (7.15 GB) DFS Remaining: 1771356160 (1.65 GB) DFS Used%: 0.00% DFS Remaining%: 17.76% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 0 Last contact: Mon Oct 25 02:17:04 UTC 2021 Last Block Report: Mon Oct 25 02:15:29 UTC 2021 Num of Blocks: 0 Name: 192.168.0.203:9866 (h3) Hostname: ip6-localhost Decommission Status : Normal Configured Capacity: 61257113600 (57.05 GB) DFS Used: 24576 (24 KB) Non DFS Used: 6547861504 (6.10 GB) DFS Remaining: 52175802368 (48.59 GB) DFS Used%: 0.00% DFS Remaining%: 85.18% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 0 Last contact: Mon Oct 25 02:17:06 UTC 2021 Last Block Report: Mon Oct 25 02:15:54 UTC 2021 Num of Blocks: 0 Name: 192.168.0.205:9866 (h5) Hostname: ip6-localhost Decommission Status : Normal Configured Capacity: 9971224576 (9.29 GB) DFS Used: 24576 (24 KB) Non DFS Used: 7665070080 (7.14 GB) DFS Remaining: 1779433472 (1.66 GB) DFS Used%: 0.00% DFS Remaining%: 17.85% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 0 Last contact: Mon Oct 25 02:17:06 UTC 2021 Last Block Report: Mon Oct 25 02:15:33 UTC 2021 Num of Blocks: 0 Name: 192.168.0.206:9866 (h6) Hostname: ip6-localhost Decommission Status : Normal Configured Capacity: 9971224576 (9.29 GB) DFS Used: 24576 (24 KB) Non DFS Used: 7667515392 (7.14 GB) DFS Remaining: 1776988160 (1.65 GB) DFS Used%: 0.00% DFS Remaining%: 17.82% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 0 Last contact: Mon Oct 25 02:17:05 UTC 2021 Last Block Report: Mon Oct 25 02:15:38 UTC 2021 Num of Blocks: 0 Name: 192.168.0.207:9866 (h7) Hostname: ip6-localhost Decommission Status : Normal Configured Capacity: 9971224576 (9.29 GB) DFS Used: 24576 (24 KB) Non DFS Used: 7667224576 (7.14 GB) DFS Remaining: 1777278976 (1.66 GB) DFS Used%: 0.00% DFS Remaining%: 17.82% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 0 Last contact: Mon Oct 25 02:17:07 UTC 2021 Last Block Report: Mon Oct 25 02:15:31 UTC 2021 Num of Blocks: 0 Test HDFS HA Visit http://192.168.0.201:50070/ You can see \u0026lsquo;h1:9000\u0026rsquo; (active)\nStop the NameNode at H1.\n1 hdfs --daemon stop namenode At this point visit http://192.168.0.204:50070/ to see \u0026lsquo;h4:9000\u0026rsquo; (active).\nEnable NameNode again in H1.\n1 hdfs --daemon start namenode At this point visit http://192.168.0.204:50070/ to see \u0026lsquo;h1:9000\u0026rsquo; (standby).\nStart YARN Start the YARN service of H2 and H5, i.e., execute the following command in H2.\n1 start-yarn.sh Get the relevant services that are currently running by executing jps.\n1 2 3 4 5 6 hadoop@h2:~$ jps 5203 JournalNode 6691 NodeManager 6534 ResourceManager 6838 Jps 5692 DataNode 1 2 3 4 5 6 hadoop@h5:~$ jps 6099 Jps 5395 DataNode 4921 JournalNode 5659 ResourceManager 5789 NodeManager Test YARN HA Visit http://192.168.0.202:8188/ to see ResourceManager HA state: active.\nStop ResourceManager in H2.\n1 yarn --daemon stop resourcemanager At this point visit http://192.168.0.205:8188/ and you can see ResourceManager HA state: active.\nThe NameNode is enabled again in H2.\n1 yarn --daemon start resourcemanager At this point visit http://192.168.0.202:8188/ and you can see ResourceManager HA state: standby.\nTest MapReduce Switch to H2.\n1 hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar pi 1 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 hadoop@h2:~$ hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar pi 1 1 Number of Maps = 1 Samples per Map = 1 Wrote input for Map #0 Starting Job 2021-10-25 07:08:04,762 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2 2021-10-25 07:08:04,977 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hadoop/.staging/job_1635145651320_0001 2021-10-25 07:08:05,934 INFO input.FileInputFormat: Total input files to process : 1 2021-10-25 07:08:06,626 INFO mapreduce.JobSubmitter: number of splits:1 2021-10-25 07:08:06,973 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1635145651320_0001 2021-10-25 07:08:06,975 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2021-10-25 07:08:07,341 INFO conf.Configuration: resource-types.xml not found 2021-10-25 07:08:07,342 INFO resource.ResourceUtils: Unable to find \u0026#39;resource-types.xml\u0026#39;. 2021-10-25 07:08:07,991 INFO impl.YarnClientImpl: Submitted application application_1635145651320_0001 2021-10-25 07:08:08,048 INFO mapreduce.Job: The url to track the job: http://h5:8188/proxy/application_1635145651320_0001/ 2021-10-25 07:08:08,051 INFO mapreduce.Job: Running job: job_1635145651320_0001 2021-10-25 07:08:25,438 INFO mapreduce.Job: Job job_1635145651320_0001 running in uber mode : false 2021-10-25 07:08:25,439 INFO mapreduce.Job: map 0% reduce 0% 2021-10-25 07:08:34,585 INFO mapreduce.Job: map 100% reduce 0% 2021-10-25 07:08:50,737 INFO mapreduce.Job: map 100% reduce 100% 2021-10-25 07:08:52,774 INFO mapreduce.Job: Job job_1635145651320_0001 completed successfully 2021-10-25 07:08:52,993 INFO mapreduce.Job: Counters: 54 File System Counters FILE: Number of bytes read=28 FILE: Number of bytes written=555793 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=255 HDFS: Number of bytes written=215 HDFS: Number of read operations=9 HDFS: Number of large read operations=0 HDFS: Number of write operations=3 HDFS: Number of bytes read erasure-coded=0 Job Counters Launched map tasks=1 Launched reduce tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=5590 Total time spent by all reduces in occupied slots (ms)=14233 Total time spent by all map tasks (ms)=5590 Total time spent by all reduce tasks (ms)=14233 Total vcore-milliseconds taken by all map tasks=5590 Total vcore-milliseconds taken by all reduce tasks=14233 Total megabyte-milliseconds taken by all map tasks=5724160 Total megabyte-milliseconds taken by all reduce tasks=14574592 Map-Reduce Framework Map input records=1 Map output records=2 Map output bytes=18 Map output materialized bytes=28 Input split bytes=137 Combine input records=0 Combine output records=0 Reduce input groups=2 Reduce shuffle bytes=28 Reduce input records=2 Reduce output records=0 Spilled Records=4 Shuffled Maps =1 Failed Shuffles=0 Merged Map outputs=1 GC time elapsed (ms)=220 CPU time spent (ms)=1050 Physical memory (bytes) snapshot=302690304 Virtual memory (bytes) snapshot=4986265600 Total committed heap usage (bytes)=138096640 Peak Map Physical memory (bytes)=202854400 Peak Map Virtual memory (bytes)=2490499072 Peak Reduce Physical memory (bytes)=99835904 Peak Reduce Virtual memory (bytes)=2495766528 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=118 File Output Format Counters Bytes Written=97 Job Finished in 48.784 seconds Estimated value of Pi is 4.00000000000000000000 Build an HBase HA cluster Switch to H2.\nModify hbase-env.sh.\n1 nano $HBASE_HOME/conf/hbase-env.sh Set JAVA_HOME。\n1 export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 💡 Similarly, if there are ARM architecture devices such as Raspberry Pi in the cluster, you should install the corresponding version of JDK and modify the corresponding version of JAVA_HOME, such as\n1 export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64 Does not use the HBase provided ZooKeeper.\n1 export HBASE_MANAGES_ZK=false Modify hbase-site.xml。\n1 nano $HBASE_HOME/conf/hbase-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 \u0026lt;configuration\u0026gt; \u0026lt;!-- Specify the path to where HBase is stored on HDFS --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hbase.rootdir\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;hdfs://ns1/hbase\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specifying if HBase is distributed --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hbase.cluster.distributed\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;true\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the ZooKeeper addresses --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hbase.zookeeper.quorum\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;h3:2181,h6:2181,h7:2181\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;!-- Specify the maximum time difference between HMaster and HRegionServer --\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hbase.master.maxclockskew\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;180000\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;property\u0026gt; \u0026lt;name\u0026gt;hbase.wal.provider\u0026lt;/name\u0026gt; \u0026lt;value\u0026gt;filesystem\u0026lt;/value\u0026gt; \u0026lt;/property\u0026gt; \u0026lt;/configuration\u0026gt; Modify regionservers。\n1 nano $HBASE_HOME/conf/regionservers 1 2 3 h3 h6 h7 Copy core-site.xml and hdfs-site.xml from H1 to H2.\n1 2 scp -r $HADOOP_HOME/etc/hadoop/core-site.xml h2:$HBASE_HOME/conf \\ \u0026amp;\u0026amp; scp -r $HADOOP_HOME/etc/hadoop/hdfs-site.xml h2:$HBASE_HOME/conf Replicate the HBase in H2 that you just configured to the other 4 machines.\n1 2 3 4 scp -r $HBASE_HOME/conf/* h3:$HBASE_HOME/conf \\ \u0026amp;\u0026amp; scp -r $HBASE_HOME/conf/* h5:$HBASE_HOME/conf \\ \u0026amp;\u0026amp; scp -r $HBASE_HOME/conf/* h6:$HBASE_HOME/conf \\ \u0026amp;\u0026amp; scp -r $HBASE_HOME/conf/* h7:$HBASE_HOME/conf Start HBase HA Switch to H1.\nStart HDFS.\n1 start-dfs.sh Get the relevant services that are currently running by executing jps.\n1 2 3 4 hadoop@h1:~$ jps 3783 NameNode 4169 Jps 4138 DFSZKFailoverController Switch to H2.\nStart HMaster.\n1 start-hbase.sh Switch to H5.\nStart HMaster.\n1 hbase-daemon.sh start master Get the relevant services that are currently running by executing jps.\n1 2 3 4 5 hadoop@h2:~$ jps 1052 JournalNode 7901 DataNode 8125 HMaster 8847 Jps 1 2 3 4 5 6 hadoop@h3:~$ jps 6368 Jps 6210 HRegionServer 2660 QuorumPeerMain 5717 DataNode 5846 JournalNode 1 2 3 4 5 hadoop@h5:~$ jps 6336 Jps 1058 JournalNode 6117 HMaster 5980 DataNode 1 2 3 4 5 6 hadoop@h6:~$ jps 4722 Jps 4408 JournalNode 4248 DataNode 4570 HRegionServer 1039 QuorumPeerMain 1 2 3 4 5 6 hadoop@h7:~$ jps 4402 DataNode 4563 JournalNode 4726 HRegionServer 1031 QuorumPeerMain 5000 Jps Test HBase HA Visit http://192.168.0.202:16010/ to see Current Active Master: h2.\nStop HMaster at H2.\n1 hbase-daemon.sh stop master At this point visit http://192.168.0.205:16010/ to see Current Active Master: h5.\nStart HMaster again in H2.\n1 hbase-daemon.sh start master At this point visit http://192.168.0.202:16010/ and you will see Current Active Master: h5 and Backup Master is h2.\nGreat work The end of the process is the beginning of the learning curve. Keep smiling and keep progressing 😊\n","permalink":"https://ethans.me/posts/2021-11-01-building-a-hadoop-distributed-environment-on-virtual-machines-and-raspberry-pi/","summary":"This process comes from a course experiment, A more detailed tutorial to fully distributed cluster deployment using multiple virtual machines for Hadoop HA, HBase HA and YARN HA. 6 virtual machines as nodes of the cluster on 2 computers and 1 Raspberry Pi.","title":"Building a Hadoop Distributed Environment on Virtual Machines and Raspberry Pi"},{"content":"Introduction The usability of Windows touchpad gestures has increased with the rollout of Windows 10, with not only a greater variety of gestures, but also an undeniable improvement in smoothness. In contrast, optimizing the touchpad experience for Linux, which is lacking on the desktop, is a concern for both Linux users and laptop users.\nThe following article focuses on Wayland\u0026rsquo;s own libinput and does not discuss alternatives such as synaptics. In addition, I have made some minor contributions to libinput\u0026rsquo;s Archwiki Chinese entry, which can be helpful too.\nEditing touchpad-related Xorg configuration files The configuration files for Linux input devices are stored in /etc/X11/xorg.conf.d and 30-touchpad.conf, as the name implies, the configuration information for the touchpad. Copy them to /usr/share/X11/xorg.conf.d and modify it. At this point you can refer to the basic framework of 40-libinput.conf and follow the following configuration format\n1 2 3 4 5 6 7 Section \u0026#34;InputClass\u0026#34; Identifier \u0026#34;touchpad\u0026#34; Driver \u0026#34;libinput\u0026#34; Option \u0026#34;Tapping\u0026#34; \u0026#34;on\u0026#34; Option \u0026#34;NaturalScrolling\u0026#34; \u0026#34;true\u0026#34; Option \u0026#34;ClickMethod\u0026#34; \u0026#34;clickfinger\u0026#34; EndSection`\u0026lt;/pre\u0026gt; where Option \u0026quot;Tapping\u0026quot; \u0026quot;on\u0026quot; turns on the \u0026ldquo;touch tap\u0026rdquo; operation of the touchpad which eliminating the need to press the touchpad.\nOption \u0026quot;NaturalScrolling\u0026quot; \u0026quot;true\u0026quot; restores a more natural scrolling orientation, allowing you to gain experience on Windows.\nOption \u0026quot;ClickMethod\u0026quot; \u0026quot;clickfinger\u0026quot; turns on the option of \u0026ldquo;two-finger click\u0026rdquo; for right-click and \u0026ldquo;three-finger click\u0026rdquo; for middle-click.\nIn addition, libinput provides more options for \u0026ldquo;tap\u0026rdquo;, \u0026ldquo;scroll\u0026rdquo;, \u0026ldquo;touchpad pointer acceleration\u0026rdquo;, etc. Other configuration options can be found in libinput\u0026rsquo;s man page\nFor more information about libinput\u0026rsquo;s various point-and-click scrolling operations, there are friendly illustrations in its documentation to help understand them. Also, you can see the configuration and support for other input devices such as \u0026ldquo;red dots\u0026rdquo; and \u0026ldquo;touch screens\u0026rdquo;, which are outside the scope of this article on \u0026ldquo;touch panels\u0026rdquo;, so they are not discussed.\nInstalling xdotool The xdotool is a tool that can simulate keyboard and mouse input. When used in conjunction with the libinput-gestures package mentioned below, you can add and optimize touchpad gestures by binding them to corresponding hotkeys or mouse actions.\nIn the ArchWiki\u0026rsquo;s Recommended for Xorg\u0026rsquo;s emulation of keystrokes, xdotool is described as \u0026ldquo;Very Buggy\u0026rdquo;. However, after using it briefly, I think xdotool is very simple and bug-free (maybe I\u0026rsquo;m only using the part that handles keyboard input 😂) Here is the basic operation of xdotool\n1 2 3 4 xdotool click 1 xdotool key Ctrl+L xdotool mousemove x y xdotool type \u0026#39;string\u0026#39; This can be considered a very clear syntax. It is worth noting that xdotool calls several special keys: 1 left mouse button, 2 middle mouse button, 3 right mouse button, Left left array button, Right right array button, Super Windows logo key, plus plus sign, minus minus sign.\nxdotool also supports window retrieval, getting window focus, etc., which can fulfill most of the keystroke automation needs.\nUsing libinput-gestures Install libinput-gestures and add the current user to the input group, copy the configuration file template /etc/libinput-gestures.conf to ~/.config/ for modification. Start libinput-gestures-setup for initialization\n1 libinput-gestures-setup start where gestures are known by libinput-gestures as swipe and pinch, which correspond to finger swipe and pinch operations on the touchpad, respectively. As we all know, the Ctrl + + and Ctrl + - hotkeys can basically zoom in and out on most software views, and it is customary to open both fingers for zooming in and pinch them for zooming out, so add them to ~/.config/libinput-gestures.conf first.\n1 2 gesture pinch in 2 xdotool key Ctrl+minus gesture pinch out 2 xdotool key Ctrl+plus For pinch, you can specify in and out and the number of fingers involved in the action. Similarly, for swipe, the number of fingers involved and the four directions of swiping (Left, Right, Up, Down) can be specified, and here is an example of swiping left and right to switch tabs in Chrome.\n1 2 gesture swipe right 3 xdotool key Ctrl+Tab gesture swipe left 3 xdotool key Ctrl+Alt+Tab Three finger swipe right goes to the next tab, three finger swipe left goes to the previous tab. There is no corresponding animation feedback, but it works fine.\nFrom reading libinput-gestures related chapter, I learned that the touchpad does not differentiate between swipe and pinch with the same number of fingers involved in the action. pinch is not completely accurate, so do not configure the same number of fingers of two different actions coexist. The later experience will be poor for the greater the probability of misjudgment. Especially when the same number of fingers swipe but in different directions, such as configuring two fingers left and right swipe to switch applications, it is likely to conflict with two fingers up and down swipe, that is, scrolling action.\nMore possibilities Of course, libinput-gestures and xdotool do not have to be used together, but you can also use keystroke-enabled ones like xte, xvkbd, and their operation is roughly the same as xdotool.\nThe following are examples of xvkbd implementing left mouse click and Ctrl+Tab respectively\n1 2 xvkbd -no-jump-pointer -xsendevent -text \u0026#39;\\m1\u0026#39; xvkbd -no-jump-pointer -xsendevent -text \u0026#39;\\Ct\u0026#39; xte operations on key combinations are implemented using keydown and keyup. Here is an example of xte\u0026rsquo;s implementation of Ctrl+Tab\n1 2 3 xte keydown Control_L xte key Tab xte keyup Control_L Similarly, you can also bind libinput-gestures to launch an application directly, such as a four-finger swipe up to call out the application shortcut portal.\nWhen you encounter a jumping and flashing touchpad pointer, or a serious gesture miscalculation, try restarting libinput-gestures with the following command\n1 libinput-gestures-setup restart Thanks for reading 💗\n","permalink":"https://ethans.me/posts/2019-03-10-optimizing-experience-of-using-touchpad-on-linux-with-xdotool-and-libinput-gestures/","summary":"Improve the experience of using touchpad on Linux by tweaking libinput, xdotool and libinput-gesture.","title":"Optimizing Experience of Using Touchpad on Linux with xdotool and libinput-gestures"},{"content":"Introduction The \u0026ldquo;Editor War\u0026rdquo; (Editor War) is an endless debate between Vim and Emacs. But in my opinion, as long as the editor is configured and put into daily production, die-hard fans of either Vim or Emacs can gain great experience. The reason for this thought is that I had a painful experiment with Vim. In particular, the configuration of YouCompleteMe, a plugin whose name even sounds a bit ambiguous, did its best to make me feel its maliciousness.\nWhile this article does not faithfully recreate the whole process of falling into the pits, I have tuned the order of operations, but I have kept the description of where the problems might occur and also explained the solutions.\nVim version \u0026amp; configuration environment The version of Vim I want to compile and install is the latest 8.1 in the official Github repository\nThe installed OS environment is Debian GNU/Linux provided by Windows Subsystem for Linux in Windows 10 version 17134. So Debian here is the stable version of 9/Stretch.\nDebian 9 provided by Windows Subsystem for Linux (hereafter WSL) has certain peculiarities, mainly in the fact that the environment is more streamlined and clean compared to larger distributions, but this does not affect how the installation process of Vim is represented.\nCompiling and installing Vim 8 Why compile and install In fact, except in the case of a missing package, downloading and installing software from a source should be the preferred option. The package manager will take care of the dependencies properly and will not tend to dirty up the system. So the recommended approach on Debian-based distributions should be.\n1 sudo apt install vim However, some Vim plugins (such as YouCompleteMe for this article) require Vim support for Python which are not enabled by default.\nSo how do you enable it? This article uses a compiled installation with the option to enable Python support to get a Vim that meets the requirements. Other solutions such as vim-nox are out of scope.\nDownloading Vim source code Install Git first. While it is possible to go directly to the official Vim repository to get the source code, Git should be the first to install, given that most Vim plugin managers also rely on Git to install and upgrade their plugins.\n1 sudo apt install git You don\u0026rsquo;t need to do a complicated configuration of Git, although you\u0026rsquo;ll have a lot of need to do so in the future. Here we\u0026rsquo;ll just use simple commands of Git to get the source code.\n1 git clone https://github.com/vim/vim Not surprisingly, this will take a few seconds of waiting.\nCompiling Vim to enable Python 3 support The need to enable Python support was stated earlier when explaining the reason for compiling and installing, so I won\u0026rsquo;t repeat it here, but first install Python 3 with its support library libpython3-dev\n1 sudo apt install python3 libpython3-dev In theory, major distributions such as Debian should already include Python3 support, but WSL does not, so install it as a supplement.\nGo back to the directory where you put the Vim source code and go to the src subdirectory. Before compiling, make sure that the build tools included in build-essential are successfully installed, otherwise you will get an error because you cannot find the build tools.\n1 sudo apt-get install build-essential Once the build tools are ready, you can start compiling and installing. Start the \u0026ldquo;build triplet\u0026rdquo;.\n1 2 3 4 5 6 7 8 . /configure --with-features=huge \\ --enable-multibyte \\ --enable-python3interp=yes \\ --with-python3-config-dir=/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu \\ --enable-rubyinterp=yes \\ --enable-perlinterp=yes \\ --enable-luainterp=yes \\ --prefix=/usr/local/vim-8.1 For the third and fourth configuration options, I chose to enable Python 3 support and tell you where the config file is located. If you omitted to install libpython3-dev above, the config file may be missing.\nOf course, both of these directories will change with the version of Python installed (for example, the installed version is 3.7), so if you get an error, look in the /usr/lib directory for the Python directory and fill in the config field.\nIf you need Python 2 support, for example if some old plugins still only support Python 2 but there are no other plugins available, you should add.\n1 2 --enable-pythoninterp=yes \\ --with-python-config-dir=/usr/lib/python2.7/config \\ Accordingly, you will need to install Python 2 and libpython2-dev in advance. Note that if you install the required files for Python 3 but incorrectly turn on the enable-pythoninterp option, Vim will fail to enable Python because Python 2 support is turned on and no Python 2 related files are found. So, be careful to install the appropriate version and turn on the appropriate options.\nYou may also notice that the Ruby, Perl, and Lua support options are turned on, and you can change them accordingly if you are sure you don\u0026rsquo;t need them.\nThe directory after -prefix specifies the target directory for the Vim installation.\nIf you run . /configure and you get an error like this.\n1 2 3 4 no terminal library found checking for tgetent()... configure: error: NOT FOUND! You need to install a terminal library; for example ncurses. Or specify the name of the library with -with-tlib. The solution is to install libncurses5-dev as stated in the error message.\n1 sudo apt install libncurses5-dev If . /configure everything went well, next you just need to execute.\n1 2 sudo make sudo make install Registering Vim as Editor Debian has the following usage to register Vim as an editor.\n1 sudo update-alternatives --install /usr/bin/editor editor /usr/local/vim-8.1/bin/vim 1 Use the generic softlink chain on vim.\n1 sudo ln -s /usr/local/vim-8.1/bin/vim /usr/bin/vim This allows you to run Vim by typing vim on the command line.\nAt this point, to ensure that Python support of Vim is enabled as planned, the following command can be verified.\n1 vim --version | grep python Not surprisingly, two of the output results are\n1 2 -python +python3 The presence of a plus sign in front of the item indicates that it is enabled, and the presence of a minus sign means that it is not enabled. So here Python 3 support is successfully enabled.\nInstalling the YouCompleteMe plugin YouCompleteMe (YCM for short) is a powerful and malicious auto-completion plugin for Vim. In the issues of YCM Official Repository, its author Valloric takes great pains to remind issues authors to read the official documentation properly, so follow the official Installation Instructions and check out the User Guide after you hit a hole \u0026ndash; that\u0026rsquo;s the process we should stick to next.\nThe goal of this article is to configure YCM to support auto-completion for C/C++, and other languages are beyond the scope of this article. It is worth mentioning that for C Family semantic autocompletion, it is officially specified that you must read this document.\nInstalling the vim-plug plugin manager Although our goal is only to install YCM, the great help of using a plugin manager for future use of Vim should not be overlooked. Here we choose vim-plug, the Vim plugin manager with the second most stars on Github, which supports asynchronous installation.\nYou need to download the plug.vim file from the repository and put it in the autoload directory.\nThe autoload folder exists in the ~/.vim directory, and both directories can simply be created if they do not exist.\nEdit the ~/.vimrc file (if it doesn\u0026rsquo;t exist, create a new one) and add a reference to YCM.\n1 Plug \u0026#39;Valloric/YouCompleteMe\u0026#39; The use of vim-plug is well covered in the official repository in README. Simply put, you can change the content between the single quotes in the line of configuration above to the address of another plugin (in the format of \u0026ldquo;Username/Plugname \u0026ldquo;) to reference the plugin.\nIf you are not familiar with Vim now, you can use nano to make changes to ~/.vimrc.\nInstalling Clang For C Family semantic completion, YCM uses the Clang semantic search engine. Although it comes with libclang, the documentation gives the recommendation to compile it yourself to get libclang. This means that it is a prerequisite that Clang is properly configured on this computer.\nClang, being a LLVM based compiler, also requires the LLVM framework system to be installed. You can install it using the software source (the latest version at the time of writing is 6.0).\n1 sudo apt install clang-6.0 This is the quickest and easiest way to install Clang + LLVM. If you download the source code separately and compile and install it, it is not really tedious. Of course, LLVM also provides pre-compiled packages for different distributions on the Download Page. So, you can also move the downloaded precompiled package to /usr/local, which is next to the Vim installation directory.\nThe following version is replaced by xxxxxxx, please refer to the actual downloaded file name.\n1 sudo mv /path/to/package/clang+llvm_xxxxxxx /usr/local Don\u0026rsquo;t forget to link to clang and clang++.\n1 2 sudo ln -s /usr/local/clang+llvm_xxxxxxx/bin/clang /usr/bin/clang sudo ln -s /usr/local/clang+llvm_xxxxxxx/bin/clang++ /usr/bin/clang++ At this point, Clang is ready. And libclang is quietly hidden in /usr/local/clang+llvm_xxxxxxx/x86_64-linux-gnu/lib.\nInstalling Cmake Cmake is the default compiler supported by YCM, so using Cmake to compile YCM is the safest and most secure way. If you don\u0026rsquo;t install Cmake to compile ycm_core, Vim will give you an error after startup:\n1 The ycmd server SHUT DOWN (restart with \u0026#39;:YcmRestartServer\u0026#39;). Unexpected error while loading the YCM core library. Also, the good news is that Cmake provides the installation script for the shell, just look for your platform in the Download Page and download it. After downloading, you can reach the directory where the script is located, move it to /usr/local, and install it.\n1 2 sudo mv /path/to/cmake-x.xx.x-Linux-x86_64.sh /usr/local sudo /usr/local/.cmake-x.xx.x-Linux-x86_64.sh x.xx.x refers to the version number, based on the downloaded file.\nEnter y to accept the agreement and enter y again when prompted to install to a separate subdirectory of the directory you are in.\nSo, Cmake is ready to go.\nInstalling YCM with vim-plug The YCM installation script requires Cmake to generate some configuration files and compile them by itself, so it will save us a lot of work if we run the installation script after Cmake is installed.\nOpen Vim, type : to enter command mode, type PlugInstall and enter to confirm. At this point vim-plug starts fetching each plugin through Git and handling the installation directory properly. Only one plugin is currently referenced, so there should only be one YouCompleteMe installation in the task list.\nYCM is a more complex Vim plugin, so it has its own unique process of fetching dependency packages from Github after downloading it, which takes some time.\nExecute the install script and add the --clang-completer parameter This step is an important one in adding semantic completion support for C-Family to YCM. So what you need to do is to run install.sh in its directory and add the -clang-completer parameter.\n1 2 cd ~/.vim/plugged/YouCompleteMe . /install.sh --clang-completer Accordingly, there are parameters such as --js-completer, --java-completer and --rust-completer, which can be added as needed.\nIf you browse through the YCM directory, you may find a Python script called -install.py. So if you want to use this script, this is also a viable option.\n1 2 cd ~/.vim/plugged/YouCompleteMe python3 install.py --clang-completer If you are using the vundle plugin manager, the YCM installation directory should be under ~/.vim/bundle, not ~/.vim/plugged as specified by vim-plug.\nCompiling ycm_core Theoretically, the installation of YCM is complete after the previous step, but you may still get several errors from YCM when you start Vim. Among other things, it can make finding a Python runtime library quite difficult.\nThe errors given by YCM here appear to be in many forms on the web, mostly mentioning Couldn\u0026rsquo;t load various Python modules.\nDon\u0026rsquo;t forget to also let YCM use the libclang we just installed. So compile ycm_core with the relevant arguments for the directory where Python is located and include -DEXTERNAL_LIBCLANG_PATH as well.\n1 2 3 4 5 6 cmake -G \u0026#34;Unix Makefiles\u0026#34; ~/.vim/plugged/YouCompleteMe/third_party/ycmd/cpp/ \\ -DUSE_SYSTEM_BOOST=ON \\ -DEXTERNAL_LIBCLANG_PATH=/usr/local/clang+llvm-xxxxxxx/lib/libclang.so \\ -DPYTHON_LIBRARY=/usr/lib/python3.7 \\ -DPYTHON_INCLUDE_DIR=/usr/include/python3.7 \\ -DUSE_PYTHON2=OFF LIBCLANG_PATH refers to the location of the libclang.so file, depending on the directory chosen for the previous installation of Clang.\nPYTHON_LIBRARY and PYTHON_INCLUDE_DIR are related to the Python version number (in this case, 3.7) or the installation directory.\nIn particular, it is worth noting that YCM\u0026rsquo;s Makefile here is compiled with Python 2 by default, so if you don\u0026rsquo;t have Python 2 installed and you set the Python runtime libraries and headers to the Python 3 directory, you will get the error.\nYou set USE_PYTHON2 to make CMake find python2 libs only, but CMake found python3 libs instead. necessary python2 headers.\nThe solution is to add a line -DUSE_PYTHON2=OFF after the Cmake command, as shown in the last line of the given reference. Of course, if you are using Python 2 from the beginning, you should remove this line and just execute the command.\nSpecify .ycmextraconf.py If you open a C/C++/ObjC/ObjC++ file with Vim at this point, there is a certain chance that you will also see an error like this, also from YCM:\nNo .ycm_extra_conf.py file detected, so no compile flags are available. Thus no semantic support for C/C++/ObjC/ObjC++. See the docs for details.\nTake a deep breath, this issue exists in various descriptions in YCM\u0026rsquo;s repository issues list. This is the result of YCM not finding the most critical C-Family autocomplete configuration file. In fact, the ycm_extra_conf.py file is located in this well-hidden directory.\n1 ~/.vim/plugged/YouCompleteMe/third_party/ycmd/cpp/ycm Just specify where it is in ~/.vimrc.\n1 let g:ycm_global_ycm_extra_conf = \u0026#39;~/.vim/plugged/YouCompleteMe/third_party/ycmd/cpp/ycm/.ycm_extra_conf.py\u0026#39; Some people also choose to copy the ycm_extra_conf.py file out of the deep directory and into a shallower part of the YCM home directory, and then modify the g:ycm_global_ycm_extra_conf variable.\nTake it a step further If nothing else happens, the installation should complete without incident. My description of the process of installing Vim and the YouCompleteMe plugin comes to an end. Apart from some strange issues in issues, this article basically covers the classic issues.\nThe beautification and further configuration of Vim is not covered in this article, so I recommend everyone to read:\nThe official Vim Tips Wiki\n.vimrc with large number of stars on Github\nThe most important thing is not to spend too much time on tuning the tool, but to get familiar with Vim\u0026rsquo;s keystrokes and get the tool into production as soon as possible.\nThanks for reading 💗\n","permalink":"https://ethans.me/posts/2018-09-01-installing-vim-with-youcompleteme-plugin-to-enable-autocomplete/","summary":"Install Vim and install the YouCompleteMe plugin to support autocomplete for multiple programming languages to improve development efficiency and provide an IDE-like experience.","title":"Installing Vim with YouCompleteMe Plugin to Enable Autocomplete"},{"content":"Introduction The enthusiasm of the Linux kernel to follow up with support for AMD Ryzen™ mobile low-voltage CPUs seems to match the enthusiasm of the few AMD Ryzen™ laptops on the market. While AMD Ryzen™ desktop CPUs have been better supported since Linux 4.11, kernels below 4.17 do not even load properly on their mobile CPUs. At the time of writing (2018.07), most distributions have \u0026lt; 4.17 as their default kernel version, so the few radical Linux distributions that exist are my only options, and Gentoo is one of them.\nAfter going through the Gentoo Handbook, which is much more friendly than the previous one, and with the need to toss it around, I decided to use Gentoo. So my interest in using Gentoo was ignited again 😂, and I have documented a few installation tips in this article.\nInstalling images with Arch Linux The official rolling update install image provided by Gentoo does not support UEFI booting. It does provide a UEFI-compatible image from 2016. However, I know that this image will not load properly on my device (the kernel is too old for AMD Ryzen™ mobile platforms), so using the Arch image with the latest kernel seems to be the best option.\nYou may want to know that Gentoo Handbook has been updated. \u0026ldquo;As of August 23, 2018 the official Minimal CDs are capable of booting in UEFI mode. Previous versions boot in BIOS (MBR) mode only. Readers looking to make their system UEFI bootable must download the latest ISO.\u0026rdquo; As it mentions, the latest ISO supports UEFI boot. It is unnecessary to use Arch Linux installation image from now on.\nPartitioning and preparing to boot with EFISTUB My partitioning scheme is to follow the Preparing the disks section of the Handbook, mount the /root partition to /mnt/gentoo and refer to EFI system partition to mount the EFI partition to /mnt/gentoo/esp and then /esp/EFI/Gentoo to /mnt/gentoo/boot.\nThis way, while keeping the ESP partition tidy, there is no need to manually update the files in /boot after the kernel files have been updated. All you need to do is copy the set of files in /boot to /esp/EFI/Gentoo the old way.\nDownloading the Stage3 image See Network configuration for the network configuration section.\nThe Handbook recommends using Links to access the mirrors to download the Stage3 image; the Arch Linux image does not come with Links, so you should first install Lynx or Links using pacman to do this. I did this by copying the downloaded Stage3 image to the boot disk and then to the /mnt/gentoo directory when I made the Arch Linux boot disk.\nModify precompilation parameters You need to add or change some parameters in make.conf to add support for AMD Ryzen™.\nAdding znver1 after -march in CFLAGS is a mandatory parameter for AMD Ryzen™ CPUs. If the device uses Vega\u0026rsquo;s GPU, then the VIDEO_CARDS parameter radeonsi should be added. Laptop users should add Synaptics support. In addition to that, the very fast Tuna 🐟 is chosen for GENTOO_MIRRORS and the MAKEOPTS parameter follows the CPU core count plus one, i.e. -j5 in the case of AMD Ryzen™ 2700U.\nThe following is an example of a complete make.conf.\n1 2 3 4 5 6 7 CFLAGS=\u0026#34;-march=znver1 -O2 -pipe\u0026#34; CXXFLAGS=\u0026#34;${CFLAGS}\u0026#34; MAKEOPTS=\u0026#34;-j5\u0026#34; VIDEO_CARDS=\u0026#34;amdgpu radeonsi\u0026#34; INPUT_DEVICES=\u0026#34;evdev synaptics\u0026#34; LANGUAS=\u0026#34;zh_CN\u0026#34; GENTOO_MIRRORS=\u0026#34;https://mirrors.tuna.tsinghua.edu.cn/gentoo\u0026#34; Information about other options can be found in Configuring compile options\nRsync Mirrors selection Gentoo allows the address of the rsync mirror server to be specified by editing the configuration file of the portage repository. You can of course choose either the tuna or USTC source.\nCuriosity led me to look through the official Gentoo page Gentoo rsync mirrors, but I did not find any mirrors from mainland China, perhaps because the existing mirrors from mainland China, both university and enterprise, do not reach Gentoo. Perhaps because the existing mirrors from mainland China, whether universities or enterprises, do not meet Gentoo\u0026rsquo;s standards. However, the speed of Taiwan\u0026rsquo;s sources in mainland China is hard to be complimented. I accidentally found the rsync source provided by KAIST. Although it only supports IPv4, the speed is very satisfactory. This is an unexpected discovery.\n🎉 Congratulations to Tsinghua University for being the only available rsync mirror in mainland China, now that\u0026rsquo;s a better choice (2019.01).\nAbout \u0026ndash;make-rslave Mounting the necessary filesystems, it is mentioned that enabling Systemd support requires -mount followed by the -make-rslave argument. The man page of the mount command indicates that this will recursively give the directory the propagation type of slave, i.e. the directory, including all its subdirectories, will follow master changes without notifying itself when it changes and does not notify master of changes when it changes itself.\nIf you are not using Systemd, you can also add this parameter so that you cannot use umount -R to forcibly unmount the partition.\nSelecting a Profile When using eselect to select a profile, it is important to keep an eye on the kernel version. For Gentoo, at the time of writing (2018.07) the 4.17 version of the Linux kernel is still an experimental kernel, so the options are at the bottom of the list.\nIf, like me, you intend to install Gentoo minimally, you should choose the option with the fewest additional installation items (i.e., the shortest text). In particular, avoid the option with the desktop environment, even if you need it later, and it is not recommended to place this task in the installation process to avoid overloading the CPU with work for a short period of time. People don\u0026rsquo;t want their laptops to personally cook their hands into ironclad dog paws.\nConfiguration of USE Flags USE Flags is the beauty of Gentoo, it offers the possibility to streamline the system, helping to keep it simple and maximize efficiency. But its configuration is not a quick fix. I tend to set fewer global USE Flags, and local USE Flags for each package.\nThis is a result of a certain amount of maintenance time, but at the beginning you can also configure -qt4 -qt5 -kde to avoid some unnecessary dependencies. In addition, the USE Flags page in the Handbook is a good place to get a deeper understanding.\nManually compiling the kernel Choosing genkernel is a relatively painless way to compile the kernel, but it means that the kernel will not be able to turn on AMD Ryzen™ related parameters. A conservative approach would be to genkernel all and then add the relevant kernel parameters and recompile the kernel once more. The disadvantage of this approach is that it is time and resource consuming, and the compiled kernel is not small and elegant.\nStart by installing the AMD Zen microcode via emerge linux-firmware.\n1 emerge --ask sys-kernel/linux-firmware Don\u0026rsquo;t forget to install AMD GPUs drivers.\n1 emerge --ask x11-drivers/xf86-video-amdgpu Use menuconfig to enable several options mentioned below to provide support for AMD Ryzen™ CPU and AMD GPUs, as well as kernel options related with EFI stub kernel.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 Processor type and features ---\u0026gt; [*] Symmetric multi-processing support [*] AMD ACPI2Platform devices support [*] /dev/cpu/microcode - microcode support [*] AMD microcode patch loading support [*] MTRR (Memory Type Range Register) support Processor family (Opteron/Athlon64/Hammer/K8) ---\u0026gt; (X) Opteron/Athlon64/Hammer/K8 [*] Supported processor vendors ---\u0026gt; [*] Support AMD processors (NEW) [*] SMT (Hyperthreading) scheduler support [*] Multi-core scheduler support [*] Machine Check / overheating reporting [*] AMD MCE features Processor family (MZEN) ---\u0026gt; (X) AMD Zen Performance monitoring ---\u0026gt; \u0026lt;*\u0026gt; AMD Processor Power Reporting Mechanism [*] AMD microcode loading support [*] EFI runtime service support [*] EFI stub support [ ] EFI mixed-mode support [*] Built-in kernel command line (root=PARTUUID=) Power management and ACPI options ---\u0026gt; CPU Frequency scaling ---\u0026gt; \u0026lt;*\u0026gt; AMD Opteron/Athlon64 PowerNow! \u0026lt;*\u0026gt; AMD frequency sensitivity feedback powersave bias Device Drivers ---\u0026gt; Generic Driver Options ---\u0026gt; (amd-ucode/microcode_amd_fam17h.bin) External firmware blobs to build into the kernel binary (/lib/firmware) Firware blobs root directory Graphics support ---\u0026gt; \u0026lt;*/M\u0026gt; Direct Rendering Manager (XFree86 4.1.0 and higher DRI support) ---\u0026gt; [*] Enable legacy fbdev support for your modesetting driver \u0026lt;\u0026gt; ATI Radeon AMD GPU [ /*] Enable amdgpu support for SI parts (only needed for Southern Islands GPUs with the amdgpu driver) [ /*] Enable amdgpu support for CIK parts (only needed for Sea Islands GPUs with the amdgpu driver) [*] Enable AMD powerplay component ACP (Audio CoProcessor) Configuration ---\u0026gt; [*] Enable AMD Audio CoProcessor IP support (CONFIG_DRM_AMD_ACP) Display Engine Configuration ---\u0026gt; [*] AMD DC - Enable new display engine [ /*] DC support for Polaris and older ASICs (only needed for Polaris, Carrizo, Tonga, Bonaire, Hawaii) [ /*] AMD FBC - Enable Frame Buffer Compression [ /*] DCN 1.0 Raven family (only needed for Vega RX as part of Raven Ridge APUs) \u0026lt;*/M\u0026gt; HSA kernel driver for AMD GPU devices \u0026lt;*/M\u0026gt; Sound card support ---\u0026gt; \u0026lt;*/M\u0026gt; Advanced Linux Sound Architecture ---\u0026gt; [*] PCI sound devices ---\u0026gt; HD-Audio ---\u0026gt; \u0026lt;*\u0026gt; HD Audio PCI [*] Support initialization patch loading for HD-audio \u0026lt;*\u0026gt; whatever audio codec your soundcard needs \u0026lt;*\u0026gt; Build HDMI/DisplayPort HD-audio codec support (2048) Pre-allocated buffer size for HD-audio driver Firmware Drivers ---\u0026gt; EFI (Extensible Firmware Interface) Support ---\u0026gt; \u0026lt;*\u0026gt; EFI Variable Support via sysfs [*] IOMMU Hardware Support ---\u0026gt; [*] AMD IOMMU support \u0026lt;*\u0026gt; AMD IOMMU Version 2 driver -*- Enable the block layer ---\u0026gt; Partition Types ---\u0026gt; [*] Advanced partition selection [*] EFI GUID Partition support Built-in kernel command line in Processor type and features needs to be filled in with paths to /root and initrd, like\n1 root=/dev/nvme0n1p3 rw initrd=/EFI/Gentoo/initramfs-linux.img About Ryzen, AMDGPU and AMD_microcode of the chapters in the Handbook provide a great deal of help for working on the content of this section.\nCreating fstab file If following configuration for EFISTUB above, state the binding mounts on the EFI partition in fstab file. Here is an example:\n1 2 3 4 /dev/nvme0n1p1 /esp vfat defaults,noatime 0 0 /esp/EFI/Gentoo /boot none defaults,bind 0 0 /dev/nvme0n1p2 none swap sw 0 0 /dev/nvme0n1p3 / btrfs ssd,noatime 0 1 It is recommended to use blkid to get the respective UUID of the file system to replace its partition or device name, remembering to add the UUID= prefix. If the SSD supports TRIM commands, you can add the mount parameter discard to improve its read/write performance. If you are using Btrfs as a file system, you can add the mount parameter ssd, but according to the Btrfs Wiki on [optimization for SSD](https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_ optimized_for_SSD.3F), adding the ssd parameter will not support TRIM/discard, and the official recommendation is to use ssd for better compatibility.\nAdd UEFI boot portal For the UEFI boot portal, the recommendation of the Gentoo Handbook and other distribution wikis is to use efibootmgr to manage it. efibootmgr is really simple and convenient, but it does not recognize nvme device partition information, e.g. /dev/nvme0n1p1 is an illegal disk parameter. Since efibootmgr has a fondness for sda, I added the entry manually using the Shell according to the section UEFI Shell.\nInstalling wireless network related packages After configuring the network according to the Handbook, don\u0026rsquo;t forget to install the wireless network related packages, it won\u0026rsquo;t be elegant to look around for a network cable and crouch by the router 😎.\n1 emerge --ask net-wireless/iw net-wireless/wpa_supplicant Conclusion That said, I probably wouldn\u0026rsquo;t have gotten on board with Gentoo if AMD Ryzen™ hadn\u0026rsquo;t given me a good reason to do so. Either way, thanks for the wonderful experience. Stay tuned for the next few months 💗\n","permalink":"https://ethans.me/posts/2018-08-15-the-best-practice-for-installing-gentoo-linux-on-amd-ryzen-mobile/","summary":"Gentoo Linux should work well on brand new AMD Ryzen™ Mobile platforms. Yes, it does but it requires some extra magic.","title":"The Best Practice for Installing Gentoo Linux on AMD Ryzen™ Mobile"},{"content":"This is an API based on the online ordering system owned by Shenzhen Experimental School.\nDevelopers can bypass the originally unnecessary but unavoidable value-passing codes, free themselves from the complicated and incomprehensible logic, and conduct online ordering interaction through the more simplistic interfaces of the packaged szsy-canteen API.\nszsy-canteen API is written in Golang which can ensure its efficiency and performance. The API is suitable for the secondary development of the online ordering system 💗\nEnglish | 简体中文 | 正體中文 Why use this API In short, the API is designed to be simple, data-saving and efficient.\nFor the original intention was aimed at reducing the cost of mobile application development of the school online ordering system, the logic-optimization and data-saving were regard as important indicators in design of the API. Here is an example. The API provides an interface for obtaining a list of \u0026ldquo;orderable dates\u0026rdquo; directly, which save the overhead of operations such as \u0026ldquo;gaining all dates including unnecessary ones\u0026rdquo; (waste of data traffic) and \u0026ldquo;judging whether the dates can be ordered\u0026rdquo; (complex logic).\nOf course, some parts of the API is roughly-designed because the developing time is limited and my lack for programming ability. Welcome new contribution, and hope that you can let me know if you made a new implementation of the ordering system and I will update your repo link in the README 😄\nHow to configure If you are a developer on 64-bit Windows or Linux, you can download the pre-compiled package of your specific system. Double-click to execute it and the API will be enabled on port 2018 on the local machine by default.\nIf you attempt to compile it and run it yourself, you need to install the developing environment required by Golang, do necessary configurations and download the complete source files to compile. You can execute the following command to generate an executable file for your platform.\n1 go build main.go Get started The szsy-canteen API provides the following interfaces:\nlogin : Login to the online ordering system to get basic information and the key for further operations\ndates : Obtain the list of all the dates that can be ordered currently (i.e. the dates before the order deadline)\nmenu : Get details and order status of the menu for the specific date\norder : Submit the order request for the specific date\nlogout : Log out of the online ordering system\nThe URL of the API in default configuration should be\n1 http://localhost:2018/ Login to online ordering system Write username and password encrypted with the MD5 algorithm to the JSON field. Here is an example.\n1 2 3 4 { \u0026#34;账号\u0026#34;:\u0026#34;2152000\u0026#34;, \u0026#34;密码\u0026#34;:\u0026#34;80d9ba95cce518bf747bda3bc98faef8\u0026#34; } For security reasons, the encrypting process of the password with MD5 is handed over to the requester on design, that is, you need to encrypted password string yourself and put it into the 密码(password) item.\nSubmit to URL with POST method\n1 http://localhost:2018/login The API will return a JSON as follows\n1 2 3 4 5 { \u0026#34;姓名\u0026#34;:\u0026#34;XXX\u0026#34;, \u0026#34;余额\u0026#34;:\u0026#34;666.66元\u0026#34;, \u0026#34;口令\u0026#34;:\u0026#34;6E19822908B2FEA56F7BF,bnfh4fsigr5z04ebdeo\u0026#34; } In fact, the value of the 口令(the key) will be much longer than the one in the example. Here, the key has been shortened for typography.\nObtain orderable dates list Write the key returned at login to the JSON field\n1 2 3 { \u0026#34;口令\u0026#34;:\u0026#34;6E19822908B2FEA56F7BF,bnfh4fsigr5z04ebdeo\u0026#34; } Submit to URL with POST method\n1 http://localhost:2018/dates You will get return data as follows\n1 2 3 4 5 { \u0026#34;可订日期\u0026#34;:[\u0026#34;2018-06-26\u0026#34;,\u0026#34;2018-06-27\u0026#34;,\u0026#34;2018-06-28\u0026#34;,\u0026#34;2018-06-29\u0026#34;, \u0026#34;2018-07-02\u0026#34;,\u0026#34;2018-07-03\u0026#34;,\u0026#34;2018-07-04\u0026#34;,\u0026#34;2018-07-05\u0026#34;, \u0026#34;2018-07-06\u0026#34;,\u0026#34;2018-07-09\u0026#34;,\u0026#34;2018-07-10\u0026#34;,\u0026#34;2018-07-13\u0026#34;] } The date list will contain all orderable dates, i.e. the API will also try requesting menus for the next month without missing orderable dates in the next month. The school updates at most two months\u0026rsquo; menus once. You can leave the job to the szsy-canteen API at ease.\nObtain the specific date menu Similarly, write the key to the JSON field.\n1 2 3 { \u0026#34;口令\u0026#34;:\u0026#34;6E19822908B2FEA56F7BF,bnfh4fsigr5z04ebdeo\u0026#34; } Submit to the URL with the \u0026ldquo;specific date\u0026rdquo; parameter with the POST method\n1 http://localhost:2018/menu/?date=2018-07-06 You can get return JSON as follows\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 { \u0026#34;不订餐\u0026#34;:null, \u0026#34;早餐\u0026#34;: [ [\u0026#34;0\u0026#34;,\u0026#34;套餐\u0026#34;,\u0026#34;早餐套餐\u0026#34;,\u0026#34;5.00\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;1\u0026#34;,\u0026#34;牛奶\u0026#34;,\u0026#34;学生奶\u0026#34;,\u0026#34;2.04\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;2\u0026#34;,\u0026#34;蛋类\u0026#34;,\u0026#34;五香鸡蛋\u0026#34;,\u0026#34;1.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;3\u0026#34;,\u0026#34;点心\u0026#34;,\u0026#34;巧克力面包\u0026#34;,\u0026#34;1.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;4\u0026#34;,\u0026#34;点心\u0026#34;,\u0026#34;玉米酥\u0026#34;,\u0026#34;1.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;5\u0026#34;,\u0026#34;点心\u0026#34;,\u0026#34;南瓜米糕\u0026#34;,\u0026#34;1.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;6\u0026#34;,\u0026#34;点心\u0026#34;,\u0026#34;鲜肉包\u0026#34;,\u0026#34;1.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;7\u0026#34;,\u0026#34;粉面类\u0026#34;,\u0026#34;水饺\u0026#34;,\u0026#34;1.60\u0026#34;,\u0026#34;0\u0026#34;] ], \u0026#34;午餐\u0026#34;: [ [\u0026#34;0\u0026#34;,\u0026#34;套餐\u0026#34;,\u0026#34;午餐套餐\u0026#34;,\u0026#34;12.00\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;1\u0026#34;,\u0026#34;水果\u0026#34;,\u0026#34;杨桃\u0026#34;,\u0026#34;1.30\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;2\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;蒜茸炒黄芽白\u0026#34;,\u0026#34;1.80\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;3\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;炒小瓜片\u0026#34;,\u0026#34;1.80\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;4\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;剁椒蒸鱼(微辣)\u0026#34;,\u0026#34;4.40\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;5\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;肉末豆角\u0026#34;,\u0026#34;3.60\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;6\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;土豆烧肉\u0026#34;,\u0026#34;5.60\u0026#34;,\u0026#34;0\u0026#34;], [\u0026#34;7\u0026#34;,\u0026#34;菜肴\u0026#34;,\u0026#34;香辣翅根2个\u0026#34;,\u0026#34;6.70\u0026#34;,\u0026#34;0\u0026#34;] ], \u0026#34;晚餐\u0026#34;:null } Each dish includes the following information points\n1 [\u0026#34;编号\u0026#34;,\u0026#34;类型\u0026#34;,\u0026#34;菜名\u0026#34;,\u0026#34;单价\u0026#34;,\u0026#34;订餐个数\u0026#34;] The item of 不订餐(not order) should be a null value in general, as the example above. Of course, if you choose not to order 早餐(breakfast) or 午餐(lunch) , then the value of 不订餐(not order) will supposed to be\n1 \u0026#34;不订餐\u0026#34;:[\u0026#34;早餐\u0026#34;,\u0026#34;午餐\u0026#34;] July 10, 2018 is Friday in this example, so the menu for dinner is missing, i.e. the value of 晚餐(dinner) is null\nSubmit an order request Write the key obtained at login and the order information for breakfast, lunch and dinner to the JSON field.\n1 2 3 4 5 6 { \u0026#34;口令\u0026#34;:\u0026#34;6E19822908B2FEA56F7BF,bnfh4fsigr5z04ebdeo\u0026#34;, \u0026#34;早餐\u0026#34;:\u0026#34;套餐\u0026#34;, \u0026#34;午餐\u0026#34;:\u0026#34;0,0,2,0,0,1,0,1\u0026#34;, \u0026#34;晚餐\u0026#34;:\u0026#34;不订餐\u0026#34; } There are three formats for order information, and the example has given all the cases.\n套餐(set menu) (i.e. equivalent to \u0026ldquo;1,0,0,0,0,0,0,0\u0026rdquo;)\n不订餐(not order) (i.e. equivalent to check the \u0026ldquo;not order\u0026rdquo; checkbox)\nWrite the number you would like to order of eight dishes with commas as separators\nTherefore, in this case, this student chose a \u0026ldquo;套餐\u0026rdquo;(set menu) for his breakfast. For lunch, he chose 2 servings of \u0026ldquo;蒜茸炒黄芽白\u0026rdquo;, 1 serving of \u0026ldquo;肉末豆角\u0026rdquo; and 1 serving of \u0026ldquo;香辣翅根2个\u0026rdquo;, and choose \u0026ldquo;不订餐\u0026rdquo;(not order) for dinner.\nOn Friday night, i.e. the menu for dinner is missing, you should serve the 晚餐(dinner) item a null value instead of omitting the 晚餐(dinner) item. Here, in order to show the 不订餐(not order) field, 晚餐(dinner) is filled in with it, but it should be null 😂\nYou may find that the first dish in the eight dishes is 套餐(set menu). In fact, if one of the dishes following is not 0, the value of 套餐(set menu) will be covered by 0 regardless of what the value is.\nThen use the POST method to submit to the URL with the \u0026ldquo;specific date\u0026rdquo; parameter\n1 http://localhost:2018/order/?date=2018-07-06 You may get types of return information\n提交成功 (the request is submitted successfully, and the order information is written to the system)\n账户异常 (this account is frozen or stuck in other abnormal situations, often means in arrear)\n超过订餐时间 (the date is over the order deadline, you can avoid this error by checking whether the date in the \u0026ldquo;orderable date\u0026rdquo; list before submitting)\n提交失败 (failed for the reason that network timeout or school server goes down, etc.)\nThe time it takes for the school server to deal with the order information is far more than the time needed normally, so you usually have to wait a few seconds to get feedback (students in SZSY should be impressed) Do not submit too often, you might fail, and even worse, get the wrong return information.\nLogout online ordering system Write the key obtained at login to the JSON field\n1 2 3 { \u0026#34;口令\u0026#34;:\u0026#34;6E19822908B2FEA56F7BF,bnfh4fsigr5z04ebdeo\u0026#34; } Submit to URL with POST method\n1 http://localhost:2018/logout You will logout after reading 登出成功.\nError about key expiration szsy-canteen API error messages are fairly legible, but there is a special explanation for this.\nIf you are using a key to request dates list, obtain menu, or submit request with receiving 口令错误或过期 error message, it means your key may have expired. In this case, all you need to do is re-login to get the new key.\nTips for ordering multiple servings The school online ordering system limits 3 servings for each dish maximum, which is conducted by the JavaScript script on the website. However, there is no such restriction if using the API. In theory, you can order more than 3 servings.\nThe practical tests show that the success rate of 5 servings or less is relatively higher, and the submissions will be accepted in most cases. If the number is too exaggerated, the success rate falls to zero basically, and there will be a 提交失败(submit failed) error returning. The maximum number which successfully submitted I have tried is 20, yes, 20 servings of 卤鸡腿.\nOf course, this is not a bug. 20 servings of 卤鸡腿 means you have to pay them as 20 servings (I won\u0026rsquo;t state this here if not so)\nThis text added here is a piece of advice: As a developer, you can offer an entry that can modify the number of servings in an explicit way when developing the client. After all, adding a dish of \u0026ldquo;净荤菜\u0026rdquo; occasionally is also a sort of pleasure.\nAcknowledgement @xlfjn\n@GrakePch\nThanks to the above two for your technical and moral supports during development 💗\n","permalink":"https://ethans.me/posts/2018-07-20-szsy-canteen-api-readme/","summary":"This is API for Shenzhen Experimental School Canteen Online Ordering System.","title":"szsy-canteen-API README"},{"content":"It seems that Windows Subsystem for Linux (WSL) is getting much more mature than the time when it first came out. Fewer and fewer people keep questioning its fitness for daily development. Tuning WSL to be a neat developing environment and how to SSH into WSL the topic I am going to share with.\nOne-Click installation The installation entry of several Linux distributions available for WSL has been moved to Microsoft Store since the Fall Creators.\nIt means you can install a Linux distribution as simple as installing an app from the store. The moment I am writing this, Ubuntu, Debian, OpenSUSE and Kali Linux are officially collected into the WSL family.\nI choose Debian as my first trial for WSL.\nIf nothing goes wrong, the familiar icon will show up in the start menu after the installation. Click it, boot it on and create your Linux user account.\nAs you can see in the Task Manager, required few seconds to get ready, WSL have very small demand for resource and please don\u0026rsquo;t forget that it is able to offer almost full Linux experience to you. So that is pretty good enough as a subsystem.\nBy the way, this command can lead you to your C:/.\n1 cd /mnt/c Similarly, /mnt/d for D:/ and so on.\nSimple configuration You can tweak it in the same way as you play with Linux as usual when you have got this far. The following are things I have done just after the installation.\nUpdate sources.list Replacing stable to testing will let your Debian have access to the latest version of software updates.\n1 sudo nano /etc/apt/sources.list Configure locale Select your language to add UTF8 locales support by doing this.\n1 sudo dpkg-reconfigure locales Create a shortcut on the desktop The path to Debian(WSL) root follows the format:\n1 C:\\Users\\username\\AppData\\Local\\Packages\\TheDebianProject.DebianGNULinux_xxxxxxxxxxxxx\\LocalState\\rootfs However, do not move or delete or do anything else on files in WSL with Windows File Explorer which might mess up the file permission on Linux File System. That causes tons of problems. Here it can only be used as a single-function graphical File Explorer.\nSSH to WSL Considering that it is a kind of awkward experience to run bash in CMD, so here I introduce MobaXterm to you. Next, we are about to build up an SSH server on WSL and pair it with MobaXterm through SSH connection.\nInstall OpenSSH server OpenSSH is the premier connectivity tool for remote login with the SSH protocol.\n1 sudo apt-get install openssh-server Edit the configuration and change the listening port for the reason that Port 22 as the default setting has been already taken by SSH service running on Windows.\n1 sudo nano /etc/ssh/sshd_config To make sure SSH server on WSL works well, change Port 22 into Port 2018 or whatever you prefer.\n1 2 3 Port 2018 ListenAddress 0.0.0.0 PermitRootLogin prohibit-password the above is the scanty lines you have to deal with in the sshd_config. Then start the SSH service.\n1 sudo service ssh start Use MobaXterm to SSH MobaXterm has not only SSH support but also full X-Server support if you need. Two editions (Home Edition and Professional Edition) are provided with free one and paid one.\nI download MobaXterm Home Edition (Portable edition) which already fulfils my demand.\nCreate a new session and edit its configuration. Remote host is set as 127.0.0.1. Remember that Port should be 2018 or the same port you decided several minutes ago. You can tweak other settings catered for your appetite.\nIf your Debian is running well with SSH enabled, it must work. Select Remember the Password to save the time you type in your password whenever booting the WSL.\nNow enjoy WSL with the excellent terminal experience supplied by MobaXterm.\nAuto-boot WSL with SSH started It is so unpleasant to manually start SSH service each time you turn WSL on and the bot to boot WSL at the same time as you log in Windows can be the thing what you want.\nAlso, you will soon notice that the SSH connection goes bad when you shut down the CMD window which is quite bothering.\nHere comes the solution to the three so-called problems.\nAuto-start SSH service Write a shell script named init.sh placed in /home.\n1 2 3 sudo -S service ssh start \u0026lt;\u0026lt; EOF \u0026lt;your_password\u0026gt; EOF Replace \u0026lt;your_password\u0026gt; to your root password.\nThe object of the tiny script is to start SSH service in sudo mode when you execute this shell.\nWrite a VBS script on your Windows host.\n1 2 Set ws = CreateObject(\u0026#34;Wscript.Shell\u0026#34;) ws.run \u0026#34;bash /home/init.sh\u0026#34;,vbhide To mention that, WSL can be start up by typing bash in CMD or Powershell. The script is so self-explain that it will obviously run the init.sh as well after booting WSL.\nvbhide at the end of the line tells that it runs silently in the background.\nActually, it doesn\u0026rsquo;t matter that what sort of name you give to the shell above and where you place it. Just to be sure the path-to-the-shell has to be identical in two scripts.\nAuto-boot WSL in background It is much easier to achieve this goal since the big steps have got settled in the previous move.\nSet the VBS script into the scheduled tasks performing when Windows host logins.\nReboot Windows and keep an eye on the Task Manager. if two processes named init (depends on the name of the shell) and sshd exist, then you can immediately start MobaXterm to SSH into WSL without any other keyboard job.\nJob done Now, the complete WSL with SSH workflow has been set up.\nThanks for reading :)\n","permalink":"https://ethans.me/posts/2018-06-10-building-ssh-for-development-on-windows-subsystem-for-linux/","summary":"A simple guide to building SSH for development on Windows Subsystem for Linux. Let\u0026rsquo;s do something on \u0026ldquo;Microsoft Linux\u0026rdquo;","title":"Building SSH for Development on Windows Subsystem for Linux"},{"content":"This is Ethan Shen.\nFriends GrakePch\nUI designer \u0026amp; Minecraft content maker ","permalink":"https://ethans.me/about/","summary":"This is Ethan Shen.\nFriends GrakePch\nUI designer \u0026amp; Minecraft content maker ","title":"About Me"}]