Bash and Linux Tool

工具

标准输入输出

文件系统

1. 工作目录(working directory)

   Command	Description
   pwd ls cd
   du	du -sh * display disk usage statistics
   df	df -h /mnt display free disk space

2. 查找文件

   Command	Description
   find	find . -type f 找到本目录下所有的文件并输出到屏幕 (包括子目录，下同) -name: find . -name "*.jpg" -maxdepth: find -maxdepth 1

3. 文件操作（移动、拷贝、创建文件夹）

   Command	Description
   cp mv rm mkdir rmdir	在~/.bashrc文件中，加入下面的命令： alias cp='cp -i' alias rm='rm -i' alias mv='mv -i' -i: 如果命令会删除或者覆盖文件，会提示用户confirm。
   chmod	chmod a-w somefile 把somefile的所有写权限去掉
   ln	ln -s $src $dest 创建一个软链接$dest, 它指向的原文件在$src

文件内容

1. 查看文件

   Command	Description
   cat	cat <file>
   head tail	head -n <file>
   less more	less是more的升级版，推荐使用less
   diff	diff file1 file2
   md5sum	md5sum <file> 计算文件的md5码，用来比较本机和远程文件是否相同

2. 文件内容操作

   Command	Description
   cut	cut -f1,3 file1 cut out selected portions of each line of a file -b 按字节 -f显示某几列
   bc	echo "1.212*3" | bc 任意精度的计算器
   wc	find . -name "*.jpg" | wc -l 数出本目录下所有的jpg文件数量
   sort	sort a.list > a.sort 对文件a.list排序并写到a.sort -u: 去重 -k: sort -k2,2 a.list, 按第2列排序 -n: 按数字排序 -V:可以理解为加强版的按数字排序 -r: 倒序 -R: 随机
   uniq	cat list1 list2 | sort | uniq > final.list 将list1, list2合并，排序后去重，写到final.list (其实sort一个命令就够了) -c: 用来统计很方便 -u -d
   sed	sed "s@origin@corrected@" a.list -i 将a.list中每行出现的第一个origin替换为corrected -i: inplace change sed -i "s/old/new/g" `grep old -rl /www` 将/www路径下递归的所有文件中包含old字符串的文件中的old都替换成new
   awk	awk -F'_' '{print $1}' a.list 将a.list中每行按'_'分隔，每行输出分隔后的第一个字符串，例如1_2_3-> 1
   xargs	find . -name "*.jpg" | xargs -i rm {} find . -name "*.jpg" | xargs rm 找到本目录下所有的jpg文件并删除 -P: 多进程 -n: 一般配合"-P"参数
   comm	comm -23 a.sort b.sort 输出在a中，不在b中的行
   grep	cat a.list | grep png 找到a.list中包含png的行 -R 递归查找 -n 列出行号 -l 列出匹配的文件名 -a -o -i不区分大小写 grep "abc" . -nr 查找当前路径下含有“abc”的文件，列出文件名，行号和匹配内容 grep "abc" . -lr 查找当前路径下含有“abc”的文件，列出文件名
   rev	rev <file> find . -type f | rev | cut -d'/' -f1 | rev # revcut'/' reverse the file(or reverse the stdin if no file provided)

系统相关

网络相关

常用工具

Linux Package Manager

Bash脚本语法

"#！": shebang。使用chmod +x file使sh脚本文件可以运行。

#!/bin/bash

变量

1. 基本用法

   des="awe and cool"
   echo $des
   echo ${des}
   echo "This is ${des}"
   

2. 内置变量

   $0, $1, $2...: 脚本参数

3. 输出赋值

   ret=`cat readme.txt`
   echo $ret
   
   ret=$(cat readme.txt)
   echo $ret
   

4. 运算

• let:

  let "a = 5 + 6"
  echo $a
  
  x=11
  y=22
  let "z = x * y"
  echo $z
  

• expr:

  $ expr 5 + 6
  11
  
  $ foo=$(expr 5 + 6)
  $ echo $foo
  11
  

  注意：运算符和各个数字之间需要有空格

• 双括号

  a=$((5+6))
  

5. 判断语句

   if <condition>
   then
       <cmd>
   else
       <cmd>
   fi
   

   tmp=35
   if [ ${tmp} -ge 30 ]
   then
       echo "It's too hot"
   fi
   

| Operator       | Description  |
| :------------- |:-------------|
|`! EXPRESSION`| The EXPRESSION is false.
|`-n STRING`| The length of STRING is greater than zero.
|`-z STRING`|The lengh of STRING is zero (ie it is empty).
|`STRING1 = STRING2`| STRING1 is equal to STRING2
|`STRING1 != STRING2`| STRING1 is not equal to STRING2
|`INTEGER1 -eq INTEGER2`| INTEGER1 is numerically equal to INTEGER2
|`INTEGER1 -gt INTEGER2`| INTEGER1 is numerically greater than INTEGER2
|`INTEGER1 -lt INTEGER2`| INTEGER1 is numerically less than INTEGER2
|`-d FILE`| FILE exists and is a directory.
|`-e FILE`| FILE exists.
|`-r FILE`| FILE exists and the read permission is granted.
|`-s FILE`| FILE exists and it's size is greater than zero (ie. it is not empty).
|`-w FILE`| FILE exists and the write permission is granted.
|`-x FILE`| FILE exists and the execute permission is granted.

test命令：
```bash
$ test 5 -ge 4
$ echo $?
0
```
test命令的结果，0表示true或命令运行成功；1表示false或运行失败。

6. 布尔运算

   &&, ||

   if [ $code_review = "pass" ] && [ $regression_test = "pass" ]
   then
   echo "ship it!"
   fi
   

7. 循环语句

   for var in <list>
   do
       <cmd>
   done
   

   for name in John, Emma, Tom
   do
       echo "My name is $name"
   done
   
   for num in {1..10}
   do
       echo "count $num"
   done
   
   for ((i=0; i<10; i++))
   do
       echo "count $i"
   done
   

8. until语句

   until [ some_test ]
   do
       <cmd>
   done
   

   suffix=1
   until [ ! -e "foo${suffix}" ]
   do
   let suffix++
   done
   echo "The file name foo${suffix} is good.
   

9. break

10. continue

11. 函数

function_name () {
<commands>
}

discuss_langauges() {
 language1=$1
 language2=$2
 echo "$1 is a good language."
 echo "$2 is also a good language."
}

discuss_langauges c++ python

调用不需括号

find_cpp_files() {
 local folder=$1
 local ret=$(find ${folder} -name '*.cpp' | wc -l)
 return $ret
}
folder=/home/hchen/code/ficus/common/machine_learning
find_cpp_files $folder
num_files=$?
echo "There are ${num_files} cpp files in ${folder}"

调试技巧

• set -e
• set -x

实用案例

1. Test if a comand outputs an empty string:
   if [[ $(YOUR_COMMAND)]]

# ls directory
if [[ $(ls -A) ]]; then
    echo "there are files"
else
    echo "no files found"
fi

# grep file context 
if [[ $(cat file | grep key_word) ]]; then
  echo "find key_word in the file"
then 
  echo "Not found"
fi

2. Check whether the last command complete successfully.
   $?: the exit code of the last command (zero for success, non-zero for failure).

files=$(ls -A)
if [[ $? != 0 ]]; then
    echo "Command failed."
elif [[ $files ]]; then
    echo "Files found."
else
    echo "No files found."
fi