Filesystem Hierarchy Standard (FHS) "

FHS actually talks about 3 major portions of Linux File System

root boot process and system management related
usr stands for Unix software resource, i.e. most of the program will be installed and configured here
var variable, system running related files

static and variable, share and unshare.png

Root Directory - Starts From "/"

reference: http://cn.linux.vbird.org/linux_basic/0210filepermission.php#dir_fhs
some of the refs have been deprecated or obsoleted

事实上FHS针对根目录所定义的标准就仅有上面的咚咚，不过我们的Linux底下还有许多目录你也需要了解一下的。 底下是几个在Linux当中也是非常重要的目录喔：

除了这些目录的内容之外，另外要注意的是，因为根目录与开机有关，开机过程中仅有根目录会被挂载， 其他分割槽则是在开机完成之后才会持续的进行挂载的行为。就是因为如此，因此根目录下与开机过程有关的目录， 就不能够与根目录放到不同的分割槽去！那哪些目录不可与根目录分开呢？有底下这些：

/etc：配置文件
/bin：重要执行档
/dev：所需要的装置文件
/lib：执行档所需的函式库与核心所需的模块
/sbin：重要的系统执行文件

• Devices and Naming Convention
  Since all entities include devices are all files within Linux System, therefore, the rules for naming convention

Things need to be aware:

1. The physical slots != device naming order

   /dev/sda  != the 1st slot of your harddisk
   

2. The number of device label != total amount of devices

   /dev/sda3   !=  there are 3 partitions in total, 
   it could be the third partition and omit the 1st and 2nd partition since the number can be assigned manually 
   

image.png

• Hardisk consist of tons of sectors, each sector has 512 bytes
• The first sector is one of the most important sectors for which it contains Master Boot Record(MBR) and partition table
• MBR takes 446 bytes, and the rest 66 bytes (partition table takes 64 bytes, terminator takes 2 bytes
• The 64 bytes can be divided into 4 partitions with each of 16 bytes. If there are more than 4 partitions then we use one 16 bytes with single linked table pointer to create an extended partition, from there the logical partition can be generated.

So Basically the system can recognize 3 major partitions at most and then a limited number of logical partitions

image.png

Filesystem

EXT2 earlier file system, it was born in 1993, not safe, obsoleted
EXT3 journaling file system, it allows data recovery but the speed is slow
EXT4 improved version of EXT3, rhel6 default file system, support 1EB(1,073,741,824GB) at most, unlimited sub-directories, assign blocks in batches(improved performance for reading and writing), etc...
XFS high-performance journaling file system, support 18EB at most

Important terms:

参考 http://linux.vbird.org/linux_basic_train/unit06.php

• superblock record of the entire structure of the file system, including inode/block amount, file system format, relative information, etc...

superblock 為整個檔案系統的總結資訊處，要讀取檔案系統一定要從 superblock 讀起。superblock 主要紀錄資料為：
block 與 inode 的總量；
未使用與已使用的 inode / block 數量；
block 與 inode 的大小 (block 為 1, 2, 4K，inode 為 128bytes 或 256bytes)；
filesystem 的掛載時間、最近一次寫入資料的時間、最近一次檢驗磁碟 (fsck) 的時間等檔案系統的相關資訊；
一個 valid bit 數值，若此檔案系統已被掛載，則 valid bit 為 0 ，若未被掛載，則 valid bit 為 1

• block the actual data, if the data is too large, then one file will take multiple blocks

檔案實際的資料存放在 data block 上面，每個 block 也都會有號碼，提供給檔案來儲存實際資料，也讓 inode 可以紀錄資料放在哪個 block 號碼內。

原則上，block 的大小與數量在格式化完就不能夠再改變了(除非重新格式化)；
每個 block 內最多只能夠放置一個檔案的資料；
承上，如果檔案大於 block 的大小，則一個檔案會佔用多個 block 數量；
承上，若檔案小於 block ，則該 block 的剩餘容量就不能夠再被使用了(磁碟空間會浪費)。

一般來說，檔案系統內的一個檔案被讀取時，流程是這樣的：

讀到檔案的 inode 號碼
由 inode 內的權限設定判定使用者能否存取此檔案
若能讀取則開始讀取 inode 內所紀錄的資料放置於哪些 block 號碼內
讀出 block 號碼內的資料，組合起來成為一個檔案的實際內容。
至於新建檔案的流程則是這樣的：

有寫入檔案的需求時，先到 metadata 區找到沒有使用中的 inode 號碼
到該 inode 號碼內，將所需要的權限與屬性相關資料寫入，然後在 metadata 區規範該 inode 為使用中，且更新 superblock 資訊
到 metadata 區找到沒有使用中的 block 號碼，將所需要的實際資料寫入 block 當中，若資料量太大，則繼續到 metadata 當中找到更多的未使用中的 block 號碼，持續寫入，直到寫完資料為止。
同步更新 inode 的紀錄與 superblock 的內容。
至於刪除檔案的流程則是這樣的：

將該檔案的 inode 號碼與找到所屬相關的 block 號碼內容抹除
將 metadata 區域的相對應的 inode 與 block 號碼規範為未使用。
同步更新 superblock 資料。

• inode record of the file property, it contains inode number, block number, etc...

每一個 inode 都有號碼，而 inode 的內容在記錄檔案的屬性以及該檔案實際資料是放置在哪幾號 block 內。 inode 記錄的檔案資料至少有底下這些：

該檔案的存取模式(read/write/excute)；
該檔案的擁有者與群組(owner/group)；
該檔案的容量；
該檔案建立或狀態改變的時間(ctime)；
最近一次的讀取時間(atime)；
最近修改的時間(mtime)；
定義檔案特性的旗標(flag)，如 SetUID...；
該檔案真正內容的指向 (pointer)；
由於每個檔案固定會佔用一個 inode，而目前檔案所記載的屬性資料越來約多，因此 inode 有底下幾個特色：

每個 inode 大小均固定為 128 bytes (新的 ext4 與 xfs 可設定到 256 bytes)；
每個檔案都僅會佔用一個 inode 而已；
承上，因此檔案系統能夠建立的檔案數量與 inode 的數量有關；
系統讀取檔案時需要先找到 inode，並分析 inode 所記錄的權限與使用者是否符合，若符合才能夠開始實際讀取 block 的內容。

http://linux.vbird.org/linux_basic_train/unit06.php

image.png

Virtual File System let the user manipulate files without considering in detail how the underlying structure works, i.e. whatever the operating system you are working with, surely there will be a command like cp or copy that allow you to copy the file in your file system.

Mounting File System

Mount command allows the user to hook the device file with the kernel, therefore the device then can be read, written, and executed.

[root@linuxprobe media]# mount /dev/sda1 /media/disks/oneG
[root@linuxprobe oneG]# cd /media/disks/oneG/
[root@linuxprobe oneG]# tree
.
└── lost+found

1 directory, 0 files

[root@linuxprobe oneG]# vim /etc/fstab

# user defined
/dev/sda1               /media/disks/oneG       etx4    defaults        0 0

In Linux system, to mount a file system you cannot count on device name + number in all situation since the name could vary, rather, a UUID often a better choice for relibility

What is UUID

A universally unique identifier (UUID) is a 128-bit number used to identify information in computer systems. The term globally unique identifier (GUID) is also used, typically in software created by Microsoft.

image.png

# user defined
# /dev/sda1                                   /media/disks/oneG       etx4    defaults        0 0
UUID=488f3d14-6582-4981-b734-d86b6c49db94     /media/disks/oneG       etx4    defaults        0 0

[root@linuxprobe oneG]# sync;
[root@linuxprobe oneG]# reboot

To make the fstab take effect, we need to run the following command

mount -a   # mount all potentially-available mount points

for network storage device, we'd better add _netdev parameter in fstab

# iscsi is based on TCP/IP protocol, so therefore, _netdev parameter can make sure the device will be
# mounted after network is connected,  in order to prevent the long waiting boot time
UUID=eb9cbf2f-fce8-413a-b770-8b0f243e8ad6 /iscsi xfs defaults,_netdev 0 0

df and lsblk

Both of them will detect and represent the hard disk information
Check the free space of your hard disk, df -h, human-readable
Making better sense of the storage system structure, lsblk, tree-shape

[root@linuxprobe ~]# df -h
Filesystem                        Size  Used Avail Use% Mounted on
devtmpfs                          3.8G     0  3.8G   0% /dev
tmpfs                             3.9G     0  3.9G   0% /dev/shm
tmpfs                             3.9G  9.9M  3.8G   1% /run
tmpfs                             3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rhel_linuxprobe-root   17G  4.1G   13G  24% /
/dev/sr0                          6.7G  6.7G     0 100% /media/dvd
/dev/nvme0n1p1                   1014M  169M  846M  17% /boot
tmpfs                             780M   16K  780M   1% /run/user/42
tmpfs                             780M  3.5M  776M   1% /run/user/1000
[root@linuxprobe ~]# lsblk
NAME                     MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                        8:0    0    5G  0 disk 
sr0                       11:0    1  6.6G  0 rom  /media/dvd
nvme0n1                  259:0    0   20G  0 disk 
├─nvme0n1p1              259:1    0    1G  0 part /boot
└─nvme0n1p2              259:2    0   19G  0 part 
  ├─rhel_linuxprobe-root 253:0    0   17G  0 lvm  /
  └─rhel_linuxprobe-swap 253:1    0    2G  0 lvm  [SWAP]

Adding new hard disk, SWAP partition

• Adding hard disk for vmware virtual machine allows us to scale the storage system in a much nice way
  

  image.png

• fdisk manipulate disk partition table, this command asks user question and gives hint which is very much user-friendly as well as versatile

  parameters	outcome
  m	complete help information
  n	add new partition
  d	delete some partition
  l	list all the viable file system type
  t	change file system type for specified partition
  p	check hte partition table
  w	save and quit
  q	exit without save

  # list all the partitions
  fdisk -l 
  
  # enter edit mode for a particular hard disk  
  [root@linuxprobe ~]# fdisk /dev/sda
  
  Welcome to fdisk (util-linux 2.32.1).
  Changes will remain in memory only, until you decide to write them.
  Be careful before using the write command.
  
  Device does not contain a recognized partition table.
  Created a new DOS disklabel with disk identifier 0x88b8f279.
  
  Command (m for help):...
  
  Command (m for help): n
  Partition type
     p   primary (0 primary, 0 extended, 4 free)
     e   extended (container for logical partitions)
  Select (default p): p
  Partition number (1-4, default 1): 
  First sector (2048-10485759, default 2048): 
  Last sector, +sectors or +size{K,M,G,T,P} (2048-10485759, default 10485759): +1G
  
  Created a new partition 1 of type 'Linux' and of size 1 GiB.
  
  Command (m for help): w
  The partition table has been altered.
  Calling ioctl() to re-read partition table.
  Syncing disks.
  
  [root@linuxprobe ~]# partprobe
  Warning: Unable to open /dev/sr0 read-write (Read-only file system).  /dev/sr0 has been opened read-only.
  
  [root@linuxprobe ~]# file /dev/sda1
  /dev/sda1: block special (8/1)
  

• mkfs - formatting the device

  [root@linuxprobe ~]# mkfs.xfs /dev/sda1
  meta-data=/dev/sda1              isize=512    agcount=4, agsize=65536 blks
           =                       sectsz=512   attr=2, projid32bit=1
           =                       crc=1        finobt=1, sparse=1, rmapbt=0
           =                       reflink=1
  data     =                       bsize=4096   blocks=262144, imaxpct=25
           =                       sunit=0      swidth=0 blks
  naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
  log      =internal log           bsize=4096   blocks=2560, version=2
           =                       sectsz=512   sunit=0 blks, lazy-count=1
  realtime =none                   extsz=4096   blocks=0, rtextents=0  
  
  
  

• check the file system type and relative information

  [root@linuxprobe ~]# df -T /dev/sda1
  Filesystem     Type 1K-blocks  Used Available Use% Mounted on
  /dev/sda1      xfs    1038336 40368    997968   4% /run/media/linuxprobe/0478a8eb-2cee-4d4f-a00a- 
  641e41aedd86
  
  # check the partition or directory usage with du command
  [root@linuxprobe ~]# du -sh /*
  0 /bin
  130M  /boot
  0 /dev
  29M   /etc
  29M   /home
  0 /lib
  0 /lib64
  6.7G  /media
  0 /mnt
  0 /opt 
  
  # cp /etc/* to new created /dev/sda1
  [root@linuxprobe ~]# cp -fr /etc/* /media/sda1/
  
  # check the file usage of new created /dev/sda1 
  [root@linuxprobe media]# du -sh /media/sda1   # du with -h parameter - human readable
  29M   /media/sda1
  

Adding SWAP Partition

similar to virtual memory, helping to acquire some free space by using the hard disk space when the physical memory is used up.

• make swap partition

  [root@linuxprobe media]# fdisk /dev/sda
  
  Welcome to fdisk (util-linux 2.32.1).
  Changes will remain in memory only, until you decide to write them.
  Be careful before using the write command.
  
  Command (m for help): m  
  
  Help:  
  
    DOS (MBR)
     a   toggle a bootable flag
     b   edit nested BSD disklabel
     c   toggle the dos compatibility flag
  
  Device     Boot   Start     End Sectors Size Id Type
  /dev/sda1          2048 2099199 2097152   1G 83 Linux
  /dev/sda2       2099200 4196351 2097152   1G 82 Linux swap / Solaris
  

• mkswap and swapon

  root@linuxprobe media]# swapoff /dev/sdb1
  [root@linuxprobe media]# mkswap /dev/db1  
  mkswap: cannot open /dev/db1: No such file or directory
  [root@linuxprobe media]# mkswap /dev/sdb1
  mkswap: /dev/sdb1: warning: wiping old swap signature.
  Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes)
  no label, UUID=2fb7a5d5-aad0-4f42-9b3f-1ad1788edeed
  [root@linuxprobe media]# swapon /dev/sdb1
  [root@linuxprobe media]# swapon  
  NAME      TYPE       SIZE USED PRIO
  /dev/sdb1 partition 1024M   0B   -3
  /dev/dm-1 partition    2G   0B   -2
  

disk quotation

In order to prevent the disk space is going to be filled with various user data, there is a command that can allocate disk space for the user in a much more appropriate way.

• xfs_quota - a quotation utility that was designed for the XFS file system particularly

  [root@linuxprobe ~]# chmod -R o+w /boot
  [root@linuxprobe ~]# useradd quotaTest
  [root@linuxprobe ~]# xfs_quota -x -c "limit bsoft=3m bhard=6m isoft=3 ihard=6   
  [root@linuxprobe ~]# xfs_quota -x -c report /boot
  User quota on /boot (/dev/sda1)
                                 Blocks                     
  User ID          Used       Soft       Hard    Warn/Grace     
  ---------- -------------------------------------------------- 
  root           130840          0          0     00 [--------]
  quotaTest           0       3072       6144     00 [--------]
  quotaTest" /boot
  

  difference between soft quotation and hard quotation

  • soft quotation only warn user, not to stop the user from writing the data into disk
  • hard quotation will stop everything, disallow the user to write data into the disk block once for all

  [quotaTest@linuxprobe boot]$ 
  [quotaTest@linuxprobe boot]$ dd if=/dev/zero of=/boot/test bs=5M count=1
  1+0 records in
  1+0 records out
  5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0111994 s, 468 MB/s
  #  bs=8M broke the rule of bhard=6M which is a hard quotation rule!!!
  [quotaTest@linuxprobe boot]$ dd if=/dev/zero of=/boot/test bs=8M count=1
  dd: error writing '/boot/test': Disk quota exceeded
  1+0 records in
  0+0 records out
  4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.00460679 s, 910 MB/s
  

• edquota edit user quotas

  parameters	outcome
  -u	target user
  -g	target group
  -p	copy original rules to the new user/group
  -t	edit the soft time limits for each file system

  # open the quotation editor (default vim) for the specified user name
  [root@linuxprobe ~]# edquota -u quotaTest
  
  [root@linuxprobe ~]# dd if=/dev/zero of=/boot/quotaTest bs=8M count=1
  1+0 records in
  1+0 records out
  8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.012372 s, 678 MB/s
  

VDO virtual Data Optimize

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/deduplicating_and_compressing_storage/maintaining-vdo_deduplicating-and-compressing-storage
manage kernel VDO devices and related configuration information

VDO will compress or delete the duplication within the user data in order to optimize the data storage system and speed up the data access. Permabit was purchased by Redhat and the latter got this new technology

• The encryption layer must be under VDO

• The VDO technology cannot be overlapped
  

  image.png

• VDO practise

  • create a brand new VDO volume

    [root@linuxprobe ~]# ls -l /dev/sdb
    brw-rw----. 1 root disk 8, 16 Jan 26 00:57 /dev/sdb
    [root@linuxprobe ~]# vdo create --name=vdo00 --device=/dev/sdb --    vdoLogicalSize=200G
    Creating VDO vdo00
    Starting VDO vdo00
    Starting compression on VDO vdo00
    VDO instance 0 volume is ready at /dev/mapper/vdo00
    

  • Check VDO status

    [root@linuxprobe ~]# vdo status --name=vdo00
    VDO status:
         Date: '2021-01-26 01:02:13-05:00'
         Node: linuxprobe.com
    

    The status includes VDO volume created time, hostname, duplication and compression flag

  • Format and Mount VDO

    # Format
    [root@linuxprobe ~]# mkfs.xfs /dev/mapper/vdo00
      
    # `udevadm settle` waits for `udevd` to process the device creation events 
    # for all hardware devices, thus ensuring that any device nodes have been
    # created successfully before proceeding.
    [root@linuxprobe vdo00]# udevadm settle
    
    # Mount
    [root@linuxprobe ~]# mount /dev/mapper/vdo00 /media/vdo/vdo00/
    

  • Check vdo stats

    [root@linuxprobe vdo00]# vdostats --human-readable
    Device                    Size      Used Available Use% Space saving%
    /dev/mapper/vdo00        20.0G      4.0G     16.0G  20%           99%
       
    [root@linuxprobe vdo00]# df -h /dev/mapper/vdo00
    Filesystem         Size  Used Avail Use% Mounted on
    /dev/mapper/vdo00  200G  1.5G  199G   1% /media/vdo/vdo00
    

  • Test for VDO
    • Before large file is copied

     [root@linuxprobe ~]# vdostats --human-readable
     Device                    Size      Used Available Use% Space saving%
     /dev/mapper/vdo00        20.0G      5.4G     14.6G  26%            7%
     [root@linuxprobe ~]# df -h /dev/mapper/vdo00 
     Filesystem         Size  Used Avail Use% Mounted on
     /dev/mapper/vdo00  200G  1.5G  199G   1% /media/vdo/vdo00
    

    • After the large file is copied

     [root@linuxprobe ~]# vdostats --human-readable
     Device                    Size      Used Available Use% Space saving%
     /dev/mapper/vdo00        20.0G      5.4G     14.6G  26%            7%
     [root@linuxprobe ~]# df -h /dev/mapper/vdo00 
     Filesystem         Size  Used Avail Use% Mounted on
     /dev/mapper/vdo00  200G  2.9G  198G   2% /media/vdo/vdo00
    

    The advantage of the use of VDO is quite obvious
    Before the large file ....mp4 is copied, the physical available space was 5.4G, After the large file ...mp4 is copied, the physical available space is still the same, where the VDO space is changed just a little bit
  • Add VDO mount point to fstab
    Notice the _netdev parameter, it makes sure the vdo device will be mounted after system and network are all set

    [root@linuxprobe ~]# blkid /dev/mapper/vdo00 
    /dev/mapper/vdo00: UUID="926f56ed-0a52-4647-9004-8edb67edcc7e"       TYPE="xfs"
    [root@linuxprobe ~]# vim /etc/fstab
    # vdo00
    UUID=926f56ed-0a52-4647-9004-8edb67edcc7e  /media/vdo/vdo00  xfs  defaults,_netdev   0 0
    

Symbolic link(soft link) and Hard link

• Symbolic link

  In computing, a symbolic link (also symlink or soft link) is a term for any file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution.
  If the actual data is deleted, then the link file loses its effect too
  

  image.png

• Hard link

  In computing, a hard link is a directory entry that associates a name with a file on a file system. All directory-based file systems must have at least one hard link giving the original name for each file. The term “hard link” is usually only used in file systems that allow more than one hard link for the same file.
  In other words, the hard link is just like a pointer in which it points to the original data block, there can be multiple such pointer, i.e. multiple hard links to the same data block
  

  image.png

  Notice the difference between the original file and the data block the hard link points to, if the data block was pointed by multiple hard link, i.e. multiple pointers, and one of the hard links was deleted, the data block would be still there, even the so called original file is deleted, it actually the hard link is deleted
  Different partition hard link is not allowed

  parameters	outcome
  -s	symbolic link
  -f	force create link file for file or directory
  -i	query for replacement
  -v	vebose mode, i.e. show the creation process

  Practise

  [root@linuxprobe linkfile]# ln -s target.txt target
  [root@linuxprobe linkfile]# ln target.txt targethard
  
  [root@linuxprobe linkfile]# ll
  total 8
  lrwxrwxrwx. 1 root root 10 Jan 26 08:42 target -> target.txt
  -rw-r--r--. 2 root root 12 Jan 26 08:41 targethard
  -rw-r--r--. 2 root root 12 Jan 26 08:41 target.txt
  

  The second column indicates how many links is linking to the file, the hard link is 2 in the above case since we have just created a hard link to the original file, so therefore, even you deleted the original file, the data still can be accessed since the hard link still there, unless the number of links turns to ZERO