fsck:

https://access.redhat.com/solutions/9541
How to repair filesystem in rescue environment for Red Hat Enterprise Linux
System won't load after boot, requests fsck of root file system
# Force fsck at reboot
# tune2fs -c 1 /dev/vg_data02/lv_data02
# tune2fs -c 1 /dev/vg_data03/lv_data03
1). What is fsck?
fsck..it is filesystem check. It will run to fix the inconsistences on a
filesystem.
The fsck can be run in two ways. Automatically and manually. The automatic run is
happening at the time of booting. A system administrator can run the utility
manually. The operations by this command will work on the data structures stored on
the disk directly. This data is internal and specific to the particular file system
in use. There are various fsck. The exact behavior of various implementation can
vary, but there is a common order of internal operations and a command line
interface.
There are both interactive and non-interactive fixing of errors. In interactive
mode, the user can choose how to fix an error, but in non-interactive mode, the
system will make a decision on the operations to do. A system administrator can
also run fsck manually if they believe there is a problem with the file system. The
file system is normally checked while unmounted, mounted read-only, or with the
system in a special maintenance mode. Modern journaling file systems are designed
so that tools such as fsck do not need to be run after an unclean shutdown (i.e.
crash). The UFS2 Filesystem in FreeBSD has a background fsck, so it is usually not
necessary to wait for fsck to finish before accessing the disk.
The ext2, ext3, and ext4 file systems use the e2fsck utility to perform file system
checks and repairs. The file names fsck.ext2, fsck.ext3, and fsck.ext4 are
hardlinks to the e2fsck utility. These binaries are run automatically at boot time
and their behavior differs based on the file system being checked and the state of
the file system.
After running fsck, it will return an exit code. These cods can be seen in fsck’s
manual
Note down the EXIT CODE into the CRQ notes
0 - No errors
1 - File system errors corrected
2 - System should be rebooted
4 - File system errors left uncorrected
8 - Operational error
16 - Usage or syntax error
32 - Fsck canceled by user request
128 - Shared library error
2) How to check the information about a filesystem?
By using tune2fs -l command.
3). What fsck will do?
Ans> It will run to fix the inconsistences on a filesystem.Fsck will run 5
phases.It will fix as shown below
# fsck /dev/sde1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
fsck.ext3: Superblock invalid, trying backup blocks...
/dev/sde1 was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #0 (7661, counted=7288).
Fix<y>?
4) what is lost+found directory?
last+found...whenever you create a filesystem by default last+found will be
created.
inconsistences(un referenced data) found wile running fsck will be stored in that
directory.
It will be used to analize by vendor.(Redhat / IBM)
5) You need to run a fack on /data01 filesystem? how you will proceed.
1. first unmount the filesystem (by taking approval, checking fuser and lsof )
2. run fsck
3 mount it back.
-----------------------------------------
7) what is the additional advantage with journaling than fsck?
fsck without journaling will check entire file system. SO, it will take huge time.
But the file system having journaling feature will take less time for fsck because
it will check only journal log and updates are copied back from the journal to the
filesystem.
## A thread writes data to the f/s and each transaction is flagged. After a crash,
updates are copied back from the journal to the filesystem and any incomplete
transactions are gone. In case of a failure, a journaled file system will ensure
that all inconsistencies are restored back to its prior state.###
8) How to see if any filesystem in read only mode?
With mount command, cat /proc/mounts , and also by writing something on that file
system , you will get the error.
9) fsck cannot check the root file system on a running machine because it cannot be
unmounted.
If you want to check or repair the root file system, you have several options at
your disposal. You can set the fsck to run on boot, boot the system in recovery
mode, or use a live CD
10) # Force fsck at reboot
# tune2fs -c 1 /dev/vg_data02/lv_data02
# tune2fs -c 1 /dev/vg_data03/lv_data03
================================================================================
housekeeping: (cleaning of space on a FS (filesystem)
>> First identify what kind of file system it is ( OS, Customer Or NFS/NAS)
Once we find the most consuming folder/file, based on the type of the FS, we
perform the housekeeping.
>> If it is NAS Filesystem, we have to find out the files consuming space and let
the customer know if any housekeeping required OR to increase the NAS volume from
storage team side.
>> If it is Customer related filesystem and local, find out the files consuming
space and owner of those files and let customers know and suggest how to proceed
further with housekeeping.
>> If it is OS related filesystem, we have to find out the files consuming space,
and to perform housekeeping based on the filesystem type.
A) If filesystem consumes space, then by using du command or find command get
the file which is consuming more space.
B) if it is /boot:
Verify how many old kernels are there.
just keep only one old kernel ( total 2 , 1 is latest and other one is old)
and remove remaining duly taking approval from the customer.
C) if it is /var, check which log is consuming space. take a copy to a
different location, nullify the existing log, compress the backup file
and move it to its original location.
D) if it is "/", find out which folder is consuming space, ignore if it is
mount point and check the directories which are consuming space.
E) if it is /tmp, do it in a normal way
- using du command or find command to get the file which is consuming more space.
- Check the owner of the file and inform him to perform housekeeping.
- If root is owning, try to understand the file and zip it, if it is not live.
- If it is owned by other users, try to understand the file and zip it, if it is
not live.If it is a live file, better not to touch and to notify
the user.
NOTE: If you are compressing/moving any file which is unknown to you, understand
who is owning that file and time stamp of the file. If it is live file do not
perform any action. If it is owning an app/db let them know about the file and take
advice on what to do.
If inodes are exhausted:
(motto is find out the folder where huge number of smallest files are there. and
check the time stamp of the files)
a) first check the inode utilization by using df -i /filesystem
b) by using find command find out which folder is having huge number of files and
consuming inodes.
# find /var -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -rn | head
c) Then based on the confirmation from customer dele few old files ..example older
than one month.
# find /path/to/files* -mtime +5 -exec rm {} \;
https://www.thegeekdiary.com/centos-rhel-67-why-the-files-in-tmp-directory-gets-del
eted-periodically/
1. one FS is consuming space but nothing is showing consumed with du -sh
command.Why ??
- check deleted files with lsof command. If you see any , kill that process.if
still issue is there, proceed with next point
# lsof |grep i deleted
- unmount and mount it back. if still issue is there, proceed with next point
- run fsck (if still issue is there, proceed with next point)
- Ask for the approval to reboot the server.