Moving around
Where am I, what is here, and how do I get somewhere else?
The shell always has one working directory: cd changes it, pwd prints it and ls lists it.
Without a file manager, how do you move around? You always stand in exactly one directory. cd with a name steps into it, and pwd confirms where you are. ls shows what is there. Two dots mean the directory above, so cd dot dot goes up one level. A full path, starting with a slash, takes you anywhere in one step. And cd on its own always brings you home.
Paths
From the root, or from where you stand
An absolute path starts at /; a relative path starts at the current directory, with . and .. for here and up.
Every file has a path. Say you are in the logs folder in ana’s home. A path that starts with a slash is absolute, read from the root, and means the same thing from anywhere. Any other path is relative to where you are: dot is here, dot dot is one level up, and tilde is your home. Scripts should use absolute paths, because they never know where they will be started from.
Filesystem hierarchy
Every file hangs from /, and each top folder has one job
The Filesystem Hierarchy Standard gives each top folder a job: /etc for configuration, /var for changing data, /usr for programs.
Linux has no drive letters. There is one tree, starting at the root, written as a slash. Each top-level folder has a job. Et-see holds configuration, home holds people’s files, var holds things that change, like logs and databases, and user holds installed software. A few are not on disk at all. Dev, proc, sys and run are the kernel showing you devices, processes and its own state as if they were files. Learn this map once and you can find your way on any Linux server.
/proc and /sys · Going deeper · 1 of 2
The kernel answers questions as if they were files
/proc and /sys are generated on each read; ps, free and sysctl are readers of them.
Filesystem hierarchy · Going deeper · 2 of 2
Disks have no letters: each joins the tree at a directory
Mounting attaches a file system at a mount point; the path does not say which disk holds a file.
So where are the disks? Each disk, or part of a disk, is attached to the tree at a directory, called its mount point. Here the first disk holds boot and the root. A second disk is mounted at home. A logical volume is mounted at var. And proc comes from the kernel. From the inside, it all looks like one tree. That has a practical upside: if var fills up with logs, only that file system is full.
Copy, move, delete
How do I make, copy, rename and delete without a mouse?
mkdir, cp, mv and rm do the work of a file manager; a wildcard names many files at once, and rm has no undo.
Five commands replace the file manager. mkdir makes a directory. cp copies: the first name is the source, the last is the destination. mv moves, and moving to a new name in the same place is how you rename. rm deletes, and there is no recycle bin: the file is gone. A star in a name stands for any characters, so one command can act on many files. That is powerful and dangerous together with rm, so list with the pattern first, then delete with the same pattern.
Copy, move, delete · Going deeper
Inside one file system, mv only renames
mv within a file system changes a directory entry; across file systems it copies the data and deletes the original.
Inodes also explain how fast file operations are. Moving a file within one file system only changes the directory entry. The inode does not move, so a gigabyte moves as fast as a byte. Moving it to another disk is different: the bytes are copied, then the original is deleted, which takes time and needs room for both. Copy with cp dash a when owners and times matter. And rm has no recycle bin.
Inodes
A name points to an inode, and the inode is the file
An inode holds a file’s type, owner, permissions, size, times and data location; the name lives only in a directory.
What is a file, really? A directory is just a list of names, each with a number. The number leads to an inode, which holds everything about the file: its type, owner, permissions, size, times, and how many names point to it. The inode points to the data blocks. The name is not part of the file. That explains a lot: renaming is instant, and a deleted file that a program still has open goes on using space until the program closes it.
Hard and symbolic links
A second name for the same inode, or a note with a path
A hard link is another name for the same inode; a symbolic link stores a path and breaks when the target goes.
Links follow directly from inodes. A hard link is a second name for the same inode. Both names are equal, and the inode counts two links. A symbolic link is a tiny file of its own, holding a path that is looked up each time. Now delete the original. The hard link still reads the data, because the inode still has a name. The symbolic link points at nothing. Symbolic links can cross disks and point at folders, so they are the ones you will use most.
Hard and symbolic links · Going deeper
Disks, terminals and sockets are files too
The first character of ls -l shows one of seven file types; devices are opened and read like any file.
The first letter of a long listing tells you what kind of file you are looking at. A dash is a regular file, d a directory and l a symbolic link. Then come the special ones: b for block devices such as disks, c for character devices such as terminals and dev null, s for sockets, and p for named pipes. Because a disk is a file, ordinary tools can read and copy it. And note that Linux ignores file extensions: the file command looks inside to tell you what a file really is.
Finding files
Walk the tree and test every file: where, which, then what
find takes a starting point, tests that must all be true, and an action for each match.
To search the tree, use find. Give it a place to start. Then tests, which must all be true: a name pattern, a size, a modification time. Then an action, run on each match. The tests cover nearly anything you would ask: type, size, age, owner, permission bits, combined with or and not. Quote the patterns, so the shell does not expand them before find sees them.
Archives and compression
tar bundles the files; the compressor trades time for size
tar packs files with owners and permissions into one archive; gzip, bzip2 and xz shrink it at different speeds.
Backups and transfers start with tar. It packs a whole tree into one file, keeping owners, permissions and times: here, twenty-one megabytes of documentation. gzip squeezes it to six and a half megabytes in about half a second. bzip2 to five megabytes in one and a half seconds. xz to three point four megabytes, but it takes nearly five seconds. The flags are worth memorising: c to create, t to list, x to extract, and z, j or capital J to choose the compressor.
Lab
Map the mounts, make both links, find and archive
findmnt, ln, find and tar on your own machine, and a backup of /etc before you change anything.
In this lab you map your own machine’s mounts, make a hard and a symbolic link and break one of them, search with find, and take a compressed backup of the configuration folder. Press the down arrow for the steps.
Lab · Going deeper
Map the tree, make links, find files, archive /etc
Work as a normal user in your home directory, with sudo where a step says so.
Work as a normal user in your home directory, with sudo where a step says so. Steps. Map the mounts. Run findmnt, then df -hT. Which file system holds /home? Which entries are not on any disk? Visit the kernel’s files. cat /proc/loadavg, /proc/meminfo | head and /sys/class/net/*/address. Compare with uptime and ip link. Practise paths. cd /var/log; cd ../lib; pwd; cd -; cd; pwd. Then reach /etc/hosts with a relative path from your home. Inspect inodes. ls -li ~; stat ~/.bashrc; df -i /. How many inodes are free on the root file system? Make both links. echo hi > a.txt; ln a.txt h.txt; ln -s a.txt s.txt; ls -li. Delete a.txt and cat both links. Then make a symbolic link to a directory. Find by test. find / -xdev -type f -size +50M 2>/dev/null; find /etc -mtime -7; sudo find / -xdev -perm -4000 -type f. What does -xdev do? Rename and copy. Time mv of a large file within /home, then cp -a it to /tmp and compare owners and times with ls -l --time-style=full-iso. Back up /etc. sudo tar -czf ~/etc-$(date +%F).tar.gz /etc; tar -tzf the file | head; extract it into /tmp/restore with -C. Compare compressors. Repeat the backup with -j and -J and compare sizes with ls -lh. Which would you use for a nightly backup, and why? Check: You can explain why the hard link survived and the symbolic link broke, and your tar of /etc lists and extracts cleanly.
Steps
- Map the mounts. Run findmnt, then df -hT. Which file system holds /home? Which entries are not on any disk?
- Visit the kernel’s files. cat /proc/loadavg, /proc/meminfo | head and /sys/class/net/*/address. Compare with uptime and ip link.
- Practise paths. cd /var/log; cd ../lib; pwd; cd -; cd; pwd. Then reach /etc/hosts with a relative path from your home.
- Inspect inodes. ls -li ~; stat ~/.bashrc; df -i /. How many inodes are free on the root file system?
- Make both links. echo hi > a.txt; ln a.txt h.txt; ln -s a.txt s.txt; ls -li. Delete a.txt and cat both links. Then make a symbolic link to a directory.
- Find by test. find / -xdev -type f -size +50M 2>/dev/null; find /etc -mtime -7; sudo find / -xdev -perm -4000 -type f. What does -xdev do?
- Rename and copy. Time mv of a large file within /home, then cp -a it to /tmp and compare owners and times with ls -l --time-style=full-iso.
- Back up /etc. sudo tar -czf ~/etc-$(date +%F).tar.gz /etc; tar -tzf the file | head; extract it into /tmp/restore with -C.
- Compare compressors. Repeat the backup with -j and -J and compare sizes with ls -lh. Which would you use for a nightly backup, and why?
Lesson complete
You finished “Files and directories”
Nicely done.