Moving around

Where am I, what is here, and how do I get somewhere else?

starting in /home/ana$ cd reports1$ pwd/home/ana/reports$ ls2q3.txt$ cd ..3$ cd /var/log4$ pwd/var/log$ cd5$ pwd/home/ana1cd namego into a directory that is here2lslist where you now stand3cd ..go up one level, to the parent4cd /var/logjump anywhere by its full path5cdalone, it takes you homeYou always stand in one directory. cd moves you, pwd says where you are, ls shows what is there.Commands act in the directory you stand in unless you name another, so a wrong location is the usual cause of "No such file".Tab completes directory names as you type them, and cd - returns to the directory you were in before.

The shell always has one working directory: cd changes it, pwd prints it and ls lists it.

Paths

From the root, or from where you stand

/homeanalogsnotes.txtetchostsyou are herePathMeans/etc/hostsabsolute: from /../notes.txtup one, then notes.txt./app.loghere, then app.log~/notes.txtmy home, then notes.txtcd -back to the last directoryA path starting with / is read from the root. Any other path is read from where you are.Every directory holds two special names: . is itself and .. is its parent, so ../notes.txt climbs one level. pwd prints where youare; cd with no argument goes home. Scripts and configuration files should use absolute paths, because they cannot know thedirectory they will be started from. Names are case-sensitive: Notes.txt and notes.txt are different files.

An absolute path starts at /; a relative path starts at the current directory, with . and .. for here and up.

Filesystem hierarchy

Every file hangs from /, and each top folder has one job

/the root: one tree, whatever the disks/etcconfigurationtext files you edit/homeusers’ files/home/ana/rootroot’s home/vardata that changeslogs, databases/usrinstalled softwareread-only in use/optadd-on softwareone folder each/bootkernel, bootloader/tmptemporary filescleared at boot/srvserved dataweb, FTP/devdevices as filesmade by the kernel/procprocessesnot on disk/syskernel, hardwarenot on disk/runruntime statein memoryEvery file on a Linux machine sits somewhere under /, and each top folder has one job.Configuration lives in /etc, things that grow in /var, installed programs in /usr. Knowing the map tells you where to look:a service’s settings are in /etc/<name>, its logs under /var/log, its data under /var/lib/<name>. The grey folders are not on anydisk: the kernel makes them up on demand to show devices, processes and its own state as files.

The Filesystem Hierarchy Standard gives each top folder a job: /etc for configuration, /var for changing data, /usr for programs.

/proc and /sys · Going deeper · 1 of 2

The kernel answers questions as if they were files

$ cat /proc/loadavg0.42 0.37 0.31 2/187 4120$ cat /proc/sys/net/ipv4/ip_forward0$ ls /proc/4120cmdline cwd environ exe fd limits maps status ...$ cat /sys/class/net/eth0/address52:54:00:3a:91:0c$ cat /sys/block/vda/size5242880001/proc/loadavgthe load averages uptime prints2/proc/sys/...kernel settings: sysctl reads these3/proc/<PID>/one folder per running process4/sys/class/net/each network interface5/sys/block/vda/sizein 512-byte sectors: 250 GiB/proc and /sys are the kernel answering questions, formatted as files.Nothing here is stored: each read asks the kernel and gets today’s answer. Most monitoring tools are just readers of these files:ps walks /proc/<PID>, free reads /proc/meminfo, and sysctl net.ipv4.ip_forward reads /proc/sys/net/ipv4/ip_forward.Some files can be written to change the kernel at once, until the next boot; lesson 14 makes such changes permanent.

/proc and /sys are generated on each read; ps, free and sysctl are readers of them.

Filesystem hierarchy · Going deeper · 2 of 2

Disks have no letters: each joins the tree at a directory

//boot/home/var/procvdavda1 → /boot, vda2 → /vdbone partition → /homeLVM volumevg0/var → /varthe kernelproc → /procA disk has no letter. It joins the tree at a directory, called its mount point.You cannot tell from a path which disk holds it: /home/ana/notes.txt may be on a different disk from /etc/hosts. findmnt showsthe whole picture and df -h the space left on each. If /var fills up, only the file system mounted there is full. That is whyservers often give /var and /home their own: a runaway log cannot stop the whole system from writing.

Mounting attaches a file system at a mount point; the path does not say which disk holds a file.

Copy, move, delete

How do I make, copy, rename and delete without a mouse?

$ mkdir backup1$ cp notes.txt backup/2$ mv notes.txt todo.txt3$ lsbackup logs reports todo.txt$ rm backup/notes.txt4$ rmdir backup$ ls logs/app.*.log5logs/app.1.log logs/app.2.log logs/app.3.log$ rm logs/app.*.log$ ls logsdb.log1mkdirmake a directory2cp from tocopy; the original stays3mv from tomove, or rename in place4rmdelete: there is no recycle bin5* stands for any characterslist with a pattern, then delete with itrm does not ask and cannot be undone. Run ls with the same pattern first, then rm.For a whole directory, cp -r copies it with everything inside and rm -r deletes it with everything inside; plain rm refuses:"rm: cannot remove 'logs.old': Is a directory". cp and mv overwrite a file of the same name without a word; -i makes them ask.

mkdir, cp, mv and rm do the work of a file manager; a wildcard names many files at once, and rm has no undo.

Copy, move, delete · Going deeper

Inside one file system, mv only renames

one file system: /homedraft.txtmvfinal.txtthe inode stays put; only the name changes1 GB or 1 KB: instant/home/srv (another disk)video.mp4mvvideo.mp4copied byte by byte, then deleted1 GB takes seconds and needs space twiceCommandBehaviour to remembercp -a src/ dst/archive: keep owner, permissions, times, linksrm -r dirno undo, no recycle bin; rm -ri asks firstmkdir -p a/b/cmake the parents too, no error if presentMoving a file within one file system is a rename. Moving it to another is a copy and a delete.

mv within a file system changes a directory entry; across file systems it copies the data and deletes the original.

Inodes

A name points to an inode, and the inode is the file

directory /home/ananotes.txt574880logs574870.bashrc574862just names and numbersinode 574880typeregular filepermissionsrw-r--r--owner, groupana, anasize2,310 bytestimesaccess, modify, changelinks1 name points hereblockswhere the data isdata blocksThe name is not in the inode.The inode has no name.A file is an inode: owner, permissions, size, times and where its data is. Names live in directories.A directory is only a list mapping names to inode numbers; ls -i shows them. This is why renaming is instant, why a deletedfile keeps using space while a program still holds it open, and why a disk can be “full” with space left: each file system hasa fixed number of inodes on ext4, and millions of tiny files can use them all up. df -i shows how many are left.

An inode holds a file’s type, owner, permissions, size, times and data location; the name lives only in a directory.

Hard and symbolic links

A second name for the same inode, or a note with a path

a.txthard.txtinode 574878links: 2 · "hi"soft.txtinode 574879holds the text "a.txt"the path is looked up again on each use$ rm a.txt$ cat hard.txthi$ cat soft.txtcat: soft.txt: No such fileor directoryln a.txt hard.txtln -s a.txt soft.txtA hard link is a second name for the same inode. A symbolic link is a small file holding a path.Delete a.txt and the inode survives, because hard.txt still counts as a name for it; the data goes only when the count reaches zero.The symbolic link now points at nothing. Hard links cannot cross file systems or point at directories; symbolic links can do both,which is why they are used everywhere: /bin is a symbolic link to usr/bin on current Ubuntu and Red Hat systems.

A hard link is another name for the same inode; a symbolic link stores a path and breaks when the target goes.

Hard and symbolic links · Going deeper

Disks, terminals and sockets are files too

ls -lTypeExampleWhat it is-regular file/etc/hostsbytes: text, a program, an imageddirectory/var/loga list of names and inode numberslsymbolic link/bin -> usr/bina path to followbblock device/dev/vdaa disk, read in blocksccharacter device/dev/null, /dev/ttya stream of bytesssocket/run/dbus/system_bus_socketa local connection pointpnamed pipemade with mkfifoone program writes, one readsEverything is a file: the first letter of ls -l says which of seven kinds.Disks, terminals and the bit bucket /dev/null are reached with the same open, read and write as a text file, which is whydd can copy a whole disk and > /dev/null throws output away. file tells you what a regular file contains, whatever its name says:Linux does not use file extensions to decide anything.

The first character of ls -l shows one of seven file types; devices are opened and read like any file.

Finding files

Walk the tree and test every file: where, which, then what

find/var/logstart here-name '*.log'-size +100M-mtime -7tests: all must be true (AND)-exec ls -lh {} +action: {} is each matchTestMeansNote-type f / -type dfiles / directories-size +100Mlarger than 100 MiB-size -1k: under 1 KiB-mtime -7modified in the last 7 days+30: over 30 days ago-user ana -perm -4000owned by ana / setuid bit setused in security audits-o, !, ( )or, not, groupingquote the brackets: \( \)find walks the tree live and tests every file; tests are ANDed, and the action runs on each match.It is the tool for questions like “which files over 100 MB changed this week?”. Quote the patterns so the shell does not expandthem first. -exec ... {} + passes many files to one command; -delete removes matches, so run without it first.locate answers faster from a database built nightly by updatedb, so it misses anything newer than the last run.

find takes a starting point, tests that must all be true, and an action for each match.

Archives and compression

tar bundles the files; the compressor trades time for size

One tar of /usr/share/doc, compressed three ways on one coresizetime to compresstar only21.1 MBgzip (tar -z)6.5 MB0.58 sbzip2 (tar -j)5.0 MB1.5 sxz (tar -J)3.4 MB4.64 star -czf docs.tar.gz /usr/share/doc create, gzip, to this filetar -tzf docs.tar.gz list tar -xzf docs.tar.gz -C /tmp extracttar bundles many files into one; a compressor then shrinks it. Smaller costs more time.gzip is fast and everywhere, xz is about half the size and eight times slower here, zstd (tar --zstd) gives near-xz size at gzip speed.Bars are measured and to scale. tar keeps owners, permissions and times, which zip does not, so it is the format for backups.

tar packs files with owners and permissions into one archive; gzip, bzip2 and xz shrink it at different speeds.

Lab

Map the mounts, make both links, find and archive

$ findmnt -t ext4,xfs1TARGET SOURCE FSTYPE OPTIONS/ /dev/vda2 ext4 rw,relatime$ echo hi > a.txt; ln a.txt h.txt; ln -s a.txt s.txt2$ ls -li a.txt h.txt s.txt574878 -rw-r--r-- 2 ana ana 3 ... a.txt574878 -rw-r--r-- 2 ana ana 3 ... h.txt574879 lrwxrwxrwx 1 ana ana 5 ... s.txt -> a.txt$ find /etc -name '*.conf' -mtime -303/etc/locale.conf$ tar -czf etc.tar.gz /etc 2>/dev/null; ls -lh etc.tar.gz4-rw-r--r-- 1 ana ana 922K ... etc.tar.gz1Which disk is wherefindmnt shows every mount2Two links, two behaviourssame inode for the hard link3Recent configuration changeswhat changed this month?4A compressed backupof /etc, before you edit itLab: map the tree, link, find and archive.Nine steps on your own machine. Press the down arrow.

findmnt, ln, find and tar on your own machine, and a backup of /etc before you change anything.

Lab · Going deeper

Map the tree, make links, find files, archive /etc

Work as a normal user in your home directory, with sudo where a step says so.

Steps

  1. Map the mounts. Run findmnt, then df -hT. Which file system holds /home? Which entries are not on any disk?
  2. Visit the kernel’s files. cat /proc/loadavg, /proc/meminfo | head and /sys/class/net/*/address. Compare with uptime and ip link.
  3. Practise paths. cd /var/log; cd ../lib; pwd; cd -; cd; pwd. Then reach /etc/hosts with a relative path from your home.
  4. Inspect inodes. ls -li ~; stat ~/.bashrc; df -i /. How many inodes are free on the root file system?
  5. Make both links. echo hi > a.txt; ln a.txt h.txt; ln -s a.txt s.txt; ls -li. Delete a.txt and cat both links. Then make a symbolic link to a directory.
  6. Find by test. find / -xdev -type f -size +50M 2>/dev/null; find /etc -mtime -7; sudo find / -xdev -perm -4000 -type f. What does -xdev do?
  7. Rename and copy. Time mv of a large file within /home, then cp -a it to /tmp and compare owners and times with ls -l --time-style=full-iso.
  8. Back up /etc. sudo tar -czf ~/etc-$(date +%F).tar.gz /etc; tar -tzf the file | head; extract it into /tmp/restore with -C.
  9. Compare compressors. Repeat the backup with -j and -J and compare sizes with ls -lh. Which would you use for a nightly backup, and why?
Check: You can explain why the hard link survived and the symbolic link broke, and your tar of /etc lists and extracts cleanly.

Lesson complete

You finished “Files and directories”

Nicely done.

Open in a new tab