Linux Foundation Certified System Administrator (LFCS) – In‑Depth Study Guide
Updated for the May 11 2023 blueprint. Two‑hour, hands‑on exam; distribution‑agnostic; remote‑proctored.
1. Exam at a Glance
| Domain | Weight | What You’ll Be Asked to Do |
|---|---|---|
| Operations & Deployment | 25 % | Kernel tuning, package & service management, virtualization, containers, SELinux policy |
| Networking | 25 % | Interface/DNS config, time sync, SSH, firewalls/NAT, routing, bonding/bridging, reverse proxy & LB |
| Storage | 20 % | LVM, filesystems, swap, automount, NBD/NFS/SMB, performance monitoring |
| Essential Commands | 20 % | Core CLI tools, Git basics, troubleshooting, SSL certificate handling |
| Users & Groups | 10 % | Local/LDAP accounts, sudo/PAM limits, ACLs, environment profiles |
- Format – 15–20 practical tasks in a single terminal.
- Two attempts + two Killer.sh simulator runs included.
- Certification valid for 2 years.
2. How to Use This Guide
- Read: digest each domain summary.
- Practice: reproduce every task in a throw‑away VM.
- Validate: finish timed mock labs (see § 7).
- Refine: follow the 12‑week schedule (see § 6).
3. Domain Deep‑Dives & Task Checklists
Below are do‑it‑yourself lab exercises that map 1‑for‑1 to every skill in the “Domain Deep‑Dives & Task Checklists.”
Each exercise is phrased as a mini‑scenario with a clear objective, the resources you must use, and a quick self‑check so you know when you’ve succeeded.
3.1 Operations & Deployment (25 %)
| Skill | Exercise |
|---|---|
| Kernel tuning | Scenario: An application fails because the system hits the default inotify watch limit. Task: Bump fs.inotify.max_user_watches to 524288 persistently.Verify: sysctl fs.inotify.max_user_watches returns 524288 and survives reboot. |
| Package operations | Scenario: Your team needs a private repo for a patched RPM/DEB. Task: Build a dummy “hello‑world” package, sign it with a local GPG key, host it via Nginx, and configure DNF/APT on the VM to install it. Verify: hello-world --version prints the custom build string. |
| Service mgmt & recovery | Scenario: A developer broke the httpd unit file.Task: Locate the error in /etc/systemd/system/httpd.service, restore a working copy from the package, and make sure the service starts at boot.Verify: systemctl is-active httpd → active and systemctl status shows no failed dependencies. |
| Job scheduling | Scenario: You must delete rotated logs older than 14 days nightly. Task: Create a systemd timer (not cron) that calls journalctl --vacuum-time=14d at 03:15 local time.Verify: systemctl list-timers shows next and last runs; force a run with systemctl start. |
| Virtualization | Scenario: QA needs a throw‑away VM snapshot before every test run. Task: Using virt-install + cloud‑init, deploy an AlmaLinux 9 guest with a second 2 GiB disk. Take an external snapshot, corrupt /etc/hosts, then roll back.Verify: After rollback, the guest’s /etc/hosts matches the pre‑corruption hash. |
| Containers | Scenario: Convert a legacy static‑site build into an OCI image. Task: Write a Containerfile (or Dockerfile) that serves the files with nginx 1.25, tag it local/static:latest, run it with Podman on port 8080, then generate a systemd unit via podman generate systemd and enable it.Verify: curl http://localhost:8080/ returns the site and systemctl is-enabled container-static.service → enabled. |
| SELinux | Scenario: You move the web root to /data/www. SELinux blocks access.Task: Add the needed context mapping with semanage fcontext, run restorecon, and make sure HTTP requests succeed without setting SELinux to permissive.Verify: curl -I http://localhost/ returns 200 OK and audit2why shows no new AVCs. |
3.2 Networking (25 %)
| Task | Exercise |
|---|---|
| Configure IPv4/6 + DNS | Assign eth0 static IPv4 192.168.57.10/24 with gateway .1 and configure SLAAC IPv6. Add Cloudflare DNS 1.1.1.1. Use nmcli (or netplan).Verify: ping -c3 2606:4700:4700::1111 and dig @1.1.1.1 linux.org +short both succeed after reboot. |
| Time sync | Replace the default NTP pool with your own LAN NTP server 10.10.10.5 for Chrony.Verify: chronyc sources shows ^* 10.10.10.5 and timedatectl status lists System clock synchronized: yes. |
| SSH hardening | Disable password logins entirely and permit only users in group sshusers.Verify: Password attempts fail with Permission denied; key‑based auth for an sshusers member succeeds. |
| Firewall/NAT | Forward incoming traffic on host port 2222 to guest VM 10.0.2.20:22 via nftables.Verify: ssh -p2222 localhost connects to the guest, and nft list ruleset shows your DNAT rule. |
| Routing & bonding | Create a bond bond0 (active‑backup) from eno1 and eno2, assign 172.31.30.50/24, and set the default route.Verify: Unplugging eno1 keeps connectivity; ip -br link shows state UP for bond0. |
| Reverse proxy/LB | Deploy HAProxy listening on 443 that terminates TLS (use a self‑signed cert) and balances to two backend containers running nginx on ports 8081/8082 (round‑robin). Verify: curl -k https://localhost/ alternates the hostname header between the two back‑ends. |
3.3 Storage (20 %)
| Objective | Exercise |
|---|---|
| Partition & FS | Attach a 4 GiB virtual disk, partition GPT with a single partition, format XFS, mount it at /srv/data by UUID through /etc/fstab.Verify: Reboot and run lsblk -f—the mount persists. |
| LVM & snapshots | Convert /home into an LVM logical volume, enable thin‑provisioning, create a snapshot, delete files, then merge the snapshot to restore them.Verify: Files are back and lvs shows snapshot merged. |
| Remote storage | Export /srv/nfs_share via NFSv4 on the host and automount it on another VM under /mnt/share using autofs.Verify: `mount |
| Swap management | Disable the existing swap partition, create a 2 GiB zram swap device, and make it permanent. Verify: swapon --show lists only the zram device; grep zram /etc/fstab (or /etc/systemd/zram-generator.conf) exists. |
| Performance | Fill /srv/data with random data, identify the top 3 directories by space, add an XFS quota of 500 MiB to one sub‑directory.Verify: Writes beyond quota fail with EDQUOT. |
3.4 Essential Commands (20 %)
| Area | Exercise |
|---|---|
| Shell & Text | Write a one‑liner that lists unique TCP ports a process named java is listening on, sorted numerically.Verify: The command outputs a list like 80 443 8080 with no duplicates. |
| Processes | Launch stress to eat 2 CPU cores, lower its priority to 19 without killing it, then send it SIGSTOP and later SIGCONT.Verify: top shows the nice value change, and CPU usage pauses/resumes as expected. |
| Monitoring | Attach strace to the PID of sshd, filter only accept syscalls, and write them to /tmp/ssh_accept.log.Verify: A new SSH connection appends a line to that log. |
| Git basics | Initialize a repo, create a branch feature/logrotate, stage changes interactively (git add -p), squash commits with an interactive rebase, and fast‑forward‑merge to main.Verify: git log --oneline --graph shows a clean history. |
| SSL certificates | Generate a CSR for CN server.lab, sign it with your own CA, and configure nginx to use the new cert.Verify: openssl s_client -connect localhost:443 -showcerts prints CN=server.lab. |
3.5 Users & Groups (10 %)
| Topic | Exercise |
|---|---|
| Local account lifecycle | Add user analyst with an expired password that must be changed at first login.Verify: First SSH login for analyst forces a password change, then proceeds. |
| Password ageing & limits | Set a policy where all new passwords must age at least 1 day before they can be changed again, and expire after 90 days. Verify: chage -l analyst reflects min 1 day and max 90 days. |
| Privilege delegation (sudo) | Allow members of group ops to restart nginx without a password.Verify: sudo -l -U <user_in_ops> lists the rule; running sudo systemctl restart nginx skips the password prompt. |
| ACLs | Grant user qa read‑only access to /srv/app/config.yml while keeping default group/world permissions unchanged.Verify: sudo -u qa cat /srv/app/config.yml succeeds; echo test >> fails with Permission denied. |
| LDAP integration | Spin up the osixia/openldap container, configure SSSD to use it, and login as LDAP user jdoe.Verify: id jdoe shows UID/GID from LDAP and getent passwd jdoe returns the entry even after reboot. |
How to Use These Exercises
- Snapshot or clone a fresh VM before each exercise.
- Time yourself—aim for ≤ 10 minutes per item once comfortable.
- Keep a lab journal: record the command sequence and any gotchas.
- When you can finish every exercise in this list confidently and quickly, you’re LFCS‑ready. Good luck!
4. Lab Environment Setup
- Hypervisor: libvirt + virt‑manager or VirtualBox.
- Base images: Ubuntu 22.04 and AlmaLinux 9 cloud images.
- Automation: Vagrant or Ansible to rebuild labs fast.
- Snapshot before every drill; destroy/revert when solved.
- Optional: shared clipboard + VS Code remote SSH.
5. Must‑Read Resources
- Official man pages (only reference allowed during exam).
- Killer.sh LFCS simulator (2 runs included).
- LFS207 e‑Learning + KodeKloud LFCS labs.
- Practice repos: https://www.linuxtrainingacademy.com/linux-commands-cheat-sheet/
- Community: LF forum, r/linuxadmin on Reddit.
6. 12‑Week Structured Plan
| Week | Focus Area | Deliverable |
|---|---|---|
| 1–2 | Lab build, CLI refresher | Bash one‑liner cheat‑sheet |
| 3–4 | Storage deep dive | LVM‑based NAS VM |
| 5–6 | Networking mastery | HAProxy + Nginx reverse proxy lab |
| 7–8 | Operations & Deployment | Containerize legacy app; enforce SELinux |
| 9 | Users & Groups | LDAP + sudo rules scenario |
| 10 | Troubleshooting drills | 20 tasks in 90 min |
| 11 | Killer.sh simulator #1 | ≥70 % score |
| 12 | Weak‑spot remediation + sim #2 | <5 min per task |
Weekly “Do This Now” Exercises
| Week | Core Focus | One Concrete Exercise |
|---|---|---|
| 1 | Lab Scaffold | Spin up a disposable lab VM (Ubuntu 22.04 and AlmaLinux 9) with cloud‑init SSH keys and nested‑virt enabled. Snapshot it and write a 5‑line bash script named reset_lab.sh that destroys/re‑provisions the VM in ≤ 90 seconds. |
| 2 | CLI Reflexes | Produce a personal “Top‑40 one‑liners” cheat‑sheet. Fill it with find/grep/sed/awk commands that you actually tested. Store it at ~/lfcs-cheats/cli.md and sync to a Git repo. |
| 3 | LVM Foundations | Add a 10 GiB virtual disk, create a new LVM VG vg_data, LV lv_nas, format XFS, mount on /srv/nas, and set an XFS quota of 2 GiB for a sub‑directory projects/. |
| 4 | NAS Services | Export /srv/nas over NFSv4 with read‑write access to a second VM. Automount it on demand using autofs (/net/nas). Validate by writing a 1 GiB test file from the client. |
| 5 | Reverse‑Proxy Basics | Inside the lab, start two dockerized Flask apps on ports 5001 & 5002. Install Nginx on the host to round‑robin HTTP 80 traffic to those back‑ends. Prove rotation with five curl hits. |
| 6 | TLS + HAProxy | Replace Nginx with HAProxy listening on 443, using a self‑signed cert (CN=lab.local). Terminate TLS and continue to load‑balance the Flask apps. Confirm with openssl s_client -connect localhost:443. |
| 7 | Containerize Legacy App | Take a static HTML site, write a Containerfile that serves it with nginx, tag it legacy:web, push to a local registry, and deploy via podman run --name legacy -p8080:80. |
| 8 | SELinux Enforcement | Move the site’s volume to /data/legacy, fix contexts with semanage fcontext + restorecon, without setting SELinux to permissive. Show curl http://localhost:8080 returns 200 OK and audit.log is clean. |
| 9 | Directory Services | Run osixia/openldap + phpldapadmin in containers, configure SSSD on the host, import an LDIF that creates group ops, user jdoe. Make jdoe able to sudo systemctl restart nginx password‑lessly. |
| 10 | Speed‑Troubleshoot | Randomly pick 10 services (e.g., SSH, chronyd, crond, NetworkManager). Stop or mis‑configure each, then restore them all in ≤ 90 minutes. Log each fault + fix in ~/fire‑drill.log. |
| 11 | Sim #1 Benchmark | Run the first Killer.sh LFCS simulator. Record score + per‑task times in ~/sim1-results.md. Identify any task that took > 8 minutes or failed. |
| 12 | Targeted Remediation + Sim #2 | Re‑practice only the > 8 minute tasks from Week 11 until each is < 5 minutes. Run Killer.sh simulator #2 and beat 70 %. Document the delta in ~/sim2-results.md. |
Complete each week’s exercise before moving on; your notes, scripts, and logs will become your pre‑exam “run book.”
7. DIY Mock Labs
https://github.com/LINUXexpert-org/lfcs-labs
- Broken Boot – break root UUID, recover via chroot.
- Network Outage – bad gateway; make fix persistent.
- Disk Full – fill
/var; rotate/compress logs, grow LV. - SELinux Denial – wrong context; design & apply policy fix.
- LDAP Fail‑over – swap primary/secondary, keep auth working.
Time yourself – aim for ≤10 min per scenario.
8. Exam‑Day Tips
- Skim all tasks first; knock out quick wins for 70 % buffer.
- Use
tmuxsplits (logs left, commands right). - Keep man pages open in another pane (
man bash,man systemd.unit). - Track progress:
echo OK > /tmp/taskX.done. - Reserve 10 min for sanity checks (
df -h,systemctl --failed).
9. Final Thoughts
The LFCS rewards muscle‑memory troubleshooting more than flag memorization. Spin up labs, time your fixes, and the exam will feel like just another practice run.
Good luck – see you on the other side with a brand‑new badge!
