Pawan Kumar - DevOps Blog
Linux System Administration & Automation

Linux System Administration & Automation

Pawan Kumar Vishwakarma's photo
Pawan Kumar Vishwakarma
·

4 min read

🚀 Introduction

Linux system administration & automation, which is a crucial skill set for every DevOps engineer. From managing users and file permissions to automating backups and log analysis, these tasks form the backbone of server management in production environments.

Here's a quick recap of what I accomplished:

✅ Managed users & SSH access 🔐

✅ Set up file permissions & storage volumes 📁

✅ Analyzed logs using AWK, Grep & Sed 📜

✅ Automated backups using shell scripting & cron 🛠️

In this article, I will walk you through the key Linux administration tasks I completed, along with the necessary commands, explanations, and automation scripts. Let's dive in! 🛠️

🏗️ Project: DevOps Linux Server Monitoring & Automation

📌 Tasks Covered:

1️⃣ User & Group Management

Managing users and groups is fundamental in system administration. Here’s what we did:

  • Created a user (devops_user************************) and added them to a group (devops_team************************).

  • Set up passwords and sudo access.

  • Restricted SSH login for certain users.

Commands:

# Create user and group
groupadd devops_team
useradd -m -G devops_team devops_user
passwd devops_user

# Grant sudo access
echo 'devops_user ALL=(ALL) NOPASSWD:ALL' | tee /etc/sudoers.d/devops_user

# Restrict SSH access
echo 'DenyUsers restricted_user' >> /etc/ssh/sshd_config
systemctl restart sshd

2️⃣ File & Directory Permissions

File permissions define access levels. Here’s how we managed them:

  • Created a directory (/devops_workspace************************) and a file (project_notes.txt************************).

  • Set permissions where owners can edit, groups can read, and others have no access.

Commands:

mkdir /devops_workspace
touch /devops_workspace/project_notes.txt
chmod 740 /devops_workspace/project_notes.txt
ls -l /devops_workspace/project_notes.txt

3️⃣ Log File Analysis with AWK, Grep & Sed

Logs are critical in troubleshooting and system monitoring. Here’s how we extracted insights:

  • Found occurrences of "error".

  • Extracted timestamps and log levels.

  • Replaced IP addresses with [REDACTED] for security.

  • Identified the most frequent log entry.

Commands:

wget https://raw.githubusercontent.com/some_repo/Linux_2k.log

# Find occurrences of "error"
grep -i "error" Linux_2k.log

# Extract timestamps and log levels
awk '{print $1, $2, $3}' Linux_2k.log

# Replace IP addresses with [REDACTED]
sed -E 's/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/[REDACTED]/g' Linux_2k.log > sanitized.log

# Find most frequent log entry
awk '{print $0}' Linux_2k.log | sort | uniq -c | sort -nr | head -10

4️⃣ Volume Management & Disk Usage

Storage management is crucial for ensuring system performance.

Tasks Completed:

  • Created a directory (/mnt/devops_data************************).

  • Mounted a new volume.

  • Verified using df -h and mount.

Commands:

mkdir /mnt/devops_data
mount /dev/xvdf /mnt/devops_data
df -h
mount | grep devops_data

5️⃣ Process Management & Monitoring

System performance depends on how well processes are managed.

Tasks Completed:

  • Started a background process (ping google.com > ping_test.log &************************).

  • Monitored it using ps, top, htop.

  • Killed the process.

Commands:

ping google.com > ping_test.log &
ps aux | grep ping
top
htop
kill $(pgrep ping)

6️⃣ Automate Backups with Shell Scripting

Backups prevent data loss and ensure disaster recovery. We automated this process with a shell script and cron job.

Shell Script:

#!/bin/bash
mkdir -p /backups
tar -czf /backups/backup_$(date +%F).tar.gz /devops_workspace
echo -e "\e[32mBackup successful!\e[0m"

Cron Job (Runs Daily at 2 AM):

crontab -e
0 2 * * * /path/to/backup_script.sh

🎯 Extra Learning!

1️⃣ Find the top 5 most common log messages

awk '{print $0}' Linux_2k.log | sort | uniq -c | sort -nr | head -5

2️⃣ Find files modified in the last 7 days

find / -type f -mtime -7

3️⃣ Extract and display only ERROR and WARNING logs

grep -E 'ERROR|WARNING' Linux_2k.log

🔥 Final Thoughts

🚀 Diving into Linux System Administration & Automation has strengthened my understanding of core DevOps principles like user management, file permissions, log analysis, volume management, process control, and automation.

This is just the beginning, and I can't wait to see what's next in this DevOps journey!

If you’re on a similar learning path, let’s connect and grow together! 🚀

😊 Thanks for reading, keep exploring and follow for more.

DevopsLinuxsysadminCloud Computing