Linux for HPC: Essential Terminal Commands

Built My Own HPC Virtual Beowulf Cluster with MPI and VirtualBox! 💻⚙️ After days of configuring, debugging, and connecting virtual machines, I finally got my High Performance Computing (HPC) Cluster running! 🎉 👉 The cluster consists of one master node and two compute nodes, built using: 🧠 MPICH (for parallel processing) 💾 NFS (for shared storage) 🔒 SSH key-based communication 💻 Ubuntu Server 24.04 on VirtualBox I can now run parallel MPI programs distributed across all nodes! It was challenging — setting up NFS, SSH, and MPI coordination wasn’t easy — but seeing all nodes working together was totally worth it. 💪 A High Performance Computing (HPC) Cluster enables parallel computation by connecting multiple computers (nodes) to work together as one. Thank you Budditha Hettige Sir for your valubale guidance. Check this guide for your reference https://lnkd.in/gsTiTmgt Check the repository https://lnkd.in/gGrZ7h5y 🔹 Objectives Learn how HPC clusters work. Set up a master node and compute nodes. Run parallel C programs using MPI. #HPC #MPICH #ParallelComputing #Linux #VirtualBox #BeowulfCluster #SoftwareEngineering #LearningByDoing

Tech in history by Roberto Capodieci Linux Kernel 0.02 Released – A Step Toward Open-Source Dominance On October 5, 1991, 21-year-old University of Helsinki student Linus Torvalds announced the release of Linux kernel version 0.02 on the comp.os.minix Usenet newsgroup, marking a key milestone after the initial 0.01 version's limited functionality. Inspired by Minix but frustrated by its restrictions, Torvalds wrote Linux as a free alternative for Intel 80386 processors, incorporating multitasking and POSIX compliance. This release allowed compilation of code directly on Linux, attracting early contributors who shared patches via email and FTP, embodying the open-source ethos amid debates over software freedom sparked by Richard Stallman's GNU project. By integrating with GNU tools, Linux formed a complete OS, influencing tech-politics as corporations like IBM adopted it for enterprise, challenging Microsoft's dominance and fueling the dot-com boom's server infrastructure. Quirky early limitations, like dependency on Minix for bootstrapping, highlighted grassroots innovation, evolving into distributions like Slackware and Debian that powered 96% of top supercomputers by 2025. Memorable flops included kernel panics in unstable builds, but community fixes accelerated growth. Today, Linux underpins Android's billions of devices and cloud services like AWS, a testament to collaborative coding that makes nerds grin at its versatility from IoT to AI. How has Linux changed your computing world? Reply and share stories! #TechHistory #Innovation #RetroTech #TechMilestones #TechNews

💻 What Happens When Linux Boots? (Step-by-Step Explained) Ever wondered what really happens when you press the power button on your Linux system? 🖥️ Here’s the boot process simplified 👇 ⚙️ 1️⃣ BIOS / UEFI The firmware runs first when you power on your system. It checks your hardware (CPU, RAM, disks, keyboard, etc.). Then it looks for a bootable device and hands control to the bootloader. 🚀 2️⃣ Bootloader (GRUB2, LILO, systemd-boot) Lives in the disk’s MBR or EFI partition. Loads the Linux kernel into memory. Lets you choose OS (dual-boot) and pass kernel parameters. 🐧 3️⃣ Linux Kernel Bootloader loads the kernel image (vmlinuz). Kernel initializes CPU, memory, and drivers. Mounts a temporary root filesystem using initramfs. Then starts the first user-space process (init or systemd). 🧩 4️⃣ initramfs (Initial RAM Filesystem) Temporary filesystem in memory. Loads essential drivers (disk, filesystem, LVM, etc.). Mounts the real root filesystem (like /dev/sda1). 🛠️ 5️⃣ Init / systemd (PID 1) Responsible for starting all services. Sets up networking, mounts filesystems, and starts daemons. Brings the system to its target (runlevel) — multi-user or graphical. 🎯 6️⃣ Runlevels / systemd Targets Old: Runlevels (3 = multi-user, 5 = graphical). New: systemd targets (multi-user.target, graphical.target). 👤 7️⃣ User Space & Login The system shows a login prompt (terminal or GUI). You log in → shell starts → system is ready to use! 💬 Learning Linux deeply helps you understand how everything connects — from hardware to shell. #Linux #DevOps #SysAdmin #CloudComputing #LearningJourney

𝐌𝐚𝐧𝐚𝐠𝐞 𝐭𝐮𝐧𝐢𝐧𝐠 𝐩𝐫𝐨𝐟𝐢𝐥𝐞𝐬 𝐢𝐬 𝐥𝐢𝐭𝐭𝐥𝐞 𝐭𝐫𝐢𝐜𝐤𝐲 𝐛𝐮𝐭 𝐧𝐨𝐭 𝐝𝐢𝐟𝐟𝐢𝐜𝐮𝐥𝐭 — 𝐋𝐞𝐭𝐬 𝐥𝐞𝐚𝐫𝐧 𝐚𝐛𝐨𝐮𝐭 𝐢𝐭. Tuned is a daemon that optimizes Linux system performance by dynamically adjusting system settings based on workload requirements. It’s used to configure performance parameters for different goals such as high performance, low latency, or energy efficiency. Here’s how you can use tuned and cpupower to manage tuning profiles effectively 👇 🔹 Common Tuned Profiles: Performance: Maximizes overall system performance. Powersave: Reduces power consumption. Throughput-performance: Optimizes for high I/O throughput. Latency-performance: Minimizes latency for real-time operations. Virtual-host: Ideal for virtualized environments. 🔹 Install Tuned: sudo apt install tuned # Ubuntu/Debian sudo yum install tuned # RHEL/CentOS 🔹 Start and Enable Tuned: sudo systemctl start tuned sudo systemctl enable --now tuned 🔹 List Available Profiles: tuned-adm list 🔹 Apply a Profile: sudo tuned-adm profile performance 🔹 Check Current Profile: tuned-adm active 🔹 Recommend Best Profile: tuned-adm recommend 🔹 Turn Off Tuning: tuned-adm off 🔹 Creating a Custom Profile: cd /etc/tuned sudo mkdir my_profile sudo nano /etc/tuned/my_profile/tuned.conf Example tuned.conf: [cpu] governor=performance [disk] elevator=deadline Apply it: sudo tuned-adm profile my_profile 🔹 Using cpupower for CPU-Specific Tuning: Install: sudo apt install linux-tools-common linux-tools-generic # Ubuntu/Debian sudo yum install cpupower # RHEL/CentOS Set CPU governor: sudo cpupower frequency-set -g performance Common governors: performance → Max performance powersave → Minimum power usage ondemand → Dynamically adjusts based on load Check CPU frequency info: cpupower frequency-info To make settings persistent, apply through tuned or add startup commands in /etc/rc.local or systemd units. #Linux #SysAdmin #CommandLine #DevOps #ITSkills #LinuxTips #Infrastructure #seismic #processing #geophysicist

💡 Why Being “Too Nice” Can Hurt — Even in Linux Scheduling Ever wondered how Linux decides who gets the CPU next? That’s the job of the Completely Fair Scheduler (CFS) — the core of process scheduling in modern Linux systems. CFS’s goal is simple: give every process a fair share of CPU time. But instead of using fixed “time slices,” it measures how much CPU each process has already received — something called virtual runtime (vruntime). 🧠 Here’s the cool part: Every runnable process has a vruntime. The one with the lowest vruntime (i.e., who’s had the least CPU recently) runs next. CFS stores processes in a red-black tree so it can quickly pick the fairest one. Now, fairness alone isn’t enough — some processes are more important than others. That’s where the nice value comes in. Each process has a nice level from -20 to +19: 🔺 Lower nice (e.g., -10, -20) → higher priority → gets more CPU 🔻 Higher nice (e.g., +10, +19) → lower priority → gets less CPU ⚙️ Default is 0 Internally, CFS converts this nice value into a weight that adjusts how fast vruntime increases: Heavy-weight (less nice) → vruntime grows slower → more CPU time Light-weight (more nice) → vruntime grows faster → less CPU time So yes — in Linux, when you’re too nice, you actually get less (CPU) attention! 😄 It’s a beautiful example of how fairness and efficiency are balanced mathematically in systems engineering — elegant, simple, and fair. Resource: Operating Systems: Three Easy Pieces. #Linux #CFS #OperatingSystems #SystemsProgramming #EngineeringInsights #Kernel

As part of our Introduction to Operating system course at kk Modi University Durg under the guidance of Dr. Wasim Ahmad Ansari sir, we explore the HBR case study "Red Hat and Linux Revolution ". Red Hat wasn’t selling a better operating system. In fact, Solaris and Windows NT were technically superior in many ways. What Red Hat sold was freedom. As CEO Bob Young said, “We were giving control of the operating system to the user.” That idea changed everything. The story begins with Richard Stallman, who rebelled against proprietary software and launched the GNU project to build a free Unix-like OS. But it was Linus Torvalds, a Finnish student, who created the missing piece—the Linux kernel. Together, GNU + Linux became a fully free operating system, powered by thousands of global contributors. Red Hat saw the chaos and turned it into clarity. They packaged Linux, added support, and made it user-friendly—like turning raw car parts into a sleek BMW. Their quirky release names (Halloween, Mother’s Day Plus One!) and rapid development cycles made them stand out. By 1999, Red Hat went public, and its IPO soared. Big players like Intel and Netscape invested. Even Microsoft took notice—and felt the heat. This case isn’t just about tech. It’s about vision, community, and challenging the status quo. As a student, it inspired me to think differently about innovation and collaboration. RedHat #LinuxRevolution #OpenSource #StudentPerspective #TechInnovation #CST102#kkmodi#btech2024

You use Linux every day, but I bet you can't define its Kernel in one sentence. It's the most critical piece of software you never directly see. 🤫 The Linux Kernel is the core of the operating system. Think of it as the ultimate translator and traffic cop between your hardware (CPU, RAM, storage) and all the software you run (Chrome, VS Code, a game). It lives in a protected area of memory, managing four essential jobs: 1. Memory Management: Deciding where to store all the data each running program needs. 2. Process Management: Allocating CPU time fairly so all applications run smoothly (that's scheduling). 3. Device Drivers: Handling communication with physical devices like your keyboard, mouse, and network card. 4. System Calls: Providing a safe, standardized way for your applications to ask the hardware for resources. Why is it used? Because without the kernel, every single application would have to include its own code to talk directly to every possible piece of hardware—a total nightmare. It provides abstraction and security. It's the invisible foundation that makes multi-tasking and stable computing possible on Linux (and Android, which runs on the Linux Kernel!). Which of the Kernel's four main jobs do you think is the most complex to manage? Let me know below! 👇 #LinuxKernel #OperatingSystems #TechExplained #DevOps #SystemAdministration #ShreyBhardwaj

This blew my mind when I first learned it 🐧 Why Linux Boots the Way It Does And Why It Matters Ever wonder what happens between pressing the power button and seeing your Linux prompt? The answer reveals one of Linux’s greatest strengths: modularity, transparency, and control. Here’s a simplified view of the magic: 1. The bootloader (like GRUB) loads the Linux kernel and a tiny temporary filesystem (initramfs) into memory. 2. The kernel wakes up: it detects hardware, configures memory, sets up CPUs, I/O, and storage all before user space even exists. 3. Once the real root filesystem is mounted, the kernel launches /sbin/init—the ancestor of every user space process on the system. 4. init (now usually systemd, but historically SysVinit) orchestrates services, manages logins, cleans up zombies, and ensures the system stays alive or shuts down gracefully. Unlike many proprietary OSes that treat booting as a black box, Linux exposes every layer You can inspect, modify, or replace any part from bootloader to init. The separation between kernel space and user space enhances stability and security. The init system’s process supervision ensures resilience failed services can be restarted automatically. Everything is scriptable, version-controllable, and reproducible perfect for servers, containers, and embedded systems. This isn’t just “how Linux works” it’s why Linux powers everything from Android phones to 90% of the public cloud. Image Credit - https://lnkd.in/gtApP3Ud #Linux #OperatingSystems #DevOps #SystemDesign #OpenSource #Engineering #SysAdmin #Kernel #BootProcess #TechDeepDive

Your computer is dying. Slowly. Every Single day. 💀 847 cache entries. Junk files everywhere. System crawling at 12% performance? Get Some Rest. 🐌 ** One Line Script + 10 seconds = Problem solved. I made 42BleachClean AKA Blc - Hybrid Cleaner "Both LINUX & MACOS" that transforms digital chaos into pristine performance. Here's the Reality: As a developer (Esp 1337/42 Students with only 5GB as total Storage), your disk is a Battlefield. VSCode caches, Chrome data, Discord files, old builds ... etc - they pile up like digital termites eating your precious gigabytes. The solution? Built 42BleachClean. So that "One Word + 10 seconds = Problem solved" This isn't just another Cache Cleaner. It's an Intelligent, Cross-Platform weapon designed for BOTH Personal machines and School Or Work environments, No Sudo Required Of Course. Here's what makes it special: 🔸 Hybrid Intelligence: Auto-detects MacOS vs Linux and targets platform-specific junk for maximum cleanup Based on each. 🔸 100% Safe Operations: No Sudo required for School Or Work computers, but scales up for personal machines with deeper cleaning wirth Extra Features. 🔸 Smart Targeting And Instant Results 🔸 Advanced Automation: Smart auto-cleaning schedules and disk space notifications for Personal laptops 🔸 Developer-Friendly: Written in pure Shell script - lightweight, fast, and completely transparent. No black boxess. Ready to reclaim your Disk Space? One installation command stands between you and Your Struggle: Curious about the code?  Explore the The REPO : https://lnkd.in/ednqep6f #ShellScripting #macOS #Linux #Productivity #OpenSource #1337school #42cursus #42network #cleaner #Automation #42BleachClean

🦀 Excited to share my latest open-source project: affinity-rs A cross-platform CPU affinity launcher built with Rust that simplifies process core pinning on Windows and Linux. The Problem -> Setting CPU affinity manually through Task Manager or calculating hex masks for PowerShell gets tedious, especially when you need to do it repeatedly for the same applications. The Solution -> affinity-rs provides a simple CLI interface with profile management, making CPU affinity control accessible and reproducible. Technical Highlights: ✅ Cross-platform using windows-sys and taskset ✅ Profile persistence with platform-specific config directories ✅ Desktop shortcut generation (.bat for Windows, .desktop for Linux) ✅ Zero-cost abstractions leveraging Rust's performance ✅ Simple human-readable syntax (no hex mask calculations) Real-World Applications: • Gaming optimization on hybrid CPU architectures (P-core/E-core) • Legacy application compatibility • Workload isolation for streaming/encoding • Reproducible benchmarking scenarios • Server process isolation Example workflow: affinity-rs myapp → Configure once → Reuse infinitely → Deploy via desktop shortcuts The tool uses Microsoft's official windows-sys API on Windows and taskset on Linux, with profile data serialized via serde_json to platform-appropriate config directories. Released under MIT license. Contributions and feedback welcome! 🔗 https://lnkd.in/eqCXqD9x #Rust #OpenSource #SystemsProgramming #PerformanceOptimization #CrossPlatform