To understand a lease, we must first understand the problem it solves.

Imagine a distributed lock as a simple, physical padlock on a shared resource, like a critical database table or a configuration file. When a server, or client, wants to use the resource, it acquires the lock, uses the resource, and then explicitly releases the lock. This ensures mutual exclusion: only one client can access the resource at a time.

Acquire → Hold → Release

In a perfect world, this works flawlessly. But distributed systems are inherently imperfect. They suffer from network latency, machine crashes, and unpredictable delays.

Consider the following scenario:

1. Client A successfully acquires the lock 🔑.

2. Client A begins working on the shared resource 🚧.

3. Before Client A can release the lock, one of two things happens:

   1. Client A crashes 😱 (a sudden power failure, for example).

   2. Client A experiences a network partition 🛜 (it gets temporarily cut off from the rest of the network).

In both cases, Client A is gone or unreachable, and the lock is never explicitly released. The resource is now deadlocked. No other client can ever access it, and the entire system grinds to a halt. The lock, intended to protect the resource, has become its prison.

The client holding the key has disappeared, leaving the door locked forever.

This is the critical vulnerability of a simple, non-expiring lock in an unreliable environment.

🔑 Leases : A Lock with an Expiration Date ⏱️

A lease is essentially a time-bound grant of authority over a resource. Think of it not as owning a key forever, but as renting a resource for a fixed period.

The fundamental difference between a lock and a lease is the automatic expiration.

The Lease Mechanism

When a client requests a lease, the Lease Grantor issues it with a TTL, say 60 seconds. The client now has exclusive access to the resource for that 60 seconds.

1. Acquire: Client A requests and receives a lease for 60 seconds.

2. Use: Client A uses the resource, knowing it has 60 seconds of guaranteed access.

3. Renew: If Client A needs more time, it must send a renewal request to the Grantor before the 60 seconds are up. The Grantor resets the timer.

4. Expire: If Client A crashes or is partitioned, it cannot renew the lease. After 60 seconds, the lease automatically expires, and the Grantor is free to issue a new lease to another client.

Where Leases Are Used

Leases are a fundamental building block in modern, highly available distributed systems. They are the core mechanism that allows systems to recover gracefully from failure.

1. Leader Election

In many distributed systems (like those using protocols such as Raft or Paxos, or systems like Kubernetes and etcd), one server must be designated as the Leader to coordinate all activity.

Mechanism: Servers compete to acquire a lease on a specific “leader” resource. The server that successfully holds the lease is the current leader.

Fault Tolerance: If the leader fails, its lease expires. The other servers (followers) detect the expiration and immediately start a new election process to acquire the now-free lease, ensuring the system quickly gets a new leader.

2. Cache Consistency

When a client caches data locally to improve performance, the central data store needs a way to ensure that cached data doesn’t become stale.

Mechanism: The central store grants a client a cache lease on a piece of data. While the lease is active, the central store promises not to allow any other client to modify that data. This allows the client to use its local copy with high confidence.

Performance: The client avoids constantly checking with the central store, leading to massive performance gains. If the central store needs to update the data, it can try to revoke the lease, or simply wait for it to expire.

3. Distributed Coordination

Leases are used to coordinate access to any shared resource, ensuring that even if a client fails mid-operation, the resource is eventually released.

The diagram below shows how a Lease Grantor (Resource Manager) manages the flow of authority between server nodes:

The Grantor ensures that when a lease expires on one node, the authority can be safely passed to the next waiting node, maintaining system continuity.

Advantages

The adoption of leases over simple locks provides two critical advantages that are essential for building robust systems:

1. Superior Fault Tolerance

This is the primary benefit. By making the default state of a resource unlocked after a timeout, leases solve the deadlock problem. The system’s recovery from failure is no longer dependent on a potentially crashed or partitioned client sending a “release” message.

2. Improved Performance and Reduced Communication

Leases allow a client to operate with a high degree of certainty for the duration of the lease. This reduces the need for constant, expensive communication with the central authority.

For example, a client with a 60-second lease on a resource knows it has 60 seconds of exclusive access. It can perform multiple operations locally without having to check back with the central lock manager for every single step. This batching of operations significantly reduces network overhead and latency, leading to a much faster and more efficient system overall.

Common Mistakes with Leases

1. Lease Too Long → Slow failover

When a node crashes, others must wait for the lease to expire before taking over.
Long leases delay recovery and keep the system unavailable longer than necessary.

2. Lease Too Short → Frequent leadership churn

Short leases require very frequent renewals to stay valid.
Small delays or GC pauses can cause leadership to change unnecessarily.

3. No Renewal Buffer → Accidental lease loss due to clock skew

If renewal happens too close to expiry, time drift or network latency can cause failure. The node may lose its lease even though it is still healthy.

💡 How to Overcome These Issues (A Safe Pattern)

Renew the lease well before it expires, instead of waiting until the last moment.
A common and safe practice is to renew at 50–70% of the TTL, leaving enough buffer for delays, clock skew, or brief pauses.

Accidentally committing a .env file is a common Git mistake, and it often happens before proper guardrails are fully established.

Fixing it properly requires understanding how Git tracks files and why deleting a file is not the same as deleting its history.

This post walks through the correct steps to completely remove a .env file from Git and prevent it from being committed again.

rm .env
git commit -m "remove env"

//or

echo ".env" >> .gitignore

This does not remove the secret. Because Git doesn’t track files the way your file system does.

How Git actually works

Each commit is a full picture of the repository at that moment.

Git has three layers:

Once a file enters Commit History, it stays there forever, unless history is rewritten.

.gitignore only affects untracked files.

If .env was ever committed:

• Git already knows about it

• .gitignore will be ignored for that file

That’s why we must explicitly remove it from tracking.

Step 0: Assume the secrets are compromised [ Important ]

Before touching Git, rotate any keys, tokens, or passwords that were in the .env file. Even if you delete the file later, anyone who saw the repo earlier may still have access.

Step 1: Add .env to .gitignore (prevents re-commit)

echo ".env" >> .gitignore
git add .gitignore
git commit -m "Ignore .env"
git push

// or create ~/.gitignore in home directory and make it global
git config --global core.excludesfile ~/.gitignore

.gitignore prevents new tracking. If .env was already committed, Git will still track it until you untrack it (next step).

Step 2: Stop tracking .env in the current state

If .env is already tracked, remove it from Git (without deleting it locally):

// Remove .env from Git tracking, but don’t delete the local file.”
git rm --cached .env


//Now commit and push:
git commit -m "Stop tracking .env"
git push

From this point forward:

• .env is no longer tracked

• .gitignore works correctly

• future commits are safe

⚠️ But the secret still exists in older commits

Step 3: Remove .env from the entire Git history

This is the step that actually removes the trace.

Install git-filter-repo

brew install git-filter-repo     # macOS
# or
pip install git-filter-repo

git filter-repo --path .env --invert-paths 

This rewrites all commits and removes .env everywhere. basically it would look like below and Commit hashes change because the history is rebuilt.

Before:
A ── B ── C ── D   (contains .env somewhere)

After:
A'── B'── C'── D'  (no .env anywhere)

Step 4: Force-push the cleaned history to remote

Now publish the cleaned history to the remote repository:

git push --force --all
git push --force --tags

This replaces the remote history with your cleaned history.

Step 5: Verify .env is gone from history

// This should return nothing
git log --all -- .env 

Important to remember

• ⚠️ Force-pushing rewritten history breaks existing clones. So, before running the final cleanup steps:

  • Inform your team that history will be rewritten

  • Ask them to pause merges and pushes

  • Make sure no active work depends on the old commit history

  • This avoids broken branches and accidental re-introductions of the file.

• ⚠️ Don’t assume GitHub “purges” secrets automatically. You should rotate secrets regardless because PR diffs/forks/caches may still expose content.

• Team members or contributors should re-clone, or hard reset

git fetch --all
git reset --hard origin/main

• ⚠️ GitHub cannot delete history from forks you don’t control.

If someone forked the repo before cleanup:

• Their fork still contains the old history

• GitHub support cannot remove that

This is another reason secret rotation matters more than cache clearing.

Tag: Engineer’s Toolbox
⏳ 10 min focus read, best read on Desktop includes code and screenshots

Hello Everyone 👋 I’m considering adding more practical, deep-dive software engineering and data related tech stack explorations alongside the usual AI posts. Would you be up for that? Thanks to everyone who took a moment to fill out my recent survey, your responses mean a lot.

Start Survey

You’ve got a new laptop, maybe it’s your own, or one from a new startup job and the first thing you want is for your terminal or IDE to feel right.
Your shell prompt ( .bashrc, .zshrc ), your aliases, your shortcuts, Vim ( .config/nvim ), everything configured perfectly to how you work.

• custom color schemes that make logs readable.

• prompt themes that show your current branch or virtualenv.

• handy aliases that cut down on repetitive typing.

But fast forward a year.

You’ve switched companies, or maybe you just want to mirror your setup across two machines. Suddenly, that finely crafted environment is gone. A blank terminal window stares back no aliases, no colors, no familiar shortcuts.

Painful, right? Let me introduce a small but powerful thing….

What is Dotfiles ?

• A dotfile is a configuration file whose name begins with a dot (e.g. .bashrc, .zshrc, .claude, .gitconfig, .tmux.conf). These are called hidden files by default in Unix-style systems.

• It is like a developer blueprint, a version-controlled way to preserve your preferences, configurations, and scripts. They turn the painful setup process into a one-command sync, so you can bring your workspace anywhere.

• Developers often collect all such customizations into a single Git repository (often named dotfiles) so they can easily back up those configs (history, diffs, rollbacks), sync them across multiple machines and share useful configs (open source your dotfiles).

🤖 Dotfiles for Agent Developers

Although AI dotfiles are generally stored within project repositories for version control, you can also maintain a personal set in your home directory for private or global agent configurations that don’t interfere with project-level setups. These personal settings can even be integrated into your IDE, AI-powered terminal, or any other workspace tools you use.

Example: .agentrc

# ~/.ai-dotfiles/.agentrc
# example
name: “analytics-engineer”
role: “AI data assistant for local projects”
personality: “curious, concise, technical”
goals:
  - Assist in analyzing, visualizing, and generate code
  - Maintain consistent style
capabilities:
  - visualize_data
  - summarize_data
startup_message: “Analyst ready to assist”

Example: .aiconfig

# ~/.ai-dotfiles/.aiconfig
model: “llama3.1”
temperature: 0.7
providers:
  openai:
    api_key: “{{ env.OPENAI_API_KEY }}”
  ollama:
    model_path: “/usr/local/share/ollama/models/”
memory_backend: “local”
log_level: “info”

Example: Role-Based Prompt

# ai-dotfiles/prompts/data_analyst.md
# Data Analyst Template

You are a data analyst AI. Your task is to understand, summarize, and visualize datasets provided in CSV, JSON, or tabular formats.

Focus areas:
- Key trends and patterns in the data
- Missing values, anomalies, or outliers
- Summary statistics (mean, median, counts, etc.)
- Suggested visualizations (histograms, bar charts, line plots, scatter plots)

Respond in this format:
1. **Summary**
   - Brief overview of the dataset
   - Key observations
2. **Insights**
   - Interesting trends or patterns
   - Any anomalies or noteworthy points
3. **Suggested Visualizations**
   - List of charts or plots that help understand the data
   - Include recommended libraries or tools (e.g., matplotlib, seaborn)
4. **Next Steps**
   - Recommendations for further analysis or modeling

🧠 Addressing Key Challenges

❓ Managing Sensitive Data in AI Dotfiles

AI dotfiles may have 2 kinds of files:

1. Public setup files : safe to share (prompt templates, skills, workflows)

2. Private secrets : must never be shared (API keys, memory snapshots, tokens)

✅ Solution

To safely version-control your repo, you can use a .gitignore file and ensures your secrets stay local while still keeping your AI dotfiles organized and portable.

# ~/.ai-dotfiles/.gitignore

# Sensitive AI configs
.aiconfig
memory/
*.env

❓ Config files must stay in the right place

Tools like Bash, Git, and nvim look for configs in very specific places like:

~/.bashrc              # Bash default location
~/.gitconfig           # Git default location
~/.config/nvim         # Nvim default location

If you move those into subfolders like ~/dotfiles/git/.gitconfig, they simply won’t be loaded anymore. Now, How do we organize the dotfiles in subfolders structures mentioned in Figure 1.

✅ Solution : Bare Git repository

Normally, when you create a Git repo, Git adds a hidden .git folder inside that directory to track changes. But you don’t want a .git folder sitting in your home folder, that would look messy.

A bare Git repository has no working directory, only the Git database (commits, branches, etc.). So instead of having .git inside your home folder, you store it somewhere else (like ~/.config) and tell Git to treat your home directory as the working tree.

• All your real config files stay where they are (~/.bashrc, ~/.zshrc, etc.)

• The dotfiles/ folder secretly holds your Git history

• You manage everything using the config command ( you can alias whatever you like )

❓ Managing Subfolder Structures in the Repository

You want to organize dotfiles into subfolders (bash/, git/, nvim/, ai-dotfiles/ ) for clarity and maintainability, but you still need the tools to load the files correctly.

✅ Solution : Hybrid Dotfiles Setup

1️⃣ Track live configs locally with a bare repo (.dotfiles)

• Keep your actual dotfiles (~/.bashrc, ~/.gitconfig, .agentrc, etc.) in their normal locations because tools expect them there.

• Use a bare Git repository (~/.dotfiles) to track these files.

• This allows you to version-control your live environment, keep full history, and roll back changes without moving or reorganizing your files.

• Your secrets remain local and are never accidentally pushed.

2️⃣ Mirror configs to a structured, shareable folder (dotfiles/)

• Create a copy or symlink structure in ~/dotfiles/ for publishing or backup.

• Organize files into subfolders (e.g., bash/, git/, ai-dotfiles/) so it’s easy to browse and maintain.

• Ignore sensitive files (like .aiconfig or memory/) to keep secrets safe.

• Use a small script (sync_dotfiles.sh) to automatically mirror changes from your live environment to the structured repo.

3️⃣ Benefits of this hybrid approach

• .dotfiles = private journal of live configs, tracks everything locally, including experiments

• dotfiles/ = curated, public-friendly snapshot ready for GitHub or sharing

• Keeps your workflow safe, organized, and reproducible on any machine, and protecting secrets.

• Your dotfiles make your developer setup reproducible.

• Your ai-dotfiles make your agent setup reproducible.

🧩 Let’s Put It Into Practice

For a clear example and all the scripts mentioned below, please check out my personal dotfiles repository: https://github.com/kannandreams/dotfiles

1️⃣ Push to Repository (Sync Dotfiles)

Step 1: Track live files in $HOME using bare repo

Step 2: Mirror into structured repo (dotfiles/)

# Run sync script to copy live configs into organized folders
~/dotfiles/sync_dotfiles.sh

Output

2️⃣ Setup a New Machine (Bootstrap)

Tools & Inspirations

As your dotfiles grow, you may want more structure or tooling. Here are common tools and few popular repos for inspiration

• chezmoi — allows templating, encryption of secrets, multi-OS support.

• YADM — “Yet Another Dotfiles Manager”

• General-purpose dotfiles utilities

• Tom Payne’s dotfiles managed with chezmoi GitHub

• thoughtbot’s dotfiles (uses rcup, local override pattern) GitHub

• holman’s dotfiles (topic-based layout, auto symlinking script) GitHub

🤖 AI runs on compute. I run on coffee. Fuel the next post if you enjoyed this one!

Buy KK a coffee

From an engineering perspective, it’s fascinating to see how this concept scales. VMTs and Python’s dynamic dispatch form the backbone of how ML frameworks like PyTorch and TensorFlow route computations, and how AI agents dynamically decide which tools to execute.

Let’s understand these concepts with practical code examples.

What is a Virtual Method Table?

A Virtual Method Table (VMT), also known as a virtual function table or vtable, is a mechanism predominantly used in object-oriented programming languages such as C++ to implement dynamic dispatch or runtime polymorphism.

Key characteristics of VMTs include:

• Dynamic Dispatch: Enables selecting and invoking the appropriate method implementation at runtime based on an object’s actual type rather than its declared type.

• Polymorphism: Allows objects from different classes to be treated as instances of a common base class, letting a single interface represent multiple underlying behaviors.

• Modularity and Extensibility: Supports modular design by making it easy to introduce new behaviors through derived classes, helping systems evolve without major refactoring.

Lets try to understand with C++ code example.

What’s Happening 👓

1. VMT generation: Both Animal and Dog class have their own VMTs (arrays of pointers to virtual functions).

2. vptr: Each object stores a hidden pointer (vptr) to its class’s VMT.

3. Dynamic dispatch: When a->speak() is called, the program looks at a’s vptr to find the correct function in the VMT. Because a points to a Dog object, Dog::speak() is executed, not Animal::speak().

Python: Dynamic Dispatch Without VTables

Every class in Python has a __dict__ , a mapping of method names to function objects. When you call a method, Python looks it up dynamically:

When you run d.speak(), Python:

1. Checks Dog.__dict__ for speak().

2. If not found, walks up the Method Resolution Order (MRO) to Animal.

   1. Disassembling the Method Call (Bytecode Proof)

This lookup chain is Python’s virtual dispatch mechanism. It’s slower than a C++ vtable but infinitely more flexible, you can even replace methods at runtime (a.k.a. monkey patching).

How ML Frameworks use Virtual Dispatch

If you thought VMTs and dynamic method lookup were just for C++ or Python classes, No, Modern ML frameworks like PyTorch and TensorFlow rely on the same core principle of virtual dispatch, they just use it at a much larger, dynamic scale.

For example, in PyTorch we have:

• Models as classes = usually subclassing nn.Module (PyTorch)

• Forward methods = Each models define computation

• Dynamic dispatch = ensures the right forward method is called, without the training loop caring about the specific model.

Base Class

Subclass (Specific Model)

Training Loop (Using Polymorphism)

What’s Happening 👓

1. Model object (MyModel) contains a reference to its class.

2. Call to model(x) triggers Python’s __call__ on nn.Module, which internally calls self.forward(x).

3. Dynamic dispatch looks in MyModel.__dict__ for forward(). If it weren’t found, it would walk the MRO up to BaseModel.

4. Training loop doesn’t care what the model is, it just uses forward().

How AI Agent Frameworks apply Dynamic Dispatch

We started by looking at VTables in C++, then saw how Python uses dynamic dispatch instead of rigid tables ( “rigid table” here refers to how C++ implements polymorphism through a fixed, precomputed structure ). We also explored ML frameworks like PyTorch, where dynamic dispatch lets the training loop call forward() without knowing the exact model.

Now, the same principle is applied in AI agent frameworks like LangChain, LlamaIndex, and OpenAI agents, extend this idea one step further.

1. Interface guarantee: Each tool provides a known interface (e.g., a run() method).

2. Dynamic method dispatch: When the agent calls tool.run(), Python dynamically dispatches to the correct implementation just like it does for a model’s forward() method.

3. Intelligent tool selection: Unlike a simple loop over tools, the agent can decide at runtime which tool to invoke based on the input. For example, it can pick the MathTool for “2 + 2” and SearchTool for “Python AI frameworks”. ( check below tool example )

So the agent framework is essentially doing two layers of dynamic dispatch:

• Method-level dispatch: Python ensures run() of the correct tool is called.

• Tool-selection dispatch: The agent’s reasoning logic decides which tool to use without needing to know in advance.

The agent doesn’t need to know the exact tool ahead of time, but Python’s dynamic dispatch ensures the correct method is executed once the agent picks the tool.

1. Tool Definitions (Base + Subclass Analogy)

• This part defines the base interface ( tool ) and specific tools (SearchTool, MathTool).

• Python dynamic dispatch ensures that func() calls the correct implementation when invoked.

• Analogous to base class + subclass in OOP.

2. Agent Initialization

• This part is like the training loop in ML: it doesn’t know the specifics of each tool.

• The agent is ready to dynamically pick a tool at runtime based on input.

3. Dynamic Tool Invocation (Runtime Dispatch)

• Layer 1 (Tool Selection): Agent decides which tool to call based on the query.

• Layer 2 (Method Dispatch): Python dynamically calls the correct func() of the chosen tool.

• This mirrors the ML model forward() dispatch and the VTable example: flexible, runtime method resolution.

Why Dynamic Dispatch Matters in AI:

• Hardware Abstraction: AI frameworks like TensorFlow or PyTorch can automatically choose the best code for the hardware you have. For example, a matrix multiplication will run on a GPU if available, or fall back to the CPU otherwise.

• Polymorphism: Different types of layers (like ConvolutionalLayer, ReLULayer, DropoutLayer) can each have their own forward() and backward() methods. Dynamic dispatch makes sure the right method runs for each layer at runtime.

• Flexibility: AI frameworks need to support many models, layers, and optimizers. Dynamic dispatch lets developers add new features, like a custom activation function, without changing the core code.

🤖 AI runs on compute. I run on coffee. Fuel the next post if you enjoyed this one!

Buy KK a coffee

Further Readings

https://legacy.python.org/workshops/1998-11/proceedings/papers/lowis/lowis.html

https://python.langchain.com/docs/how_to/callbacks_custom_events/

I don’t want this article to be another one of those. Instead, let’s focus on one groundbreaking concept that has the entire Python community excited.

So, what’s the big deal?

What does it means?
🔥 True parallel execution across multiple CPU cores
⚡ Faster multi-threaded performance
🚀 Advantages in data science, AI, and data engineering workloads.

Let’s break down what this really means starting from the basics to benchmark comparison and visualize it.

Understanding Threads and Concurrency

In computing, a thread is simply a sequence of instructions that a processor can run independently. A single Python process can have multiple threads, each doing some task like reading files, making network calls, or crunching numbers.

Think of a program like a recipe. Some steps can happen at the same time . For example, boiling water while chopping vegetables. Each of those simultaneous activities represents a thread. Threads allow a program to handle multiple tasks at once.

It’s often achieved through multithreading, where a single program creates several threads that appear to run at the same time. True parallelism happens only when multiple processor cores execute threads simultaneously.

However, even on a single core, a computer can rapidly switch between threads creating the illusion of parallel execution and making programs feel faster and more responsive.

The Global Interpreter Lock (GIL)

For many years, Python’s primary implementation, CPython, has included a mechanism known as the Global Interpreter Lock (GIL). The GIL is a mutex (a type of lock) that protects access to Python objects, ensuring that only one thread can execute Python bytecode at any given time, even on multi-core processors.

Why the GIL Exists?

The GIL was originally added to Python to make memory management easier and to prevent race conditions, situations where multiple threads try to change the same data at once. Without the GIL, CPython’s memory handling would be much more complicated and prone to deadlocks or data corruption. At the time, most Python programs were single-threaded, so the GIL made the interpreter simpler, safer, and easier to maintain.

How the GIL Works

When a Python program runs multiple threads, the GIL ensures that only one thread can execute Python bytecode at a time. Before running, a thread must first acquire the GIL. After executing a set number of operations or performing an I/O task, it releases the lock so another thread can take over. As a result, even on a multi-core CPU, only one Python thread can run Python code at any moment and the rest are simply waiting for their turn.

Performance Implications

CPU-bound tasks

The GIL greatly affects Python’s performance, especially for CPU-bound tasks, those that spend most of their time doing calculations rather than waiting for I/O. In these cases, Python’s multithreading doesn’t achieve true parallel execution across multiple CPU cores. Instead, threads compete for the GIL, and the constant switching between them can actually make a program slower than a single-threaded one due to context-switching overhead.

I/O-bound tasks

On the other hand, for I/O-bound tasks such as network requests or file reads, the GIL’s impact is much smaller. When a thread performs an I/O operation, it usually releases the GIL, allowing other threads to run in the meantime. This makes it possible to achieve good concurrency for I/O-heavy workloads, even with the GIL in place.

Python 3.14: (No-GIL) Python

After years of R&D, Python 3.14 marks a pivotal moment in Python’s history: the free-threaded (No-GIL) version of the Python interpreter.

PEP 703 – Making the Global Interpreter Lock Optional in CPython

PEP 779 – Criteria for supported status for free-threaded Python

This means that developers can now officially leverage Python without the constraints of the Global Interpreter Lock, opening up new possibilities for true parallel execution in multithreaded Python applications.

CPython’s internal architecture has undergone major changes, particularly in memory management and object safety, to enable thread-safe execution without depending on a global lock.

• True Parallelism: CPU-bound tasks can finally benefit from multiple cores, potentially leading to significant speedups.

• Simplified Concurrency Models: Developers no longer need to rely on multiprocessing (which bypasses the GIL by running separate Python processes) for CPU-bound parallelism, simplifying application design.

• Improved Responsiveness: Applications can remain more responsive, as long-running computations in one thread won’t block other threads from executing.

• Architecture Changes:

Benchmark Comparison

Prime Number computation

Let’s look at a classic CPU-bound problem, prime number computation.
This task involves heavy mathematical operations and minimal I/O, making it ideal for testing true parallel execution.

• My mac configuration: macOS Sequoia arm64 | Mac mini ( M1, 2020 ) | 16 GB Mem.

• Benchmark code : https://github.com/kannandreams/when-engineers-meet-ai/tree/main/code/python-gil

Python == 3.12.10 (with GIL)

Even though I launched 4 threads, the total execution time is almost the same as the single-threaded version. That’s because of the GIL

Python == 3.14 (no-GIL)

The below chart shows

• Python 3.14t shows significant multi-threading performance gains, with 4-thread execution being 2.8x faster at the largest range.

• Python 3.12.10 shows minimal benefit from multi-threading, likely due to GIL limitations, with multi-threaded execution being slightly slower.

• Python 3.14t is consistently faster in both single and multi-threaded scenarios, with the performance gap widening significantly with multi-threading.

The below chart shows how much faster multi-threaded (4 threads) execution is compared to single-threaded execution.

• Python 3.14t: Shows increasing speedup with larger ranges, achieving ~2.8x speedup at 20M primes

• Python 3.12.10: Shows speedup close to 1.0x (no improvement), indicating GIL prevents effective parallelization

Python feels effortless on the surface due to its clean syntax and productivity are a big reason it’s so popular. But like an iceberg, there’s a lot hidden underneath.

When performance challenge us, the usual reaction is to scale up with more cloud resources instead of optimizing core logic at low level.

Python struggles with things like parallel execution because of the Global Interpreter Lock (GIL). It lacks strong static typing, offers limited control over low-level operations, and doesn’t support native GPU execution. While there are just-in-time compilers like PyPy, they’re not commonly used in the AI ecosystem.

Two-Language / Two-World Problem

Especially in AI development, things often split into two phases:

Data scientists and Researchers typically use Python with libraries like NumPy, Pandas, ML Libraries or PyTorch to quickly experiment and build models. It’s great for prototyping. But when we need to train large models or to deploy those models to production and run them at scale, performance issues start to surface.

In the end, teams often have to rewrite performance-critical parts of their code in faster, lower-level languages like C++, Rust, or CUDA to run efficiently.

Take PyTorch, for example. Although it is Pythonic interface, many of its core operations and critical components are written in C++ and CUDA.

The torchscript and custom CUDA extensions are used when Python isn’t fast enough.

This gap between high-level development and low-level optimization slows teams down. To keep up with the rapid pace of AI, there's also a growing need to optimize for hardware efficiency, energy usage, and adopt AI-native hardware platforms.

Imagine Python’s Simplicity with the Speed of Rust

What if we had a language that looked like Python, but was built on a high-performance compiler infrastructure designed to optimize code at the systems level, and to run efficiently on CPUs, GPUs, and other hardware platform without needing to rewrite the code?

def main() {
    print("Hello, World!, Welcome to Mojo");
}

# run mojo hello.mojo

What is Mojo ?

Mojo is a programming language developed by Modular Inc., a company co-founded by Chris Lattner, the creator of LLVM, Clang, and the Swift programming language.

• MOJO’s development began with the ambitious goal of creating a unified language that could serve the entire AI development lifecycle from initial research and prototyping to production deployment and optimization.

• The language was designed to be a superset of Python, meaning that existing Python code could run on MOJO with minimal or no modifications, while new code could take advantage of MOJO's advanced performance features when needed.

If I had to define what Mojo is in one line:

Python vs Mojo vs Rust - Speed Test

Mojo claims to be 68,000 times faster 🚀 than Python

The following chart shows the avg. time (in secs) it takes to compute the n-th fibonacci number in Python, Mojo, and Rust for different values of n:

How Mojo Works?

Architectural Foundations

Mojo is built on MLIR (Multi-Level Intermediate Representation) compiler framework, which is a key reason it stands out in accelerating Python, especially for AI workloads.

MLIR, also created by Chris Lattner during his time at Google, represents a more sophisticated approach to compiler design that allows for multiple levels of abstraction within a single framework. This architecture enables MOJO to perform optimizations that would be difficult or impossible with traditional LLVM-based compilers.

Before we go any further, it’s important to understand two foundational concepts at a high level: LLVM and MLIR.

In the coming months, I plan to publish a detailed blog post that not only explains “How LLVM works” but also explores the core principles of compiler infrastructure as I continue learning.

High-Level Overview of LLVM and MLIR

LLVM is a popular compiler infrastructure used by many programming languages. Think of it as a powerful toolkit that helps converting your high-level code (like Python) into efficient machine-level instructions that computers can run. It’s great at optimizing code for performance.

MLIR builds on LLVM but takes a step further. Instead of working with just one low-level representation of code, MLIR introduces multiple levels of intermediate representations. This multi-level design makes it easier to handle complex programming tasks, support new languages, and optimize code across different hardware platforms.

Hardware-Agnostic Approach

Mojo, built on MLIR, uses multi-level intermediate representations (IRs) to target a wide range of hardware, including:

• General-purpose CPUs

• Parallel and accelerated processors like GPUs, TPUs, and NPUs

• Custom hardware such as FPGAs and ASICs

• Emerging computing platforms like QPUs

Because of MLIR, developers don’t need to learn different programming styles for each type of hardware. Unlike CUDA, which only works with NVIDIA GPUs and has its own programming language, Mojo can generate optimized code for many kinds of devices, all from the same simple code.

📌 This hardware-agnostic approach is particularly valuable as the AI hardware landscape continues to diversify. Companies like AMD, Intel, and many startups are developing specialized AI chips, and cloud providers are offering their own custom silicon (specialized semiconductor chips) for machine learning workloads. This is a significant advantage for anyone looking to avoid vendor lock-in or optimize for specific hardware configurations.

Key Features of Mojo

MOJO's design philosophy is to provide developers with the flexibility to choose their level of abstraction and performance optimization. This approach is manifested through several key features that distinguish it from both traditional Python and other high-performance programming languages.

1. Dual Functions : def vs fn

Mojo supports two keywords to declare functions: def and fn.

Use def for Python-style dynamic functions, or fn when you want higher performance and stricter behavior.

2. Memory Management and the Borrow Checker

Mojo offers a streamlined approach to memory management, inspiration from Rust's ownership system to ensure memory safety without the performance cost of garbage collection.

• Mojo's memory management is a “borrow checker” that analyzes code as it is compiled. This tool prevents common memory-related bugs such as using memory after it has been freed, freeing memory twice, and data races.

• The goal is to provide these safety guarantees in a way that is more user-friendly for Python developers and provide clear error messages to fix memory issues quickly.

3. Automatic Vectorization and Parallelization

• Mojo has built-in features to manage concurrent operations and makes parallel programming easier by providing built-in tools take full advantage of hardware.

• It automatically manages threads and uses SIMD ( Single instruction, multiple data ) vectorization, so developers can concentrate on their program’s logic without worrying about low-level details.

For example, its parallelize function efficiently splits loop iterations across multiple CPU cores, which is perfect for tasks that can run in parallel on data.

Below sample code using parallelize in Mojo that squares each element of an array in parallel using all available CPU cores.

4. Struct-Based Object Model

Mojo's struct is similar to Rust’s struct, designed for speed, safety, and low-level control.

Unlike Python classes, Mojo structs are value types with a fixed layout known at compile time. This makes them highly efficient for performance-critical tasks like AI and systems programming. Yet, they retain object-oriented features like methods and operator overloading.

Mojo’s Use Cases

• AI Workflow & Edge AI: Mojo optimizes tensor operations, backpropagation, and gradient descent during training. Its efficient runtime and hardware-agnostic codegen make it ideal for edge AI, real-time systems, and model serving.

• Systems Programming for AI Infrastructure: Mojo isn’t just for model code. It can also power AI infrastructure like data pipelines, custom drivers, and high-performance runtimes.

References

1. Multi-Level IR Compiler Framework

2. Mojo: Powerful CPU+GPU Programming

3. Why Mojo - https://docs.modular.com/mojo/why-mojo

4. Understanding the Global Interpreter Lock in Python

5. Mojo vs. Rust: what are the differences?

6. Python vs Mojo vs Rust: A Fibonacci Speed Test

💡 Enjoyed this post? A coffee helps fuel the next one!

Buy KK a coffee

The Service

The service folder usually contains the business logic of the application. It is responsible for performing operations that the application is designed to do. For instance, in a simple e-commerce application, a service might handle functions such as adding or removing products from the cart, placing an order, processing payment, and updating the order status.

Here is an example of a simple service function in Go:

package service

func PlaceOrder(order Order) error {
    // Validate order details
    // Process payment
    // Update order status
    // Send confirmation email to the user
    return nil
}

The Handler

The handler folder typically contains the HTTP request handlers. These handlers are responsible for receiving HTTP requests from the user, performing any necessary validation or parsing, and then passing control to the appropriate service function.

Here is an example of a simple handler function in Go:

package handler

func PlaceOrderHandler(w http.ResponseWriter, r *http.Request) {
    // Parse order details from the request body
    // Call the PlaceOrder function from the service package
    // Return appropriate response to the user
}

The Controller

The controller folder typically contains the functions responsible for handling user input, such as form data or user input data from the command line. These functions parse the input data, validate it, and then call the appropriate service function to perform the desired operation.

Here is an example of a simple controller function in Go:

package controller

func PlaceOrderController() {
    // Prompt user to input order details
    // Parse and validate the input
    // Call the PlaceOrder function from the service package
    // Display appropriate response to the user
}

How the process works

1. When the user submits a form or clicks a button on a web page, an HTTP request is sent to the server.

2. The request is routed to the appropriate handler function based on the URL or route specified in the request.

3. The handler function receives the request and any associated data, such as form data or URL parameters.

4. The handler function then passes the data to the appropriate service function to perform any necessary business logic.

5. The service function performs any necessary operations, such as saving data to a database or performing calculations, and returns the result to the handler function.

6. The handler function then returns an HTTP response to the user, typically in the form of an HTML page or JSON data.

7. If there are any errors or exceptions during this process, they are handled appropriately, such as by returning an error message to the user or logging the error for later analysis.

The controller function acts as an interface between the front-end and the backend, handling user input and calling the appropriate handler function to process the request. The handler function is responsible for handling the HTTP request and invoking the appropriate service function to perform any necessary business logic. Finally, the response is returned to the controller function, which can then display the appropriate response to the user on the front-end.

So, in summary, the process involves a request being handled by the appropriate handler function, which calls the necessary service function to perform any business logic, and returns a response to the controller function, which can then display the result to the user on the front-end.

Does Router consider as Controller?

The router can be thought of as a part of the controller layer, as it is responsible for routing incoming HTTP requests to the appropriate handler function based on the URL or route specified in the request.

However, in the context of a typical MVC (Model-View-Controller) architecture, the router would be considered a part of the infrastructure or framework layer, rather than the controller layer.

The controller layer is typically responsible for handling user input and managing the flow of data between the front-end and the backend. This includes tasks such as validation, authentication, and authorization.

The handler functions are responsible for handling HTTP requests and invoking the appropriate service function to perform any necessary business logic.

So, while the router can be considered a part of the controller layer in the sense that it routes requests to the appropriate handler function, it is not typically considered to be a part of the controller layer in the context of a typical MVC architecture.

Advantage of using Controller?

By using a controller, we can separate the handling of HTTP requests from the business logic of the application. The controller handles tasks such as parsing request parameters, performing validation, and returning appropriate HTTP responses to the client. The handler function is responsible for performing the actual business logic of the application.

This separation of concerns makes the code more modular, easier to test, and easier to maintain. It also allows for more flexibility in the application design, as different controllers can be used to handle different types of HTTP requests or to interact with different parts of the application.

Additionally, using a controller allows us to encapsulate any application-specific logic, such as authentication or authorization, within the controller layer. This can help to keep the business logic layer more focused on the core functionality of the application.

So, while it is possible to call the handler function directly from the HTTP request handler, using a controller provides an additional layer of abstraction that can be beneficial for the overall design and maintainability of the application.

Organize these components in Go project

myapp/
├── cmd/
│   ├── main.go
│   └── ...
├── internal/
│   ├── config/
│   │   ├── config.go
│   │   └── ...
│   ├── controllers/
│   │   ├── user_controller.go
│   │   └── ...
│   ├── handlers/
│   │   ├── user_handler.go
│   │   └── ...
│   ├── models/
│   │   ├── user.go
│   │   └── ...
│   ├── services/
│   │   ├── user_service.go
│   │   └── ...
│   └── ...
├── static/
│   ├── css/
│   │   └── ...
│   ├── js/
│   │   └── ...
│   └── ...
├── templates/
│   ├── index.html
│   └── ...
└── ...