File Locking in Go: Cross-Process Synchronization

Summary

In the world of concurrent programming, Go developers are well-versed in using sync.Mutex or channels to manage access to shared resources. However, these tools only work within a single process. When you have multiple processes—such as a distributed agent, multiple instances of a microservice, or a cron job overlapping with a CLI tool—trying to access the same file, you enter the danger zone of race conditions. To solve this, we need Operating System-level file locking.

The Problem: The Process Boundary

Imagine you have a CLI tool that appends logs to a file. If you run two instances of this tool simultaneously, they might try to write to the file at the exact same time. The OS might interleave their writes, resulting in garbled log lines. A sync.Mutex in your Go code won't help because the two instances are running in completely separate memory spaces. They don't know about each other's locks.

Insight: This is a classic distributed systems problem on a local scale. To coordinate, processes need an external source of truth. The file system itself can act as this coordinator through a mechanism called Advisory Locking.

The Solution: syscall.Flock

On Unix-like systems (Linux, macOS), the flock system call allows you to place an advisory lock on an open file descriptor. "Advisory" means it works like a gentleman's agreement: it only prevents other processes from writing if they also check for the lock. It doesn't stop a rogue process from ignoring the lock and writing anyway (though "mandatory" locking exists, it's rarely used due to complexity).

Exclusive vs. Shared Locks

• Exclusive Lock (LOCK_EX): Only one process can hold this. Used for writing. If another process holds it, you wait.
• Shared Lock (LOCK_SH): Multiple processes can hold this. Used for reading. Prevents anyone from acquiring an Exclusive lock while active.

Implementation in Go

Here is how you implement a robust file lock in Go. Note that file locking is OS-specific. The example below focuses on Unix-based systems using syscall.

package main

import (
    "fmt"
    "os"
    "syscall"
    "time"
)

// AcquireLock attempts to acquire an exclusive lock on a file.
// It blocks until the lock is obtained.
func AcquireLock(file *os.File) error {
    // LOCK_EX: Exclusive lock (for writing)
    // 0: Wait until lock is available (blocking)
    return syscall.Flock(int(file.Fd()), syscall.LOCK_EX)
}

// ReleaseLock releases the lock.
func ReleaseLock(file *os.File) error {
    // LOCK_UN: Unlock
    return syscall.Flock(int(file.Fd()), syscall.LOCK_UN)
}

func main() {
    // Open the file we want to lock
    // We use a separate lock file usually, e.g., "app.lock"
    f, err := os.Create("app.lock")
    if err != nil {
        panic(err)
    }
    defer f.Close()

    fmt.Println("Waiting for lock...")
    
    // This will block if another process holds the lock
    if err := AcquireLock(f); err != nil {
        panic(err)
    }

    fmt.Println("Lock acquired! Doing critical work...")
    
    // Simulate work
    time.Sleep(5 * time.Second)

    // Always release the lock!
    // In practice, closing the file descriptor also releases the lock,
    // but explicit unlocking is cleaner.
    if err := ReleaseLock(f); err != nil {
        panic(err)
    }
    
    fmt.Println("Lock released.")
}
                

Cross-Platform Considerations (Windows)

The syscall.Flock function is not available on Windows. Windows uses LockFileEx via the golang.org/x/sys/windows package. If you are building a cross-platform application, you should use build tags (e.g., lock_unix.go and lock_windows.go) to abstract the implementation, or use a library like gofrs/flock which handles these differences for you.

When to Use (and Avoid) File Locks

Use Cases

• Single-Instance Daemons: Ensuring only one copy of your background worker is running.
• Local Databases: SQLite uses file locks extensively to manage concurrent access.
• Configuration Updates: Preventing a reader from reading a config file while a writer is halfway through updating it.

When to Avoid

• Network File Systems (NFS): File locking over NFS is notoriously unreliable and can cause processes to hang indefinitely if the network glitches. Avoid flock on shared network drives.
• High Concurrency: File locks are slow compared to memory locks. If you need to coordinate thousands of goroutines, do it in memory or use a proper database/queue, not the file system.