33. Syscalls from Python: what open() really does

33.1 Python is a thin layer over libc

The os module is a nearly one-to-one wrapper over POSIX system calls. The high-level APIs (open, pathlib, socket, subprocess) are built on it.

import os

fd = os.open("/tmp/py-sys.txt", os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o644)  # open(2)
n = os.write(fd, b"hello kernel\n")                                            # write(2)
os.fsync(fd)                                                                   # fsync(2)
os.close(fd)                                                                   # close(2)

st = os.stat("/tmp/py-sys.txt")                                                # stat(2)
print(n, st.st_size, oct(st.st_mode & 0o777), st.st_ino)
print(os.getpid(), os.getppid(), os.getuid(), os.uname().sysname)

33.2 Watch it with strace

cat > /tmp/hello.py <<'EOF'
with open("/tmp/out.txt", "w", encoding="utf-8") as f:
    f.write("hi\n")
EOF
strace -f -e trace=openat,write,close,fstat,ioctl,lseek python3 /tmp/hello.py 2>&1 | tail -8

openat(AT_FDCWD, "/tmp/out.txt", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
ioctl(3, TCGETS, 0x7ffd...) = -1 ENOTTY (Inappropriate ioctl for device)   # "is this a terminal?"
lseek(3, 0, SEEK_CUR) = 0
write(3, "hi\n", 3) = 3
close(3) = 0

Things to notice, all explained in the OS guide Ch 2 and Ch 14:

• Python opens every file with O_CLOEXEC, so child processes do not inherit your file descriptors by accident (PEP 446).
• f.write("hi\n") did not call write(2); Python's io layer buffered it, and the single write happened at close. Text files are line-buffered only when attached to a terminal (the ioctl checks that).
• Run strace -c python3 -c pass to see how many syscalls the interpreter makes just to start (hundreds, mostly stat and openat while searching sys.path for modules). Start-up time is why short Python CLIs feel slower than Go ones.

33.3 Buffering: the io stack

 open("f", "w", encoding="utf-8")
   └─ TextIOWrapper     str ⇄ bytes (encoding, newline translation)
       └─ BufferedWriter  collects bytes; flushes when its buffer (8 KiB+) fills
           └─ FileIO       one write(2) per flush
               └─ kernel page cache ──(later)──▶ disk

f.flush() moves data from Python's buffer to the kernel. os.fsync(fd) moves it from the kernel's page cache to the disk. Those are two different promises, and the next chapter depends on the difference.

print() to a pipe is block-buffered, which is why a Python program's output appears late (or not at all, if it is killed) under docker logs or systemd. Set PYTHONUNBUFFERED=1 in containers, or use print(..., flush=True).