This article can be found at: https://www.ebpf.top/post/no_space_left_on_devices 1. Understanding “no space left on device” Error 2. Problem Analysis and Localization 2.1 Preliminary Identification of Problematic Function 2.2 Locating the Root Cause of the Issue 2.3 Identifying the Root Cause of the Issue 3. Analysis of Discrepancies Between Code Flow and Tracing Process References Recently, there have been cases of failures in creating containers with the error “no space left on device” in the production environment. However, during the investigation, it was found that disk space and inodes were quite normal. In cases where conventional troubleshooting methods have failed, is there a quick and universal approach to pinpointing the root cause of the problem? This article records the analysis and troubleshooting process using eBPF + Ftrace in a separate environment. Considering the general applicability of this approach, it has been organized in the hope of serving as a stepping stone for further exploration.……

Continue reading