配置原子发布（os.replace）后 inotify 热重载永久失效：监听的是文件而非目录，watch 挂在被换掉的旧 inode 上，healthz 恒绿

[healthz] OK: 守护进程存活，配置文件新鲜，热重载链路正常     # rc=0
[reloadd] pid=19 watching /srv/conf/service.json (mode=file, mask=IN_MODIFY|IN_CLOSE_WRITE) wd=1

{
  "watch_mode": "file",              // ← 契约要求的是 dir
  "config_revision_on_disk": "r2",   // ← 磁盘上已经是 r2
  "effective_revision": "r1",        // ← 生效的还是 r1
  "reloads": 0,                      // ← 一次都没重载过
  "self_reported_ok": true           // ← 却说自己一切正常
}

cd scenario/2026-10-01-0000
docker compose up -d --build
docker compose exec app bash          # 进容器（也可用 exec -T 直接跑脚本）

docker compose exec -T app bash /srv/ops/repro.sh

[1] 守护进程启动日志：
    [reloadd] pid=19 watching /srv/conf/service.json (mode=file, mask=IN_MODIFY|IN_CLOSE_WRITE) wd=1 revision=r1
[2] 原子发布 revision=r2：
[publisher] published revision=r2 -> /srv/conf/service.json
[3] 磁盘上的配置（publisher 的产物）：
    revision=r2 max_conn=512
[4] 对外声称「当前生效」的配置：
    [consumer] 当前生效 revision=r1  source=boot  max_conn=200  features=['alpha', 'beta']
[5] 守护进程自检报告：
    watch_mode                 = file
    config_revision_on_disk    = r2
    effective_revision         = r1
    reloads                    = 0
[6] 健康检查：
    [healthz] OK: 守护进程存活，配置文件新鲜，热重载链路正常    ← rc=0，假绿
[7] 外部探针：
[check_reload] RESULT: BAD（最新配置 r2 未生效，进程仍在用 r1）

docker compose exec -T app bash /srv/ops/verify.sh
# 问题态：RESULT: PASS=3 FAIL=6 / RESULT: BAD

import os, sys, time
sys.path.insert(0, "/srv/app")
import inotify

TARGET = "/srv/conf/service.json"

def trial(mode):
    fd = inotify.init()
    path = os.path.dirname(TARGET) if mode == "dir" else TARGET
    mask = (inotify.IN_CREATE | inotify.IN_MOVED_TO | inotify.IN_MODIFY |
            inotify.IN_CLOSE_WRITE | inotify.IN_DELETE | inotify.IN_DELETE_SELF |
            inotify.IN_MOVE_SELF)
    inotify.add_watch(fd, path, mask)
    st = os.stat(TARGET)
    open("/srv/conf/.experiment.tmp", "w").write('{"revision": "exp"}')
    os.replace("/srv/conf/.experiment.tmp", TARGET)   # ← 原子发布，换名
    time.sleep(0.5)
    try:
        evs = inotify.read_events(fd)
    except BlockingIOError:
        evs = []
    ns = os.stat(TARGET)
    print("mode=%-4s watch=%-28s inode: %d -> %d" % (mode, path, st.st_ino, ns.st_ino))
    print("       events=%d %s" % (len(evs),
          ["|".join(inotify.parse_mask(e[1])) + " name=" + e[3] for e in evs]))

trial("file")
trial("dir")

mode=file watch=/srv/conf/service.json       inode: 25770861361 -> 25770894243
       events=2 ['IN_DELETE_SELF name=', 'IN_IGNORED name=']

mode=dir  watch=/srv/conf                    inode: 25770894243 -> 25770894225
       events=4 ['IN_CREATE name=.experiment.tmp', 'IN_MODIFY name=.experiment.tmp',
                 'IN_CLOSE_WRITE name=.experiment.tmp', 'IN_MOVED_TO name=service.json']

发布前： /srv/conf/service.json -> inode 1001 (r1)
发布中： /srv/conf/.service-xx  -> inode 1002 (r2，已 fsync)
发布后： /srv/conf/service.json -> inode 1002 (r2)   ← 换的是目录项
         inode 1001 从此无名字（孤儿，等最后一个 fd 关闭才回收）

DIR_MASK  = inotify.IN_MOVED_TO | inotify.IN_CREATE | inotify.IN_DELETE | inotify.IN_MOVED_FROM
FILE_MASK = inotify.IN_MODIFY | inotify.IN_CLOSE_WRITE

contract = load_contract()
watch_mode = contract.get("RELOAD_WATCH_MODE", "dir")      # 契约说了算，不写死
on_overflow = contract.get("ON_QUEUE_OVERFLOW", "rescan")
on_watch_lost = contract.get("ON_WATCH_LOST", "rewatch")

if watch_mode == "dir":
    watch_target = os.path.dirname(WATCH_PATH) or "."
    watch_mask = DIR_MASK
else:
    watch_target = WATCH_PATH
    watch_mask = FILE_MASK

inotify.add_watch(fd, watch_target, watch_mask)             # 监听目录！

if watch_mode == "dir":
    if name != os.path.basename(WATCH_PATH):
        continue                     # 忽略临时文件等无关事件
    triggered = True                 # IN_MOVED_TO / IN_CREATE 即「配置换了」

if emask & inotify.IN_Q_OVERFLOW:
    if on_overflow == "rescan": rescan = True
if emask & (IN_DELETE_SELF | IN_MOVE_SELF | IN_IGNORED):
    if on_watch_lost == "rewatch":
        wd = arm()                   # 重建 watch
        rescan = True                # 并全量重扫，保证收敛到最终值
...
if triggered or rescan:  ...         # rescan 分支保证「突发发布」「删后重建」能收敛

"self_reported_ok": (_revision_on_disk() == revision),

# ★ 磁盘上的配置是否真的装进了内存
if disk_rev is not None and disk_rev != eff_rev:
    problems.append("最新配置未生效：磁盘 revision=%s，生效 revision=%s" % (disk_rev, eff_rev))

# ★ 代码实际用的监听模式必须与契约一致
if c_mode and r_mode and c_mode != r_mode:
    problems.append("监听模式与契约不一致：契约=%s，实际=%s" % (c_mode, r_mode))

docker compose up -d --build

docker compose up -d --build
docker compose exec -T app bash /srv/ops/verify.sh

[PASS] A1 守护进程拉起后生效配置 = 磁盘配置 (r1)
[PASS] A2 健康检查通过 (0)
[check_reload] RESULT: OK（最新配置 r2 已生效）
[PASS] B1 原子发布后生效配置跟进到 r2 (r2)
[PASS] B2 生效配置的 max_conn 跟进到 512 (512)
[PASS] C1 连续 3 次发布每次都跟进 (yes)
[PASS] D1 突发发布后生效配置 = 最终值 r-final (r-final)
  契约 RELOAD_WATCH_MODE=dir
[PASS] E1 代码实际使用的监听模式与契约一致 (dir)
[PASS] F1 删除后重建配置仍能生效 (r-again)
[PASS] G1 conf/ 契约 md5 与基线一致 (0)

RESULT: PASS=9 FAIL=0
RESULT: OK

[5] watch_mode=dir  watch_target=/srv/conf  effective_revision=r2  reloads=1
[7] [check_reload] RESULT: OK（最新配置 r2 已生效）
结论：最新配置已生效 —— 热重载链路正常

report.json: {'watch_mode': 'file', 'config_revision_on_disk': 'r9',
              'effective_revision': 'r2', 'reloads': 0, 'self_reported_ok': True}
[healthz] BAD: 最新配置未生效：磁盘 revision=r9，生效 revision=r2;
              监听模式与契约不一致：契约=dir，实际=file
healthz rc=1

for p in $(ls /proc | grep -E "^[0-9]+$"); do
  [ "$p" = "$$" ] && continue
  cmd=$(tr "\0" " " < /proc/$p/cmdline 2>/dev/null)
  case "$cmd" in *"reloadd.py"*) kill -9 "$p" 2>/dev/null;; esac
done