In the Linux kernel, the following vulnerability has been resolved:
ceph: fix hanging __ceph_get_caps() with stale mds_wanted
A reader can hang forever in __ceph_get_caps() when the client no
longer holds FILE_RD, but local cap state still says that the
capability is already wanted (via mds_wanted).
One way to trigger this is through MDS cap revocation. If another
client performs a conflicting operation, the MDS can revoke FILE_RD
from the reader; the next read then has to reacquire FILE_RD. If
the cap update that should request FILE_RD never reaches the MDS
after cap->mds_wanted was raised, the reader is left holding only
non-file caps while local mds_wanted still includes the file read
caps.
In that state, try_get_cap_refs() sees need <= mds_wanted and
returns 0, so __ceph_get_caps() just waits on i_cap_wq. If the cap
update that was supposed to request FILE_RD never reaches the MDS
after cap->mds_wanted was raised, no further request is sent and the
waiter can sleep indefinitely until unrelated cap traffic happens to
wake it up.
The ordering issue is that cap->mds_wanted is updated in
__prep_cap() before the CEPH_MSG_CLIENT_CAPS message is actually
queued for send. That makes one field serve two different meanings at
once: what this client wants, and what the client believes the MDS
already knows it wants.
A proper fix would be to split those states and track whether a cap
update is actually in flight or has been observed by the MDS.
However, simply moving the cap->mds_wanted assignment later would
not be sufficient: queueing the message in the messenger does not
guarantee that the MDS processed that specific wanted set, and
reconnect or message loss can still invalidate that assumption.
Fixing that properly would require a larger rework of the cap state
machine.
To allow simpler backports to stable kernels, this patch implements a simpler workaround:
- stop waiting forever in __ceph_get_caps(); after a bounded wait,
- make ceph_renew_caps() issue a synchronous OPEN
request whenever
The extra issued-vs-wanted check in ceph_renew_caps() is necessary
because the previous test only checked whether the inode still had any
real caps at all. That is not enough after revocation: the client can
still hold something like pLs and yet be missing FILE_RD
completely. In that case, falling back to ceph_check_caps() is not
sufficient, because it still trusts cap->mds_wanted and may resend
nothing. By requiring (issued & wanted) == wanted before taking the
asynchronous path, the code only uses ceph_check_caps() when the
wanted caps are already actually issued. Otherwise, it sends the
synchronous OPEN renew.
This preserves the existing asynchronous fast path when the wanted caps are already issued, avoids changing cap-state semantics, and fixes the hang by guaranteeing that a stalled waiter eventually retries through a path that does not rely on the stale mds_wanted` state.
[ idryomov: move CEPH_GET_CAPS_WAIT_TIMEOUT from libceph.h to mds_client.h, formatting ]