rook-discover does not discover physical disks with LVM partitions
Author: stygian-coffeeCreated May 19, 2025Updated Sep 14, 2026
Labelsbug
Is this a bug report or feature request?
- Bug Report
Deviation from expected behavior:
The rook-discover DaemonSet does not detect physical disks that contain LVM partitions. This seems to be a deliberate choice, but a consequence of this choice is that disks used with an OSD with encryptedDevice: true as well as disks used as a metadataDevice for multiple OSDs are not detected. These physical disks show up neither in the Ceph dashboard nor when running ceph orch device ls. On the other hand, dm-crypt devices as well as partitions of disks used by Ceph are displayed as physical disks. A consequence of this is that I cannot use features that rely on the physical disks being detected such as blinking the disk light and disk health monitoring (I think?).
See this screenshot from my test cluster. Note that in my case there are no OSDs with separate metadata devices, and sdb, sdc, and sdd are intentionally not used for Ceph.
Expected behavior:
The physical disks should be detected, and the random partitions should not be detected.
How to reproduce it (minimal and precise):
Using Rook, create a host storage cluster on worker nodes with physical disks that you want Ceph to consume. Either enable encryptedDevice for an OSD or use a separate metadata device for an OSD. After the Ceph cluster is up and running, restart the rook-discover DaemonSet: kubectl -n rook-ceph rollout restart ds/rook-discover. Run ceph orch device ls in the toolbox pod.
File(s) to submit:
cluster.yaml:
apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
name: rook-ceph
namespace: rook-ceph
spec:
cephVersion:
allowUnsupported: false
image: quay.io/ceph/ceph:v19.2.2
cleanupPolicy:
allowUninstallWithVolumes: false
confirmation: ""
sanitizeDisks:
dataSource: zero
iteration: 1
method: quick
continueUpgradeAfterChecksEvenIfNotHealthy: false
crashCollector:
disable: false
dashboard:
enabled: true
ssl: true
dataDirHostPath: /var/lib/rook
disruptionManagement:
managePodBudgets: true
osdMaintenanceTimeout: 30
healthCheck:
daemonHealth:
mon:
disabled: false
interval: 45s
osd:
disabled: false
interval: 60s
status:
disabled: false
interval: 60s
livenessProbe:
mgr:
disabled: false
mon:
disabled: false
osd:
disabled: false
logCollector:
enabled: true
maxLogSize: 500M
periodicity: daily
mgr:
allowMultiplePerNode: false
count: 2
modules:
- enabled: true
name: pg_autoscaler
- enabled: true
name: rook
mon:
allowMultiplePerNode: false
count: 3
monitoring:
enabled: false
network:
addressRanges:
public:
- 192.168.0.0/16
connections:
compression:
enabled: true
encryption:
enabled: true
requireMsgr2: false
provider: host
priorityClassNames:
mgr: system-cluster-critical
mon: system-node-critical
osd: system-node-critical
removeOSDsIfOutAndSafeToRemove: false
resources:
cleanup:
limits:
memory: 1Gi
requests:
cpu: 500m
memory: 100Mi
crashcollector:
limits:
memory: 60Mi
requests:
cpu: 100m
memory: 60Mi
exporter:
limits:
memory: 128Mi
requests:
cpu: 50m
memory: 50Mi
logcollector:
limits:
memory: 1Gi
requests:
cpu: 100m
memory: 100Mi
mgr:
limits:
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
mgr-sidecar:
limits:
memory: 100Mi
requests:
cpu: 100m
memory: 40Mi
mon:
limits:
memory: 2Gi
requests:
cpu: 1000m
memory: 1Gi
osd:
limits:
memory: 4Gi
requests:
cpu: 1000m
memory: 4Gi
prepareosd:
requests:
cpu: 500m
memory: 50Mi
skipUpgradeChecks: false
storage:
config:
encryptedDevice: "true"
nodes:
- devices:
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:0
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:1
name: worker-01
- devices:
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:0
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:1
name: worker-02
- devices:
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:0
- name: /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:1
name: worker-03
useAllDevices: false
useAllNodes: false
upgradeOSDRequiresHealthyPGs: false
waitTimeoutForHealthyOSDInMinutes: 10Logs to submit:
rook-discover logs on initial run:
2025/05/19 06:34:45 maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
2025-05-19 06:34:45.583617 I | rookcmd: starting Rook v1.17.2 with arguments '/usr/local/bin/rook discover --discover-interval 60m --use-ceph-volume'
2025-05-19 06:34:45.583638 I | rookcmd: flag values: --discover-interval=1h0m0s, --help=false, --log-level=INFO, --use-ceph-volume=true
2025-05-19 06:34:45.584258 I | rook-discover: updating device configmap
2025-05-19 06:34:45.591303 W | inventory: skipping device "loop0". unsupported diskType loop
2025-05-19 06:34:45.593322 W | inventory: skipping device "loop1". unsupported diskType loop
2025-05-19 06:34:45.595423 W | inventory: skipping device "loop2". unsupported diskType loop
2025-05-19 06:34:45.597185 E | sys: failed to execute lsblk. output: .
2025-05-19 06:34:45.597196 W | inventory: skipping device "loop3". exit status 32
2025-05-19 06:34:45.599163 E | sys: failed to execute lsblk. output: .
2025-05-19 06:34:45.599175 W | inventory: skipping device "loop4". exit status 32
2025-05-19 06:34:45.601159 E | sys: failed to execute lsblk. output: .
2025-05-19 06:34:45.601170 W | inventory: skipping device "loop5". exit status 32
2025-05-19 06:34:45.603125 E | sys: failed to execute lsblk. output: .
2025-05-19 06:34:45.603137 W | inventory: skipping device "loop6". exit status 32
2025-05-19 06:34:45.604905 E | sys: failed to execute lsblk. output: .
2025-05-19 06:34:45.604914 W | inventory: skipping device "loop7". exit status 32
2025-05-19 06:34:45.609889 W | inventory: uuid not found for device /dev/sda. output=Creating new GPT entries in memory.
Disk /dev/sda: 62914560 sectors, 30.0 GiB
Model: QEMU HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 6FC75AF9-672D-4E98-A9A2-98D86D230CAB
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62914526
Partitions will be aligned on 2048-sector boundaries
Total free space is 62914493 sectors (30.0 GiB)
Number Start (sector) End (sector) Size Code Name
2025-05-19 06:34:45.623757 W | inventory: uuid not found for device /dev/sdb. output=Creating new GPT entries in memory.
Disk /dev/sdb: 41943040 sectors, 20.0 GiB
Model: QEMU HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 8A2358FC-0A8F-44D7-86BE-CC8E7ACFF42C
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 41942973 sectors (20.0 GiB)
Number Start (sector) End (sector) Size Code Name
2025-05-19 06:34:45.637028 W | inventory: uuid not found for device /dev/sdc. output=Creating new GPT entries in memory.
Disk /dev/sdc: 41943040 sectors, 20.0 GiB
Model: QEMU HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 9E5DF70F-06E6-4FDB-A8D7-182A9C3E4A74
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 41942973 sectors (20.0 GiB)
Number Start (sector) End (sector) Size Code Name
2025-05-19 06:34:45.651187 W | inventory: uuid not found for device /dev/sdd. output=Creating new GPT entries in memory.
Disk /dev/sdd: 41943040 sectors, 20.0 GiB
Model: QEMU HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 4EB20AEE-CB11-4146-8553-EB05145D0D86
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 41942973 sectors (20.0 GiB)
Number Start (sector) End (sector) Size Code Name
2025-05-19 06:34:45.665188 W | inventory: uuid not found for device /dev/sde. output=Creating new GPT entries in memory.
Disk /dev/sde: 62914560 sectors, 30.0 GiB
Model: QEMU HARDDISK
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 1049C5DC-30CC-406F-8BBE-459548A00C71
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 62914526
Partitions will be aligned on 2048-sector boundaries
Total free space is 62914493 sectors (30.0 GiB)
Number Start (sector) End (sector) Size Code Name
2025-05-19 06:34:45.675069 W | inventory: skipping device "sr0". unsupported diskType rom
2025-05-19 06:34:45.690378 I | inventory: skipping device "vda" because it has child, considering the child instead.
2025-05-19 06:34:45.742547 I | rook-discover: localdevices: "sda, sdb, sdc, sdd, sde, vda1, vda2, vda3, vda4, vda5, vda6"
2025-05-19 06:34:45.742567 I | rook-discover: Getting ceph-volume inventory information
2025-05-19 06:34:48.309752 I | sys: Output: NAME="sda" SIZE="32212254720" TYPE="disk" PKNAME=""
2025-05-19 06:34:48.309791 I | sys: Device found - sda
2025-05-19 06:34:48.318632 I | sys: Output: NAME="sdb" SIZE="21474836480" TYPE="disk" PKNAME=""
2025-05-19 06:34:48.318654 I | sys: Device found - sdb
2025-05-19 06:34:48.325983 I | sys: Output: NAME="sdc" SIZE="21474836480" TYPE="disk" PKNAME=""
2025-05-19 06:34:48.326001 I | sys: Device found - sdc
2025-05-19 06:34:48.335869 I | sys: Output: NAME="sdd" SIZE="21474836480" TYPE="disk" PKNAME=""
2025-05-19 06:34:48.335891 I | sys: Device found - sdd
2025-05-19 06:34:48.343119 I | sys: Output: NAME="sde" SIZE="32212254720" TYPE="disk" PKNAME=""
2025-05-19 06:34:48.343140 I | sys: Device found - sde
2025-05-19 06:34:48.351923 I | sys: Output: NAME="vda1" SIZE="104857600" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.351943 I | sys: Device found - vda1
2025-05-19 06:34:48.357758 E | rook-discover: ceph-volume information for device "vda1" not found
2025-05-19 06:34:48.360195 I | sys: Output: NAME="vda2" SIZE="1048576" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.360229 I | sys: Device found - vda2
2025-05-19 06:34:48.366354 E | rook-discover: ceph-volume information for device "vda2" not found
2025-05-19 06:34:48.368419 I | sys: Output: NAME="vda3" SIZE="1048576000" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.368433 I | sys: Device found - vda3
2025-05-19 06:34:48.374525 E | rook-discover: ceph-volume information for device "vda3" not found
2025-05-19 06:34:48.376702 I | sys: Output: NAME="vda4" SIZE="1048576" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.376716 I | sys: Device found - vda4
2025-05-19 06:34:48.382878 E | rook-discover: ceph-volume information for device "vda4" not found
2025-05-19 06:34:48.385090 I | sys: Output: NAME="vda5" SIZE="104857600" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.385106 I | sys: Device found - vda5
2025-05-19 06:34:48.391318 E | rook-discover: ceph-volume information for device "vda5" not found
2025-05-19 06:34:48.393568 I | sys: Output: NAME="vda6" SIZE="106111696896" TYPE="part" PKNAME="vda"
2025-05-19 06:34:48.393586 I | sys: Device found - vda6
2025-05-19 06:34:48.399079 E | rook-discover: ceph-volume information for device "vda6" not found
2025-05-19 06:34:48.399176 I | rook-discover: available devices: [{Name:sda Parent: HasChildren:false DevLinks:/dev/disk/by-id/wwn-0x05abcde5d8a74c77 /dev/disk/by-diskseq/14 /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:0 /dev/disk/by-id/scsi-305abcde5d8a74c77 Size:32212254720 UUID: Serial:305abcde5d8a74c77 Type:disk Rotational:false Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor:QEMU Model:QEMU_HARDDISK WWN:0x05abcde5d8a74c77 WWNVendorExtension:0x05abcde5d8a74c77 Empty:true CephVolumeData:{"path":"/dev/sda","available":true,"rejected_reasons":[],"sys_api":{"removable":"0","ro":"0","vendor":"QEMU","model":"QEMU HARDDISK","rev":"2.5+","sas_address":"","sas_device_handle":"","support_discard":"4096","rotational":"0","nr_requests":"256","partitions":{},"device_nodes":["sda"],"actuators":null,"scheduler_mode":"none","sectors":0,"sectorsize":"512","size":32212254720.0,"human_readable_size":"30.00 GB","path":"/dev/sda","devname":"sda","type":"disk","parent":"/dev/sda","id_bus":"scsi"},"lvs":[]} RealPath:/dev/sda KernelName:sda Encrypted:false} {Name:sdb Parent: HasChildren:false DevLinks:/dev/disk/by-diskseq/15 /dev/disk/by-id/scsi-305abcdb5f9fdda43 /dev/disk/by-id/wwn-0x05abcdb5f9fdda43 /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:4 Size:21474836480 UUID: Serial:305abcdb5f9fdda43 Type:disk Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor:QEMU Model:QEMU_HARDDISK WWN:0x05abcdb5f9fdda43 WWNVendorExtension:0x05abcdb5f9fdda43 Empty:true CephVolumeData:{"path":"/dev/sdb","available":true,"rejected_reasons":[],"sys_api":{"removable":"0","ro":"0","vendor":"QEMU","model":"QEMU HARDDISK","rev":"2.5+","sas_address":"","sas_device_handle":"","support_discard":"4096","rotational":"1","nr_requests":"256","partitions":{},"device_nodes":["sdb"],"actuators":null,"scheduler_mode":"none","sectors":0,"sectorsize":"512","size":21474836480.0,"human_readable_size":"20.00 GB","path":"/dev/sdb","devname":"sdb","type":"disk","parent":"/dev/sdb","id_bus":"scsi"},"lvs":[]} RealPath:/dev/sdb KernelName:sdb Encrypted:false} {Name:sdc Parent: HasChildren:false DevLinks:/dev/disk/by-id/scsi-305abcd5ee937b78c /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:3 /dev/disk/by-id/wwn-0x05abcd5ee937b78c /dev/disk/by-diskseq/16 Size:21474836480 UUID: Serial:305abcd5ee937b78c Type:disk Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor:QEMU Model:QEMU_HARDDISK WWN:0x05abcd5ee937b78c WWNVendorExtension:0x05abcd5ee937b78c Empty:true CephVolumeData:{"path":"/dev/sdc","available":true,"rejected_reasons":[],"sys_api":{"removable":"0","ro":"0","vendor":"QEMU","model":"QEMU HARDDISK","rev":"2.5+","sas_address":"","sas_device_handle":"","support_discard":"4096","rotational":"1","nr_requests":"256","partitions":{},"device_nodes":["sdc"],"actuators":null,"scheduler_mode":"none","sectors":0,"sectorsize":"512","size":21474836480.0,"human_readable_size":"20.00 GB","path":"/dev/sdc","devname":"sdc","type":"disk","parent":"/dev/sdc","id_bus":"scsi"},"lvs":[]} RealPath:/dev/sdc KernelName:sdc Encrypted:false} {Name:sdd Parent: HasChildren:false DevLinks:/dev/disk/by-id/wwn-0x05abcdada818afe6 /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:2 /dev/disk/by-diskseq/17 /dev/disk/by-id/scsi-305abcdada818afe6 Size:21474836480 UUID: Serial:305abcdada818afe6 Type:disk Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor:QEMU Model:QEMU_HARDDISK WWN:0x05abcdada818afe6 WWNVendorExtension:0x05abcdada818afe6 Empty:true CephVolumeData:{"path":"/dev/sdd","available":true,"rejected_reasons":[],"sys_api":{"removable":"0","ro":"0","vendor":"QEMU","model":"QEMU HARDDISK","rev":"2.5+","sas_address":"","sas_device_handle":"","support_discard":"4096","rotational":"1","nr_requests":"256","partitions":{},"device_nodes":["sdd"],"actuators":null,"scheduler_mode":"none","sectors":0,"sectorsize":"512","size":21474836480.0,"human_readable_size":"20.00 GB","path":"/dev/sdd","devname":"sdd","type":"disk","parent":"/dev/sdd","id_bus":"scsi"},"lvs":[]} RealPath:/dev/sdd KernelName:sdd Encrypted:false} {Name:sde Parent: HasChildren:false DevLinks:/dev/disk/by-diskseq/18 /dev/disk/by-id/wwn-0x05abcd39c57655d1 /dev/disk/by-id/scsi-305abcd39c57655d1 /dev/disk/by-path/pci-0000:00:04.0-scsi-0:0:0:1 Size:32212254720 UUID: Serial:305abcd39c57655d1 Type:disk Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor:QEMU Model:QEMU_HARDDISK WWN:0x05abcd39c57655d1 WWNVendorExtension:0x05abcd39c57655d1 Empty:true CephVolumeData:{"path":"/dev/sde","available":true,"rejected_reasons":[],"sys_api":{"removable":"0","ro":"0","vendor":"QEMU","model":"QEMU HARDDISK","rev":"2.5+","sas_address":"","sas_device_handle":"","support_discard":"4096","rotational":"1","nr_requests":"256","partitions":{},"device_nodes":["sde"],"actuators":null,"scheduler_mode":"none","sectors":0,"sectorsize":"512","size":32212254720.0,"human_readable_size":"30.00 GB","path":"/dev/sde","devname":"sde","type":"disk","parent":"/dev/sde","id_bus":"scsi"},"lvs":[]} RealPath:/dev/sde KernelName:sde Encrypted:false} {Name:vda1 Parent:vda HasChildren:false DevLinks:/dev/disk/by-label/EFI /dev/disk/by-partuuid/4ccfeaec-ae12-43ee-bec5-1143afeb4085 /dev/disk/by-partlabel/EFI /dev/disk/by-uuid/06D7-EEFB Size:104857600 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem:vfat Mountpoint: Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/vda1 KernelName:vda1 Encrypted:false} {Name:vda2 Parent:vda HasChildren:false DevLinks:/dev/disk/by-partuuid/31f25f39-2502-4e3a-af0e-b7517366c56b /dev/disk/by-partlabel/BIOS Size:1048576 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor: Model: WWN: WWNVendorExtension: Empty:true CephVolumeData: RealPath:/dev/vda2 KernelName:vda2 Encrypted:false} {Name:vda3 Parent:vda HasChildren:false DevLinks:/dev/disk/by-uuid/3c409a90-2e17-4c2c-a121-99c145466716 /dev/disk/by-partlabel/BOOT /dev/disk/by-label/BOOT /dev/disk/by-partuuid/28013210-960c-4a36-90f6-69dc4b77e8bd Size:1048576000 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem:xfs Mountpoint: Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/vda3 KernelName:vda3 Encrypted:false} {Name:vda4 Parent:vda HasChildren:false DevLinks:/dev/disk/by-partlabel/META /dev/disk/by-partuuid/3a57facf-085e-42b9-9605-c874bb1592f7 Size:1048576 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem: Mountpoint: Vendor: Model: WWN: WWNVendorExtension: Empty:true CephVolumeData: RealPath:/dev/vda4 KernelName:vda4 Encrypted:false} {Name:vda5 Parent:vda HasChildren:false DevLinks:/dev/disk/by-partlabel/STATE /dev/disk/by-uuid/46754f12-c252-4c9c-9674-afca83c5e825 /dev/disk/by-label/STATE /dev/disk/by-partuuid/fbc69150-29fe-4a8c-8c58-c9a4c8c0e591 Size:104857600 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem:xfs Mountpoint: Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/vda5 KernelName:vda5 Encrypted:false} {Name:vda6 PSource: rook/rook