infiniband collector fails on MANA RDMA devices before device exclusion is applied

Author: chmill-zzCreated Sep 4, 2026Updated Sep 4, 2026

Host operating system: output of uname -a

Observed on an Azure Ubuntu 22.04 VM using a Microsoft Azure Network Adapter (MANA). The public MANA documentation notes that Linux kernel 6.2 adds InfiniBand/RDMA support:

https://learn.microsoft.com/azure/virtual-network/accelerated-networking-mana-linux

The upstream driver is also under drivers/infiniband/hw/mana:

https://github.com/torvalds/linux/tree/master/drivers/infiniband/hw/mana

node_exporter version: output of node_exporter --version

node_exporter, version v1.12.1

This is the Microsoft Dalec-built node-exporter-kubernetes package from the upstream v1.12.1 source.

node_exporter command line flags

The collector is enabled by default. We also tried the new v1.12 device exclusion:

--collector.infiniband.device-exclude=^mana_.*

node_exporter log output

level=ERROR source=collector.go:168 msg="collector failed" name=infiniband err="error obtaining InfiniBand class info: read /sys/class/infiniband/mana_0/ports/1/rate: invalid argument"

Are you running node_exporter in Docker?

No. It runs directly as a systemd service on the VM.

What did you do that produced an error?

Scraped /metrics on a MANA-enabled Azure VM. MANA registers mana_0 under /sys/class/infiniband, but reading /sys/class/infiniband/mana_0/ports/1/rate returns EINVAL.

Adding --collector.infiniband.device-exclude=^mana_.* does not avoid the error. In v1.12.1, infinibandCollector.Update first calls c.fs.InfiniBandClass(), which fully parses every device, and applies c.deviceFilter.ignored(device.Name) only afterward. The failure therefore happens before the exclusion is evaluated.

Relevant ordering:

https://github.com/prometheus/node_exporter/blob/v1.12.1/collector/infiniband_linux.go

The underlying procfs parser treats rate as required and propagates its read error:

https://github.com/prometheus/procfs/blob/v0.21.1/sysfs/class_infiniband.go

What did you expect to see?

At minimum, the device exclusion should prevent node_exporter from parsing mana_*, allowing operators to avoid unsupported device attributes while retaining metrics for other HCAs on mixed-device nodes.

Ideally, procfs/node_exporter could also support MANA directly or tolerate an unavailable rate attribute if the remaining MANA RDMA data is useful.

What did you see instead?

The whole InfiniBand collector fails and reports:

node_scrape_collector_success{collector="infiniband"} 0

On mixed MANA and Mellanox nodes, the late filter means there is currently no configuration-only way to ignore MANA and retain collection from the other HCAs. The only working operator mitigation is --no-collector.infiniband, which disables all InfiniBand/RDMA metrics.

This resembles #3265 / prometheus/procfs#704, but that fix skips EINVAL for individual counter files. This failure is on the required port rate file and occurs before the v1.12 device filter is applied.

Source: prometheus/node_exporter