[Feat]: snmp_topology: per-device, per-protocol control of topology collection (skip FDB/ARP walks on weak SNMP agents)

Problem

snmp_topology decides what to walk on each device purely through profile selectors (sysObjectID/sysDescr). There is no user-facing configuration to exclude a device from topology collection, or to enable/disable individual topology protocols (LLDP vs CDP vs bridge FDB vs Q-BRIDGE vs ARP vs STP) per device.

This matters because some SNMP agents serve large dynamic tables very poorly. RouterOS is the canonical example:

• MikroTik's own SNMP documentation describes an internal per-service broker with a 30 s per-service timeout and states: "If, for some reason, communication between SNMP and some service is taking longer time than expected ... the proper solution in such cases is to skip such OIDs on your monitoring tool." (https://help.mikrotik.com/docs/spaces/ROS/pages/8978519/SNMP)
• Community reports show incomplete/slow dot1dTpFdbTable reads on RouterOS 7.13–7.15 (first walk returns a fraction of the table; complete only ~30 s later): https://forum.mikrotik.com/t/snmp-outputs-incomplete-values-on-bridge-host-table/178887 — and CPU saturation during SNMP polling on CRS switches.

snmp_topology runs its walks on separate gosnmp sessions from the metric jobs (src/go/plugin/go.d/collector/snmp_topology/collector.go), so it cannot block metric collection in our code — but both sessions hit the same device concurrently. On a device whose SNMP agent degrades under an FDB/ARP walk, the metric session is starved device-side and the SNMP charts get gaps. The operator currently has no way to say "map this device via LLDP only, never walk its FDB/ARP".

Current configuration surface (for reference)

• snmp job config has no topology-related field; every discovered device is registered into the topology device store unconditionally (collector/snmp/device_state.go).
• snmp_topology job config has only update_every and refresh_every (collector/snmp_topology/config.go).
• The only global switch is disabling the whole snmp_topology module in go.d.conf; the only per-protocol lever is overriding profile YAMLs, which is not a reasonable operator interface.

Proposal

Allow per-device, per-protocol control of topology collection, e.g. a topology: section on the snmp job (inherited by the device's registration in the device store):

jobs:
  - name: mikrotik-router
    hostname: 10.20.4.1
    topology:
      enabled: true            # false = metrics only, never register for topology
      protocols:
        lldp: true
        cdp: true
        fdb: false             # bridge-MIB + Q-BRIDGE FDB walks
        arp: false             # ipNetToMedia / ipNetToPhysical
        stp: true

Equivalent per-device overrides on the snmp_topology job would also work. Useful extras: per-device walk pacing/timeout overrides for topology walks (currently inherited from the metric job's options), and surfacing per-device refresh duration so operators can spot slow devices.

Prior art

• LibreNMS: per-device and per-OS discovery_modules / poller_modules toggles, including fdb-table, arp-table, mac-accounting (https://docs.librenms.org/Support/Discovery%20Support/, https://docs.librenms.org/Support/Poller%20Support/).
• Observium: $config['discovery_modules']['fdb-table'], ['arp-table'] equivalents.
• Zabbix and PRTG ship no FDB/ARP polling for MikroTik at all — avoidance by omission.

Environment

Observed against a MikroTik CCR2004-16G-2S+ (RouterOS 7.24) and CRS-class switches in a lab with ~12 SNMP devices; netdata nightly (v2.11 dev), snmp_topology with refresh_every: 30m, refresh duration 17–19 s.