NVME drives are all null/zero

[dlangille]

Hello,

This is what smart pulls for an NVME drive:

[13:03 r730-01 dvl /usr/local/etc/cron.d] % sudo smartctl -a /dev/nvme0   
smartctl 7.5 2025-04-30 r5714 [FreeBSD 14.3-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 990 EVO Plus 4TB
Serial Number:                      S7U8NJ0Y716854P
Firmware Version:                   2B2QKXG7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 4,000,787,030,016 [4.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       2.0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          4,000,787,030,016 [4.00 TB]
Namespace 1 Utilization:            0
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 5751411bcb
Local Time is:                      Fri Oct  3 13:04:57 2025 UTC
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x00df):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp Verify
Log Page Attributes (0x3f):         S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg Log0_FISE_MI
Maximum Data Transfer Size:         128 Pages
Warning  Comp. Temp. Threshold:     81 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     6.00W       -        -    0  0  0  0        0       0
 1 +     6.00W       -        -    1  1  1  1     1700    2100
 2 +     6.00W       -        -    2  2  2  2     3000    6500
 3 -   0.0800W       -        -    3  3  3  3     1600    4600
 4 -   0.0070W       -        -    4  4  4  4     1600   43000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        37 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    3 [1.53 MB]
Data Units Written:                 0
Host Read Commands:                 60
Host Write Commands:                0
Controller Busy Time:               0
Power Cycles:                       3
Power On Hours:                     44
Unsafe Shutdowns:                   1
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               55 Celsius
Temperature Sensor 2:               37 Celsius

Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

Self-test Log (NVMe Log 0x06, NSID 0xffffffff)
Self-test status: No self-test in progress
No Self-tests Logged

This is what the smart script displays for that drive:

S7U8NJ0Y716854P,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0,0,0,0,0,0,0

perl is not my strong suit... how can I help?

[dlangille]

This might be useful. It does NVMe temperatures: https://github.com/vermaden/sensors/blob/master/sensors

[VVelox]

@dlangille Howdy! Sorry about the late reply.

So looks like you are using the old one there. The new one generates JSON returns.

https://github.com/librenms/librenms-agent/blob/master/snmp/smart-v1

Just finished updating it. This should have far better support for both NVMe and SAS drives.

[dlangille]

Here's my run. I picked one drive:

S7KGNU0Y722875X,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0,0,0,0,0,0,0

That drive is:

[22:32 r730-01 dvl /usr/local/etc/snmp] % grep S7KGNU0Y722875X /var/run/dmesg.boot 
nda0: <Samsung SSD 990 PRO 4TB 4B2QJXD7 S7KGNU0Y722875X>
nda0: Serial Number S7KGNU0Y722875X

[VVelox]

@dlangille You are still using the old extend. You really need to use the new one. It outputs json to /var/cache/smart and the extend return to /var/cache/smart.snmp .

The output in /var/cache/smart should something like this...

{
  "data": {
    "dev_error": 0,
    "disks": {
      "nvme0": {
        "10": null,
        "12": 56,
        "173": null,
        "177": null,
        "183": null,
        "184": null,
        "187": null,
        "188": null,
        "190": null,
        "194": null,
        "196": null,
        "197": null,
        "198": null,
        "199": null,
        "231": 100,
        "232": null,
        "233": null,
        "5": null,
        "9": 5399,
        "completed": "0",
        "conveyance": "0",
        "dev_error": 0,
        "disk": "nvme0 -d nvme",
        "exit": 0,
        "extended": "0",
        "from_factor": null,
        "fw_version": "281010WD",
        "health_pass": 1,
        "interrupted": "0",
        "json_err": 0,
        "json_err_str": "",
        "max_temp": null,
        "model_name": "WD Blue SN580 1TB",
        "offline": "0",
        "over_temp": 0,
        "read_failure": 0,
        "selective": "0",
        "selftest_log": null,
        "serial": "24144M801847",
        "short": "0",
        "temp_limit": null,
        "unknown_failure": "0"
      }
    },
    "exit_nonzero": 0,
    "unhealthy": 0,
    "useSN": "0"
  },
  "error": 0,
  "errorString": "",
  "version": 1
}

While /var/cache/smart.snmp should look something like this...

H4sIAOYSqmkAA2VRy27CMBC89ysin6lk8yhJjlWv5UIljpZFNuDirJHtBGjFv3edkBipl2R2PbOzHv+ySgXFyl9WQSfBOetYyWes0v7kYxu7BngEgr7YGjNjYs7K1Rv914uptV5PME/dfJngEyGfYJGGFolbvCWYZMWTrBjhfCFYKTiPaJ6a0wqrEZBitSiKGdvb5mwgQMVKxlmssYObwj08Gv+DoIM+huy1yiIgElx16M/hGgCraVrtbCNrtQ9RPjjXF9mB89oicea54ILvPoh6BGXCUZ6Vp6DFjGkMZNue02rf3mJcpTcaC+kDNRgdN+oqAzTn0aixFRiJqok32X1k76aFbLtZ5TwTX+8ksHVtNI73tLTVQ0/jHaiKFtemddA3PBjYB92NdKrrAD5IYw+jowenlYnXWorl8jPn9OLryD1aFx666CCNbmJeg6rFE9oLJjfi3e9DphIt/oCz/QotDhndhsrDdjOQiTs9UI+2wWk8DLFMYYv7yx+XJXCm3wIAAA==

[dlangille]

Here's something interesting:

https://github.com/librenms/librenms-agent/blob/master/snmp/smart-v1 shows it is 1051 lines.

When downloaded, it is 1068 lines: wget -S https://raw.githubusercontent.com/librenms/librenms-agent/refs/heads/master/snmp/smart-v1

For the drive in question from the original post:

      "S7U8NJ0Y716854P": {
        "5": null,
        "9": 3737,
        "10": null,
        "12": 19,
        "173": null,
        "177": null,
        "183": null,
        "184": null,
        "187": null,
        "188": null,
        "190": null,
        "194": null,
        "196": null,
        "197": null,
        "198": null,
        "199": null,
        "231": 99,
        "232": null,
        "233": null,
        "completed": 0,
        "conveyance": 0,
        "dev_error": 0,
        "disk": "nvme5",
        "exit": 0,
        "extended": 0,
        "from_factor": null,
        "fw_version": "2B2QKXG7",
        "health_pass": 1,
        "interrupted": 0,
        "json_err": 0,
        "json_err_str": "",
        "max_temp": null,
        "model_name": "Samsung SSD 990 EVO Plus 4TB",
        "offline": 0,
        "over_temp": 0,
        "read_failure": 0,
        "selective": 0,
        "selftest_log": null,
        "serial": "S7U8NJ0Y716854P",
        "short": 0,
        "temp_limit": null,
        "unknown_failure": 0
      },

[VVelox]

Weird. Both show up as 1068 for me.

That said, awesome. Looks like it is working. Need to fix form_factor detection for NVMe it looks like.

[VVelox]

Okay form factor info for NVMe is now returned. Also fixed some temp handling stuff.

https://github.com/librenms/librenms-agent/blob/master/snmp/smart-v1

[VVelox]

@dlangille Hey, how is looking now? All good?

[dlangille]

@VVelox When I put in the new version (yesterday), the Apps | SMART panel stopped displaying any graphs. No devices, no graphs, just empty. About 7 minutes ago, I put the most recent version of smart-v1 into that host.

Another host, still on the old stuff (pre yesterday) is still displaying SMART data. However, there is a lot of NAN showing there.

Is there a corresponding back-end update I need for this? I'm on 26.2.0 (not using daily updates; running from a FreeBSD package).

[VVelox]

@dlangille Nothing needed on the backend. Although you should have snmpd catting /var/cache/smart.snmp for the results instead of /var/cache/smart like previously.

[dlangille]

A welcome point.

No, I still had it doing this:

 % grep smart /usr/local/etc/snmpd.conf
extend smart            /usr/local/etc/snmp/smart

I see I already had a cronjob for this:

[12:26 r730-01 dvl /usr/local/etc/cron.d] % cat snmpd-smart
# for LibreNMS via /usr/local/etc/snmp/smart
#minute	hour	mday	month	wday	who	    command
*/5       *     *        *      *       root    /usr/bin/lockf -t 0 /tmp/.snmp-smart /usr/local/etc/snmp/smart -u

I have since changed my snmpd config to use:

extend smart            /bin/cat /var/cache/librenms/smart.snmp

snmpd has been restarted.

My smart.config file specified the above referenced directory.

[VVelox]

Awesome. Yeah. That should get it goind.

On fun note something else fun to do with it. Check out the the -t flag. You can use it to run tests on a regular basis.

[dlangille]

Now I have graphs. Of note, for the drive in question:

• Age - when clicked on, becomes Smart Big5 - all nan
• Temperature - says temp is 30 - seems reasonable
• SSD-specific - showing red letters, no graph: librenms/rrd/HOSTNAME
• Smart Other - all nan
• Smart Tests Status - all zero
• Smart Tests Ran - all zero
• Smart Runtime - power on hours, 4k - seems about right

Then I ran: sudo smartctl -t short /dev/nvme5 - waited to confirm it had completed:

Num  Test_Description  Status                       Power_on_Hours  Failing_LBA  NSID Seg SCT Code
 0   Short             Completed without error                3761            -     -   -   -    -

Waited a bit more.. Now we have:

• Smart Tests Status - Completed 103m (no idea where that value comes from)
• Smart Tests Ran - also 103m