NVMe/TCP Initiator for Linux

Prerequisites

Kernel 5.0 or later
RHEL 9.2 or later
Ubuntu 24.04 or later
SLES 15 SP3 or later

Installation

Install the nvme-cli:

yum install nvme-cli

Load the NVMe-oF module:
```
modprobe nvme-fabrics
```

Verify the NVMe/TCP target is reachable:

nvme discover -t tcp -a GATEWAY_IP -s 8009

Connect to the NVMe/TCP target. For High-availability use the connect-all command:
```
nvme connect-all --traddr GATEWAY_IP --transport tcp -l 1800 -s 8009
```
- ‘-l 1800’ is recommended to allow the initiator to continue trying to connect to GWs for 1800 seconds. This is helpful in cases that the GW is temporarily unavailable for any reason.
- ‘-s 8009’ is the port address of the Discovery controller. The connect-all command will connect to the DC first, and then will use the information it retrns to connect to the GWs.

Next steps

Verify that the initiator is set up correctly:

Verify that the initiator is connected to all NVMe-oF gateways and subsystems in the gateway group

nvme list-subsys

example output:

nvme-subsys<X> - NQN=<NQN>
\
    +- nvmeX tcp traddr=<GW IP>,trsvcid=4420 live
    +- nvmeY tcp traddr=<GW IP>,trsvcid=4420 live
    +- nvmeZ tcp traddr=<GW IP>,trsvcid=4420 live
    +- nvmeW tcp traddr=<GW IP>,trsvcid=4420 live

List the NVMe block devices.
```
nvme list
```
Create a filesystem on the desired device:
```
mkfs.ext4 NVME_NODE_PATH
```

Mount the filesystem:

mkdir /mnt/nvmeof

mount NVME_NODE_PATH /mnt/nvmeof

List the NVME-oF files:
```
ls /mnt/nvmeof
```

Create a text file in the /mnt/nvmeof directory:

echo "Hello NVME-oF" > /mnt/nvmeof/hello.text

Verify that the file can be accessed:
```
cat /mnt/nvmeof/hello.text
```

Brought to you by the Ceph Foundation

The Ceph Documentation is a community resource funded and hosted by the non-profit Ceph Foundation. If you would like to support this and our other efforts, please consider joining now.