Notice

This document is for a development version of Ceph.

File System Shares Over SMB

CephFS access can be provided to clients using the SMB protocol via the Samba suite and samba-container images - managed by Ceph.

The smb manager module provides an interface for deploying and controlling clusters of Samba services as well as managing SMB shares. In the smb manager module a cluster is a logical management unit that may map to one or more managed Samba service - by itself a cluster may or may not be using any high-availability mechanisms.

If the module is not already enabled on your cluster you can enable by running ceph mgr module enable smb.

There are two ways of interacting with the smb module. The imperative method uses commands like ceph smb cluster create ... and ceph smb share rm ... and should be very familiar to those who have used Ceph’s nfs manager module on the command line. The declarative method uses the command ceph smb apply to process “resource descriptions” specified in YAML or JSON. This method should be familiar to those who have used Ceph orchestration with cephadm, just using SMB specific resource types.

Note

Ceph managed Samba only supports SMB2 and SMB3 versions of the protocol. The SMB1 version of the protocol, sometimes known as CIFS, is not supported. Some systems, such as the Linux kernel, provide tooling for both SMB1 and SMB2+ under the CIFS moniker. Check the documentation of the software packages used to ensure they support SMB2+ regardless of how the tool is named.

Note

At this time, the smb module requires cephadm orchestration. It does not function without orchestration.

Management Commands - Imperative Style

Cluster Commands

Create Cluster

ceph smb cluster create <cluster_id> {user|active-directory} [--domain-realm=<domain_realm>] [--domain-join-user-pass=<domain_join_user_pass>] [--define-user-pass=<define_user_pass>] [--custom-dns=<custom_dns>] [--placement=<placement>] [--clustering=<clustering>] [--password-filter=<password_filter>] [--password-filter-out=<password_filter_out>]

Create a new logical cluster, identified by the cluster ID value. The cluster create command must specify the authentication mode the cluster will use. This may either be one of:

Custom users and groups, also known as a standalone server, with the user keyword
An Active Directory (AD) domain member server, with the active-directory keyword

Options:

cluster_id: A short string uniquely identifying the cluster
auth_mode: One of user or active-directory
domain_realm: Required for active-directory clusters and ignored by user clusters. The domain/realm value identifying the AD domain. Required when choosing active-directory
domain_join_user_pass: Required for active-directory clusters and ignored by user clusters. A string in the form <username>%<password> that will be used to join Samba servers to the AD domain.
define_user_pass: Optional. Ignored by active-directory clusters. A string of the form <username>%<password> that will be used for authentication in user auth_mode. Can be specified multiple times to define more than one user.
custom_dns: Optional. Can be specified multiple times. One or more IP Addresses that will be applied to the Samba containers to override the default DNS resolver(s). This option is intended to be used when the host Ceph node is not configured to resolve DNS entries within AD domain(s).
placement: Optional. A Ceph orchestration placement specifier
clustering: Optional. Control if a cluster abstraction actually uses Samba’s clustering mechanism. The value may be one of default, always, or never. A default value indicates that clustering should be enabled if the placement count value is any value other than 1. A value of always enables clustering regardless of the placement count. A value of never disables clustering regardless of the placement count. If unspecified, default is assumed.
public_addrs: Optional. A string in the form of <ipaddress/prefixlength>[%<destination address>]. Supported only when using Samba’s clustering. Assign “virtual” IP addresses that will be managed by the clustering subsystem and may automatically move between nodes running Samba containers. Can be specified multiple times to assign more than one public address to the SMB cluster.
password_filter: Optional. One of none or base64. If the filter is none the password values on the command line are assumed to be plain text. If the filter is base64 the password values are assumed to be obscured with base64 encoding the string. If --password-filter-out is not specified this filter will also be applied to the output.
password_filter_out: Optional. One of none, base64, or hidden. If the filter is none the password fields in the output are emitted as plain text. If the filter is base64 password fields will be obscured by base64 encoding the string. If the filter is hidden the password values will be replaced by an invalid generic replacement string containing only asterisks.

Examples

Create a cluster with two locally defined users:

ceph smb cluster create bob user \
    --define-user-pass=bob%Passw0rd1 \
    --define-user-pass=carol%Passw0rd2

Create a cluster with a single user and an explicit placement value for cephadm (hosts labeled smb):

ceph smb cluster create test1 user \
    --define-user-pass=test%Passw0rd1 \
    --placement="label:smb"

Create a cluster connected to an active directory system. Use a custom DNS server:

ceph smb cluster create test2 active-directory \
    --domain-realm=MYDOM.EXAMPLE.ORG \
    --domain-join-user-pass=Administrator%Ph0nyPassw0rd \
    --custom-dns=192.168.76.210

Create a cluster connected to an active directory system, similar to the previous example. Set three CTDB public address values and a custom placement:

ceph smb cluster create test3 active-directory \
    --domain-realm=MYDOM.EXAMPLE.ORG \
    --domain-join-user-pass=Administrator%Ph0nyPassw0rd \
    --custom-dns=192.168.76.210 \
    --public-address=192.168.76.110/24 \
    --public-address=192.168.76.111/24 \
    --public-address=192.168.76.112/24 \
    --placement="3 label:smb"

Remove Cluster

ceph smb cluster rm <cluster_id> [--password-filter=<password_filter>]

Remove a logical SMB cluster from the Ceph cluster.

Options:

cluster_id: A cluster_id value identifying a cluster resource.
password_filter: Optional. One of none, base64, or hidden. If the filter is none the password fields in the output are emitted as plain text. If the filter is base64 password fields will be obscured by base64 encoding the string. If the filter is hidden the password values will be replaced by an invalid generic replacement string containing only asterisks.

List Clusters

ceph smb cluster ls [--format=<format>]

Print a listing of cluster ids. The output defaults to JSON, select YAML encoding with the --format=yaml option.

Share Commands

Update Share QoS

ceph smb share update cephfs qos <cluster_id> <share_id> [--read-iops-limit=<int>] [--write-iops-limit=<int>] [--read-bw-limit=<str>] [--write-bw-limit=<str>] [--read-burst-mult=<int>] [--write-burst-mult=<int>]

Update Quality of Service (QoS) settings for a CephFS-backed share. This allows administrators to apply per-share rate limits on SMB input/output (I/O) operations, specifically limits on IOPS (Input/Output Operations per Second) and bandwidth (in bytes per second) for both read and write operations. Additionally, burst multipliers can be configured to allow temporary bursts above the configured limits.

Options:

read_iops_limit: Optional integer. Maximum number of read operations per second (0 = disabled). Valid range: 0 to 1,000,000. Values above this will be capped.
write_iops_limit: Optional integer. Maximum number of write operations per second (0 = disabled). Valid range: 0 to 1,000,000. Values above this will be capped.
read_bw_limit: Optional string. Maximum allowed bandwidth for read operations (0 = disabled). This can be specified as a plain integer representing bytes per second, or as a human-readable string with bytes per second as a unit. Example: "1M" = 1 MiB/s (1,048,576 bytes/s). Valid range: 0 to 1 << 40 (≈1 T). Numeric values above this will be capped.
write_bw_limit: Optional string. Maximum allowed bandwidth for write operations (0 = disabled). This can be specified as a plain integer representing bytes per second, or as a human-readable string with bytes per second as a unit. Example: "1M" = 1 MiB/s (1,048,576 bytes/s). Valid range: 0 to 1 << 40 (≈1 T). Numeric values above this will be capped.
read_burst_mult: Optional integer. Burst multiplier for read operations (value ÷ 10 = multiplier), allowing temporary bursts above the configured limit. Example: 20 = 2* the configured limit. Range: 10-100 (1* to 10*), default: 15 (1.5*).
write_burst_mult: Optional integer. Burst multiplier for write operations (value ÷ 10 = multiplier), allowing temporary bursts above the configured limit. Example: 20 = 2* the configured limit. Range: 10-100 (1* to 10*), default: 15 (1.5*).

Behavior:

All limits are optional
Setting a limit to 0 disables that specific QoS limit
Setting all four limits (IOPS and bandwidth) to 0 completely removes QoS configuration
Burst multipliers only apply when their corresponding limit is enabled (non-zero)
Bandwidth limits can be specified with human-readable units (e.g., "10M", "5G")
Burst multipliers are expressed in tenths (e.g., 15 = 1.5*, 20 = 2*, 30 = 3*)

Burst Behavior

The burst multiplier allows short-term I/O bursts above your configured limits. For example, if you set read_iops_limit = 1000 and read_burst_mult = 20, your share can handle bursts up to 2000 read operations per second for short periods, while maintaining an average of 1000 IOPS over time.

This is useful for workloads that have occasional spikes in activity. The appropriate burst multiplier depends on your workload - higher values allow larger bursts but may temporarily consume more resources.

Note

The burst multiplier only affects short-term spikes. The long-term average throughput remains limited by your configured IOPS and bandwidth limits.

Examples:

Set QoS limits with burst multipliers for a share:

ceph smb share update cephfs qos foo bar \
  --read-iops-limit=100 \
  --write-iops-limit=200 \
  --read-bw-limit="10M" \
  --write-bw-limit="20M" \
  --read-burst-mult=20 \
  --write-burst-mult=15

In this example: - Read burst multiplier of 20 means 2* the read IOPS limit (allowing bursts up to 200 read IOPS) - Write burst multiplier of 15 means 1.5* the write IOPS limit (allowing bursts up to 300 write IOPS)

Disable QoS for a share:

ceph smb share update cephfs qos foo bar \
  --read-iops-limit=0 \
  --write-iops-limit=0 \
  --read-bw-limit=0 \
  --write-bw-limit=0

List Shares

ceph smb share ls <cluster_id> [--format=<format>]

Print a listing of share ids. The output defaults to JSON, select YAML encoding with the --format=yaml option.

Management Commands - Declarative Style

In addition to the basic imperative management commands the smb manager module supports configuration using declarative resource specifications. Resource specifications can be written in either JSON or YAML. These resource specifications can be applied to the cluster using the ceph smb apply command, for example:

ceph smb apply -i /path/to/resources.yaml

In addition to the resource specification the apply sub-command accepts options that control how the input and output of the command behave:

ceph smb apply [--format=<format>] [--password-filter=<password_filter>] [--password-filter-out=<password_filter_out>] -i <input>

Options:

format: One of json (the default) or yaml. Output format can be selected independent of the input format.
password_filter: Optional. One of none or base64. If the filter is none the password fields in the input are assumed to be plain text. If the filter is base64 the password fields are assumed to be obscured with base64 encoding the string. If --password-filter-out is not specified this filter will also be applied to the output.
password_filter_out: Optional. One of none, base64, or hidden. If the filter is none the password fields in the output are emitted as plain text. If the filter is base64 password fields will be obscured by base64 encoding the string. If the filter is hidden the password values will be replaced by an invalid generic replacement string containing only asterisks.
input: A file name or - to use the standard input (aka stdin).

Resources that have already been applied to the Ceph cluster configuration can be viewed using the ceph smb show command. For example:

ceph smb show ceph.smb.cluster.cluster1

The show command can show all resources, resources of a given type, or specific resource items. Options can be provided that control the output of the command.

ceph smb show [resource_name...] [--format=<format>] [--results=<results>] [--password-filter=<password_filter>]

Options:

resource_name: One or more strings specifying a resource or resource type. See description below.
format: One of json (the default) or yaml.
results: One of collapsed (the default) or full. When set to collapsed the output of the command will show only the resource JSON/YAML of a single item if a single item is found. When set to full even if a single item is found the output will always include a wrapper object like (in pseudo-JSON): {"resources": [...Resource objects...]}.
password_filter: Optional. One of none, base64, or hidden. If the filter is none the password fields in the output are emitted as plain text. If the filter is base64 password fields will be obscured by base64 encoding the string. If the filter is hidden the password values will be replaced by an invalid generic replacement string containing only asterisks.

resource_name arguments can take the following forms:

ceph.smb.cluster: show all cluster resources
ceph.smb.cluster.<cluster_id>: show specific cluster with given cluster ID
ceph.smb.share: show all share resources
ceph.smb.share.<cluster_id>: show all share resources part of the given cluster
ceph.smb.share.<cluster_id>.<share_id>: show specific share resource with the given cluster and share ids
ceph.smb.usersgroups: show all Users & Groups resources
ceph.smb.usersgroups.<users_goups_id>: show a specific Users & Groups resource
ceph.smb.join.auth: show all join auth resources
ceph.smb.join.auth.<auth_id>: show a specific join auth resource

For example:

ceph smb show ceph.smb.cluster.bob ceph.smb.share.bob

Will show one cluster resource (if it exists) for the cluster “bob” as well as all share resources associated with the cluster “bob”.

Note

The show subcommand prints out resources in the same form that the apply command accepts, making it possible to “round-trip” values between show and apply.

Composing Resource Specifications

A resource specification is made up of one or more Ceph SMB resource descriptions written in either JSON or YAML formats. More than one resource can be specified if the resources are contained within a JSON/YAML list, or a JSON/YAML object containing the key resources with a corresponding list value containing the resources. Additionally, a YAML specification may consist of a series of YAML documents each containing a resource.

An example YAML based simple list looks like the following:

- resource_type: ceph.smb.cluster
  cluster_id: rhumba
  # ... other fields skipped for brevity ...
- resource_type: ceph.smb.cluster
  cluster_id: salsa
  # ... other fields skipped for brevity ...
- resource_type: ceph.smb.share
  cluster_id: salsa
  share_id: foo
  # ... other fields skipped for brevity ...

An example JSON based simple list looks like the following:

[
  {"resource_type": "ceph.smb.cluster",
   "cluster_id": "rhumba",
   "...": "... other fields skipped for brevity ..."
  },
  {"resource_type": "ceph.smb.cluster",
   "cluster_id": "salsa",
   "...": "... other fields skipped for brevity ..."
  },
  {"resource_type": "ceph.smb.share",
   "cluster_id": "salsa",
   "share_id": "foo",
   "...": "... other fields skipped for brevity ..."
  }
]

An example YAML based resource list looks like the following:

resources:
  - resource_type: ceph.smb.cluster
    cluster_id: rhumba
    # ... other fields skipped for brevity ...
  - resource_type: ceph.smb.cluster
    cluster_id: salsa
    # ... other fields skipped for brevity ...
  - resource_type: ceph.smb.share
    cluster_id: salsa
    share_id: foo
    # ... other fields skipped for brevity ...

An example JSON based resource list looks like the following:

{
  "resources": [
    {"resource_type": "ceph.smb.cluster",
     "cluster_id": "rhumba",
     "...": "... other fields skipped for brevity ..."
    },
    {"resource_type": "ceph.smb.cluster",
     "cluster_id": "salsa",
     "...": "... other fields skipped for brevity ..."
    },
    {"resource_type": "ceph.smb.share",
     "cluster_id": "salsa",
     "share_id": "foo",
     "...": "... other fields skipped for brevity ..."
    }
  ]
}

An example YAML resource list consisting of multiple documents looks like the following:

---
resource_type: ceph.smb.cluster
cluster_id: rhumba
# ... other fields skipped for brevity ...
---
resource_type: ceph.smb.cluster
cluster_id: salsa
# ... other fields skipped for brevity ...
---
resource_type: ceph.smb.share
cluster_id: salsa
share_id: foo
# ... other fields skipped for brevity ...

Each individual resource description must belong to one of the types described below.

Note

For brevity, all following examples will use YAML only. Assume that the equivalent JSON forms are valid.

Cluster Resource

A cluster resource supports the following fields:

resource_type

A literal string ceph.smb.cluster

cluster_id

A short string identifying the cluster

auth_mode

One of user or active-directory

intent

One of present or removed. If not provided, present is assumed. If removed all following fields are optional

domain_settings

Object. Ignored/optional for user auth. Required for active-directory Fields:

realm: Required string. AD domain/realm name.
join_sources: Required list. Each element is an object with join source fields

user_group_settings

List. Ignored/optional for active-directory. Each element is an object with user group source fields

custom_dns

Optional. List of IP Addresses. IP addresses will be used as DNS resolver(s) in Samba containers allowing the containers to use domain DNS even if the Ceph host does not

custom_ports

Optional. A mapping of service names to port numbers that will override the default ports used for those services. The service names are: smb, smbmetrics, ctdb, and remote-control. If a service name is not present in the mapping the default port will be used. For example, {"smb": 4455, "smbmetrics": 9009} will change the ports used by SMB for client access and the metrics exporter, but not change the port used by the CTDB clustering daemon. Note - not all SMB clients are able to use alternate port numbers.

bind_addrs

Optional. A list of objects indicating what IP address or IP network the SMB and related services may bind to. The fields described for these objects are mutually exclusive, but at least one field is required. (The behavior of this option changes when used with clustering and public_addrs. See note below.) Fields:

address: Optional. A single IP address represented as a string. For example, 192.168.7.50.
network: Optional. A single IP network represented as a string. A network can be used to specify a range of many IP addresses. The network string always includes a “/” character before a prefix length. For example, 192.168.7.0/24.

placement

Optional. A Ceph Orchestration placement specifier. Defaults to one host if not provided

clustering

Optional. Control if a cluster abstraction actually uses Samba’s clustering mechanism. The value may be one of default, always, or never. A default value indicates that clustering should be enabled if the placement count value is any value other than 1. A value of always enables clustering regardless of the placement count. A value of never disables clustering regardless of the placement count. If unspecified, default is assumed.

public_addrs

List of objects; optional. Supported only when using Samba’s clustering. Assign “virtual” IP addresses that will be managed by the clustering subsystem and may automatically move between nodes running Samba containers. (The behavior of this option changes when used with bind_addrs. See note below.) Fields:

address: Required string. An IP address with a required prefix length (example: 192.168.4.51/24). This address will be assigned to one of the host’s network interfaces and managed automatically.
destination: Optional. String or list of strings. A destination defines where the system will assign the managed IPs. Each string value must be a network address (example 192.168.4.0/24). One or more destinations may be supplied. The typical case is to use exactly one destination and so the value may be supplied as a string, rather than a list with a single item. Each destination network will be mapped to an interface on a host. Run cephadm list-networks for an example of these mappings. If destination is not supplied the network is automatically determined using the address value supplied and taken as the destination.

remote_control

Optional object. This object configures an SMB cluster to deploy an extra remote control service. This service provides a gRPC server that can be used to enumerate connected clients and disconnect clients from shares. This service uses mTLS for authentication. By default, this service uses port 54445. The port can be configured using the custom_ports parameter in the cluster resource. If the service is enabled and any of the cert, key, or ca_cert fields are not populated mTLS will be disabled and the service will operate in a read-only mode. Running the service with mTLS disabled is not recommended. Fields:

enabled: Optional boolean. If explicitly set to true or false this field will enable or disable the remote control service. If left unset the TLS fields will be checked - if the TLS fields are filled automatically enable the service.
cert: Optional object. The fields are described in tls source fields
key: Optional object. The fields are described in tls source fields
ca_cert: Optional object. The fields are described in tls source fields
locally_enabled: Optional boolean. If set to true this field will enable the remote control service local listener. The local listener lets processes on the Ceph cluster host communicate with the remote control service independently of the default TCP/mTLS listener. The TLS certificates configuration values do not apply to this unix socket based listener.

external_ceph_cluster:

Optional object. The fields are described in external Ceph cluster source fields. This is an advanced option and should be used with caution.

debug_level:

Optional object. Specify subsystem based default logging level values. Supported keys are samba and ctdb. Supported values include numbers (1 through 10 typically) or level names such as INFO or DEBUG. The system will translate names to numbers (for samba) or vice-versa as needed. Example YAML snippet: debug_level: {smb: 8, ctdb: INFO}.

custom_smb_global_options

Optional mapping. Specify key-value pairs that will be directly added to the global smb.conf options (or equivalent) of a Samba server. Do not use this option unless you are prepared to debug the Samba instances yourself.

This option is meant for developers, feature investigators, and other advanced users to take more direct control of a share’s options without needing to make changes to the Ceph codebase. Entries in this map should match parameters in smb.conf and their values. A special key _allow_customization must appear somewhere in the mapping with the value of i-take-responsibility-for-all-samba-configuration-errors as an indicator that the user is aware that using this option can easily break things in ways that the Ceph team can not help with. This special key will automatically be removed from the list of options passed to Samba.

Warning

Setting the clustering option allows an administrator to choose exactly when Samba’s CTDB clustering will be used. By default, the use of Samba’s clustering is derived from the placement count. If you choose to set clustering make sure you understand how clustering interacts with placement. In particular, be aware that running multiple instances of the same smb service without clustering enabled can cause unexpected behavior.

Warning

The behavior of the system when combining bind_addrs and public_addrs on a cluster could lead to unexpected results. The smbd process can only dynamically add/remove public addresses when assigned to monitor a network interface (e.g. eth0) versus a specific address. If the network interface is assigned multiple addresses and those addresses overlap with a different SMB cluster it is possible the services may fail to start. Currently, one must manually ensure that the interfaces used by an IP or network is exclusively used for that network to ensure SMB services start properly.

A join source object supports the following fields:

source_type: Optional. Must be resource if specified.
ref: String. Required for source_type: resource. Must refer to the ID of a ceph.smb.join.auth resource

A user group source object supports the following fields:

source_type: Optional. One of resource (the default) or empty
ref: String. Required for source_type: resource. Must refer to the ID of a ceph.smb.usersgroups resource

A TLS source object supports the following fields:

source_type: Optional. Must be resource if specified.
ref: String. Required for source_type: resource. Must refer to the ID of a ceph.smb.tls.credential resource

An external Ceph cluster source object supports the following fields:

source_type:: Optional. Must be resource if specified.
ref:: String. Required for source_type: resource. Must refer to the ID of a ceph.smb.ext.cluster resource

Note

The source_type empty is generally only for debugging and testing the module and should not be needed in production deployments.

The following is an example of a cluster configured for AD membership:

resource_type: ceph.smb.cluster
cluster_id: tango
auth_mode: active-directory
domain_settings:
  realm: DOMAIN1.SINK.TEST
  join_sources:
    # this join source refers to a join auth resource with id "join1-admin"
    - source_type: resource
      ref: join1-admin
custom_dns:
  - "192.168.76.204"
placement:
  count: 1

The following is an example of a cluster configured for standalone operation:

resource_type: ceph.smb.cluster
cluster_id: rhumba
auth_mode: user
user_group_settings:
  - source_type: resource
    ref: ug1
placement:
  hosts:
    - node6.mycluster.sink.test

An example cluster resource with intent to remove:

resource_type: ceph.smb.cluster
cluster_id: rhumba
intent: removed

Share Resource

A share resource supports the following fields:

resource_type

A literal string ceph.smb.share

cluster_id

A short string identifying the cluster

share_id

A short string identifying the share. Must be Unique within a cluster

intent

One of present or removed. If not provided, present is assumed. If removed all following fields are optional

name

Optional string. A longer name capable of supporting spaces and other characters that will be presented to SMB clients

readonly

Optional boolean, defaulting to false. If true no clients are permitted to write to the share

browseable

Optional boolean, defaulting to true. If true the share will be included in share listings visible to clients

comment

Optional string. A single line description used to provide human-readable explanation or notes about the share.

max_connections

Optional integer. Specifies the maximum number of simultaneous client connections to a specific share. The default value is 0 and it indicates that there is no limit on the number of connections

cephfs

Required object. Fields:

volume

Required string. Name of the cephfs volume to use

path

Required string. Path within the volume or subvolume to share

subvolumegroup

Optional string. Name of a subvolumegroup to share

subvolume

Optional string. Name of a subvolume to share. If subvolumegroup is not set and this value contains exactly one / character, the subvolume field will automatically be split into <subvolumegroup>/<subvolume> parts for convenience

provider

Optional. Selects how CephFS storage should be provided to the share. The value may be one of samba-vfs, samba-vfs/classic, samba-vfs/new or samba-vfs/proxied. If unspecified, samba-vfs is assumed.

Selecting samba-vfs/new selects the new Samba VFS plugin to connect to CephFS and samba-vfs/proxied uses the new VFS plugin but routes the connections through a proxy. Using the proxy allows for a greater number of simultaneous client connections to the share, but it comes at the cost of performance. samba-vfs/classic uses the older Samba VFS plugin to connect to CephFS. samba-vfs automatically selects the preferred VFS based implementation, currently samba-vfs/proxied. This option is suitable for the majority of use cases and can be left unspecified for most shares.

qos

Optional object. Quality of Service settings for the share. Fields:

read_iops_limit: Optional integer. Maximum number of read operations per second (0 = disabled). Valid range: 0 to 1,000,000. Values above this will be capped.
write_iops_limit: Optional integer. Maximum number of write operations per second (0 = disabled). Valid range: 0 to 1,000,000. Values above this will be capped.
read_bw_limit: Optional string. Maximum allowed bandwidth for read operations (0 = disabled). This can be specified as a plain integer representing bytes per second, or as a human-readable string with bytes per second as a unit. Example: "1M" = 1 MiB/s (1,048,576 bytes/s). Valid range: 0 to 1 << 40 (≈1 T). Numeric values above this will be capped.
write_bw_limit: Optional string. Maximum allowed bandwidth for write operations (0 = disabled). This can be specified as a plain integer representing bytes per second, or as a human-readable string with bytes per second as a unit. Example: "1M" = 1 MiB/s (1,048,576 bytes/s). Valid range: 0 to 1 << 40 (≈1 T). Numeric values above this will be capped.
read_burst_mult: Optional integer. Burst multiplier for read operations (value ÷ 10 = multiplier), allowing temporary bursts above the configured limit. Example: 20 = 2* the configured limit. Default: 15 (1.5*).
write_burst_mult: Optional integer. Burst multiplier for write operations (value ÷ 10 = multiplier), allowing temporary bursts above the configured limit. Example: 20 = 2* the configured limit. Default: 15 (1.5*).

restrict_access

Optional boolean, defaulting to false. If true the share will only permit access by users explicitly listed in login_control.

login_control

Optional list of objects. Fields:

name: Required string. Name of the user or group.
category: Optional. One of user (default) or group.
access: One of read (alias r), read-write (alias rw), none, or admin. Specific access level to grant to the user or group when logging into this share. The none value denies access to the share regardless of the restrict_access value.

hosts_access

Optional list of objects. Items in the hosts_access list are used to restrict the share to use by specific client addresses. If any allow entries are found all other hosts will be denied. Fields:

access: Required string. One of allow or deny.
address: Optional string. Required if network field is not supplied. The string value must be either an IPv4 address or an IPv6 address. The specific host will be allowed or denied access to the share.
network: Optional string. Required if address field is not supplied. The string value must be either an IPv4 network or an IPv6 network (for example 192.0.2.0/24). If the client’s IP address is found within the specified network that host will be allowed or denied access to the share.

custom_smb_share_options

Optional mapping. Specify key-value pairs that will be directly added to the smb.conf (or equivalent) of a Samba server. Do not use this option unless you are prepared to debug the Samba instances yourself.

This option is meant for developers, feature investigators, and other advanced users to take more direct control of a share’s options without needing to make changes to the Ceph codebase. Entries in this map should match parameters in smb.conf and their values. A special key _allow_customization must appear somewhere in the mapping with the value of i-take-responsibility-for-all-samba-configuration-errors as an indicator that the user is aware that using this option can easily break things in ways that the Ceph team can not help with. This special key will automatically be removed from the list of options passed to Samba.

The following is an example of a share with QoS settings including burst multipliers and human-readable bandwidth limits:

resource_type: ceph.smb.share
cluster_id: tango
share_id: sp1
name: "Staff Pics"
cephfs:
  volume: cephfs
  path: /pics
  subvolumegroup: smbshares
  subvolume: staff
  qos:
    read_iops_limit: 100
    write_iops_limit: 50
    read_bw_limit: "10M"
    write_bw_limit: "5M"
    read_burst_mult: 20
    write_burst_mult: 15

Another example with plain byte values:

resource_type: ceph.smb.share
cluster_id: tango
share_id: sp1
cephfs:
  volume: cephfs
  path: /pics
  qos:
    read_iops_limit: 100
    write_iops_limit: 50
    read_bw_limit: 10485760      # 10 MB/s
    write_bw_limit: 5242880       # 5 MB/s
    read_burst_mult: 10           # 1× burst
    write_burst_mult: 20          # 2× burst

Another example, this time of a share with QoS disabled:

resource_type: ceph.smb.share
cluster_id: tango
share_id: sp2
cephfs:
  volume: cephfs
  path: /data
  qos:
    read_iops_limit: 0
    write_iops_limit: 0
    read_bw_limit: 0
    write_bw_limit: 0
    # Note: burst multipliers are ignored when limits are disabled

And finally, a share with an intent to be removed:

resource_type: ceph.smb.share
cluster_id: tango
share_id: sp2
intent: removed

Join-Auth Resource

A join auth resource supports the following fields:

resource_type

A literal string ceph.smb.join.auth

auth_id

A short string identifying the join auth resource

intent

One of present or removed. If not provided, present is assumed. If removed all following fields are optional

auth

Required object. Fields:

username: Required string. User with ability to join a system to AD
password: Required string. The AD user’s password

linked_to_cluster:

Optional. A string containing a cluster ID. If set, the resource may only be used with the linked cluster and will automatically be removed when the linked cluster is removed.

Example:

resource_type: ceph.smb.join.auth
auth_id: join1-admin
auth:
  username: Administrator
  password: Passw0rd

Users-and-Groups Resource

A users & groups resource supports the following fields:

resource_type

A literal string ceph.smb.usersgroups

users_groups_id

A short string identifying the users and groups resource

intent

One of present or removed. If not provided, present is assumed. If removed all following fields are optional.

values

Required object. Fields:

users

List of objects. Fields:

name: A user name
password: A password

groups

List of objects. Fields:

name: The name of the group

linked_to_cluster:

Optional. A string containing a cluster ID. If set, the resource may only be used with the linked cluster and will automatically be removed when the linked cluster is removed.

Example:

resource_type: ceph.smb.usersgroups
users_groups_id: ug1
values:
  users:
    - name: chuckx
      password: 3xample101
    - name: steves
      password: F00Bar123
    groups: []

TLS Credential Resource

TLS credential resources store copies of TLS files such as Certificates, Keys, or CA Certificates. A TLS credential resource supports the following fields:

resource_type: A literal string ceph.smb.tls.credential
tls_credential_id: A short string identifying the TLS credential resource
intent: One of present or removed. If not provided, present is assumed. If removed all following fields are optional
credential_type: Required string. The value may be one of cert, key, or ca-cert. This value indicates what type of TLS credential the value field holds.
value:: A string containing the TLS certificate or key value in PEM encoding.
linked_to_cluster:: Optional. A string containing a cluster ID. If set, the resource may only be used with the linked cluster and will automatically be removed when the linked cluster is removed.

Example:

resource_type: ceph.smb.tls.credential
tls_credential_id: mycert1
credential_type: cert
# NOTE: The value below is truncated to make the documentation more
# consise. A real embedded certificate is expected to be valid and
# will be longer than this example.
value: |
  -----BEGIN CERTIFICATE-----
  MIIFDjCCA/agAwIBAgISBtFQfoXc4RmyVabbv28RClKdMA0GCSqGSIb3DQEBCwUA
  MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
  EwNSMTAwHhcNMjUwNTE5MTAyNzUyWhcNMjUwODE3MTAyNzUxWjASMRAwDgYDVQQD
  EwdjZXBoLmlvMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAx6fif6PQ
  LOTdnO8d1JHcF7D+oB/mQlplFz4vwq/GB6Y4oWK3uCQ4PPz/qyvE4wyvc5EPhjfg
  d8XNc4ajEBcSUoRj3UwWwiA4oht0SyoJIfwVGp/kF5jxHhVCLdoaaqAxv7nAghWM
  6Dg=
  -----END CERTIFICATE-----

External Ceph Cluster Resource

This resource can be used to configure an SMB Cluster hosted on Ceph cluster to use CephFS volumes provided by an external Ceph cluster. The values provided below allow the SMB server to connect to a cluster other than the one it is running on.

Warning

This is an advanced feature that should be used with care. It allows SMB servers to contact CephFS on a different cluster. Because of that, many values provided below can not be validated and other validations that smb mgr module normally does are disabled. In addition, automatic subvolume to path mapping is disabled. Shares in SMB clusters making use of an external Ceph cluster must not specify a subvolume by name and must specify an absolute path to a subvolume.

An external ceph cluster resource supports the following fields.

resource_type

A literal string ceph.smb.ext.cluster

external_ceph_cluster_id

A short string identifying the cluster

intent

One of present or removed. If not provided, present is assumed. If removed all following fields are optional

fsid

String. The UUID/FSID of the external cluster

mon_host

String. The mon_host string (as sourced from a ceph.conf file)

cephfs_user

Object. Fields:

name: String. A ceph user name indicating the cephx user that will access the CephFS volume(s) on the external cluster
key: String. The Base64 encoded key value corresponding to the cephx user name provided

A Declarative Configuration Example

Using the resource descriptions above we can put together an example that creates a cluster and shares from scratch based on a resource configuration file. First, create the YAML with the contents:

resources:
  # Define an AD member server cluster
  - resource_type: ceph.smb.cluster
    cluster_id: tango
    auth_mode: active-directory
    domain_settings:
      realm: DOMAIN1.SINK.TEST
      join_sources:
        - source_type: resource
          ref: join1-admin
    custom_dns:
      - "192.168.76.204"
    # deploy 1 set of samba containers on a host labeled "ilovesmb"
    placement:
      count: 1
      label: ilovesmb
  # Define a join auth that our cluster will use to join AD
  # Warning: Typically you do not want to use the Administrator user
  # to perform joins on a production AD
  - resource_type: ceph.smb.join.auth
    auth_id: join1-admin
    auth:
      username: Administrator
      password: Passw0rd
  # A share that uses the root of a subvolume
  # The share name is the same as its id
  - resource_type: ceph.smb.share
    cluster_id: tango
    share_id: cache
    cephfs:
      volume: cephfs
      subvolumegroup: smb1
      subvolume: cache
      path: /
  # A share that uses the a sub-dir of a subvolume
  # The share name is not the same as its id
  - resource_type: ceph.smb.share
    cluster_id: tango
    share_id: sp1
    name: "Staff Pics"
    cephfs:
      volume: cephfs
      path: /pics
      subvolumegroup: smb1
      subvolume: staff
      qos:
        read_iops_limit: 100
        write_iops_limit: 50
        read_bw_limit: "10MiB"
        write_bw_limit: "5MiB"
        read_burst_mult: 20
        write_burst_mult: 15

Save this text to a YAML file named resources.yaml and make it available on a cluster admin host. Then run:

ceph smb apply -i resources.yaml

The command will print a summary of the changes made and begin to automatically deploy the needed resources. See Accessing Shares for more information about how to test this example deployment.

Later, if these resources are no longer needed they can be cleaned up in one action with a new file removed.yaml containing:

resources:
  - resource_type: ceph.smb.cluster
    cluster_id: tango
    intent: removed
  - resource_type: ceph.smb.join.auth
    auth_id: join1-admin
    intent: removed
  - resource_type: ceph.smb.share
    cluster_id: tango
    share_id: cache
    intent: removed
  - resource_type: ceph.smb.share
    cluster_id: tango
    share_id: sp1
    intent: removed

By issuing the command:

ceph smb apply -i removed.yaml

SMB Cluster Management

The smb module will automatically deploy logical clusters on hosts using cephadm orchestration. This orchestration is automatically triggered when a cluster has been configured for at least one share. The placement field of the cluster resource is passed onto the orchestration layer and is used to determine on what nodes of the Ceph cluster Samba containers will be run.

At this time Samba services can only listen on port 445. Due to this restriction only one Samba server, as part of one cluster, may run on a single Ceph node at a time. Ensure that the placement specs on each cluster do not overlap.

The smb clusters are fully isolated from each other. This means that, as long as you have sufficient resources in your Ceph cluster, you can run multiple independent clusters that may or may not join the same AD domains/forests. However you should not share a directory with multiple different clusters that may have different authentication modes and/or identity mapping schemes.

Note

Future versions of the smb module may programatically attempt to prevent such conditions.

Accessing Shares

Once a cluster and it’s component Samba containers have been deployed and the shares have been configured clients may connect to the servers. Microsoft Windows systems have SMB support built in and using Windows Explorer a share can be specified like so: \\<hostname>\<sharename>. For example: \\ceph0.mycluster.sink.test\Staff Pics. The Windows node should automatically attempt to log into the share. If the cluster and Windows client are both configured for the same AD Domain then a password-less single sign-on login will automatically be performed. If the cluster is configured for user auth, a username and password prompt should appear. Enter one user name and password combination that was specified in the cluster and/or ceph.smb.usersgroups resource.

MacOS X systems and many Linux based systems also support connecting to SMB shares. Consult the documentation for those Operating Systems and Distributions for how to connect to SMB shares.

A Ceph cluster operator wanting to quickly test a share is functioning may want to install smbclient or use the Samba Client Container image available from the samba-container project with the image quay.io/samba.org/samba-client:latest. On a client or within the container run smbclient -U <username> //<hostname>/<sharename> and enter the password at the prompt. Refer to the smbclient documentation for more details.

Brought to you by the Ceph Foundation

The Ceph Documentation is a community resource funded and hosted by the non-profit Ceph Foundation. If you would like to support this and our other efforts, please consider joining now.

File System Shares Over SMB

Management Commands - Imperative Style

Cluster Commands

Create Cluster

Examples

Remove Cluster

List Clusters

Share Commands

Create Share

Examples

Update Share QoS

Burst Behavior

Remove Share

List Shares

Management Commands - Declarative Style

Composing Resource Specifications

Cluster Resource

Share Resource

Join-Auth Resource

Users-and-Groups Resource

TLS Credential Resource

External Ceph Cluster Resource

A Declarative Configuration Example

SMB Cluster Management

Accessing Shares