Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.shiftlabs.dev/llms.txt

Use this file to discover all available pages before exploring further.

The Services & Daemons page provides visibility into all Ceph services running in your cluster and allows you to manage individual daemon processes. Each service type (monitors, managers, OSDs, etc.) groups related daemons that you can start, stop, restart, or redeploy.

​
Key Concepts

Service

A logical grouping of daemons of the same type (e.g., all monitors form the mon service).

Daemon

An individual Ceph process running on a specific host (e.g., mon.node1, osd.5).

Service Health

Determined by how many daemons are running: Healthy (all), Degraded (some), Down (none).

Redeploy

Recreates a daemon container with a fresh configuration, useful for troubleshooting.

​
Required Permissions

ActionPermission
View Servicesiam:project:infrastructure:ceph:read
View Daemonsiam:project:infrastructure:ceph:read
Start Daemoniam:project:infrastructure:ceph:execute
Stop Daemoniam:project:infrastructure:ceph:execute
Restart Daemoniam:project:infrastructure:ceph:execute
Redeploy Daemoniam:project:infrastructure:ceph:execute

​
Service Types

TypeIconDescription
monDatabaseMonitor daemons - maintain cluster maps and quorum
mgrActivityManager daemons - provide monitoring and orchestration
osdHard DriveObject Storage Daemons - handle data storage on disks
mdsLayersMetadata Server - manages CephFS metadata
rgwBoxRADOS Gateway - provides S3/Swift object storage API
crashCircleCrash collector - gathers daemon crash reports
node-exporterActivityPrometheus node metrics exporter
prometheusActivityMetrics collection and alerting
grafanaActivityMetrics visualization dashboards
alertmanagerActivityAlert routing and notification

​
Service Status

StatusIndicatorDescription
HealthyGreenAll daemons are running
DegradedAmberSome daemons are running, some are stopped
DownRedNo daemons are running

​
How to View Services

1

Select Cluster

Choose a Ceph cluster from the cluster dropdown. Only ready (bootstrapped) clusters are available.
2

View Statistics

Review the summary cards showing:
  • Services: Total number of service types
  • Daemons: Total daemon processes
  • Running: Number of healthy daemons
  • Stopped: Number of stopped daemons
  • Types: Distinct service types deployed
  • Hosts: Number of hosts running daemons
3

Filter Services

Use filters to narrow down the view:
  • Type Filter: Show specific service types (mon, mgr, osd, etc.)
  • Status Filter: Show healthy, degraded, or stopped services
4

Expand Service Details

Click on a service row to expand it and view all individual daemons in that service.

​
How to View Daemon Details

1

Expand a Service

Click on any service row to expand it and reveal the daemon table.
2

Review Daemon Information

The daemon table shows:
  • Daemon: Full daemon name (e.g., osd.5)
  • Host: Server where the daemon runs
  • Status: Running or Stopped
  • Version: Ceph version of the daemon
  • Last Refresh: When the daemon status was last updated

​
How to Start a Daemon

1

Find the Daemon

Expand the service and locate the stopped daemon you want to start.
2

Open Actions Menu

Click the actions menu (three dots) on the daemon row.
3

Click Start

Select Start from the menu. The daemon will begin starting up.
4

Verify Status

The page refreshes automatically. Verify the daemon shows as Running.
Starting a daemon may take a few moments as the container needs to initialize and connect to the cluster.

​
How to Stop a Daemon

1

Find the Daemon

Expand the service and locate the running daemon you want to stop.
2

Open Actions Menu

Click the actions menu on the daemon row.
3

Click Stop

Select Stop from the menu.
4

Verify Status

The daemon status changes to Stopped. The service status may change to Degraded if other daemons are still running.
Stopping daemons can affect cluster availability. Ensure the cluster has sufficient redundancy before stopping critical daemons like monitors or OSDs.

​
How to Restart a Daemon

Restarting is useful for applying configuration changes or recovering from transient issues.
1

Find the Daemon

Expand the service and locate the daemon to restart.
2

Open Actions Menu

Click the actions menu on the daemon row.
3

Click Restart

Select Restart from the menu. The daemon will stop and start again.
4

Monitor Recovery

The daemon briefly shows as stopped, then returns to running state.
Restart is preferred over stop+start for configuration changes, as it’s a single atomic operation that minimizes downtime.

​
How to Redeploy a Daemon

Redeploying recreates the daemon container entirely, which can resolve issues with corrupted containers or apply new container images.
1

Find the Daemon

Expand the service and locate the daemon to redeploy.
2

Open Actions Menu

Click the actions menu on the daemon row.
3

Click Redeploy

Select Redeploy from the menu.
4

Wait for Completion

The orchestrator removes the old container and creates a new one. This takes longer than a simple restart.
Redeploy removes and recreates the daemon container. Use this for troubleshooting persistent issues, not for routine restarts.

​
Statistics Cards

​
Services

The total number of distinct service types in the cluster (mon, mgr, osd, etc.).

​
Daemons

The total count of all daemon processes across all services and hosts.

​
Running

Number of daemons currently in running state and serving the cluster.

​
Stopped

Number of daemons that are not running. Non-zero values may indicate issues.

​
Types

Count of unique service types deployed (same as Services count).

​
Hosts

Number of distinct hosts running at least one daemon.

​
Troubleshooting

  • Check the host is online and accessible
  • Verify there’s sufficient disk space for logs
  • Check for port conflicts on the host
  • Review daemon logs on the host: journalctl -u ceph-<type>@<id>
  • Ensure the container image is available
  • One or more daemons are stopped
  • Expand the service to identify which daemons are down
  • Try starting the stopped daemons
  • Check host availability if start fails
  • The cluster may not be properly bootstrapped
  • Check network connectivity to cluster nodes
  • Verify the orchestrator is running
  • Check cluster health on the Clusters page
  • The underlying host may be having issues
  • Check disk space and memory on the host
  • Try a redeploy instead of restart
  • Check systemd service status on the host
  • Container image may be downloading
  • Network issues between nodes
  • Large OSD daemons take longer to initialize
  • Check orchestrator logs for progress
  • Normal during a cluster upgrade
  • Use the Upgrade page to complete version alignment
  • Consider pausing workloads during upgrade

​
FAQ

Restart: Stops and starts the daemon process. Quick and preserves the container.Redeploy: Removes the container entirely and creates a new one from scratch. Slower but resolves container-level issues.Use restart for routine maintenance; use redeploy for troubleshooting.
Currently, daemon actions are performed individually. This is intentional to prevent accidental service outages. For maintenance, consider using node maintenance mode instead.
Some services may be defined but not yet deployed:
  • osd: No disks have been added as OSDs
  • mds: CephFS is not configured
  • rgw: Object gateway is not deployed
Use the appropriate page to deploy these services.
Consider these factors:
  • mon: Need majority for quorum (e.g., 2 of 3)
  • mgr: Need at least 1 active manager
  • osd: Data should be replicated to other OSDs
  • mds: Standby MDS can take over
Check cluster health before stopping critical daemons.
This typically occurs during or after an upgrade:
  • Daemons upgrade one at a time
  • Some daemons may not have restarted yet
  • Manual intervention may be needed for stuck daemons
Complete the upgrade process to align versions.
These are auxiliary services that support cluster monitoring:
  • crash: Collects crash dumps from failed daemons
  • node-exporter: Exports host metrics to Prometheus
  • prometheus: Stores and queries metrics
  • grafana: Visualizes metrics dashboards
  • alertmanager: Handles alert routing
They don’t affect core storage but help with observability.
The page shows a “Last Refresh” timestamp for each daemon. Status is updated:
  • When you load or refresh the page
  • After performing daemon actions
  • The orchestrator periodically checks daemon health
Use the refresh button for the latest status.
Currently, daemon logs are not viewable directly from this page. To view logs:
  • SSH to the host running the daemon
  • Use journalctl -u ceph-<type>@<id> for systemd logs
  • Check /var/log/ceph/ for Ceph-specific logs
  • Use the Ceph Dashboard for centralized log viewing