Upgrading Ceph Reef to Squid in Proxmox VE: A Comprehensive Guide
For administrators running a Proxmox VE hyper-converged infrastructure, maintaining the health and security of the underlying Ceph cluster is paramount. With the Ceph Reef release approaching its end-of-life, the upgrade to Ceph Squid is not just a recommendation—it is an essential step to ensure continued stability, security, and access to the latest features.
This guide provides a clear, step-by-step process for safely upgrading your Proxmox Ceph cluster from Reef to Squid, minimizing downtime and protecting your data integrity.
Before You Begin: Essential Pre-Upgrade Checks
A successful upgrade begins with thorough preparation. Rushing this process can lead to cluster instability or data inaccessibility. Before starting the upgrade, you must complete the following checks:
Full System Backup: This cannot be overstated. Ensure you have a complete, verified backup of all critical data stored on your Ceph cluster and your Proxmox VE configurations. While the upgrade process is generally safe, having a reliable recovery point is a non-negotiable insurance policy.
Cluster Health Verification: Your Ceph cluster must be in a perfectly healthy state before you proceed. Run the following command on a monitor node and confirm the output:
ceph statusThe cluster state should report
HEALTH_OK. If it showsHEALTH_WARNorHEALTH_ERR, you must resolve all underlying issues before attempting an upgrade. Do not proceed until the cluster is fully healthy.Proxmox VE Updates: Ensure all nodes in your Proxmox cluster are fully updated. Running the latest stable version of Proxmox VE helps guarantee compatibility with the new Ceph packages.
Read the Release Notes: Familiarize yourself with the official Ceph Squid release notes. They contain important information about deprecated features, configuration changes, and potential upgrade-related issues.
The Ceph Reef to Squid Upgrade Process
The Ceph upgrade must follow a specific order to maintain quorum and cluster availability. The general sequence is: Monitors (MONs), then Managers (MGRs), and finally Object Storage Daemons (OSDs).
Step 1: Set the noout Flag
To prevent Ceph from rebalancing data while OSDs are temporarily offline during their individual upgrades, you must first set the noout flag. This flag tells the cluster not to mark OSDs as “out” if they fail to report in.
ceph osd set noout
This is a critical safety measure that prevents unnecessary data migration and potential performance degradation during the upgrade.
Step 2: Update Ceph Repositories
On each Proxmox node, you need to change the Ceph repository from Reef to Squid. This is typically done through the Proxmox VE web interface under Datacenter -> [Node] -> Updates -> Repositories or by editing the Ceph repository file in /etc/apt/sources.list.d/.
Change the target repository to point to the Squid release channel. After updating the repository configuration, refresh your package lists on every node:
apt update
Step 3: Upgrade the Monitors (MONs)
Upgrade the Ceph monitor packages one node at a time. This ensures that the monitor quorum is always maintained.
For each monitor node:
- Upgrade the Ceph packages:
bash
apt dist-upgrade
- Restart the monitor daemon:
bash
systemctl restart ceph-mon.target
- Wait for the monitor to rejoin the quorum and verify cluster health with
ceph statusbefore moving to the next monitor node.
Step 4: Upgrade the Managers (MGRs)
After all monitors have been successfully upgraded, proceed with the manager daemons. The process is identical: upgrade the packages and restart the service on each manager node, one by one.
- Upgrade the Ceph packages:
bash
apt dist-upgrade
- Restart the manager daemon:
bash
systemctl restart ceph-mgr.target
Step 5: Upgrade the Object Storage Daemons (OSDs)
This is the most time-intensive part of the upgrade. Upgrade OSDs one Proxmox node at a time to ensure data redundancy is never compromised.
For each OSD host:
- Upgrade the Ceph packages:
bash
apt dist-upgrade
- Restart all OSD daemons on that node. Proxmox typically manages this, but you can trigger it manually if needed:
bash
systemctl restart ceph-osd.target
- After upgrading all packages and restarting services on a node, monitor the cluster status closely. Wait for all placement groups (PGs) to return to an
active+cleanstate before moving to the next node.
Step 6: Final Verification
Once all components (MONs, MGRs, and OSDs) have been upgraded across all nodes, verify that all daemons are running the new Squid version:
ceph versions
The output should show that all daemons have been migrated to the new version. Also, run ceph status again to confirm the cluster is HEALTH_OK.
Step 7: Unset the noout Flag
With the upgrade complete and the cluster stable, you can now unset the noout flag to allow Ceph to resume its normal rebalancing and recovery operations.
ceph osd unset noout
Post-Upgrade Actions
After the upgrade, it’s a good practice to finalize the process by enforcing the new release features. This command prevents older, pre-Squid daemons from joining the cluster and enables features specific to the Squid release.
ceph osd require-osd-release squid
Monitor your cluster’s performance and health over the next few days to ensure everything is operating as expected.
By following this structured and methodical approach, you can perform a smooth and safe upgrade from Ceph Reef to Squid, ensuring your Proxmox VE environment remains robust, secure, and ready for the future.
Source: https://nolabnoparty.com/proxmox-aggiornare-ceph-reef-a-squid/
