Restoring to Replicated Storage
#
Steps to Restore from cStor Backup to Replicated Storage for Distributed DBs (Cassandra)Cassandra is a popular NoSQL database used for handling large amounts of data with high availability and scalability. In Kubernetes environments, managing and restoring Cassandra backups efficiently is crucial. In this article, we will walk you through the process of restoring a Cassandra database in a Kubernetes cluster using Velero, and we will change the storage class to Replicated Storage (a.k.a Replicated Engine or Mayastor) for improved performance.
info
Before you begin, make sure you have the following:
- Access to a Kubernetes cluster with Velero installed.
- A backup of your Cassandra database created using Velero.
- Replicated Storage configured in your Kubernetes environment.
#
Step 1: Set Up Kubernetes Credentials and Install VeleroSet up your Kubernetes cluster credentials for the target cluster where you want to restore your Cassandra database. Use the same values for the BUCKET-NAME and SECRET-FILENAME placeholders that you used during the initial Velero installation. This ensures that Velero has the correct credentials to access the previously saved backups. Use the gcloud command if you are using Google Kubernetes Engine (GKE) as shown below:
Install Velero with the necessary plugins, specifying your backup bucket, secret file, and uploader type. Make sure to replace the placeholders with your specific values:
#
Step 2: Verify Backup Availability and Check BackupStorageLocation StatusConfirm that your Cassandra backup is available in Velero. This step ensures that there are no credentials or bucket mismatches:
Check the status of the BackupStorageLocation to ensure it is available:
#
Step 3: Create a Restore RequestCreate a Velero restore request for your Cassandra backup:
#
Step 4: Monitor Restore ProgressMonitor the progress of the restore operation using the below commands:
Velero initiates the restore process by creating an initialization container within the application pod. This container is responsible for restoring the volumes from the backup. As the restore operation proceeds, you can track its status, which typically transitions from in progress to Completed.
In this scenario, the storage class for the PVCs remains as cstor-csi-disk
since these PVCs were originally imported from a cStor volume.
note
Your storage class was originally set to cstor-csi-disk
because you imported this PVC from a cStor volume, the status might temporarily stay as In Progress and your PVC will be in Pending status.
Inspect the status of the PVCs in the cassandra namespace:
#
Step 5: Back Up PVC YAMLCreate a backup of the Persistent Volume Claims (PVCs) and then modify their storage class to mayastor-single-replica
.
note
The statefulset for Cassandra will still have the cstor-csi-disk
storage class at this point. This will be addressed in the further steps.
Edit the PVC YAML to change the storage class to mayastor-single-replica
. You can use the provided example YAML snippet and apply it to your PVCs.
#
Step 6: Delete and Recreate PVCsDelete the pending PVCs and apply the modified PVC YAML to recreate them with the new storage class:
#
Step 7: Observe Velero Init Container and Confirm RestoreObserve the Velero init container as it restores the volumes for the Cassandra pods. This process ensures that your data is correctly recovered.
Run this command to check the restore status:
Run this command to check if all the pods are running:
#
Step 8: Verify Cassandra Data and StatefulSet#
Access a Cassandra Pod using cqlsh and Check the Data- You can use the following command to access the Cassandra pods. This command establishes a connection to the Cassandra database running on pod
cassandra-1
:
- The query results should display the data you backed up from cStor. In your output, youare expecting to see the data you backed up.
- After verifying the data, you can exit the Cassandra shell by typing
exit
.
#
Modify your Cassandra StatefulSet YAML to use the Replicated Storage-Single-Replica Storage Class- Before making changes to the Cassandra StatefulSet YAML, create a backup to preserve the existing configuration by running the following command:
- You can modify the Cassandra StatefulSet YAML to change the storage class to
mayastor-single-replica
. Here is the updated YAML:
- Apply the modified YAML to make the changes take effect:
#
Delete the Cassandra StatefulSet with the --cascade=orphan FlagDelete the Cassandra StatefulSet while keeping the pods running without controller management:
#
Recreate the Cassandra StatefulSet using the Updated YAML- Use the kubectl apply command to apply the modified StatefulSet YAML configuration file, ensuring you are in the correct namespace where your Cassandra deployment resides. Replace <path_to_your_yaml> with the actual path to your YAML file.
- To check the status of the newly created StatefulSet, run:
- To confirm that the pods are running and managed by the controller, run: