Storing terabytes of database backups in Object Storage? Is the data accessed rarely or mostly saved for archive purposes? Now we have the opportunity to save some budget and use Archive Storage!
The general difference between data backup and data archiving is how long we need to retain the data and how frequent we are accessing it. Backup solutions are provided to quick access, recover from human error or hardware failure. Archiving provides long term storage and limited access. Oracle offers the Archive Storage option as a part of Object Storage technology.
Oracle Cloud Infrastructure (OCI) introduced an automatic database backup service as an option for customers using database services. Automatic backup functionality enables customers to move database backup management to Oracle. This is a very effective solution and provides data protection from the starting point of the database lifecycle. Just after it is created. Customers then have the option to disable or enable it whenever needed.
In fact, all backup which are manage by Oracle are stored in Object Store bucket managed by Oracle and customer has no access to this area. Also, retention window for databases backup are limited up to 60 or 90 days as of now. It is important to mention all automatic incremental backups are deleted when database instance is terminated, along with all its resources. Oracle also offer an on-demand full backup which can be started any time using console UI or API. This option enables to create a full backup which will remain in Object Store even after database is terminated.
So, what to do when you would like to create a full backup stored in your namespace? With full access and use the most cost-effective, secure solution?
OCI offer Oracle Archive Storage service. It provides long-term retention periods with lifecycle management rules as well as secure and cost-effective. Customers using a huge amount of Object Storage, with data rarely accessed, have a good opportunity to implement archiving for database backups and save budget on IT operations. It is worth to mention that archived data are offline and not accessible until a restore is signaled. This provides additional security for stored objects.
This blog post will explain how to do a database backup to Customer managed bucket when an automatic backup is enabled and archive them using lifecycle policy to Oracle Archive Storage.
Using Archive Storage will provide cost-savings, secure and long retention periods for all your database backups.
How does it work
Enable automatic backup for your database is ‘one click’ operation. Process is based on incremental backups running level 0 backup once a week and daily incremental level 1. It is running in fully automated way and manage by Oracle. In case we are looking to create a full backup for archive purpose we are talking about full backup in Customer Managed Bucket using Archive Storage. First of all, during preparation we need to create regular Object Store bucket for database as we are not able to write directly to Archive Storage. Next, disable current automatic backup temporarily to not overlap backup data with archiving process. When all configurations are ready, we need to create our full RMAN backup writing to Customer Managed Bucket. Finally, revert changes related to automatic backup to let it run for regular daily bases.
To properly manage our backups by RMAN and lifecycle polices in Archive Storage we need to agree backup tagging model which help us filter metadata. We will talk about that shortly.
On architecture diagram below we can see multiple databases configured to use automatic backups sending data to Oracle mange Buckets – represented by green lines. Orange lines show temporary backups redirecting to Object Storage buckets defined and managed by customer. Finally, there are lifecycle polices which move backup pieces to Archive Storage for cost effective archiving. Retention defined by lifecycle may differ and are defined during implementation period. In my case these are 1- and 5-year retention.
Using dbcli tool we can managed database backups and its configuration on DB System. On OCI platform we need oci-cli to manage and configure our buckets, enable and disable automatic backup. Follow documentation to get details how to install and configure OCI CLI. Details regarding managing, updating and usage of dbcli can be found on Oracle Database CLI Reference page. Oracle tools which are used in this post are fully supported by OCI services.
dbcli - it is already installed and configured on DB System (except Exadata Cloud Services)
oci cli - it is a command line tool to manage entities in Oracle Cloud Infrastructure
- Collect the following OCID from the OCI console
- Create bucket for your database backup on Oracle Object Store
Login to DB System host and run OCI commands below:
- Configure access to your bucket in Oracle Object Store using dbcli tool
Login to DB System host and run as root user:
To know more how to create auth token please check Managing Users and Credentials
- Prepare backup configuration:
At this moment we are ready to create full backup in customer manage bucket. Configurations are completed. Now we need to temporarily disable automatic backups and create long term database backup in Archive Storage
Running full backup
- Disable automatic backup using oci cli:
- Update database config with new configured long term backupconfig
- Create long term backup
Tagging is very important to be set in agreed format
This notation will help us to use lifecycle policy filters and in the future during restore process.
- Mark long term backup unavailable in database Control File to prevent RMAN search for that backup pieces on daily activities. Run RMAN as oracle user:
- Restore previous autobackup configuration
- Enable autobackup
Now we have full backup created in Customer managed bucket. Autobackup config has been reverted to original one and is enabled for next automatic cycles. Now, all we need to do is setup of lifecycle rules to move our data to Archive Storage and delete them after configured time period.
Create Archive Storage lifecycle rulesRetention polices for my specific use case have been defined for 1 and 5 years. After that time backup data should be automatically removed from Archive Storage to reduce occupied space and costs of storage. Using vi editor I am preparing JSON file listed in Appendix and provide it as an input to oci command.
After policy setup is configured you can see them in Console UI
Uploaded JSON file can be found
I have shown you how to use OCI services to take advantages of Archive Storage. Following that short post, you have an overview of how to save IT budget with proper storage management and keep backups secure. There are advantages of autobackup kept and additional options explored to enable users to get the most from OCI.
Use Object Storage for data to which you need fast, immediate, and frequent access. Data accessibility and performance justifies a higher price point to store data in the Object Storage. For more information, see Overview of Object Storage.
Use Archive Storage for data to which you seldom or rarely access, but that must be retained and preserved for long periods of time. The cost efficiency of the Archive Storage tier offsets the long lead time required to access the data. For more information, see Overview of Archive Storage.
Database Backup Cloud Service provide automatic backup for databases running on DB System (except Exadata Services – ExaCS ) For more information, see Backing Up a Database to Oracle Cloud Infrastructure Object Storage
Database Cloud Service – about DBaaS details on OCI, see Bare Metal and Virtual Machine DB Systems