Oracle Database backup using Archive Storage in OCI
January 10, 2020 | 7 minute readIntroduction
Storing terabytes of database backups in Object Storage? Is the data accessed rarely or mostly saved for archive purposes? Now we have the opportunity to use Archive Storage! This is cost-effective, long term, secure, policy managed storage.
The general difference between data backup and data archiving is how long we need to retain the data and how we need to access them. Archiving provides long term storage and limited access. Oracle offers the Archive Storage option as part of Oracle Cloud Infrastructure Object Storage service. Taking advantage of Archive Storage we can retain database backups for years without any additional actions and delete them automatically after their lifecycle retention policy applies. It is not only a cost-effective solutions, but we can also retain backups for regulatory and compliance purposes. In this blog, I will explain how to use Archive Storage for database backups on DB System in OCI
Oracle Cloud Infrastructure (OCI) introduced an automatic backup feature as an option for customers using database cloud services. This is a very effective solution and provides data protection from the start of the database lifecycle. Just after it is created.
In fact, automatic backups are stored in an Object Store bucket which is managed by Oracle and customers have no access to this area. The retention window for database backup is limited up to 60 or 90 days as of now (January 2020). Automated backups are stored in a Standard Object Storage bucket only. Using Archive Storage requires some manual adjustments, but enables us to create a full backup which will be stored in a Customer managed bucket with an archiving policy attached to it, remain after the database is terminated and provide full customer access whenever needed.
How does it work
Enabling automatic backup for your database is a ‘one-click’ operation. The process is based on incremental backups running a level 0 backup once a week and daily incremental level 1. It runs in a fully automated way and is managed by Oracle on your behalf. In cases where we want to create a full backup for archiving purposes we will take a full backup in the Customer Managed Bucket using Archive Storage. First of all, during the preparation phase, we need to create a regular Object Storage bucket for the database as we are not able to write directly to Archive Storage. Long running database backups may interfere with each other, which will lead to mixing the location of data backups, so next, disable the current automatic backup temporarily to avoid overlapping backup data with the archiving process. When all configurations are ready, we need to create our full RMAN backup writing to the Customer Managed Bucket. Finally, revert changes related to automatic backup to let it run on a regular daily basis.
To properly manage our backups by RMAN and lifecycle policies in Archive Storage we need to agree with the backup RMAN tagging model which helps us filter metadata. We will talk about that shortly.
Architecture
In the architecture diagram below we can see multiple databases configured to use automatic backups sending data to Oracle manged Buckets (1) Orange lines show temporary backups redirecting to Object Storage buckets defined and managed by the customer (2). Finally, there are lifecycle policies (3) these move backup pieces to Archive Storage for cost-effective archiving. Retention defined by the lifecycle may differ and are defined during the implementation period. In our example case, these are 1- and 5-year retention.
Creating an Archive Backup scenario
- Configuration:
- Collect the required OCID details for your tenancy
- Create a bucket using OCI-CLI
- Configure access to your bucket
- Prepare the backup configuration
- Execute the long term retention backup for archiving purpose:
- Disable automatic backup in order to avoid writing backup files to different buckets
- Update the database with long term configuration
- Create long term backup
- Mark backup unavailable in ControlFile (CF) or RMAN Catalog (CAT)
- Restore previous autobackup configuration
- Enable automatic backup
Configuration
Using the dbcli tool we can manage database backups and its configuration on the DB System. On the OCI platform we need oci-cli to manage and configure our buckets, enable and disable automatic backup. Follow the documentation to get details on how to install and configure OCI CLI. Details regarding managing, updating and usage of dbcli can be found on Oracle Database CLI Reference page. Oracle tools that are used in this post are fully supported by OCI services.
-
dbcli - Is already installed and configured on DB System (except Exadata Cloud Services)
-
oci cli - Is a command-line tool to manage entities in Oracle Cloud Infrastructure. OCI-CLI tool can be installed on DB VM or any other host, even a local PC to manage an OCI tenancy.
- Collect the following OCID from the OCI console
- User OCID
- Tenancy OCID
- Region name (note that it's not the OCID)
- Compartment OCID
- key_file and fingerprint
- Create a bucket for your database backup on the Oracle Object Store
Login to DB System host and run OCI commands below:
- Configure access to your bucket in Oracle Object Store using dbcli tool
Login to DB System host and run as root user:
For more details on how how to create en auth token please check Managing Users and Credentials
- Prepare backup configuration:
At this point we are ready to create a full backup in the customer managed bucket, as the configurations are completed. Now we need to temporarily disable automatic backups and create a long term database backup in Archive Storage
Execute long term backup
- Disable automatic backup using oci cli:
- Update database config with new configured long term backupconfig
- Create long term backup with RMAN tag
RMAN tagging is very important to be set in agreed format
<DBNAME_#retentionYears_DATE>
This notation will help us to use lifecycle policy filters and in the future during the restore process.
- Mark long term backup as unavailable in the database Control File to prevent RMAN from searching for that backup pieces.
During regular day to day activities backups done for long term should not be verifed or catalog as they could not be accessed. Run RMAN as oracle user:
- Restore previous autobackup configuration
- Enable autobackup
Now we have a full backup created in Customer managed bucket. Autobackup config has been reverted to the original one and is enabled for next automatic cycles. Now, all we need to do is a setup of lifecycle rules to move our data to Archive Storage and delete them after configured time period.
Create Archive Storage lifecycle rules
Retention polices for this specific use case have been defined for 1 and 5 years. After that time backup, data should be automatically removed from Archive Storage to reduce occupied space and the associated costs of using the storage. Using the vi editor I prepare JSON file listed below and use it as an input to the following oci command:
The uploaded JSON file can be found attached LTP_rman_dbname_archive.json
After policy setup is configured you can see them in Console UI for verification.
Summary
In the article above I have shown you how to use OCI services to take advantages of Archive Storage. Following this short post, you have a basic overview of how to save IT costs with a reliable data management that keeps backups secure. Here are some advantages of autobackup being kept and additional options you might consider to enable users to get the most from OCI.
Reference
-
Use Object Storage for data to which you need fast, immediate, and frequent access. Data accessibility and performance justifies a higher price point to store data in the Object Storage. For more information, see Overview of Object Storage.
-
Use Archive Storage for data to which you seldom or rarely access, but that must be retained and preserved for long periods of time. The cost efficiency of the Archive Storage tier offsets the long lead time required to access the data. For more information, see Overview of Archive Storage.
-
Database Backup Cloud Service provide automatic backup for databases running on DB System (except Exadata Services – ExaCS ) For more information, see Backing Up a Database to Oracle Cloud Infrastructure Object Storage
-
Database Cloud Service – about DBaaS details on OCI, see Bare Metal and Virtual Machine DB Systems
-
Authentication and Authorization, If you're new to policies, see Getting Started with Policies and Common Policies.
Tomasz Buchwald
A-Team - Cloud Solution Architects
Next Post