Avoid These Top 10 Mistakes When Migrating Databases to DBaaS- Part 1

Welcome to part one of this two-part series featuring the top 10 mistakes organizations make when migrating databases to DBaaS platforms. In addition to diving into these mistakes and how they happen, we’ll take a look at the impacts they may have on your environments. We’ll then provide recommendations based on our experience with cloud migrations so that you gain the insights you need to mitigate these common mistakes and have a successful cloud journey.

1. Underestimating cloud migration and ongoing costs

One of the biggest challenges shops will face with cloud conversions is that the costs often exceed initial budge estimations. DBaaS systems aren’t products; they are entirely new architectures. Like all new architectures, they will have a wide-ranging impact that affects all areas of IT and also how technology supports business operations.

DBaaS platforms will change how your organization designs, develops, accesses, administers, monitors and documents their database systems. The budget to convert a database to DBaaS cannot be finalized until a detailed evaluation of the database being migrated is performed. The analysis often identifies conversion issues that need to be overcome.

Some issues will be “easy fixes.” Others will be more challenging, and some will cause a significant impact on migration timelines and costs. Depending on the vendor and DBaaS product chosen, you may find that you will be tailoring your database configuration, tools/utilities, interfaces, SQL statements and procedural languages to work in that vendor’s environment. These customization activities take time, and in some cases, a lot of time. Funding for these migration activities needs to be included in the overall budget estimation. You will also need to factor in costs for training, documentation and organizational changes. We’ll learn more about budget implications throughout the course of this article.

During a CIO conference discussion RDX hosted, a CIO from a large organization that has a cloud-first strategy was asked how he budgeted for cloud migrations. His response was, “It’s simple – I just take whatever my team gives me and double or triple it.” Although it generated laughter from the audience, there were many head nods in agreement.

IMPACT:
Cost overruns, unhappy management team, taking shortcuts to reduce costs, poor implementations

RDX RECOMMENDATIONS:
Perform a robust evaluation of the new architecture‘s impact on your organization. Allocate human capital, create project plans and allocate funding to the migration process and ongoing DBaaS support. Then be prepared for the surprises that can occur when you migrate to any new architecture.

Consider hiring consultants who have a well-documented background in cloud architectures to provide you with advice. Work with a cloud database services provider like RDX. Don’t go it alone as mistakes are costly, and cloud migrations have high visibility in all organizations.

View RDX’s Insights Series webinar recording titled Cloud’s Hidden Impact on IT Support Organizations. There are several webinar recordings on RDX’s YouTube Channel that provide hints, tips and migration demos to help you convert on-premises databases to several DBaaS platforms.

2. Underestimating the organizational and procedural changes required to support DBaaS environments

DBaaS systems may require changes to your support team’s organizational infrastructure. DB and application architects play important roles in the selection, configuration and implementation of public cloud-based DBMS platforms. Existing staff will have new responsibilities, and new positions may need to be created to support the new architecture.

Because the environment is different than on-premises systems, you will have to update your change management processes and support documentation. How long that takes and the number of techs you will have to dedicate to the project depend on the architecture you choose and how stringent your change management processes and documentation requirements are.

Larger shops traditionally have more documentation and procedures in place than smaller organizations. In addition, organizations that are governed by internal, industry-specific or governmental regulatory compliances are required to have more documentation and standardized administrative procedures.

IMPACT:
Failed audits, not adhering to internal best practices and change management procedures, poor support, compliance issues, increased security vulnerabilities

RDX RECOMMENDATIONS:
Add organizational and procedural documentation evaluation steps to your cloud migration project plan. Consult with on-premises support, security and auditing teams to review current documentation and procedural requirements.

Here’s a starter set to jumpstart your evaluation: security, disaster recovery, change management, turnover approvals, monitoring, problem resolution, job scheduling, administrative best practices, repeatable processes, internal, industry specific, governmental regulatory compliance, naming conventions – [[add your additional required documentation here]]…

3. Not providing internal personnel with sufficient time and training to learn the new architecture

Your staff will need to be trained to learn the new DBaaS architecture. You will have to commit resources and then provide them with the time required to learn the new platform. Provisioning, administering, tuning, securing, monitoring, backup/recovery, disaster recovery. It’s the same basic principles of good database administration that apply to all DBMS environments, but your administrators will be using different tools, utilities and commands to perform those support operations.

IMPACT:
Migration issues, poor ongoing support, continued problems after migration

RDX RECOMMENDATIONS:
Don’t expect your staff to become cloud experts overnight. There will be a learning curve that varies according to vendor and DBaaS product. Build test DBaaS systems and provide sufficient time to allow your support teams to learn the new platform. Encourage staff to take vendor- specific training courses and pursue certifications. Hire personnel who have experience supporting cloud systems to act as advisors and mentors to fellow personnel. Leverage consultants and third-party service providers, like RDX, to quickly acquire additional expertise.

4. Not fully understanding the vendor’s costing models

Evaluating initial and ongoing costs is critical. You aren’t going to be purchasing your environment; you’ll be renting it. Those rental fees will vary based on vendor, DBaaS product, system configuration and workloads.

You can search the web and find numerous cases of shops that didn’t do their due diligence, created DBaaS systems and were surprised at the charges they were incurring. Each vendor has different ways of charging the customer. Although the costing criteria will vary according to each cloud vendor, here are some of the more common factors that affect pricing: CPUs and CPU processing power, memory and disk sizes, amount of IO generated, network bandwidth and disk subsystem performance guarantees, storage for backups that exceed vendor allocations, HA configurations and data transfers.

Understanding the impact data transfers have on monthly costs is often overlooked by many organizations. Most vendors will charge you for data transfers to the cloud system from the internet, out of the cloud system to the internet and transfers between availability zones and regions.

IMPACT:
Cost overruns, poor system performance

RDX RECOMMENDATIONS:
Take the time to fully understand the cloud vendor’s pricing models – they can be complex. Follow the advice in the RDX recommendations for problem #5 below.

5. Not sizing the cloud environment correctly

Measuring the on-premises database’s resource consumption is required to initially configure the cloud system’s performance tiers and estimate monthly rental fees. A big challenge for most shops is accurately measuring their on-premises DB system’s resource utilization. Technical personnel who are responsible for supporting the on-premises database will need to measure key resource consumption indicators including CPU, memory, disk storage, IO and data transfers into and out of the environment.

Performance and resource utilization monitoring becomes even more critical in the cloud. Your organization will be renting the DBaaS system, and the vendor will charge you based on resource consumption. Planning for database growth is an important component when forecasting future rental fees. Technical personnel will need to identify any changes (concurrent user increases, new application programs, new queries, data storage growth) that have the potential of increasing resource utilization. They will use this information to forecast the impact the change may have on current performance tiers and resource allocations.

We all know that it can only take a few improperly tuned queries added to the system to impact the resource utilization of any database environment – on-premises or the cloud. In addition to the traditional poor application performance and transaction time outs that can occur, sustained increases in resource utilization may result in your shop being forced to upgrade your performance tiers and configurations.

The process to select higher performance tiers and increase resource allocations is pretty straightforward in most cases. The key phrase in that last sentence is “most cases.” Vendors understand how important it is for customers to be able to scale their DBaaS resource allocations, but, depending on the vendor chosen, there may be an outage required to apply the change. Monthly costs will also increase.

IMPACT:
Cost overruns, poor system performance

RDX RECOMMENDATION:
The first step to accurately size your DBaaS instance and estimate your rental fees is to fully understand what metrics the provider uses to calculate costs. Work with your on-premises support teams to measure your current resource utilization and estimate future workloads. Take your time during instance configuration and leverage vendor utilities that provide cost estimates.

Most cloud vendors provide a cost calculator that you can use to estimate your monthly charges before you provision your DBaaS Instance. Use the cost calculator to perform “what if” scenarios. Select different payment options, regions, instance classes, performance tiers, disk allocations and HA configurations. Read the calculator’s help information to better understand the calculations and costs.

After you migrate, one of your top priorities, in addition to making sure everything is working, is to monitor resource consumption. This includes setting up resource consumption, monitoring and billing alerts. If you don’t, you may end up with a budgetary surprise or two.

Thanks for reading. Check back soon for mistakes 6-10!

DBaaS or IaaS? Database Cloud Comparison

Introduction

Technology leaders are being inundated with a flood of new cloud architectures, strategies and products – all guaranteed by vendors and various industry pundits to solve all of our database challenges. This seemingly endless array of public cloud based DBMS offerings can quickly become bewildering.  One of the top questions our customers have is whether they should choose DBaaS or IaaS as their preferred database cloud architecture.  This post is intended to peel back the veil of the 2 primary cloud based DBMS platforms by providing readers with our experiences with IaaS and DBaaS architectures.

One of the benefits of working for a remote DBA services provider is that our shop’s collective knowledge is not constrained by any one organization’s technology implementation.  We have customers that have technology strategies that range from “bleeding edge” to “yesterday’s technology tomorrow.”

We know what products work and which ones don’t, what tech stack combinations play well together and what database technologies and features provide the most benefits for a given business or technical need.  In addition, we are required to administer virtually every database feature you can think of for every product we support, and we work with dozens of cloud systems.  This provides us with a wealth of knowledge that  includes cloud strategies, technologies, architectures, product offerings and vendor-specific features.

 DBaaS and IaaS Defined

Let’s continue our discussion by learning more about the two primary cloud architectures – Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS).  Since we are talking about database management systems, let’s use the term Database-as-a-Service or DBaaS to define that architecture.

We all have experience with on-premises systems.  We have to build the server rooms,  provide heating, cooling, redundant connectivity and power.  We are required to purchase, install and administer all of the components to provide the safest environment we can for our systems.

We are then required to buy the server hardware, install it and maintain it.  When it breaks or we want to increase horsepower, we have to open the chassis and work on the server components. We add CPU, memory, disk – whatever we need.  To perform those activities, we either have to take an outage or make plans to shift the system and workloads to another server to ensure availability.  We also buy and administer the OS and DB software we need to run our database- driven applications.

In addition, we evaluate, buy, install and support all of the other products we need, which often includes monitoring, security, auditing and third-party reporting products.  Look at the two stars in the the graphic above.  We have to buy and support everything- both hardware and software.

Let’s move on to the cloud. Most IaaS and DBaaS environments are multi-tenant, which means we are sharing the vendor’s compute and storage architecture with other customers.  In addition, depending on the architecture and vendor chosen, the system will vary in degrees of scalability, elasticity, automated administrative services and self-service.

The architecture that is the closest to on-premises is Infrastructure-as- a-Service,   the architecture defined in the bottom center of our graphic.  With Infrastructure-a- a-Service (IaaS), the vendor provides the compute and storage infrastructure and may offer some level of system maintenance activities. Customers have direct access to the cloud system, which includes both compute and storage components.  Think of it as a server, more often than not, a virtual server in the cloud.

You don’t have to  build your server support environment that provides air conditioning, light, multiple power providers, UPS systems, generators and redundant connections to the internet.  All of those features are provided by the vendor, but IaaS customers will continue to maintain full ownership of their software stack’s administration, including the operating system and database.   Customers install and administer their software of choice on the Infrastructure-as-a-Service platform.

It’s important to note that depending on the IaaS provider and the offering chosen, customers are able to take advantage of the vendor’s features to reduce the time to support the environment. Microsoft Azure, for example, provides builds you can use to get a jumpstart on provisioning a new DB environment.  However, you will need to tailor that generic build to meet your needs.

Now that we know that IaaS is pretty much a server in the cloud, let’s move on to PaaS, or in our case, DBaaS – Database-as-a-Service. DBaaS vendors provide all of the server environmental benefits that their IaaS counterparts do.

DBaaS providers increase their level of control and responsibility by assuming ownership of the operating system and database software as well as the hardware. DBaaS customers perform little to no operating system and database software administration.  The vendors will be constantly enhancing their architecture’s automation capabilities to further reduce the human labor involved to administer their environments.

The DBaaS vendors also make modifications to their database software for two reasons:

#1-is to ensure their product will work in their shared cloud  environment
#2- to leverage the benefits that the cloud, and their architecture, inherently provides

Geographic data redundancy would be an example as  it allows customers to leverage the cloud to more easily create DR and HA systems.  It’s important to note that as we discuss IaaS and DBaaS architectures, there can be a lot of variations in the vendor offerings.

Comparing On-Premises, IaaS and DBaaS Architectures

Each environment, on-premises, IaaS and DBaaS, has strengths and weaknesses that are inherent to their architectures. I’ve provided this comparison information for the 3 DB environments shops can choose from – on-premises, the database running on IaaS and the DBaaS offering.  Each environment has pros and cons, benefits and drawbacks.

IaaS allows customers  to maintain tighter administrative control over their environment.  They can also more easily leverage  their  favorite internal third-party products on IaaS systems than they can with DBaaS.  IaaS is just a server that is provided to you over the cloud.

Many of the third-party tools that include monitoring, security, application development, auditing –  can be challenging to integrate into a DBaaS architecture because of the modifications the vendors make to their systems. All of the DBaaS vendors provide monitoring tools. Some vendors, like Amazon, charge extra if the customer wants a more robust monitoring solution than what is offered in their base package. In general, DBaaS monitoring tools aren’t as robust as their on-premises counterparts, but they are catching up very quickly.

If the DBaaS provider determines that its own internal underlying software, OS or database needs a critical availability, security or performance patch, shops may not have a choice on its implementation.  If that patch requires an outage, the customer will need to schedule that outage and oftentimes before a certain date.  DBaaS offerings allow customers to more easily configure complex architectures such as high availability and disaster recovery. All of the major DBaaS providers offer geo data redundancy.

Leveraging DBaaS Environments to Reduce DBA Labor Costs

Migrating to a DBaaS environment does reduce the amount of time a DBA spends administering the database environment, but it doesn’t reduce that administrative time to 0. Customers will experience the most significant time savings in OS administration and hardware support.

DBAs do spend time installing, patching and upgrading the DBMS software as well as tuning the environment and setting up and monitoring maintenance and backup utilities. Many of these administrative tasks can be provided by the DBaaS vendor depending on the vendor and offering selected.

The majority of a DBA’s  time is spent working within the database systems themselves. DBAs  build schemas, grant security, assist developers with SQL and procedural program tuning, provide advice, enforce business logic using database features, tune application workloads and debug issues. DBaaS vendors don’t provide these services as part of their basic package. There are literally dozens (and dozens) of administrative activities that DBAs are required to perform in DBaaS environments.

It’s important to remember that although the vendor may provide the mechanisms and processes to automate administrative activities, this also does not reduce DBA support responsibilities for them to 0. Personnel may need to configure how they are to be performed and when they are scheduled. All of the major DBaaS vendors provide patching, maintenance utilities and automated backup processes, but it is up to the DBA to review scheduling, configure custom schedules and monitor the execution for all automated tasks.

An increasingly competitive DBaaS marketplace forces all vendors to maximize their product’s inherent feature set. Constant innovation and integration of new features that differentiate their product from other vendors is an absolute requirement for their continued competitive survival.

During my tenure in the IT field, I’ve found the following equation to be true:

New Features
+ New Functionality
+ New Technologies
+ New Architectures
 + New Business Requirements
 = Increased IT Support Complexity

As the cloud vendors add features to their DBaaS offerings, the products will increase in complexity.  The features may automate some of the administrators’ support  activities but, at a global level,  DBAs with a high-level of technical knowledge will continue to be needed  to support  DBaaS environments.

Here’s a quick example. Most cloud vendors provide features that allow administrators to more easily configure HA and DR environments, but it still requires an extensive knowledge of these technologies and business requirements  to configure, implement, administer and monitor.   The activities may be performed much faster in DBaaS environments than on-premises systems, but that foundational base of knowledge is still required.

Well-trained administrators will determine what regions are best to house the failover systems, work with developers and business users to select a failover strategy that meets their combined business, technical and budgetary needs, monitor the actual failover process, identify the root cause of the failover to ensure it doesn’t recur and restore the system to its original configuration once the problem is corrected.

Impact on Existing Change Management Procedures and Documentation

Because cloud environments are different than on-premises systems, organizations leveraging these new architectures may have to update their existing  change management processes and documentation. How long that takes and the number of techs needed for the project, depends on, once again, the cloud architecture chosen and how stringent the organization’s change management processes and documentation requirements are. Because DBaaS is administered much more differently than on-premises, it will have a greater impact than IaaS, which is just a server on the cloud.

Additional documentation that may also need to be modified includes: security, disaster recovery, monitoring, problem resolution, job scheduling, administrative best practices, repeatable processes, internal, industry specific, governmental regulatory compliance, etc…

The amount of documentation changes required will depend upon the breadth and depth of documentation your particular organization requires as a best practice. Once again, this is a greater impact for DBaaS.

Impact on Existing Toolsets

Shops migrating to DBaaS environments will need to identify all of the build, administration, monitoring and access tools that they use to interact with their on-premises databases. All shops usually have a couple of “must have” tools that are frequently used.  Administrators will need to identify which of their organization’s existing toolsets will continue to work in the cloud – and which ones won’t. The popularity of the cloud is driving most software product vendors to make sure their offerings work with cloud systems, but it is not something that should be taken for granted.

RDX’s recommendation is to create a list and verify that the preferred  tools will continue to work with the cloud versions of the database.  The majority of the tools will work with IaaS (remember, it is just a server in the cloud), but for DBaaS, shops will need to perform the evaluation to determine if they can integrate and the level of effort needed to integrate them.

Comparing On-Premises DB Feaures to IaaS and DBaaS

We learned that IaaS allows us to install on-premises DB software in the cloud.   There’s not much analysis required to verify that all of the features we leverage in our on-premises databases are available in the IaaS cloud. Before shops migrate their favorite on-premises databases to DBaaS environments, they will need to identify the list of DB features that are not available in the cloud DBaaS offerings.

Let’s compare on-premises SQL Server to the 2 leading cloud vendors, Amazon and Microsoft, both of which offer SQL Server DBaaS. Amazon’s SQL Server DBaaS offering doesn’t support PolyBase, stretch database, backing up to blob storage and importing data into the msdb database. Also, administrators can’t rename a database if it is used in an Amazon Mirroring deployment, and it doesn’t allow customers to increase storage on a SQL Server database.  If the customer needs to increase the storage of a SQL Server DB Instance, they are required to back the database up, create a new DB instance with increased storageand then restore the databases into the new DB instance.

It doesn’t support SSIS, SSAS or SSRS, and it doesn’t have SQL Server server-level security roles for sysadmin, serveradmin, securityadmin, dbcreator and bulkadmin.  It also doesn’t support database mail, maintenance plans, distributed queries, log shipping, change data capture, SQL Server Audit or bulk insert.

How about Azure SQL DB vs SQL Server on-premises? These are two DB products offered by the same vendor. Microsoft’s Azure DBaaS offering doesn’t support attaching a database, backup and restore statements, change data capture, database mail, database mirroring, database snapshots, extended stored procedures, filestream, linked servers, log shipping, resource governor, the profiler, SSIS, SSAS or SSRS.

Now instead of database mirroring and log shipping, it does provide Active Geodata-Replication, which could be a better alternative for many customers – but its different! I’m not stating that these environments aren’t as effective as on-premises, but they have different features that we all need to be aware of.

Wrapup

Recent RDX customer surveys have shown that although most of our clients have defined their high-level cloud migration strategies, they are continuing to evaluate and compare IaaS and DBaaS platforms.  Some of our smaller and mid-sized customers have decided to go “all in” and migrate all of their systems to the cloud, but the majority of our customers has stated that they will execute a “best fit” strategy which includes implementing  new databases that use on-premises platforms as well as IaaS and DBaaS architectures.

In addition to choosing the most appropriate cloud architecture for a given database-driven application, there are a host of other considerations that must be evaluated when migrating to the cloud. Will you need to transfer data into and out of that cloud DB environment?  How much data? Does the database being migrated depend on data from other on-premises systems? How will you ensure that the database and data transfers are secure? What level of application changes are you comfortable with?  Are you permitted to have a downtime for the migration or is it required to be a “flip the switch” process? This is just a quick sampling.  An entire article could be written on all of the issues that must be considered and evaluated when migrating to cloud architectures.

RDX has successfully converted dozens of on-premises systems to the cloud and changed DB products along the way.  RDX offers a wide range of cloud DB services – from strategic analysis and architecture design to migration and ongoing support.  If you would like our assistance, please visit our Cloud Services page for more info.

 

 

Is Microsoft Azure Cosmos DB the NoSQL Competition Killer?

Most database pros are aware that the era of the one-size-fits-all database has been over for some time. We realized that not all of the data our shops are required to store, process and present to our end user communities neatly fits into relational rows and columns.

The need to store unstructured data in conjunction with the ability to provide almost absurdly high degrees of scalability, data distribution and availability were the business drivers behind the genesis of NoSQL offerings. Many of the NoSQL products were specifically designed to leverage low cost hardware to provide the ability to store semi and non-structured data and provide extremely high horizontal scalability and data redundancy at an affordable price point.

Continue reading Is Microsoft Azure Cosmos DB the NoSQL Competition Killer?

Automation’s Impact on IT Knowledge Workers

Process automation, because of its wide range of application, takes many forms. Manufacturing companies have been using industrial robots to replace activities traditionally performed by humans for some time. Business process automation shares the same goal: to replace business functions performed by humans with software applications. Work activities that are repetitive in nature and require little intelligent analysis and decision making to complete are prime candidates for process automation.

Continue reading Automation’s Impact on IT Knowledge Workers

RDX 2017 Top Database Trends- The Rising Interest in Open Source Database Offerings

In the final installment of our 2017 Database Trends analysis series, we evaluate the database community’s rising interest in open source database offerings.   We discuss the impact these upstart competitors will have on the industry heavyweights that include Oracle, Microsoft and IBM.

Continue reading RDX 2017 Top Database Trends- The Rising Interest in Open Source Database Offerings

RDX 2017 Top Database Trends- Multi-Model Databases & NoSQL Vendor Consolidation

In this third installment of our 2017 Database Trends analysis series, we focus our attention on the emergence of multi-model database management systems.  We examine the impact the larger database vendors will have on their smaller, NoSQL competitors.  In addition, we also discuss the premise that the vendor with the best technology doesn’t always win the battle for market share – or even survive as a viable competitor in some cases.

Continue reading RDX 2017 Top Database Trends- Multi-Model Databases & NoSQL Vendor Consolidation

RDX 2017 Top Database Trends- SQL Server on Linux

In this second article of our 2017 Database Trends series, we analyze Microsoft’s SQL Server on Linux offering.   We evaluate the impact the new offering will have on the database market arena and, more specifically, what the consequences will be for the Microsoft vs Oracle war for DBMS market dominance.

Continue reading RDX 2017 Top Database Trends- SQL Server on Linux

RDX 2017 Top Database Trends Series- Microsoft & Oracle Hybrid DBMS Clouds

The database market landscape no longer consists of a handful of traditional database vendor offerings. The database market arena has exploded with dozens of new database offerings and architectures from organizations that range the spectrum. Competition from super-sized, nontraditional database vendors, open source offerings and startups are feeding the rapid escalation of advancements in database technologies.

Continue reading RDX 2017 Top Database Trends Series- Microsoft & Oracle Hybrid DBMS Clouds

Who Will Win the Database Wars? Open Source Vs Commercial Database Systems

This is the final installment in the Who Will Win the Database Wars series which analyzes the most significant areas of database vendor competition. In addition to this comparison, the series also includes the following competitive assessments:

Continue reading Who Will Win the Database Wars? Open Source Vs Commercial Database Systems

Who Will Win the Database Wars? Relational vs NoSQL Database Models

This is the fourth installment in the Who Will Win the Database Wars series which analyzes the most significant areas of database vendor competition. In addition to this comparison, the series also includes the following competitive assessments:

Continue reading Who Will Win the Database Wars? Relational vs NoSQL Database Models