Putting A Price On The New DBA

Issue Date: 01-11-97
Issue Number: 8.06
Category: INDUSTRY

DBAs have long been associated with running the mechanics of database operation. That was already a full time job, before the complexities of client-server architectures and data warehousing were involved. In this report we look at the changing face of database administration.

As the central player in database operation, the database administrator (DBA) is clearly becoming a more strategic player in the destiny of the enterprise. The DBA is the core technical player, who interacts, not only with application developers, but network and systems administrators, and in many cases, business analysts.

Traditionally, the DBA's job has been to run the database engine. Yet the mechanics of this sharply differ, depending on whether the database is mainframe or distributed client-server, relational or hierarchical, transactional or data warehouse. Therefore, the question of who runs the database grows more complex. As databases are developed, not simply to process routine transactions, but to add value to the business, the logical design of the database becomes just as important as the physical. The issue also arises with data warehousing, which operates under different rules than transaction databases. Does the DBA offer added value here?

For many IT organizations, the hottest staffing issue boils down to this: is the DBA simply a mechanic or a strategic cog in the IT infrastructure? In this report, we surveyed ten organizations active in client-server and data warehouse development to find out if the DBA is still running the show, and if not, do the people supplementing the DBA cost more.

NEW FACTORS IN DATABASE DESIGN

As relational databases scaled from workgroup to enterprise, and with the emergence of rapid-fire client-server applications, optimization of physical and logical database architectures became a balancing act. The question became, could a traditional DBA - concerned with normalization to conserve DASD and reduce database size, performance monitoring, backups, restores, and system recoveries - adequately satisfy content-related needs of users from different business units? Should this person figure out which data belongs in which tables and whether specific indexes are needed?

Further, the types of data stores necessary to support data warehouses are as different from relational databases as relational databases were distinguished from their hierarchical or flat file predecessors. In a data warehouse, the goal is providing multiple business views which are often overlapping. For instance, a sales analyst might first examine sales by product and then by region, and then drill down to product sales per account manager by quarter in each region.

To obtain such flexibility, it is often necessary to run multiple copies of high-use data, a scheme which harkens back to the days before Codd and Date. These summaries or aggregations are because it would be difficult to piece them together on the fly using conventional SQL statements.

Therefore, unlike transactional databases, which feature high volumes of relatively simple data that is usually highly normalized, data warehouses usually house complex data types that are often denormalized to make the information more accessible.

Furthermore, data warehouse usage patterns are just the opposite of their transactional counterparts. While transactional databases are used routinely, featuring large, predictable volumes of relatively simple inserts, updates, and queries, data warehouses usually are not updated except in batches, and queries are less predictable and often more complex. Furthermore, unlike the transaction database, which runs the business, data warehouses are used only when business needs dictate, a situation that changes rapidly. 'Business processes do not have long life cycles anymore,' noted Mike Tremblay, CIO for Andersen, a midwestern manufacturer.

Just ask MCI. Maintaining an enterprise operational data store (a staging area for data warehouses and data marts, which receives transactional data), MCI encourages its business units to set up data marts on an ad hoc basis. The life of a typical MCI data mart is measured in months because of the turbulence of the telecommunications business - and this was before Worldcom or GTE entered MCI's (and BT's) picture.

Therefore, compared to transaction databases, data warehousing adds some administration issues and removes others. The range of reports and data may vary more in a data warehousing environment; for the DBA, that means placing additional access controls based on query data sample size and time of day. Conversely, with some tools such as RedBrick, creating indexes is simpler and more easily automated than with transaction databases.

The question becomes - who runs the data warehouse? DBAs must unlearn virtually all the 'normal' rules of running a database. Not surprisingly, in our research, we found that DBAs usually maintained an arms-length distance from data warehouses, unless the project was enterprise scale. If data had to be migrated from a legacy system, a DBA was required to schedule the data migrations. But in most cases, they had tangential roles - if any - in running data warehouses.

According to Ken Rudin, managing partner for Emergent, a California-based data warehouse consulting firm, it requires a team approach, where responsibilities would be divided up as follows:

* DBAs. DBAs handle data movement, while systems administrators are responsible for the Unix systems functions that accompany them. (In fact, the systems administrator would also be required for platform operations and maintenance with transactional databases as well.)

* Data conversion analysts. This is in effect a content-oriented job. Someone must 'own' data quality and define how data is transformed (aggregated) for decision support. This position requires knowledge of business processes, calling for skills similar to those of a business analyst.

* Front end (application) manager. The person who designs the front-end application.

* Security manager. Because they provide business intelligence, data warehouses theoretically contain competitive data that should not fall into the wrong hands. Enterprise data warehouses should piggyback on security measures maintained for transactional databases; the person charged with security (usually the DBA) should maintain access control policies for data warehouses as well. For data marts, this may be a challenge since, as workgroup applications, security for them is often treated casually. In the long run, Rudin believes that access control issues, both for transactional and decision support databases, should be managed by a specialist.

EMERGING TRENDS IN PRACTICE

Andersen: Data models reflect the business today, and are expected to fluctuate rapidly. By implication, that raises the importance of the data model as competitive strategy. Therefore, noted Tremblay, CIO, while physical functions such as database tuning remain important, they do not dominate the agenda when designing applications anymore.

For data warehouses, Tremblay envisions development of a new position, 'information mapper', that resembles the data administrator (DA), a new breed of professional who specializes in sitting with the end-user community, then translating application requirements into data models. Information mappers are designers or architects who understand business requirements and translate them into denormalized data models. Tremblay believes that DBAs, who were trained in techniques for normalizing data, would have a difficult time with this role. Information mapper positions do not yet exist at Andersen; the most likely source for them would be software engineers who understand design issues.

Blue Cross and Blue Shield Association: A special data warehouse project team, including three DBAs is building and operating the largest federal employee medical insurance program - a bet-the-enterprise, terabyte-scale data warehouse using RedBrick. A team, which also includes a systems analysts, is jointly responsible for creating data models, sizing the database, creating the RedBrick star schema indexes, performance tuning, and interface with a Brio front-end query environment.

Compared to their former experience with mainframe transaction systems, DBAs on the data warehouse project spend more time with users; due to RedBrick's automated indexing, they spend less time building indexes. Finally, because there are no constant inserts, there is no need to constantly manage and juggle factors such as tablespace allocations. In the latter case, allocations can be planned for, since they occur in conjunction with batch data feeds that are scheduled at preset intervals, and whose sizes are already known.

Data modeling included a mix of DBAs, analysts, and users in JAD (joint application development) sessions; significantly, the DBA assumed the lead. The rationale was, since the data warehouse group was a relatively small team, it didn't make sense to retain separate DAs for the task. 'The less bodies, the less time it would take us to do the turnaround,' noted Kathy Puglise, project leader. The data model is expected to remain fairly stable for at least two or three years. Other tasks such as data transformations were the responsibility of application developers, while data quality was lead by business analysts.

Fingerhut Companies: A leading national direct marketer operates a combination of DB2 and Oracle transaction databases, along with a 300-GB plus data warehouse, which is expected to exceed several terabytes within the next 18-24 months. The data warehouse, currently residing on Oracle, will migrate to DB2 UDB (version 5 for Unix), running on a massively parallel IBM SP2 platform. (Several smaller data marts will remain on Oracle.)

DBAs perform most of the usual tasks - performance monitoring, table reorganizations, DASD allocations, backup and recovery, etc. DBAs also participate in the database logical design, do the database physical design, and review application programs to ensure they respect data integrity and make proper use of system resources. Senior developers (application designers) are responsible for developing data models to meet user requirements (a task performed by DAs in other organizations) and writing stored procedures (often the domain of DBAs elsewhere). However, although developers write stored procedures, DBAs prepare them on the database, monitor the testing, and implement them on the production database. 'It's very difficult to test stored procedures unless you know the database, especially because both DB2 and Oracle lack stored procedures debugging tools,' said Jesus Rodriguez, director of systems architecture.

Data warehouse projects are currently run by a team equally comprised of application developers, business analysts, and DBAs. After the database design is completed with the guidance of the DBAs, the running of data extractions draw minimal DBA involvement since they are automated using Prism (DBAs are involved only when initial specifications for migrations are being developed). There are no special payscales or job titles for developers who handle special tasks such as data modeling or data extraction; instead, these tasks are usually assigned to senior personnel, whose lengthier experience qualifies them for higher salaries.

Kaiser Permanente/Southeast Division: A Sybase environment that is migrating to Oracle for transaction systems, and Cognos OLAP and query tools for the data warehouse, uses three principal players: the DBA, the DA, and application developer. As would be expected, DAs perform the logical design, DBAs handle the physical aspects of running Oracle (but not Cognos). Meanwhile, developers are used, both for front-end and back-end applications and middleware. 'I am more interested in the back end,' notes Mark Broome, information resource manager, who oversees the group. 'We are very replication-oriented, and our main challenge is how do you get huge amounts of data from several databases from one point to another with near real-time interfaces?'

Developers have C and Unix backgrounds, and the ability to work with different environments such as MUMPS (an operating environment specialized to the healthcare industry). 'Data modelers must have development experience, not just theoretical design,' said Broome, because 'people with good development backgrounds know the value of data'. According to Broome, a DA is not a business analyst. Although both are user-focused, he believes that DAs are too rare - and expensive - to waste time in issue resolution. Business analysts, not DAs, are the ones who go out into the user community and assess needs and resolve conflicts. 'It makes sense to have the analyst prioritize the issues while the DA goes on to the next technical design issue,' he said.

As for DBAs, those are the people who are event-driven, and willing to wear a beeper, he said. They are involved in running transactional databases and data warehouses.

Knight Ridder Information Services: An organization which provides commercial reference services maintains a relatively modest size Oracle installation staffed by three DBAs and six Unix programmers, the organization relies heavily on stored procedures. (Applications running on Oracle support information retrieval operations on the company's mainframe.)

Initially, the approach to data modeling was to design Oracle tables to mimic the ISAM file structures on the mainframe. The result was a relational database which was highly denormalized. Growth in the core business revealed the scalability problems in this approach, and the need for data modeling expertise. DBAs in the organization are trying to learn this skill, noted Charles Dye, lead DBA. 'Maybe there are people specially trained to do data modeling. Maybe I've met one in my life,' noted Dye. Thanks to automated database administration tools, Dye spends only about 20% of his time solving performance problems. Most of his time is spent helping developers generate triggers that work in Oracle.

A Midwestern Manufacturer: A large Oracle site with roughly 300 production databases, DBAs are split into two groups: system DBAs, who handle classical DBA functions (managing and maintaining the physical database engine), and application DBAs, who deal with data modeling. According to Chris Foote, the senior DBA - who has had 11 years experience first with IMS and later with Oracle - the application DBAs are true DBAs, not DAs. Unlike DAs, who come from development backgrounds and focus strictly on logical design, application DBAs also must perform physical table maintenance tasks such as table generation and reorganizations. 'Data modeling cannot be performed in a vacuum,' he maintains, since both logical and physical design impact the performance of the database, and applications tied to them.

Foote adds that, during his job hunting days, he was always asked about his data modeling experience. As for data warehouses, DBAs advise data warehouse administrators (who come mostly from the development organization) on the best techniques to use for data extracts, such as deciding which ways to delimit a file that is being moved. However, DBAs do not run the data migrations, and do not get involved with operation or maintenance of the data warehouse.

National Association of Securities Dealers (NASD): A large Oracle shop which manages 150 databases, 30 of which are in production (the others are for QA and development), along with a few Informix instances, the DBA staff numbers nearly a dozen. 'We try to stay away from the logical side,' said Oscar Zavala, associate director of database administration. Table creation is left to the developers. 'There is constant pressure to expand our duties to the application side,' said Zavala, who maintains that his group is plenty busy as it is.

However, once the application, complete with database table structure, is turned over to production, DBAs are responsible for troubleshooting, even if the problem was due to a flaw in the data model. Zavala estimates that about half of his group's time is spent in that capacity.

MCI: An Informix parallel XPS-based, 3-terabyte enterprise operational data store, operates on a 112-node IBM SP2 platform (the overall platform has 128 nodes). The philosophy is that transactional data is poured into the data store, which in turn feed a series of data marts that are used for 99% of all decision support queries.

In fact, the Informix parallel database more closely resembles a transactional database than a data warehouse. Although it doesn't receive inserts (data is moved in batch on a scheduled basis from multiple sources), some of the data marts are mission-critical in nature and are initially staged in the Informix database before being migrated to target servers. Some of them are as large as 100 million rows, approximately 80 GB. Therefore, challenges of allocating disk and table space, which are more characteristic of transactional databases, occur several times each week. The bleeding edge nature of MCI's Informix parallel database has impacted the organization. When the company first implemented the system three years ago, no commercial tools were available to handle such a huge parallel client-server database. This led MCI to develop its own tools, which in turn raised the required skill levels for Unix developers. Consequently, there is only a 'fuzzy' distinction between DBAs and Unix programmers, according to Dave Johnson, senior manager of development for WarehouseMCI. Developers are required to have DBA skills because they develop management tools used by DBAs.

The resulting payscales have been skewed in that Unix developers are paid almost at DBA levels (in most organizations, DBA salaries are higher). An added factor: many of the developers are outside consultants. Johnson hopes the staff (and pay) situation will hopefully normalize once the organization adopts third-party management tools (it is currently evaluating tools from Ab Initio).

Midwestern Retailer: An organization currently migrating from a Cincom Supra database to a combination of DB2/MVS and Sybase was developing an enterprise information model from the ground up, application by application. Although the organization consisted of both logical database modelers/DAs and DBAs, the dominant mindset was that application developers could handle the data modeling.

After a recent reorganization, the role of the DA was significantly decreased and in some cases the application analyst has taken over the data modeling function as an extension of their duties with the DA as an order taker and a documentation specialist. There was no need for DAs, according to Shirley Menacher, data architecture program manager. 'The real problem was a lack of management buy-in for data modeling,' she said. She adds that data modeling specialists have been extremely hard to find. 'People tend to avoid this field because nobody really understands what data analysts do,' she said.

Menacher notes that the drawback of leaving data modeling to developers is that they take a very narrow view of the database. In effect, the enterprise perspective with the benefits of data reuse and reduced data duplication are lost, because they design tables for their application and nobody else's.

Signature Group: This midwest-based direct marketing organization, has a data mart using the Applix TM/1 OLAP database with an owner who is not a classic DBA. The database owner, or DBO, performs a cross section of tasks that resembles the DA, including knowledge of data modeling, front-end application development, and reporting tools.

Even if the DBO does not actually develop the reports, he or she must be familiar with the tool in order to know which data to denormalize in the database. According to Greg Shireman, director of financial and business information systems, the DBO is almost perpetually in design mode because users needs dictate frequent changes to the data mart.

DBAs are involved with the source database, which include three VSAM systems. The DBA runs all data migrations.

As for background, the DBO was originally a VSAM COBOL programmer who subsequently received training in relational databases and 4GLs. 'The DBA might know how to write a stored procedure but wouldn't know the business rules behind them,' said Shireman.

SO HOW MUCH DO THEY COST?

We found mixed results when comparing payscales of DBAs to application developers and DAs.

The general rule is that DBAs are the highest paid, and they are extremely hard to find. According to Meta Group's 1997 IT salary survey (which covers 14 US metro regions), DBAs are paid from $46,347 to $112,796, with a national median of around $71,000. By comparison, senior business application developers (the class that typically performs design) is paid on a range of $40,092 to $93,813 (national median: $61,000). Meta did not have any data for DAs. (Figures taken from the 1997 Report on the New IT Professional. Available from Meta Group on +1 800 945 META, or +1 203 973 6700.)

Our findings were consistent, although more conservative for organizations in midwestern cities, where payscales were lower. Oracle DBAs in the Pittsburgh area with barely a year or two of experience are receiving over $50,000; in a major midwestern metropolitan area, Oracle DBAs with at least four years experience command $65,000 to $75,000. Developers range from $35,000 (for entry level) and up, with DAs positioned in between (at the level of senior software engineer).

In another midwest metro area, payscales for DBAs and DAs were at rough parity, $80,000 to $85,000 for lead and $60,000 to $65,000 for staff level. In Silicon Valley, entry level DBA salaries began at $70,000. However, in some cases, DBAs were not at the top of the salary heap. At MCI, there was near parity between them and Unix developers owing to the special situation: the development of proprietary programs used by DBAs. However, once MCI transitions from proprietary to third-party tools, it expects to restore normal order. That should be feasible because most of those writing the database management tools are outside consultants.

At the other end of the spectrum, there may be demand for a new class of database professional - the database architect. Foote has lobbied to create such a position at his firm, because the choice of databases is also involving platform, application architecture, and even network decisions. In the Pittsburgh area, Foote has seen database architects commanding over $80,000.