Do Enterprise Frameworks Pay?

Issue Date: December 1997
Issue Number: 8.07
Category: DSM

Distributed client-server networks require significant infrastructure coordination. Do enterprise distributed systems management (DSM) frameworks deliver value for their complexity? In part I of this new series, we examine the experiences of six CA Unicenter customers.

Unicenter and the Tivoli Management Environment have emerged as the two leading enterprise frameworks. Analysts expect that in 1998 the marketing battle for supremacy will dramatically escalate.

Yet, as the marketing war breaks out, it pays to remember the context. It took nearly 20 years for mainframe systems management tools to mature - are DSM frameworks (most of which are less than five years old) yet delivering value to customers?

As we mentioned in the adjoining report, Meta Group expects that most large organizations will embark on some form of DSM project, and experience bumps in the road.

Because of the huge impact that DSM has - and will have - we are beginning a multi-part series examining the return on investment (ROI) of distributed management technologies. Our initial focus is on enterprise frameworks.

This month, we kick off the analysis with a look at ROIs reported by Computer Associates (CA) Unicenter users. In coming months, we will study CA's rival - Tivoli, and later on will explore point solutions.

CA UNICENTER: PRODUCT OVERVIEW

Launched in 1991, CA's Unicenter originated as a systems management product with a number of core functional modules, including:

* Security/access control.
* Event monitoring and automation.
* Job scheduling.
* Performance management.
* Problem management.
* Backup.
* Storage media management.

The product has gradually added functionality in these areas and others. For instance, its security and access control features now allow single sign-on capabilities. With CA's 1995 acquisition of Legent, Unicenter added agent technology. The result is that, instead of installing full Unicenter clients, lightweight intelligent agents can be deployed on any device, including servers, desktops, network devices, and output devices. In essence, agents use Unicenter to penetrate beyond the server.

Probably the most hyped addition is the TNG ('The Next Generation') 3-D, virtual reality interface, which allows console operators to visually point and click their way onto any device on the network without having to know arcane commands. With the new interface, the product has been re-christened 'Unicenter TNG'.

When agents are combined with the TNG virtual reality interface, users can see the impact of database, server, and network bottlenecks. As agents are developed for enterprise applications (for instance, agents for SAP's R/3 have been developed), users can view events not as system occurrences, but business processes. For instance, if a database or application server is bogged down, TNG could inform the operator that processing of customer orders from yesterday are being delayed.

This feature alone could make Unicenter useful, not only to database administrators (DBAs) or Unix administrators, but business workgroup managers. In the long run, this could be a tool for determining the economic value of operations or the cost of downtime.

Other recent enhancements to Unicenter include:

* Network management functions. These overlap with capabilities delivered by products such as Cabletron's Spectrum.

* APIs for linking third-party tools. This is a response to critiques that Tivoli has superior support of best-of-breed tools (see later).

* Support of Desktop Management Task Force DMI interfaces. They support such features as pre-loading and waking up of PCs, remote control, software delivery and asset management. CA claims that this is a superset of Intel's Wired for Management initiative and Microsoft's Zero Administration Kit.

* A new framework of basic services (e.g. console display, messaging, etc.) which provide all the capabilities necessary for monitoring and automatically responding to events. The new framework, which was released in summer 1997, is being given away for free. This reflects CA's stance that the framework alone delivers little value without the functional modules. It is also an aggressive marketing ploy against Tivoli, which sells a rival framework that must then be integrated by users or consultants to third-party tools.

CA has always positioned Unicenter as an integrated, out-of-the-box solution. By comparison, it dismisses Tivoli as a toolkit which must be painstakingly pieced together.

CA argues that integrated solutions are far more effective than point solutions, or solutions that must be integrated by the customer. For instance, job scheduling is moot unless it coordinates with network management; otherwise, the best-laid job schedules are subject to delay if the network is bottlenecked.

Third-party integration has traditionally been Unicenter's question mark. Tivoli, by comparison, has always promoted itself as a best-of-breed solution, which heavily leverages third-party tools. (We will discuss Tivoli in more detail in the next report of this series.) CA's recent promotion of Unicenter APIs is an admission that many users prefer sticking with third-party tools, although it is difficult to buy only portions of Unicenter.

For all practical purposes, the APIs are for customers planning to use Unicenter for all except one or two functions. The obvious example is network management, where products such as Cabletron's Spectrum, HP OpenView, and Sun Net Manager already have an established presence.

CA's position is that customers are free to use Unicenter's native capabilities or those of third parties supporting the Unicenter APIs.

CA'S ROI STUDY FOR UNICENTER

Not surprisingly, given its position as an out-of-the-box solution, CA claims that it produces faster ROI. It recently sponsored a study, which was released summer 1997.

The report claims that investments can be paid back in as little as 69 days. The survey targeted large companies (average $3 billion revenues, 9,000 employees), with managers responsible for, on average, 4,300 desktops and 470 servers spread over 40 sites. It also assumed that full-time, burdened salaries for IT professionals were $102,000 (which is very close to our $50/hour assumptions). The average investment in Unicenter (including purchase, installation, training) was $59,000 per 100 users.

The study focused on three areas:

* Downtime reduction. With Unicenter, the survey concluded that downtime was reduced from 6% to 1%. For end users, productivity improvement was estimated to amount to $120,000 per 100 users over five years. It also estimated opportunity costs, a softer figure reflecting lost business. With the downtime reductions, based on an average $115 per user, per hour (across all industries), recovered revenue averaged $248,000 annually over five years.

* Improved IT productivity via automation. On average, 10 hours per month of IT staff time (per 100 users) was saved for routine initial system setup and configuration of routine platform and software upgrades, amounting to average savings of $17,000/year per 100 users. Savings for end user problem resolution averaged $29,000/year per 100 users for security administration and virus protection alone. Additionally, IT organizations were asked to estimate the number of fewer system administrators necessary to accommodate future growth; the average savings cited were $18,000 per 100 users over five years.

* Value of business process views of the IT system. In the survey, this was the most difficult to quantify. It focused on the impact to help desk staff, which would be better able to pinpoint system faults presented in a more intuitive manner. Respondents claimed that the incidence of problems that could be resolved remotely, thanks to these improved viewing capabilities, would grow from 45% to 90%. With related costs (e.g. less travel and administrative time) generated average savings of $62,000/year per 100 users.

The reported figures combine tangibles and intangibles. We agree that not all savings or value can be represented by black and white productivity numbers. However, all 'soft' figures, such as IT staff growth reductions, must be taken with grains of salt - although we don't question that frameworks such as Unicenter can produce such benefits.

We also know that such benefits are not automatic. The complexity of installation of any enterprise framework means that such benefits are possible only if your group surmounts a series of technical hurdles.

As mentioned earlier, based on feedback from its clients, Meta Group claims that many will encounter serious challenges implementing anything as ambitious as Unicenter or the Tivoli Management Environment.

USER EXPERIENCES

We surveyed a half dozen Unicenter users which we picked from reference companies posted on CA's Web site. We picked older sites, using a cutoff date of January 1997 (we tried to weed out sites that had just announced purchase of the product).

We found that implementation was complex, and that in most cases, benefits would increase with the learning curve. The sample was evenly divided between users with positive and negative experiences. In most cases, problems were due, not only to Unicenter's complexity, but to inadequate staffing or system infrastructure.

Because this was a snap survey to validate Unicenter savings, most respondents restricted themselves to citing IT productivity gains. We did not get into the softer categories probed by the CA sponsored study.

Central Vermont Public Service: One of the oldest Unicenter sites, CVP has been using the system for three years. The company, which covers 85% of the state, is a $280 million organization with an IT budget of roughly $3 million. In addition to legacy mainframe and AS/400-based systems which run the core business, CVP operates a Sybase client-server environment for decision support and work order management. The client-server environment includes Unix, a headquarter's Novell NOS migrating slowly to NT, with Citrix WinFrame linking Windows terminals at over 200 district offices statewide. Overall user population is about 500.

Unicenter is primarily used for job scheduling and event management, managing batch loads from legacy systems to Sybase, along with notifications for alarm conditions in server and disk utilization. CVP also uses HP's OpenView to monitor client-server network conditions.

Recently, CVP has integrated Unicenter's scheduling with JobTrack, CA's counterpart tool for the mainframe. (CVP uses a number of CA management tools on the mainframe side.) The result is that mainframe jobs in turn can trigger activities in the Unix environment based on preset relationships. CVP plans to implement Unicenter TNG to provide application-oriented views of client-server operations.

According to Tracy Adams, who manages the Unicenter installation, results have been positive, but not without a few bumps. 'CA has been slow to deliver [many functions], but Unicenter is still the best product available today,' he said. The biggest hurdle was initial implementation. 'We assumed that this would be plug and play, but that was not the way it worked.'

Unicenter required more internal staff resources than planned (CVP did use an outside consulting group to help). Although he feels that their installation was something of a guinea pig for CA at times, he notes that 'CA has bent over backwards to help me make this work'.

Adams also criticizes Unicenter's poor support for Novell, which until now provided binary emulation, not full support of Novell's NDS directory structure. This may be a moot point; Unicenter TNG promises improved Novell support, while CVT is gradually migrating toward NT.

Adams estimates that licensing costs for Unicenter over the past three years have totaled about $250,000; additionally, CVP retains Carty Mahieux, a New York-based consulting firm, for about 10-15 days of consulting per year for upgrades and troubleshooting; our estimate is that they are paying roughly $10,000-$15,000 annually on consulting.

On the other hand, Adams estimates that Unicenter has avoided the need for an additional 1-1.5 FTEs of IT administrator staff since they do not have to constantly rotate among consoles for different server platforms to ensure that operations are running smoothly. Based on local prevailing salaries of $30,000-$40,000 (or about $50,000-$60,000, burdened), that works out to $150,000-$270,000 in three-year savings. That roughly pays back licensing costs.

The data center operates on a 20-hour schedule; thanks to Unicenter, off-peak operations late at night can run unattended, thereby saving another 0.5 FTE (or about $25,000 annually, burdened, or $75,000 over three years).

Admittedly, some savings have come about independently of Unicenter. For instance, because DASD is so cheap, they no longer pay an FTE to manage DASD. Therefore, they do not require any DASD management options. Additionally, security audits have been eliminated since they consider that the security within Sybase to be sufficient for their access control needs. That has probably saved about another 0.25 FTE, or roughly $12,500 (burdened).

Over the past three years, the help desk added two FTEs (for a total of five) as terminals were replaced by PCs; they are paid at similar payscales as operators (about $35,000-$40,000 salary, or $55,000-$60,000 burdened). Although Unicenter had nothing to do with the additional help desk staff costs, it might in the long run reduce them. As they implement Unicenter's automated software distribution functions, Adams expects that help desk staffing could return to its initial level of three. Security might be another area for help desk savings, if automated via Unicenter. Today, log-on problems are a high proportion of help desk calls. As help desk personnel are relieved of some of their load, Adams believes they could double, performing operations tasks such as mounting tapes, which would save roughly 2-4 administrative staff FTEs.

However, the long-term effect is that, although there will be fewer personnel to run operations, they will have to be more highly paid since they are performing more functions, and higher-level functions requiring a broader view. In all likelihood, that will mean minimum salaries of $40,000, compared to $25,000-$30,000 for entry-level operators previously (both sets of figures unburdened).

The bottom line is that Unicenter has easily paid for itself through labor savings, but not without struggle.

Delta Dental Plan of Kentucky: A $38 million dental insurer and managed healthcare provider. The 85-employee organization based in Louisville, operates an IBM RS/6000 server, 40 PCs running Windows 95, and five PCs using OS/2 (for a legacy application), and 40 terminals linked via token ring network. The company's Unix environment is relatively new, resulting from a 3-year migration away from a mainframe environment, maintained by a former business partner. That transition was completed as of January 1997.

Unicenter was chosen over four years ago because it was the only product that offered tape management, security, job scheduling, file management, and report management in one package. An easy to use, integrated solution was preferred as the Information Services department was essentially created with the arrival of the RS/6000.

'It was easy to learn the discipline, and the screens were all consistent,' said Don Kavanaugh, IS manager. It took less than a month to get Unicenter up and running.

Delta is currently evaluating an upgrade to TNG, as well as its experience with Unicenter itself. 'We know we have seen less labor costs because of Unicenter, but we also know we have not taken full advantage of what it has to offer,' explained Paul MacCready, Vice President of Operations.' MacCready estimates that, so far, Unicenter has saved them $50,000 annually.

Functions in TNG that Delta finds particularly attractive are automatic paging of administrators (much easier than in Unicenter 1.1) and improved network monitoring and management. They propose to use TNG's agents to monitor desktop parameters such as memory utilization. With terminals scheduled for replacement by PCs, system administration workloads are expected to rise. By virtue of TNG's agent and monitoring capabilities, Kavanaugh expects much of its cost to be offset by reduced IS staff growth.

Midwest Financial Institution: A distributed network operates with 10 sites, and 4,000 end users, including headquarters and regional offices nationwide, it uses Unix applications (PeopleSoft and others) and database servers (Oracle and Sybase), and Novell workgroup servers. Unicenter has been used for two years, primarily for job scheduling and event management. There are roughly 300 major jobs daily, with peak usage occupying about 60% of the 10-hour workday (when all time zones are factored in). Unicenter was chosen because, at the time, it was the only product integrating job scheduling with events.

Implementation was performed internally, without consultants. The first server was implemented in two months using two people, including a systems and an application specialist (four person months total); succeeding servers required half the effort. Based on estimated payscales of approximately $60,000 and $50,000 (unburdened) for systems and applications staff respectively, we estimate that implementation costs (assuming 50% burden) averaged $27,500 for the first server, and half that for subsequent platforms.

According to Cheryl Krause, an internal staff consultant for architecture and implementation, if remaining Unicenter modules (beyond scheduling and events) were implemented, it would take roughly 2-3 people six months, (12-18 staff months), which we would guess costs about $80,000-$120,000 (burdened staff time).

Unicenter proved unexpectedly difficult to implement; their experiences were similar to CVP, in that the modules were not as well integrated as advertised. Documentation was also poor. 'We have a Unicenter problem at least once a week, and it's really difficult to go through 15 manuals to find out what happened,' said Krause. A sticking point was writing the interfaces from Unix applications to Unicenter. Initially, features such as cyclic scheduling (scheduling an event to run regularly at short intervals) did not work well. Upgrades have been difficult, but primarily because there are few off-peak periods (not Unicenter's fault).

The major problem, however, has been staff commitment. 'With CICS, we have a full FTE dedicated to knowing that product inside and out. We haven't done that with Unicenter,' Krause admitted.

Overall, roughly two staff years were invested in the full implementation (about $160,000-$180,000 burdened, our estimates), not including license costs. Krause does not believe that her organization has yet realized a payoff from Unicenter. 'We have not been able to reduce staff, but we have had some reduction in downtime.' Ultimately, said Krause, the true returns would come if a critical mass of remaining functions such as performance management and problem resolution were implemented. She concedes that staffing commitments have been the prime hurdle. 'Application work has always taken priority,' she said.

Mineral Resources Company: Prior to a recent merger, a $300 million mining firm installed Unicenter four years ago when the data center was migrated from Prime to an HP 9000 platform. At the time, the firm operated a headquarters facility which ran payroll, financials, shipments, and corporate quality management applications, while two other sites were responsible for running operations management applications pertaining to mining activities. Pre-merger, there were roughly about 300 users overall, most of whom were using PCs. Unicenter was used for security, tape management, and job scheduling.

According to Unix administrator Lori Laber, Unicenter 1.0 was extremely difficult to install back in 1993. 'You had to know a lot of Unix to operate Unicenter,' she noted. Part of the problem was that Unicenter's SQL database repository was her first exposure to relational database technology. Another problem was keeping track of all the interfaces to the Unix operating system itself. 'We made the mistake of not having a consultant help us,' she said.

Additionally, the organization did not need all of the functionality on Unicenter to run on each server; for instance, outside financials and payroll, security was not deemed critical. Yet, Unicenter would not allow them to implement features only on specific servers. She noted, however, that Unicenter's tape management was extremely effective. 'We could get [data] recoveries from tape within two minutes,' she noted. The flip side, however, was that all the necessary indexing took up a full gigabyte of their 10 GB system, which at the time was considered a significant overhead cost.

Having been recently acquired, data center functions are slowly being migrated to another facility out of state. Unicenter is being phased out. The new entity, a $750 million company, will implement Oracle Financials and MIMS (a mining information system). It will rely on Oracle's own security and a point solution - Omniback - for backups.

Rocky Mountain Healthcare: The regional Blue Cross Blue Shield organization for the intermountain region, has a client-server network which serves over 1,800 desktops in the Denver headquarters, along with eight regional sites in Colorado, New Mexico, and Nevada.

There are roughly 25 Unix servers (including Sequent, Sun, HP, IBM SP/2 and RS/6000) running a mix of financial, special policies, Web, EDI, COLD (computer output to laser disk), and other applications. Additionally, there are several HP 3000s (MPE) running a membership and claims applications, and mainframe-based dental and specialty health program services (financials are slowly being migrated to Oracle).

Unicenter is still brand new, with only two modules live: software delivery and asset management tracking. In the near future, it will add security/single sign-on, configuration management, job scheduling, and alarms. Additionally, it plans to integrate its Platinum Apriori help desk application with Unicenter.

The organization just completed a complete rollout of Windows 95 and NT Workstation. LANs will remain Novell for the next three years, owing to NT Server security and performance issues, according to Steve Hill, who manages the technical project office.

The client-server environment is managed by 10 senior Unix and NT administrators; additionally, there are 4-5 administrators for Novell, NT, and email systems at workgroup level. All administrators are also involved with project work.

Unicenter's biggest contributions to productivity are from software distribution (eliminating technician visits to end users) and the ability of Unix and database administrators to monitor several servers apiece; the company did not have estimates as to the savings from these features. Regarding software distribution, the organization had an existing policy to lease PCs, so it already had effective asset management practices. The difference with Unicenter is that asset management is now automated, eliminating the trip to the desktop.

The organization is still rolling Unicenter out, primarily by consultants. The goals are to reduce current IT expenditures for desktop support by roughly 10%. Unicenter is to supply part of those savings; for instance, dramatic savings are also expected from an automated document retrieval and imaging system. Hill estimates that without Unicenter, they would have to recruit 3-4 additional desktop support personnel.

The most challenging part of implementation has been writing scripts that integrate applications, databases, and other utilities to Unicenter, according to Lonnie Schaffer, technical project manager. He cites poor documentation and does not like the pre-TNG user interface. So far, it has required about 120 days of consulting for implementation. Although the project is roughly six weeks behind schedule, the delay is not unreasonable given the project scope.

Travel Services Company: A privately-held travel center organization, based in the southeast, numbering 5,500 employees and over 600 retail sites across 38 states has implemented Unicenter for report distribution only. It has been in operation for 18 months. Ideally, the firm would have also liked to use Unicenter for network management, but in the meantime the small IT staff was caught up in other projects.

The headquarters office, numbering 350 people, is a 'vendors heaven', according to Jeff Swiney, systems manager. It includes a mix of Novell, NT - and even OS/2 - LAN servers, along with two HP 9000 Unix servers. At retail sites, there are an average of seven PCs, of which four are used as point-of-sale (POS) terminals.

Swiney concedes that Unicenter was not installed properly, even with the consulting help. So it's been the best of times and worst of times for Unicenter. The chief benefit, cited in a CA press release, was that printing and distribution time for reports was cut 67%, from 16 hours down to only six. The organization issues reports on a weekly basis. There's no question of productivity advantages there. If we estimated time savings, it would amount to 500 hours annually, or roughly $12,500 savings based on $25/hour (burdened) rate for a staff operator.

However, outside of report distribution, Unicenter has been more of a burden, bottlenecking operations due to performance overhead. Consequently, the firm only boots up Unicenter on Thursdays when they generate the reports. Swiney concedes that he requires a Unix administrator to baby-sit the system all day Thursday while Unicenter is in operation.

The site's problems extend beyond Unicenter. Thanks to the polygot hardware and platform situation and inadequate network wiring, his staff of five system and telecom administrators are often fighting fires almost full time, at a cost that we estimate is at least $250,000 annually.

Changes are on the way, including building a 100 MB fiber backbone with 10 MB drops at the home office. Switches may be next for replacement. And the organization is accelerating a migration to NT as the LAN standard. Only then will a properly executed migration to Unicenter TNG make economic sense, said Swiney.

UNICENTER'S (DIS)ADVANTAGES

Although our sample was not scientific, it is clear that Unicenter provides clear advantages when an organization commits the necessary resources for infrastructure, training, and implementation. The easiest to quantify are labor savings and productivity.

However, Unicenter cannot fix something broken unless the organization is already taking steps to get its policies and procedures in shape. Implementing Unicenter - like any enterprise application - is not trivial, but neither are the potential rewards.

Our research indicates that many organizations have had to learn these lessons the hard way. In our next report, we'll see if Tivoli users have had similar experiences.