In a study sponsored by Tibco, open source test automation vendor PushToTest has found that developer productivity with Tibco’s ActiveMatrix SOA platform is 49% lower than BEA, 35% lower than Oracle, and 22% lower than IBM.

Of course, these comparative studies are always minefields, as rivals can always point to factors such as scoping and choice of parameters that may skew the results one way or another. No single vendor ever has the best tooling for addressing 100% of all use cases. For those reasons, we normally hate writing about these things. The PushToTest study is interesting; it points to some key Tibco strengths, but by scope, its analysis didn't cover some of Tibco’s weaknesses (for instance, that IBM has consolidated run times for SOA and BPM, while Tibco hasn’t).

What makes the study interesting that it tackles one key stumbling block of SOA: the available tools don't make composing services very easy.

 
The PushToTest study pitted developers working in each of the SOA stacks from Tibco, plus BEA, IBM and Oracle, and had them work on a problem of putting together the kind of procurement and inventory management services that would be typical to a manufacturing company. The results were based on the skills and technologies that were required by each tool to compose and deploy services, and the amount of time it took to develop and deploy them.

ActiveMatrix provides a virtualization environment so you can deal at the business logic, rather than the Java or .NET plumbing level, a benefit that the study highlighted. And that was instrumental in a key conclusion: that Tibco provides a much simpler environment because it abstracts service composition from the Java, .NET implementation environment or plumbing beneath it.

Conversely, the PushToTest study slammed competing platforms as warmed over J2EE servers that were repurposed, and not built for composing services. (It did not compare Microsoft’s environments, which of course have their own .NET technology dependencies.) It listed the alphabet soup that you must wade through with rival SOA platforms, including Java, XQuery, XSLT, EJB, BPEL, XML, and WSDL.

Of course, there are those such as ZapThink who see a downside to all this; cited in Intelligent Enterprise, ZapThink managing partner Ron Schmelzer claimed ActiveMatrix simply adds yet another level of middleware that makes the architecture ultimately more complex.

But cutting back to the chase, the PushToTest study provided examples: with BEA, workflows were treated as external containers; IBM requires four products to build a service, while Oracle which requires five. By comparison, because Tibco has virtualized the service container, it requires only a couple products (BusinessWorks for modeling the process, and ActiveMatrix Service Grid for composing and deploying the service).

But the downside is that Tibco’s tooling throughout its product line is not consistent: some is Eclipse-based, while some tooling isn’t. There are good reasons for that (the Eclipse framework lacks a metadata mechanism and isn't suitable for deploying from .NET), but it means there are different developer UIs to contend with. But because the PushtoTest study limited itself to the new Tibco products, that wasn't factored into the analysis.

When it came to building services, the study noted that BEA’s own AquaLogic Service Bus ESB imposed a critical limitation in not supporting the kinds of service “callbacks” (or acknowledgements) that you would like when submitting transactions like a purchase order. In other words, it lacked a provision for the receiving party to acknowledge receiving the PO. (That is theoretically supposed to be covered by WS-BaseNotification, which happens to be one of the more obscure, and least-implemented, Oasis web services standards.)

Ironically, there’s a good reason why AquaLogic Service Bus lacks this capability, as it was designed as a lightweight pipeline for mediating and routing services. At the time, we slammed it for not supporting BPEL service orchestration, but history has proven us wrong in that organizations have preferred to orchestrate services from the server rather than on the bus. However, while BEA’s lightweight ESB design was on the right side of history, it had a cost that was illustrated by the PushToTest use case.

As for Oracle, the PushToTest study critiqued it for requiring you to manually drop down to using Apache ANT scripts for building SOAP messages, while IBM, in the words of PushToTest CEO Frank Cohen, “wanted to give you 3 - 4 hrs of architect’s time to show you how to use their tool.” Furthermore, when it came to specifying policies for newly created services, BEA had issues because AquaLogic Service Bus and WebLogic Integration were not sync’ed on the same version of WebLogic Server.

When it came to service orchestration, the study stated that Oracle had a good editor for BPEL process design, but those advantages were offset by the need to drop down to coding and generating configuration files for specifying transformations. IBM had a decent integration developer product with assembly diagrams, but Cohen criticized it for not having a true Eclipse look and feel (as we stated earlier, neither do some of Tibco’s tools), and more importantly, that the visual design tools weren’t terribly intuitive. Nonetheless, IBM’s tools took only a few more hours to assemble orchestrations compared to Tibco’s ActiveMatrix tooling.

Probably our main bone of contention with the study’s analysis of what it takes to reuse services. Admittedly, we may be making a mountain out of a molehill as we, like others, believe that the importance of reuse in SOA has been overblown. They framed ease of reuse by the ease to which you can access a service registry. However, regardless of our cynicism over reuse, we still believe that PushToTest failed to an adequate comparison on ease of reuse across SOA platforms. Specifically, excluding IBM, each of the players covered in the survey OEM HP/Systinet’s UDDI registry, but PushToTest omitted the fact that Tibco is also one of them. It simply states that “TIBCO AM [ActiveMatrix] provides good integration to registry,” without mentioning how the others integrate with the registry, or why their integration is not as good.

But one can conclude from the study some basic insights to Tibco’s approach: that it pushes the plumbing beneath the hood so developers of services can simply develop. By comparison, you have and IBM, which has put SOA, BPM, and EAI on the same platform, but for whom tooling has not always been a strength; BEA, which, distracted by Oracle, hasn't sync’ed its runtimes; and Oracle, whose Fusion middleware is a work in progress. Unfortunately, the study omitted Software AG/webMethods, a rising player whose inclusion which would have made the comparison more compelling.

Obviously, the choice of platform will depend greatly on external factors such as existing IT environment and prevailing skillsets (e.g., if your shop has lots of Oracle skills, it’s obvious which vendor is your best bet). But the PushToTest study points to one important fact: developer productivity remains a major factor in time-to-benefit, and it’s an area that most SOA vendors have not paid adequate attention.

---