SOA Explained

I recently came across this nifty little SOA primer animation. It is a really good, condensed “SOA Elevator Pitch”. If you want to convey the essentials of SOA to your boss or any other person that doesn’t seem to get it, you may want to try this one.

Mr Baseline’s EAI Crash Course – Part VI

The final installment of this EAI crash course isn’t about EAI but about SOA. So what’s the connection? To be sure, EAI and SOA are not the same thing at all. In fact, they address different domains. EAI is very much a technical discipline, mostly very IT-centric, delivering the technical solution to how disparate systems should talk to each other. SOA, on the other hand is (or should be) mostly about delivering business value through a complete overhaul of the way a business is managed, where the architecting of the business is the starting point for the implementation of this architected design using IT.

This description, as many of you know, rings pretty hollow in the mundane corridors that many of us inhabit. Mostly, business management still view SOA as primarily an IT issue, and IT struggles to “deliver SOA” despite the fact that it simply can’t be fully done without extensive management commitment.

However, what can be done while we’re waiting for the rest of humanity to see the light is to start putting an SOA-enabling infrastructure in place. And this is where EAI and SOA converges, in the shape of the ESB. It turns out that a well structured, well documented ESB where the Service concept is part of the basic design patterns is an ideal foundation for SOA. There is a journey of learning to be done, where the technical aspects of designing, implementing, deploying and maintaining services can be mastered. Using Baseline, the community involved in EAI can prepare themselves technically and organizationally for SOA, so that when the business folk eventually are ready to start requiring some real business centric services, the skills and delivery capabilities are in place, and requirement realization can be swift.

We believe strongly in this approach to SOA, so much so that we formalized this in the Baseline SOA Roadmap. Its purpose is to help companies start on their SOA journey. It won’t be SOA at first, but the transition to SOA will be that much easier when the basic skill-set is in place.

The roadmap itself is pretty simple:

1. Put an ESB in place
2. Make sure it is designed and documented using a structured, service-centric methodology (such as Baseline)
3. In the day-to-day EAI work, start identifying and implementing reusable services, applying appropriate design patterns
4. Eagerly await the rest of the business to jump aboard!

We actually got inspired to do this by one of our clients who said: “We started of doing systems integration using Baseline, and when the SOA buzz arrived, we realized that, hey, we already have an SOA!” I would have to assume that they meant the SOA-enabling infrastructure – the organizational overhaul issues are squarly outside Baseline’s domain!

This post concludes the EAI crash course. If yo want to know more about Baseline, please visit our website.

Vegas – Monady

This is my first experience with a really big IBM event, and boy is it big! Some 6.000 people buzzing about like busy ants getting to and from sessions, of which there are an overwhelming number.

The opening session was a hilarious show, mixing professional presentations with the wits of Drew Carey. He had gathered a bunch of comedians doing improvisations prompted by input from the audience. The most impressing of these was when one of the guys cracked a title of a song on a given theme (“veterinarian songs”, no less). Then two other guys proceeded to sing while allegedly composing the lyrics as they went along. Titles included the challenging “Put your hand in your dog and cough”…

Well, enough of these ramblings, what I wanted to focus on today is a session with a customer panel telling about how they got their respective SOA initiatives of the ground. I must say I did not have high expectations for this session, but the guys in the panel turned out to drop quite a few gems.

What was shining through was the fact that the challanges of SOA rarely, if ever, are technical. Instead, it’s all about organization and perception. One of the panel members stressed the fact that one of his most efficient tools to gain acceptance for SOA was inclusion, because, as he put it, “everyone wants governance, but no one wants to be goverened”. This is just a variation on the age-old not-invented-here-syndrome, but it seems to me that in this we find one of the most basic reasons for resistance to SOA. So, what to do? Well, his recommendation was to create some kind of virtual organisation, maybe an SOA reference board, have both integration specialist and application owners and developers meeting regularly to discuss all matters concerning shared services and integration. Also, he suggested, let the chair of this body circulate among its members to strengthen a sense of common ownership. In this context the same guy also quoted a simple but potetially powerful principle that could foster this sense: “Only build services that you don’t consume, only consume services that you don’t build”. I realize that this contains all sorts of political and other pitfalls, but as a general idea, it strikes me as brilliant. It highlights the essentially collaborative nature of SOA, and, if promoted, would serve as an antidote to silos and fiefdoms.

The proverbial Low Hanging Fruits were also addressed, and a recommendation that I have seen elsewhere was put forward by the panel. In the best of worlds, the SOA initiative is actively sponsored by top management, but when this is not the case, actors on lower rungs can still create success stories around SOA by applying SOA principles to infrastructure issues like security and compliance. Armed with those success stories, SOA champions can use them to prove the real value of SOA. And, one member stressed, never ever sell SOA as such to non-IT folks. Always focus on solutions to business problems and promote SOA as an enabler.

In my opinion, wise counsel was given, and also encouragement. This was three real companies realizing real SOA benefits. To be sure, they had a long way left to go. When asked what percentage of their total application functionality was delivered through shared services, answers ranged from 10 to about 50 percent, and, as one member put it, we may never get to a 100 percent. But they all agreed that the journey is worthwhile and maybe even unavoidable.

Does SOA require an ESB?

For those of you not subscribing to InfoQ, I just want to point you to this article, dealing with one of the cornerstones of how we recommend our customers to design their ESB, with or without an SOA perspective: Don’t Let Consumers and Service Providers Communicate Directly. It ends with a conclusion that I see pop up over and over again (and one that I agree with): No, an ESB is not strictly required to build an SOA, but it can sure help.

A Sobering SOA Sermon

Through InfoQ I got another juicy lecture from QCon (I should go there next year!). Anne Thomas Manes puts SOA in its proper strategic perspective. It contains lots of gems for us disillusioned practitioners of SOA (in one of my presentations I have a slide with the heading “Why is it so damn hard?”). Well, when Anne puts the SOA life cycle time span in the 15-20 years bracket, maybe we shouldn’t weep if SOA doesn’t happen over night…It is good to keep in mind, as Anne points out, that the legacy we’re up against have been in the making for decades.

I bring two particular important bits of wisdom from this presentation. The first is the concept of applying a structured analysis of an enterprise’s application portfolio. Anne is talking a lot about the fat, unhealthy state of most IT organizations, basically originating in massive redundancy of functionality and data, driving obscene amounts of cost in integrating all those redundant bits and pieces. Anne suggests regularly applying a cost/benefit analysis to each application, and ruthlessly decommission any application not earning its keep, thereby shrinking the application portfolio. The other side of that coin, obviously, must be to identify any functionality the obsolete application provided and extract that functionality (as a reusable service) from somewhere else, with the ultimate goal of only having one provider of a particular business capability. This will obviously take some time…

The other bit I bring with me is the analogy between SOA and a new, healthy life style. Just as it’s fairly easy to go on a diet, loose a few ponds, and then gain them again, plus a few extra pounds, kicking off SOA in fits and starts is easier than digging in, gaining strategic sponsorship from management and preparing for the long haul. Yet to be of any real use, persistence will be essential to success. The paradox here, as can be gleaned from Anne’s lecture, is that you can’t mount a massive, all-encompassing SOA attack on your enterprise, because of legacy, politics and funding constraints.

So all in all, SOA is damn hard, yet the promises it holds if you persevere are of such magnitude that you really have no choice in the long run but to get on the SOA band-wagon.

The Next Big Thingy?

With much apparent hullabaloo, IBM recently acquired AptSoft, a small US software vendor focusing on CEP (Complex Event Processing). Googling on AptSoft will generate reams of hits mostly linking to various IT publications commenting on the acquisition; here is but one of them. It’s funny how comments about this refer to how CEP is superior to “traditional BPM tools”, as if these were more or less commodity among the companies of the world, when the fact of the matter is that these tools are only starting to be employed by vanguard enterprises, at least from our vantage point here in the Nordics. Maybe it’s different in other parts of the world, although I wouldn’t bet on it.

In typical IBM fashion, this is also SOA. And eventually, since we’re an IBM partner in the SOA sphere, this will be another technology to investigate. Hooray!

Plumbing for SOA

For quite some time, we’ve been promoting the image that what we’re mostly about at Zystems is plumbing. We even have a presentation called “Plumbing for SOA”. This would indicate that we draw the pipes but do not necessarily have much contact with genuine business value (such as sinks an faucets). I ran across this analogy in a most illuminating article by Bobby Woolf of IBM. I never really made the distinction between a Message Bus and an Enterprise Service Bus as such, but I think the definition Bob is making makes a whole lot of sense, and it is one that we’ve somehow implicitly managed to place firmly in the center of our concept model. One of the cornerstones of this model is the Service concept. The beauty of it is that we distinguish between public services and private services. Those customers who basically use the middleware (in our case mostly WebSphere MQ and Message Broker, but increasingly also WebSphere ESB) for traditional, technically oriented point-to-point integrations will do this using private services. Private services are exactly what Bob is referring to when he talks about the Message Bus. Private services are whatever technical goo we need to apply to pass the correct bits to the correct destination. What we manage to do is to offer our customers a seamless (from a technical point of view) migration path from a Message Bus to an ESB, by simply applying some sound design principles to the services, thereby creating public (SOA) services.

There is (unfortunately some might say) still a massive need for a well structured, well documented Message Bus. Our job is to make sure that it can mature together with the customer into an ESB that will truly be a technical linchpin of an SOA. So, yes, we’re plumbers, but when the customer is ready, magically there will appear sinks and faucets firmly (i.e. loosely ) attached to the pipes.

Debunking the ESB

Anyone who’s been struggling with SOA would do well to spend some time together with Jim Webber when he talks about Guerilla SOA. For a company that promotes the ESB concept like Zystems does, the guy delivers some heavy punches. Nevertheless, I believe there are some really important lessons for us in his message. When we talk about the virtues of the ESB, we do emphasize the key benefit of accessing back-end legacy data. What I think we would all do well to consider is the ‘upper level’, i.e. where the business services reside. What Jim’s driving at, I think, is to make sure that anything on this level is standards based and open, not proprietary, which makes a whole lot of sense to me. And in a future where most applications have standards based interfaces, what, indeed, would be the purpose of a proprietary chunk of software in the middle? I do believe, however, that while we have lots of legacy apps to integrate, and lots of it outside of the SOA domain, methodologies to handle the ESB will be very much in demand.

But his most central insight, I think, is that all our efforts are “spaghetti agnostic”, i.e. our schemes and plans defy the spaghettiness of reality. We hope that magnificent tools and large-scale methodologies will somehow defeat the inherent messiness of reality. What we need is SOA – Spaghetti Oriented Architecture – hence his use of the term Guerilla SOA. I understand it as once and for all giving up the One Big Scheme pipe dream and find ways to embrace change. Quite a mouthful when you make your living trying to bring some orderliness to the integration world .

As an aside, it warms my heart to hear Jim talk about the ESB as an Erroneous Spaghetti Box! That’s exactly what we claim might happen if you don’t have a simple and straight-forward way of developing integrations (such as Baseline). Never mind that the guy wants to get rid of the ESB altogether, or nearly so. As for governance, he promotes the idea of a wiki based solution, not a high-tech UDDI-monster. Again, hello Baseline Service Registry (our wiki-based documentation).