Archive for the “Web Services” Category

Originally posted Tuesday, June 08, 2004

Literally, a pain in the neck

The brain damage required to expose IS Flow services as document literal or Soap Message style web services is in sharp contrast to the ease with which you can create Soap RPC web services using Integration Server.

I generally recommend using document literal web services to my clients because they offer more explicit control over error generation, ability to validate the schema(s) associated with the soap messages before processing its payload and the ability to log the raw soap messages received from the business partner or internal customer.

However, the work required to create and test the wrapper services that extract and validate the soap body elements, convert the body into the correct inputs for the “wrapped” service and then map the outputs of that service (or exceptions it throws) into a valid soap response is far beyond what is reasonable to expect developers to do these days.

I’ve toyed with the idea of creating a few utility services to make creating the wrapper service much, much easier, but knowing that WM is planning on replacing the IS soap stack with Glue over the next several releases, that might be a lot of wasted effort.

Generating a WSDL that will invoke your wrapper service is also harder than it should be. For some reason, you can’t just refer to the input and output documents of your wrapped service. You have to create a new document type that contains a “holder” document type. That document type contains only one child, a document reference to the document type you really want to use. When you inspect the generated WSDL the holder document is there, but when you generate a Flow or java client from the WSDL the method only uses the input document of the wrapped service.

BTW, my favorite way to generate Glue java clients from WSDL files is to use Eclipse with the Glue plugins. Just create a new Glue project and then add a web reference. The new web reference wizard prompts you for the location of the WSDL and generates the classes you need. Just add a new class with a main method to invoke your web service and you’re set. Easy!

Mark

Tags: , ,

Comments No Comments »

New project, new functional domain. Wildfire incident management.

I have been updating the design of my custom SOAP processor for webMethods Integration Server to add support for XML Signature and XML Encryption. I’m also adding support for additional WS-Security token types to include X509CertificateTokens.

The project calls for using LDAP-based authentication to authenticate service consumers. Since LDAP authentication requires plain text passwords, using tokens containing only password digests in wsse:PasswordDigest format won’t do the trick. I’m considering building a very simple Security Token Service (STS) that would issue tokens (over SSL) that would contain encrypted user credentials as well as other desired user attributes. When supplied on subsequent service requests the tokens could be decrypted and used to authenticate against LDAP or the STS could be called again by the custom soap processor to validate the token.

In the last couple of months, I’ve also joined and become a voting member of the OASIS Emergency Management TC. The TC is working on finalizing the EDXL DistributionElement standard which we will probably use as an envelope for the payload of our Soap messages. The TC is also working on EDXL-RM which will eventually describe a set of emergency management resource managemnet messages that will be useful in ordering and assigning resources to various types of incidents such as wildfires, hurricanes or the inevitable avian flu outbreak.

We’re finally getting some much needed snow here in the foothills of Colorado so hopefully we’ll avoid burning down this summer. It’s tempting to add some trap doors to the web services I’m working on to sramble an entire squadron of air tankers to any fire that is close to our house. I guess it woudl look a bit suspicous with that many planes and helos circling over one house though! ;-)

The wMUsers software migration went very well and the response from the webMethods user community has been great with over 800 new user registrations and nearly 2000 active users since the move and higher message activity levels than ever before.

Comments No Comments »

In a recent post on his “Bliki“, Martin Fowler commeted:

Whenever ThoughtWorks rashly lets me out in front of a client,
one question I’m bound to be asked is “what do you think of SOA
(Service Oriented Architecture)?” It’s a question that’s pretty much
impossible to answer because SOA means so many different things to
different people.

  • For some SOA is about exposing software through web
    services. This crowd further sub-divides into those that expect the
    various WS-* standards and those that will accept any form of XML
    over http (and maybe not even XML).
  • For some SOA implies an architecture where applications
    disappear. Instead you have core services that supply business
    functionality and data separated by UI aggregators that apply
    presentations that aggregate together the stuff that core services
    provide.
  • For some SOA is about allowing systems to communicate over
    some form of standard structure (usually XML based) with other
    applications. In it’s worse form this is “CORBA with angle
    brackets”. In more sophisticated forms this involves coming up with
    some form of standard backbone for an organization and getting
    applications to work with this. This backbone may or may not involve
    http.
  • For some SOA is all about using (mostly) asynchronous messaging to
    transfer documents between different systems. Essentially this is
    EAI without all the expensive EAI vendors locking you in.

I’ve heard people say the nice thing about SOA is that it
separates data from process, that it combines data and process, that
it uses web standards, that it’s independent of web standards, that
it’s asynchronous, that it’s synchronous, that the synchronicity
doesn’t matter….

Comments No Comments »

Ajax is a technology that I have bumped into a few times over the last several months, but haven’t (yet) needed to learn in my normal course of work.

According to it’s Wikipedia entry, Ajax or Asynchronous JavaScript And XML is a standardized approach to building rich client applications that depends on having a service-oriented architecture of some sort to provide the business logic.

Slashdot alerted me to a recent Dion Hinchcliffe post, in which he writes that

Ajax is a web client programming style which eschews traditional HTML web pages, which are only sprinkled lightly with JavaScript and reload pretty much every time they are updated or clicked on. Instead, an Ajax web client receives an Ajax JavaScript library into a hidden frame which provides run-time visuals on the main browser window that look and feel very much like a native application. Ajax web clients, once loaded, communicate with XML services on the back end (via a browser’s built-in powerful XMLHttpRequest API), and then use JavaScript to manipulate what the users sees programmatically via DHTML.

All of this allows Ajax to provide a compelling user experience because 1) it doesn’t reload the web page, and 2) it runs asynchronously allowing background server-side requests for information to be issued, all while the users clicks, types, and otherwise interacts with the application in the foreground. Google Maps is the pre-eminent example of a modern Ajax application: rich, interactive, easy-to-use, and predictive in that it loads the map tiles that are just offscreen in case you need them.

Don concludes

As Ajax proves to be the application model that allows compelling composite applications to be developed quickly, I expect that 2006 will be the year that Ajax emerges as both a key driver of SOA architecture (important: because, without a SOA, there can be no Ajax applications, by definition) and a must-have application development model.

I hear that Ajax support is on the agenda for inclusion in a future version of webMethods Portal. When combined with webMethods Integration Server, developers can already quickly create web-based applications using services exposed from any back-end application or platform. I look forward to seeing the applications that could be created using a Portal-on-Ajax version and the already strong SOA capability of webMethods IS.

Comments No Comments »

Mark Griffin noticed a new open-source Enterprise Service Bus entry called ServiceMix 1.0.

Another open-source ESB enters the mix. These are becoming more numerous. None of these are of the quality or feature set of a webMethods or Tibco. But will they eventually put any pressure on the pure play vendors? Only time will tell but certainly something to watch.

So who will be the JBoss of the ESB space and what will that do to the existing providers of ESB tools? While I would certainly see a negative impact on the traditional integration players like webMethods, Tibco and IBM, I would suggest that the impact wouuld be greater on new entrants to the ESB / SOA space who get 100% of their revenue from that sector and who lack the sales or VC funding to keep going in the wave of comparable open-source functionality.

Comments No Comments »

I wrote a while back about my frustrations with UDDI as an adequate repository for web services metadata. Recently, Ascential’s Michael Curry wrote in his SOA Blog recently

From what I’ve seen, UDDI is very seldom used by most companies beyond departmental implementations. The real problem is that the information in a UDDI directory is not very useful, or not useful enough, and therefore developers don’t bother using it to find services.

He received a bit of pushback from Systinet’s Tom Erickson who wrote

Most of our customers are certainly looking for capabilities beyond UDDI, thus our development efforts to create Systinet Registy 5.0 a year ago and 6.0 which we announced last week. Our new platform Blizzard, available in early access next month, takes things a step further by adding a repository and capabilities to model the SOA

I think both of these guys are saying similar things. UDDI as it exists today is woefully inadequate for most large-scale users. Sure some will use it, but not happily and not without building a long list of complaints.

As I design and build intelligent intermediary-style SOA architecture, I almost always need a services repository that goes far beyond what is offered by UDDI today.

Vendors, left to their own devices, will solve this problem by releasing products that create rich services metadata repository while providing a UDDI API to fill the “checkbox” requirement.

This is better than making their customers roll this on their own as I’ve been forced to do, but not nearly as useful as banding together to create a better UDDI and then tweak their new registry tools to adhere to a new spec.

I know that, in the real world, specs like UDDI often significantly lag industry and vendor best practices. Playing the specs and standards game is also a constant balance between creating competitive advantage with your own implementation and generating enough standardization to drive broad adoption of your, now-compliant, product.

UDDI just seems to be a particularly painful example of this.

Comments No Comments »

The primary purpose of an Enterprise Services Bus or ESB is to implement a layer of enterprise-class infrastructure that provides a standard way for an organization to expose services so that they can be discovered and executed by authorized users.

The services exposed by an ESB can be created by web-service enabling portions of existing applications, leveraging new web-services API’s provided by existing packaged apps or creating new web services from scratch.

The combination of development language, hardware platform or package vendor on which a service depends is not important to the ESB other than that the combination must provide a satisfactory level or service to consumers.

One approach for implementing ESB infrastructure is to create an “intelligent intermediary” (or fabric of intermediaries) through which all requests for services are routed.

This approach enables consistent implementation of the organization’s security framework, provides a mechanism to seamlessly fail over to a new instance of a requested service and supports horizontal scalability by routing requests to “farms” or “grids” of enterprise services.

The intelligent intermediary provides these things without impacting the individual services or the legacy applications and COTS packages that provide them. In essence, the ESB becomes a loosely coupled “container” for enterprise services analogous in many ways to a J2EE application server but without the restrictions of developing the services themselves as Enterprise Java Beans.

The intelligent intermediary approach is often criticized because it adds a layer between the service consumer and the service provider. This layer seems (and probably is) excessive when only a few departmental-class services need to be exposed and consumed. However, when one measures the work (and heavy handed enforcement) required to push enterprise security, transparency and quality of service standards to each developer or provider of a service, the so-called overhead looks like a bargain by comparison.

For years organizations have been creating rat’s nests of point-to-point integrations that were a nightmare to maintain. Creating a point-to-point mesh of web services invocations is just this generation’s way of re-inventing the wheel. Try adding a WS-Security based security framework to one of those messes. It can be done, but at much greater cost (time and money) than spending upfront to design and build or acquire an ESB to act as the intelligent intermediary.

We’re still in the intense hype phase of the Enterprise Services Bus and Service Oriented Architecture adoption cycle. Organizations are struggling to plot a course that avoids pitfalls of being an early adopter, emergent vendors are slapping out immature products to gain market and mind share and the mature integration vendors are adapting their core products to address the space through new releases and acquisitions.

While the product cycle takes its normal course, organizations can continue to web-service enable existing applications that provide value as enterprise services. During this preparation phase, however, organizations should implement an intermediary of some sort even if it does nothing but get developers (and vendors) in the habit of routing requests there first.

The initial version of the intermediary may not have all of the organizations desired capabilities, but it will help avoid pushing complexities of enterprise class infrastructure down to individual services and pave the way for the (perhaps not-so-distant) future when the tool vendors have once again caught up with the needs of their customers.

Comments No Comments »

Several of my recent projects have involved building web services architecture using webMethods Integration Server. A couple of these have required some level of support for WS-Security.

WS-Security support is not included in webMethods Integration 6.x although it is included in the soon-to-be-released ServiceNet 6.5. So, what do you do when you need to expose IS services as web services and protect those services using WS-Security?

From the abstract of the WS-Security Soap Message Security specification, WS-Security “describes enhancements to SOAP messaging to provide message integrity and confidentiality” and “provides a general-purpose mechanism for associating security tokens with message content”.

Message integrity and confidentiality are accomplished by digitally signing or encrypting portions of the soap message. These goals are accomplished through the use of XML Digital Signature and XML Encryption respectively.

The goal of supporting security tokens is addressed in the Username Token Profile, X509 Certificate Profiles and SAML Token Profile respectively.

These profiles describe the correct syntax for including one of these tokens in the soap header as well as the rules for how the tokens are to be processed. Because the WS-Security framework is extensible, you can also define and specify other types of security tokens to suit your specific requirements.

The initial phase of my current project requires support of UsernameTokens. This type of token provides a method of including username and password information in the soap header such that the password is never sent in clear text. Use of Username Tokens also provides countermeasures against replay attacks.

Because webMethods IS does not provide support for headers in SoapRPC messages, you are limited to using the Document/Literal style to create IS web services which support WS-Security. I have written about ways to simplify creation of IS document/literal web services before. Basically, adding WS-Security support involves creating a custom soap processor that understands how to extract the wsse:Security element from a soap header, extract the security token from that element and execute the appropriate service to authenticate the token.

Because SAML tokens can also contain assertions about resources the user is entitled to access or invoke, it can also be used to determine whether the (now authenticated) user is authorized to invoke the service specified in the body of the soap message.

Since IS 6.5 has shipped without WS-Security support and the next major release is still months away, more and more companies will need to find an approach for adding WS-Security support to IS 4.6 and 6.x. This approach requires a bit of careful design and some up front architecture development, but, once implemented, provides the enhanced web services security with almost no impact to developers of IS web services.

Mark

Comments No Comments »

Recently, I have been working on a project that requires a standalone UDDI server, specifically the one that is included with webMethods Glue 5.02.

The Glue implementation of UDDI is quite nice and allows for the directory to be persisted either in XML files or in the JDBC database of your choice.

I’m finding UDDI V2, though, to be overly complex for simple things due to its focus on flexibility. There is also very little out of the box that seems to help categorize web services other than by industry codes. I guess that’s a relic of UDDI’s early focus on public web services directories.

UDDI V3 offers greater functionality in terms of defining custom taxonomies to classify web services. Since Glue 5.x doesn’t yet support UDDI V3, I’ll have to wait until the next release of Fabric… er, I mean ServiceNet, to get into the details (or download an eval version of a competitor’s product, I suppose).

UDDI V2: harder to work with than a real database, not as flexible as you need it to be.

Comments No Comments »

Graham’s talk provided a bit of history for SOA and building new applications (composite applications) from “parts” comprised of existing services.

He stated that the interesting question is not whether shared infrastructure services should be provided, but in which layer they should reside. The application server was originally a good place to do this, but unless an enterprise is using a single vendor and single version, it is difficult or impossible to provide infrastructure services for SOA in an app server container. Graham suggests that the appropriate place for these services to live is in the services bus itself. Commonly cited shared infrastructure services are security, transformation, failover and versioning.

The second half of Graham’s presentation focused on future directions for the product. I have to reformat those notes before posting.

Comments No Comments »