Java and PHP interoperability – a web services fable

aesopfox.jpgBy definition Web Services (WS) have been designed to support interoperable Machine to Machine interaction over a network. In order to promote interoperability of Web Services, in 2002 the Web Services Interoperability Organization (WS-I) was founded which basically is an industry consortium that publishes profiles. A profile is a set of core specifications (SOAP, WSDL, etc.) in a specific version (SOAP 1.1, UDDI 2, etc.) with some additional requirements to restrict the use of the core specifications.

One of the three profiles you can actually find in their deliverables is the Basic Profile (official abbreviation is BP), which uses Web Services Description Language (WSDL) to enable the description of services as sets of endpoints operating on messages. Almost everybody is BP compliant: Glassfish Metro (JAX-WS RI), IBM WebSphere, Apache Axis 1.2+, JBossWS, ASP.NET 2.0.

The keyword here is “almost”…

My Project in a Nutshell

There is a team at my workplace that administers the Greek School Network Directory Service which currently contains more than 170,000 entries including school accounts for email and dialup access, teacher accounts and several student accounts. For various reasons that you can find at my colleague’s blog there was a need to perform LDAP operations over WSs – in fact we had to “Move LDAP writes to Web Services”.

The server-point

The server-point would actually perform the LDAP “writes” to the Directory Server and have a PHP WSs interface. For the PHP implementation, NuSOAP was chosen. NuSOAP is probably the most popular PHP SOAP implementation and is widely used, eg. in Amazon Web Services.

It is a group of PHP classes that allow developers to create and consume SOAP web services. It does not require any special PHP extensions. The current release version of NuSOAP at the time supports much of the SOAP specification. It can generate WSDL and also consume it for use in serialization. Both rpc/encoded and document/literal services are supported.

Here follows the skeleton of the PHP configuration for our deployment:


<?php

// The library
require_once('lib/nusoap.php');
// The services exposed
require_once('myinterface.functions.php');


$server = new soap_server();
$server->configureWSDL('Interface',"http://server/interface.php");
$server->wsdl->schemaTargetNamespace="http://server/interface.php?wsdl";

$server->register('ExistsUser',
      array('SubsystemID' => 'xsd:integer', 'UserNumber' => 'xsd:string'),
      array('Status' => 'xsd:integer', 
	  'ErrorMessage' => 'xsd:string', 'UserName' => 'xsd:string'),
      "http://server/interface.php",false,false,'literal',
      'ExistsUser will return Status=0 if user does not exist, 
	  Status=1 if user exists and set UserName accordingly, 
	  Status=-1 and set ErrorMeessage if an error occurs. It 
	  should be used to check if user exists before creating 
	  username. UserNumber is the number the user has in the 
	  subsystem. SubsystemID is the numerical ID of the subsystem');

$server->service($HTTP_RAW_POST_DATA);

?>

<Spoiler/>

NuSOAP was not BP compliant until a few days ago when CVS revision 109 of nusoap.php was introduced. The motivation for this article was the hard way we had to find out…

The client-point

The system my team was responsible for was the client which would have to wrap the WS related code into Java POJOs that would either be used as library code or through portlets. Implementing the functionality we needed in Jetspeed portlets was another issue but it goes beyond the scope of this post.

For the Java WS framework there were several candidates like Apache Axis and JAX-RPC but we choose JAX-WS 2.1, both because of its elegant programming model and the fact that in the newly published Java EE 6 proposal JAX-RPC will be proposed for future deprecation.

Our development platform was NetBeans 5.5 which provided a powerful wizard that starting from the WSDL that NuSOAP published, created the necessary Java stub code for our operations.

This is a small portion of the WSDL for the operation ExistsUser:


…
<message name="ExistsUserRequest">
<part name="SubsystemID" type="xsd:integer" />
<part name="UserNumber" type="xsd:string" />
</message>
<message name="ExistsUserResponse">
<part name="Status" type="xsd:integer" />
<part name="ErrorMessage" type="xsd:string" />
<part name="UserName" type="xsd:string" />
</message>
…
<operation name="ExistsUser">
<soap:operation soapAction="http://server/interface.php/ExistsUser" style="rpc"/>
<input>
<soap:body use="literal" namespace="http://server/interface.php"/>
</input>
<output>
<soap:body use="literal" namespace="http://server/interface.php"/>
</output>
</operation>
… 

Of course to debug Web Services you need tools like:

Of course the fact that the above development stack was found appropriate for our project doesn’t mean that it is suitable for every WS project. BTW a nice introduction on making similar technical decisions for Java projects can be found in the book Expert One-on-One J2EE Design and Development (AKA “The J2EE Bible”) by Rod Johnson in chapter 2 entitled “J2EE Projects: Choices and Risks”.

Holder <T> and Java’s heavy burden from C

Java Web Services make use of the Holder concept that was introduced since WSDL can specify interfaces that return multiple parameters (rather than just one single return value). In Java you can only return directly a single new object through a return value. To return multiple objects you have to return a collection object (which is still only a single object) or some other "container" object. To return a new object through a parameter, you need a holder object. Java’s object reference parameters only allow the modification of existing object instances.

Holders are necessary because the Java language designers stuck with the "pass-by-value" semantics of the C-family of languages. If a parameter is a simple type its value is copied, if it is an object instance the reference (value) to it is copied. As a result you are unable to modify the value in the original simple type or the reference (value) to the object from the called method (while you can change the passed instance, you cannot replace the passed instance). The holder mimics "pass-by-reference" semantics which in C/C++ is is done with a pointer-to-a-pointer and in C# directly through the ref keyword.

Some consider attempts to imitate call by reference in OO languages a sign of poor object-oriented design. Dale King points out that attempts to fake call by reference are usually a sign of poor object-oriented design: “a function should not be trying to return more than one thing”. He uses the term thing because it is proper to return more than one value (e.g. returning a Point that contains two values). If you are trying to return two values, the test he like to apply is whether you can come up with a logical name for the values as a group. If you can’t, you had better look to see if maybe you what you have is really two functions lumped together. WSDL never strived towards OO so that is where this mismatch comes from. It may have been a better idea for Java to wrap the WSDL method return value and all of the OUT parameters into a single wrapper object which becomes the return value of the Java interface method.

For our implementation we had the WS method:

@WebMethod(operationName = "ExistsUser", action = "http://server/interface.php/ExistsUser")
public void existsUser(
@WebParam(name = "SubsystemID", partName = "SubsystemID")
BigInteger subsystemID,
@WebParam(name = "UserNumber", partName = "UserNumber")
String userNumber,
@WebParam(name = "Status", mode = Mode.INOUT, partName = "Status")
Holder status,
@WebParam(name = "ErrorMessage", mode = Mode.OUT, partName = "ErrorMessage")
Holder errorMessage,
@WebParam(name = "UserName", mode = Mode.OUT, partName = "UserName")
Holder userName);

This method would return the status in the client:

BigInteger status = new BigInteger("0", 10);
Holder statusHolder = new Holder(status);
…
// Actual call of the WS
port.existsUser(subsystemID, userNumber, statusHolder, errorMessage, userName);
…
status = statusHolder.value
System.out.println(status);

If you fire up the JAX-WS wizard in NetBeans and feed it with the WSDL you get 100% of the necessary code for an “almost” working demo.

Again the keyword is “almost”…

Bumps on the Road

If you have Java SE 6 installed and start of with NetBeans and JAX-WS there is a big chance you’ll run into problems. Java 6 ships with JAX-WS 2.0 API in rt.jar (so as we had to find out) you need to download the latest stable version of JAX-WS (2.1.1) and copy jaxws-api.jar and jaxb-api.jar into JRE endorsed directory. Obviously you still need other JAX-WS jars in your classpath.

The above problem was relatively simple to solve compared to the fact that statusHolder.value would not alter after the WS invocation no matter what!

Although the WS would get invoked properly on the server and we would receive a SOAP response:


<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope …>
<SOAP-ENV:Body>
<ExistsUserResponse xmlns="http://server/interface.php">
<Status>1</Status>
<ErrorMessage></ErrorMessage>
<UserName>john</UserName>
</ExistsUserResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>


the @$%$# statusHolder.value would remain the same!

Weeks of debugging passed during which we had to dig into specs, JAX-WS RI code and forums to find out that:

  1. The SOAP response we got from NuSOAP was not compliant with BP since according to WS-I “An ENVELOPE described with an rpc-literal binding MUST place the part accessor elements for parameters and return value in no namespace.” In our case "Status" is in http://server/interface.php&quot; namespace instead of no namespace, and
  2. Since JAX-WS RI 2.1 would encounter a response not valid according to BP, would simply ignore it without even spawning an error message.

Number one (1) has been reported to NuSOAP mailing list and we can confirm that it been fixed in CVS revision 109 of nusoap.php that should now conform to the WS-I Basic Profile (thanks Scott).

Number two (2) has been filled as an “enhancement” of JAX-WS RI for the version 2.1.3: “Ignore rpc/lit part accessors’ namespace, there are quite a few implementations that do not obey BP”.

The Morale of the Story

At this point I cannot help but to quote Dave Podnar’s Five Stages of Dealing with Web Services:

  1. Denial – It’s Simple Object Access Protocol, right?
  2. Over Involvement – OK, I’ll read the SOAP, WSDL, WS-I BP, JAX-RPC, SAAJ, JAX-P,… specs. next, I’ll check the Wiki and finally follow an example showing service and client sides.
  3. Anger – I can’t believe those #$%&*@s made it so difficult!
  4. Guilt – Everyone is using Web Services, it must be me, I must be missing something.
  5. Acceptance – It is what it is, Web Services aren’t simple or easy.

 

 

Social Bookmarks:

Advertisements

15 thoughts on “Java and PHP interoperability – a web services fable

  1. I’d like to mention step 6 (as other have already done int the aforementioned blog):
    Nirvana – find out that using XMLRPC or REST is exactly as simple and effective as the hype that got your attention in the first place

  2. Knowing even more details about my project than can be described in a a blog, I have to agree with you that there would be no theoretical reason to go with a heavy-weight solution (XML-RPC could theoretically do fine). There would be though several practical ones like the fact that many of the implementations you mention require a larger up front commitment and risk and also the fact that until the last time I had seen the available PHP implementations, they were more of a POC than production SW. After your comment I will have to revisit these projects and see if they have evolved.

    • Webservices are also simple if you use java and java.;-)
      The purpose of webservices should be to communicate with other webservices, no matter what program language implemented them.

  3. Thanks for the detailed recount.

    The first SOAP implementation I wrote was based on a SOAP 0.9.something spec. The spec was about 4 pages. Its spirit and intention could be easily understood, and it was quite possible for a single person to write code that implemented virtually everything in it.

    Life was still pretty good with the 1.0 spec, the first submitted to W3C. However, as multiple people wrote implementations and tried to get them to work together, it became clear that the spec needed to be enhanced, and those who had worked on RPCs before knew that some sort of interface definition was necessary (which became WSDL).

    Unfortunately, at this point SOAP was hijacked by (1) lovers of all things XML and (2) lovers of big-bang distributed computing (namely DCE and DCOM technology).

    I was a developer on Apache SOAP, which implemented the SOAP spec, but not even WSDL, let alone the other specs that were already being envisioned by the people who gave birth to the Apache Axis project. As Microsoft, IBM, Sun, et al. implemented all the specs being generated, those of us who had worked on small SOAP implementations knew we had been outflanked and were fighting a losing battle. There was no way we could keep up.

    I subsequently needed SOAP for a PHP project, discovered NuSOAP, got to know Dietrich through the mailing list, contributed code, and became a developer on the project. NuSOAP aims about as high as a small part-time team can: implement as much of SOAP 1.1 and WSDL 1.1 as people really seem to need, react to new needs and issues as best as possible, and hope that Microsoft and the rest of them don’t stop supporting these earlier specs.

  4. @Jon

    Since the project was based on infrastructure that was already there (eg. Jetspeed portal) Java was the only solution, but even if I had to design the whole thing from scratch I would still make similar technical decisions.

    The reasons why one would go open source or Java for WS in 2007 should be self-evident 🙂

  5. Pingback: rascunho » Blog Archive » links for 2007-08-08

  6. @Jon

    You completely missed the purpose of web services. It’s a “simple” way to communicate data between interfaces (here languages) through HTTP (eliminating firewall issues, etc.). An entire application written on a J2EE platform, running an untold number of servlets isn’t (and shouldn’t) jump and buy a Windows box so they can “easily” implement web services with C#.

    Here are three reasons companies use Java are:

    1. It can run on Linux. (don’t scream Mono here, it’s only a good theory).
    2. It’s free.
    4. It’s more popular for web development

    When I’m writing a Windows desktop app, C# is my choice almost every time, but I’m specifically targeting Windows’ users.

  7. Pingback: Olkenava » Java and PHP interoperability - a web services fable

  8. You mentioned that a few days ago (now, two years ago :)), NuSOAP became compatible with WS-I BP 1.0.

    To me it seems that it still misses something: reading the WS-I BP 1.0 specifications:

    http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html#refinement16556272

    you can see that the way NuSOAP produces its WSDL in case you exchange an array (either as input or as output of your service), is exactly the INCORRECT way to codify arrays in WSDL (following WS-I).

    As I wrong? Do you have any idea about how to solve this issue? I’m facing traying to produce a webservice that could be consumed by a SAP installation, that only accepts WS-I BP 1.0 services…

    Thanks for the attention, and for the fable 🙂

  9. @Francesco in practice you don’t have to worry with 100% compliance. Most implementations (eg. JAX-WS RI) will not give you any problems for minor issues. Also common erros are usually supressed. If it is possible try a demo NuSOAPSAP WS to see if there is a real-world issue.

    Having said that I’m not sure if the NuSOAP project is still maintained. Maybe you should also take a look at WSO2 which seems to be carrying the torch these days.

    That’s my 0.02$

  10. @Dionysios

    Thanks for your 2 cents 🙂 I’m going to take a look to WS02, thanks for the hint.

    Unfortunately, IT IS a real-world problem, since when you try to generate a service which takes as input an array of structures, for example, you obtain a WSDL containing the following type description (or something similar):

    and the SAP system, on the other side, cannot generate (automatically) the proxy toward your service. Instead of that, you should obtain a WSDL type description similar to this:

    But then you have to “teach” the library to correct deserialize the incoming SOAP request coming from the SAP system.

    For this reason right now I’m modifying the NuSOAP library. Since I thought it could be of some interest to other developers I tried to contact Scott Nicholson, the mantainer, but at now I didn’t receive any reply. It would be helpful to test these modifications with him.

    BTW, if he does not reply me, probably I’ll move my architecture over a more updated library.

    Thanks for your interest 🙂

  11. Pingback: Amazon Web Services fera plus d’1 milliard de dollars en 2012 : Mistra Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s