##Chapter 10 - SOAP Revisited ###Introduction We will implement our own SOAP clients and servers in this chapter. We will use a PHP SOAP library called NuSOAP (http://www.scottnichol.com/soap) that works with PHP4. Again, PHP5 has much better libraries available, but we are going to use gl which only has the pervious version. This exercise will give us more insight into how distributed systems are put together and the focus is not on programming. You will be given the programs and expected to make only very minor modifications. It will require some effort, however, to understand how the programs work and how all the pieces fit together, which is our goal. We will see in greater detail how the technology of XML web services fits with implementation technologies. SOAs that use SOAP-based web services consist of clients and servers that understand SOAP. The back-end processing for the servers must, of course, be implemented in some programming language within a library- supported framework that will connect the XML of the XML web services to those programs. We will be doing a simple SOA using PHP as the back-end language and the NuSOAP library as the connector piece in the architecture. Since we are only concerned with services to begin with, we will not use a presentation-oriented framework such as CI until the exercises. ###Installing NuSOAP NuSOAP is very easy to install. There are instructions in the introduction at the NuSOAP site and I have customized them here for our gl environment. - Put the zip file of the downloadable release (from SourceForge) in the www directory of your gl account. The easiest way to do this is to use the wget command. You can also scp it up to your account. - Use the unzip command to unzip the file. - Rename the long directory name to something shorter such as nusoap. - It is installed! It really only required copying the files to the server. You should now read the simple Hello, World tutorial at the NuSOAP site. We will now go over a different example that uses WSDL. ###Creating a Client in NuSOAP Listing 10.1 shows a NuSOAP client for the Shakespeare service we used in chapter 4. You can see the relevant documentation at the NuSOAP site under Programming with NuSOAP Using WSDL but all details are given below. Each program comment is further explained below: 1. The library is included in our program with the built-in require_once function. This function includes a file, but checks to see if it has already been included first. Since the page may be loaded multiple times, this saves steps. Note that the pathname to the library file assumes that this program is in the root of your nusoap directory. You can put it anywhere and update the path. 2. All the parameters for the SOAP must be put into a PHP array. Here we only have one parameter - the string for the line of Shakespeare, but there can be multiple parameters. 3. The NuSOAP library works just like the SoapUI program that we used in chapter 4. It reads the WSDL and creates the SOAP client object automagically from that description. The boolean true parameter just means that we are using WSDL. 4. Now we can use that soapClient object's call method (recall that the arrow notation references an object's methods and properties) with parameter arguments to create the XML SOAP (#5-8 below). 5. Argument1 is the GetSpeech method along with the other parameters to needed to create the XML SOAP. 6. Argument2 is $request which holds the array of parameters for the SOAP message (see #2 above). In this case, it is the string- typed line of Shakespeare. 7. Argument3 is the namespace for the requested web service. 8. Argument4 is the soapaction for the HTTP header. 9. The code is now complete for sending a SOAP message to the Shakespeare web service. The rest of the code is to print out the actual SOAP request and response messages created by NuSOAP and the Shakespeare web service respectively for debugging purposes. Note that the print_r PHP function prints human- readable information about a complex variables so in this case, we do not have to iterate through the array programmatically to see it. 10. The request and response properties of the soapClient object are printed to the browser using the PHP function htmlspecialchars that will convert special characters to HTML entities (such as <) for display. Lookup this function at php.net to see what the ENT_QUOTES parameter does. So we can see that NuSOAP is an object-oriented library that handles creating an XML SOAP message for us (from the WSDL) and sending it to the web service. We see the request SOAP created by this SOAP client in listing 10.2 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> <title>PHP NuSoap Shakespeare Client</title> </head> <body> <h3>PHP program to access the Shakespeare web service.</h3> < ?php require_once('lib/nusoap.php'); // 1. pull in nusoap library $request = array('Request' => 'Winter of our discontent '); // 2. create an array $wsdl="http://www.xmlme.com/WSShakespeare.asmx?wsdl"; $soapClient = new soapclient($wsdl, true); // 3. create a soapClient object $result = $soapClient->call( // 4. use the call method from the soapClient object 'GetSpeech', // 5. the method name from the shakespeare web service $request, // 6. the Request tag parameter for the method defined above 'http://xmlme.com/WebServices', // 7. the namespace 'http://xmlme.com/WebServices/GetSpeech' // 8. the soapaction ); echo '<h2>Shakespeare Web Service Response from a PHP Client</h2>'; echo '<h2>PHP Array Response</h2>'; print_r($result); // 9. php function that displays an array variable // 10. Display the request and response soap with 'htmlspecialchars' echo '<h2>Request</h2>'; echo '<pre>' . htmlspecialchars($soapClient->request, ENT_QUOTES) . '</pre>'; echo '<h2>Response</h2>'; echo '<pre>' . htmlspecialchars($soapClient->response, ENT_QUOTES) . '</pre>'; ?> </body> </html> Listing 10.1. The NuSOAP shakespeare client. POST /WSShakespeare.asmx HTTP/1.0 Host: www.xmlme.com User-Agent: NuSOAP/0.7.3 (1.114) Content-Type: text/xml; charset=ISO-8859-1 SOAPAction: "http://xmlme.com/WebServices/GetSpeech" Content-Length: 475 < ?xml version="1.0" encoding="utf-8"?> <SOAP-ENV:Envelope xmlns:SOAPENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="http://xmlme.com/WebServices"> <SOAP-ENV:Body> <GetSpeech xmlns="http://xmlme.com/WebServices"> <Request>Winter of our discontent </Request> </GetSpeech> </SOAP-ENV:Body> </SOAP-ENV:Envelope> Listing 10.2. The SOAP request. We should notice a few things about this SOAP request: - It uses the POST method and the SOAP is contained in the HTTP entity body. - The HTTP User-Agent is NuSOAP - The SOAP body uses the namespace for a SOAP encoding. Recall that this is a depreciated way of doing things as RPC/encoded in the WSDL. The Shakespeare service is Document/literal and so notice that the soap encoding namespace is never actually used, it is just boilerplate. A Document/literal web service does require the third argument to call (#7 above - the namespace). We would need this server-side client for an HTML page we created for a user to access this web service. Recall that a request for a service is subject to the same domain rule for web browsers. A web browser cannot request the Shakespeare web service from the xmlme.com domain when the web page comes from the umbc.edu domain, but using the listing 10.1 program as a proxy client, it will work since server-side programs have no such restriction. Listing 10.3 shows such an HTML page. You can see working links to all these programs in the on-line syllabus for chapter 10. <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> <title>HTML form for Shakespeare web service</title> </head> <body> <h2>GetSpeech</h2> <p> GetSpeech requires a string formatted phrase from one of Shakespeare's plays as input. The speech, speaker, and play will be returned as an XML string. Sample Shakespeare Phrases: To be, or not to be | My kingdom for a horse | Get thee to a nunnery | ... </p> <h3>HTML Form for Shakespeare Web Service</h3> <form action="nusoap/shakes2.php" method="get"><!--1. http GET method --> Enter a line: <input type="text" size="40" name="sline" /><!--2. parameter --> <input type="submit" /> </form> </body> </html> Listing 10.3. The HTML page that calls the PHP client. Each program comment is further explained below: 1. The form action calls our program from listing 10.1 with a slight modification. It uses the HTTP GET method that will put the parameter in an HTTP querystring in the request URL. 2. There is an input tag that allows a user to type in a line from the displayed choices. Since the name of the input tag is sline, the request URL would be: `http://userpages.umbc.edu/~canfield/nusoap/shakes2php?sline=Et+tu%2C+Brute` (which should be all on one line). The funny characters in the URL for "Et tu, Brute" are URL encoding. The browser has replaced illegal URL characters such as space and comma with other characters specified in the W3C URL specification. A slight modification is required to the program in listing 10.1 to make this web application work. Note that in the program, the $request is hard coded. It will only work with that line 'Winter of our discontent'. We need it to work with any line submitted from the HTML form. So we must replace that line with the $request with: `$request = array('Request' => $_GET["sline"]);` This gets the parameter from the HTTP querystring that was created from the line entered in the HTML form. You can review the use of the $_GET function in the PHP tutorial at w3schools. A programming note about PHP is that this array is called an associative array or hash. This means that the indexes of the array are strings rather than integers. PHP uses the hash rocket notation (=>) to associate them. If you needed more than one parameter in the array because the SOAP has more than one, PHP uses a comma separated list of any length as below: < ?php $arr = array("foo" => "bar", 12 => true); echo $arr["foo"]; // prints bar echo $arr[12]; // prints 1 (for true) ?> ###Creating a Server in NuSOAP NuSOAP also allows one to create a server. This server will automatically generate the XML for the WSDL and for response SOAP messages. You can read the documentation at the NuSOAP site for creating a server with WSDL and we will detail another slightly more complex example here that we will use in the end of chapter exercises. The scenario for our case study will be an e-commerce system where wholesalers create SOAP servers to receive POs from retailers and return an invoice. Listing 10.4 shows the SOAP server code. The on-line syllabus has links for working code. Each program comment is further explained below: 1. A new object for the SOAP server is created with all its properties and methods from the NuSOAP library. 2. Stores the namespace in a variable for later use. 3. The server object creates a WSDL for the service. If you go to the URL for the service in a browser, you will see the WSDL XML document that is generated. That URL is just the one to this PHP file (cd.php) for this server code. 4. The register method creates the service according to its 8 parameters: - The name of the service (cdStore). - The array of input names and types (the ID of the cd and the quantity in this case). - The array of output names and types (a single string in this case). - The namespace for the service. - The soapaction for the HTTP header. - The document style for the WSDL (instead of rpc). - The use attribute for the WSDL is literal (instead of encoded). - The final parameter is just a documentation string for display. 5. A separate PHP file (cdx.php) is included that has the actual code for the service. See the top of the file. Listing 10.5 shows the cdx.php file with the code for the service. This is a very unrealistic service in the interest of keeping it simple for us to concentrate on the structure of the application. < ?php require_once("nusoap/lib/nusoap.php"); include('cdx.php'); $server = new nusoap_server; //1. creates a nusoap server object $ns = "http://userpages.umbc.edu/~canfield/cds"; //2. namespace //3. creates a WSDL automatically as document/literal $server->configureWSDL( 'cds', 'http://userpages.umbc.edu/~canfield/cds', '', 'document'); $server->register( 'cdStore', //4. creates a service array('cd' => 'xsd:string', 'quant' => 'xsd:int'), array('invoice' => 'xsd:string'), $ns, // namespace $ns.'#cdStore', // soapaction 'document', // style 'literal', // use 'Submit a PO for cdStore' // documentation ); // 5. cdx.php has the invoice function for cdStore $HTTP_RAW_POST_DATA = isset($HTTP_RAW_POST_DATA) ? $HTTP_RAW_POST_DATA : ''; $server->service( $HTTP_RAW_POST_DATA); //6. receives data ?> Listing 10.4. SOAP server (cd.php). Note the following about listing 10.5: - The catalog is just 3 hard-coded arrays. In real-life, of course, this would come from either a relational or XML database. - We are not updating the inventory. So if someone buys something, the inventory numbers are not decremented - very unrealistic. - This service simply uses the inputs of the ID of the cd and the needed quantity to check the 'database' and see if there are enough to meet the demand. If so, an 'Invoice' is generated with the total price in dollars. No code lines ending with semi-colons should actually wrap as some do in this book format. < ?php function cdStore($cd,$quant) { //array for price $price=array('cd1'=>12, 'cd2'=>9, 'cd3'=>11, 'cd4'=>15,'cd5'=>20); //array for artists $name=array('cd1'=>'Led Zepplin', 'cd2'=>'Buddy Miller','cd3'=>'AC/DC', 'cd4'=>'Neko Case', 'cd5'=>'Motorhead'); //array for inventory $stock=array('cd1'=>8, 'cd2'=>10, 'cd3'=>6, 'cd4'=>2, 'cd5'=>4); if ($quant <= $stock[$cd]) // calculate invoice $result="In Stock - ".$quant." ".$name[$cd]."@$".$price[$cd]." - Price= $" . $quant*$price[$cd]; else $result="Sorry, Out of Stock."; return array('invoice'=>$result); } ?> Listing 10.5. The SOAP service PHP code (cdx.php). The SOAP client (cdc.php) to access this web service is given in listing 10.6. It should be familiar from our previous discussion of SOAP clients. Retailers send 2 parameters to the wholesaler's SOAP server - the ID of the cd and the desired quantity. < ?php require_once( "nusoap/lib/nusoap.php" ); $ns="http://userpages.umbc.edu/~canfield/cds"; $client = new soapclient('http://userpages.umbc.edu/~canfield/cd.php?wsdl', true); $params=array("cd"=>"cd1", "quant"=> 2); // hard-coded $result = $client->call( "cdStore", array("parameters"=>$params), $ns); //echo print_r($result); echo "<h3>Invoice</h3>" . $result['invoice']; if ( $client->getError() ) { print "<h2>Soap Constructor Error:</h2><pre>".$client->getError()."</pre>"; } ?> Listing 10.6. The SOAP client for the cdStore service (cdc.php). We notice a few undesirable things about the code in listing 10.6: - The client parameters are hard-coded. This will not really work since we want to be able to order any of the CDs. We will remedy this in the exercises. - In a real B2B situation, we would want to be able to submit an XML PO to the wholesaler instead of just the values for a single cd. We would want to have a whole list of items as we saw in the shiporder PO from chapter 4. In the interests of simplicity, we will discuss this below but not fix it. - The output is a dynamic web page that shows the simple invoice information. This is fine for humans, but in real-life, we would want to get back an invoice XML document with a standard XMLSchema for processing on our own information system. We will also discuss, but not fix this issue below. We will leave the presentation of the service using CI as an exercise, but briefly outline the HTML solution here. We need to replace the hard- coded, line in listing 10.6 with a $_POST function that gets the parameters from an HTML form with action="POST". The HTML form would have 2 inputs in this case that would have different name attributes. Then we can simply replace the entries in the associative array with the relevant variables as we saw in our earlier Shakespeare example. `$params = array('cd' => $_POST["cdinput"], 'quant' => $_POST["qinput"]);` The HTML result for the hard-coded cd1 and quantity of 2 (see listing 10.6) is the following. You can try it from my account. `<h3>Invoice</h3>In Stock - 2 Led Zepplin@$12 - Price= $24` The simple web service in this case study is really adapted to humans with this kind of interface. This is fine for our toy system, as long as we understand the issues. We will refactor this system in the homework.  Figure 10.1. The purchase order scenario. Figure 10.1 shows a more realistic scenario for a purchase order to the cdStore. In many B2B use cases, large retailers would want to have computer programs generate XML POs perhaps from a just-in-time inventory system where the program is monitoring their inventory levels. These XML POs would be sent as document/literal SOAP as we saw in chapter 4. The responses would also be XML where the invoice uses a standard schema that computer programs can work with in the billing and payment system. Our little system is more of a C2B e-commerce system where a public customer or a small retailer uses a web form to order. In the more complex B2B case, the computer programs would have to parse the XML as a first step for input to the remainder of processing instead of getting all the parameters separately. All web services libraries have powerful functions for XML parsing. We will see XML parsing in the next chapter. The step numbers for the message flow in figure 10.1 are given below: 1. The client takes the purchase order information and packages it into a SOAP message. The format of the SOAP message is defined using a WSDL. The client could be a web browser for a human as in the figure or it could be a computer program with no human user. The SOAP message is sent to a web service endpoint using HTTP. 2. The web component transforms the SOAP message into a normalized message. The normalized message is sent to the normalized message router. A message router is a service given by the Enterprise Service Bus (ESB). Normalized means that the whitespace in a message has been standardized. Recall that XML retains whitespace as significant. For example, if given "<customer> John Smith </customer>", after normalization, it would be "<customer>John Smith</customer>" with the extra spaces removed. Normalization is also called canonicalization. The standard algorithm at the W3C is called C14n (there are 14 letters between C and N). 3. The normalized message router routes the normalized message to the BPEL service engine. 4. The BPEL service engine would handle billing, shipping, inventory, etc. using the information in the PO. 5. The BPEL service engine creates a response message in the form of a normalized message. The normalized message is sent to the normalized message router. 6. The web component receives the response message and converts it to a SOAP message. The SOAP message is sent back to the client as a proper response as defined by the WSDL. The ESB offers services that are common to many web serices on the same application server. Figure 10.2 shows the BPEL workflow. A received PO must be verified against a set of business rules that check purchase amounts, credit and payment history, etc. This verification is done automatically by the cdStore. After verification, the PO either begins the shipping process if in stock or it enters the back-ordered state. When the back-ordered items are received by the supplier, the order ships and the process is complete. Since these steps are largely automated and there are delays, the messages are sent asynchronously. There is a callback message when a step is completed. You can easily see how such a workflow can be modeled in BPEL XML and implemented in a back-end programming language framework in principle from our previous study. Listing 10.7 shows the SOAP message for the submission of the PO. It is very similar to the document-style message we saw in chapter 4. The headers for WS-Addressing have been added. The namespace for the PO would be defined with XMLSchema and referenced in the WSDL types section. The WSDL would also define the asynchronous operation in the portType section for the CreatePO operation that you see in the wsa:Action tag in the SOAP. There would be other operations besides CreatePO defined in a real web service such as GetPOStatus, UpdatePO, and CancelPO and each portType operation would be made up of defined messages. Listing 10.8 shows the SOAP response. Note that there is po:StatusURL tag that offers a RESTful interface to getting a status update on the PO. This could be implemented as an RSS feed as we discussed earlier.  Figure 10.2. BPEL workflow. < ?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"> <soap:Header> <wsa:MessageID>889923-1</wsa:MessageID> <wsa:To>http://userpages.umbc.edu/~canfield/po_service</wsa:To> <wsa:Action>http://userpages.umbc.edu/~canfield/po/CreatePO</wsa:Action> <wsa:From> <wsa:Address>http://userpages.umbc.edu/~customer</wsa:Address> </wsa:From> </soap:Header> <soap:Body> <shiporder orderid="889923" xmlns:po="http://userpages.umbc.edu/~canfield/po"> <po:orderperson>John Smith</po:orderperson> <po:shipto> <po:name>Ola Nordmann</po:name> <po:address>Langgt 23</po:address> <po:city>4000 Stavanger</po:city> <po:country>Norway</po:country> </po:shipto> <po:item> <po:title>Empire Burlesque</po:title> <po:note>&lt; Special Edition &gt;</po:note> <po:quantity>100</po:quantity> <po:price>10.90</po:price> </po:item> </po:shiporder> </soap:Body> </soap:Envelope> Listing 10.7. SOAP document for cdStore scenario. <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://www.w3.org/2005/08/addressing" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <soapenv:Header> <wsa:MessageID>889923-2</wsa:MessageID> <wsa:To> http://userpages.umbc.edu/~customer</wsa:To> <wsa:RelatesTo>889923-1</wsa:RelatesTo> </soapenv:Header> <soapenv:Body> <po:OrderInfo xmlns:po="http://userpages.umbc.edu/~canfield/po"> <po:OrderID>889923</po:OrderID> <po:OrderDate>2008-11-11T21:32:36.718-05:00</po:OrderDate> <po:OrderPrice>100</po:OrderPrice> <po:OrderStatus>Verified</po:OrderStatus> <po:LastUpdate>2008-11-11T21:32:31.700-05:00</po:LastUpdate> <po:StatusURL> http://userpages.umbc.edu/~canfield/po/GetPOStatus </po:StatusURL> </po:OrderInfo> </soapenv:Body> </soapenv:Envelope> Listing 10.8. SOAP response So we should remember that the W3C describes a Web Service as "a software system designed to support interoperable machine-to-machine interaction over a network". They allow a connection to be established between two computers regardless of operating system or programming language. As long as both sides recognize the XML protocol used, they can communicate. There can be a human involved or not. ###Chapter 10 Exercises Do the end-of-chapter exercises for each chapter of the book by following the link in the on-line syllabus.