Skip to content

SadService

Jump to bottom Edit New page
akil-rails edited this page Nov 1, 2014 · 9 revisions

The SadService demostrates transaction management and failure processing in the broker.

Setting It up

This sample requires 2 ODBC datasources SAMPLE & SAMPLE_AUDIT. It has been tested on Oracle, but should work with all databases.
The sample needs one table AUDITRECORDS to be present in the database. The flow inserts records in this table. The data design project is available to run the DDL directly against the database.
Use SOAPUI Toolkit to load the WSDL , the URL specifier is http://host:port/SadService/SadService.

TimeOut processing:

Running the Sample

The timeOut operation waits for a given number of seconds and then replies back.
The wait is implemented via a ESQL SLEEP statement.
The sleep interval is passed as an input parameter (timeoutIntevalInSeconds) to the service.

Note that the SOAPInput node is configured for 5 second maximum client wait time, what this means is the following:

  1. when you invoke the operation with a timeout less than 5 seconds, the flow completes successfully.
  2. when you invoke the operation with a timeout more than 5 seconds, the SOAPInput node returns a timeout message.
  3. when you invoke the operation with a timeout equal to 13 seconds, the SOAPInput node returns a different timeout message (as the timeout flow fails to complete in a reasonable time

The message flow

There are 3 compute nodes in the flow, each on inserts a record in the AUDITRECORDS table. Each nodes inserts the entire message as a bitstream, along with the timestamps and a tag that's specific to the node. There is 1 compute node attached to the Timeout terminal, this also inserts a record in the AUDITRECORDS table with a different tag.

  • 2 nodes are configured with transation mode = commit. This means that they will always issue a commit.
  • 1 node is configured transaction mode - automatic. This means that this node will delegate its commit to the message flow.
  • The commit of the message flow is controlled by the Input node, and this node issues a rollback in case of a timeout, and a commit in case of a successful completion of the message flow within the stipulated time.

Now, when you run the sample you'll see that:

  1. in case of a timeout, 3 records are inserted in the AUDITRECORDS table, these records are the ones put by the nodes that have transaction = commit, and one by the node attached to the timeout terminal.
  2. in case of a success, 3 records are inserted in the AUDITRECORDS table, as the message flow commits, thereby committing the data inserted by the node with transaction mode = automatic.
  3. in case of a timeout processing failure (when the sleepInterval is passed as 13), 2 records are inserted in the AUDITRECORDS table, these are the ones put by the nodes that have transaction = commit.

Takeaways

  1. A timeout event does not abort the execution of the message flow prematurely, the message flow always runs its entire course.
  2. The timeout sends a reply to the client as a separate thread (you can see in this the trace)
  3. After a timeout, the Input node will rollback the message flow (instead of committing), and any nodes that have transaction=automatic will rollback.
  4. The message flow will infect try and send a reply after its completion, but will fail with BIP3745E, Invalid Identifier, as a reply has already been sent.
  5. Nodes can be attached to the timeout terminal of the Input node, these nodes get executed in a separate thread and a transaction from the main flow. This means that you can create an audit of only the messages that are timeout if you so wish.
  6. Just like the main flow, the nodes attached to the timeout terminal also observe the transaction property (either they participate in the transaction of the flow or they can commit by themselves).
  7. The timeout flow does not have a configurable timeout, the broker will time out in a maximum of 10 seconds and return a SOAP fault if the timeout flow takes an unreasonable time, so make sure this flow is short.

For the curious one

The reply sent by the Input node can be tailored by a compute node connected to the Timeout terminal.

The default message

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
         <faultcode>SOAP-ENV:Server</faultcode>
         <faultstring>A timeout occurred during processing</faultstring>
         <faultactor>/SadService/SadService</faultactor>
         <detail>
            <text>Timeout. Broker IB9NODE did not provide a response within the specified time interval (5 seconds). The message reached the main flow, but timed out during processing.</text>
         </detail>
      </SOAP-ENV:Fault>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The tailored message. Here you'll see that the original message has been added as a second text element in the detail. This is done by a compute node that's attached to the Timeout terminal. (the original message is made available in the LocalEnvironment by the broker).

<NS1:Envelope xmlns:NS1="http://schemas.xmlsoap.org/soap/envelope/">
   <NS1:Body>
      <NS1:Fault>
         <faultcode>SOAP-ENV:Server</faultcode>
         <faultstring>A timeout occurred during processing</faultstring>
         <faultactor>/SadService/SadService</faultactor>
         <detail>
            <text>Timeout. The Broker IB9NODE did not provide a response within the specified time interval (5 seconds). The message reached the main flow, but timed out during processing. It was then given to the the timeout flow for processing.</text>
            <text>
               <inboundMessage>
                  <XMLNSC>
                     <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:sad="http://SadService">
                        <soapenv:Header/>
                        <soapenv:Body>
                           <sad:timeOut>
                              <sleepIntervalInSeconds>12</sleepIntervalInSeconds>
                           </sad:timeOut>
                        </soapenv:Body>
                     </soapenv:Envelope>
                  </XMLNSC>
               </inboundMessage>
            </text>
         </detail>
      </NS1:Fault>
   </NS1:Body>
</NS1:Envelope>

So what happens in case the nodes that are connected to the Timeout terminal also timeout ? Well, the broker handles this case as well, it waits a maximum of 10 seconds, and if no response is sent by the timeout flow, then it sends a reply by itself, this message is easy to identify as its descriptive as follows.

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
         <faultcode>SOAP-ENV:Server</faultcode>
         <faultstring>A timeout occurred during processing</faultstring>
         <faultactor>/SadService/SadService</faultactor>
         <detail>
            <text>Timeout. Broker IB9NODE did not provide a response within the specified time interval (5 seconds). The message reached the main flow, but timed out during processing. It was then processed by the timeout flow, but timed out again during processing.</text>
         </detail>
      </SOAP-ENV:Fault>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Catch Flow:

Running the Sample

The catch operation returns a soap fault. The soap fault is constructed by a compute node in the catch flow.

Takeaways

  1. Exceptions should reach the Catch terminal of the Input node for a rollback to be issued. This is really the only way. There is no other way to issue a rollback at the flow level.
  2. Once an exception reaches the Input node, it will rollback and will send a reply to the client, but only if a reply has not already been sent.
  3. To send a custom message (like for example in case of a REST API, or if you want to hide the detailed soap fault message from the client), you will need to create a catch flow. In this catch flow you need to include a Throw Node to ensure that a rollback is issued, and you should do that AFTER you have sent the reply via a Reply node.
  4. The default flow structure generated by the Integration Service does not make it easy to include custom messages for Soap Faults, as the Catch sub flow attaches to a common SoapReply, you need to change this to include an additional SoapReply node for the catch flow, and Throw after the SoapReply node to ensure a rollback.

For the curious one

Following the default SOAP fault, notice the detailed information included in the fault.

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
       <soapenv:Fault>
          <faultcode>soapenv:Server</faultcode>
          <faultstring>BIP3113E: Exception detected in message flow gen.SadService.SOAP Input (broker IB9NODE)</faultstring>
          <detail>
             <Text>BIP2230E: Error detected whilst processing a message in node 'gen.SadService.gen/catch_Request_Response.Throw'. 
 The message broker detected an error whilst processing a message in node 'gen.SadService.gen/catch_Request_Response.Throw'. An exception has been thrown to cut short the processing of the message. 
 See the following messages for details of the error. : /build/slot2/S000_P/src/DataFlowEngine/SQLNodeLibrary/ImbComputeNode.cpp: 497: ImbComputeNode::evaluate: ComIbmComputeNode: gen/SadService#FCMComposite_1_12.gen/catch_Request_Response#FCMComposite_1_1
 BIP2488E:  ('gen.catch_Request_Response_Compute.Main', '3.3') Error detected whilst executing the SQL statement ''THROW EXCEPTION  SEVERITY 3 MESSAGE 3001 VALUES( 1, 2, 3, 4, 5, 6, 8);''. 
 The message broker detected an error whilst executing the given statement. An exception has been thrown to cut short the SQL program. 
 See the following messages for details of the error. : /build/slot2/S000_P/src/DataFlowEngine/ImbRdl/ImbRdlStatementGroup.cpp: 666: SqlStatementGroup::execute: : 
 BIP3001I: Exception thrown by throw node '2'; text is '1'. 
 The throw node '2' has received a message and thus has thrown an exception as this is its normal behavior.  The message text associated with this exception is '1'. 
 Since this is application generated (by message flow behavior), the user action is determined by the message flow and the type of exception generated. : /build/slot2/S000_P/src/DataFlowEngine/ImbRdl/ImbRdlThrowExceptionStatements.cpp: 238: SqlThrowExceptionStatement::execute: ComIbmComputeNode: gen/SadService#FCMComposite_1_12.gen/catch_Request_Response#FCMComposite_1_1</Text>
          </detail>
       </soapenv:Fault>
    </soapenv:Body>
 </soapenv:Envelope>

And following is the custom error message thats returned back (including a rollback) from the catch flow. This also demonstrates parsing the ExceptionList in 2 different ways.

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
       <soapenv:Fault>
          <faultcode>Client</faultcode>
          <faultactor>Intentional Error</faultactor>
          <faultstring>Demostration Purposes Only</faultstring>
          <detail>
             <Message>This is a custom error to demostrate a transaction rollback with a custom message.</Message>
             <Exceptions>
                <Exception>
                   <Number>2488</Number>
                   <Text>Error detected, rethrowing</Text>
                </Exception>
                <Exception>
                   <Number>3001</Number>
                   <Text>1</Text>
                </Exception>
             </Exceptions>
             <ExceptionList>
                <RecoverableException>
                   <File>/build/slot2/S000_P/src/DataFlowEngine/SQLNodeLibrary/ImbComputeNode.cpp</File>
                   <Line>497</Line>
                   <Function>ImbComputeNode::evaluate</Function>
                   <Type>ComIbmComputeNode</Type>
                   <Name>gen/SadService#FCMComposite_1_12.gen/catch_Request_Response#FCMComposite_1_1</Name>
                   <Label>gen.SadService.gen/catch_Request_Response.Throw</Label>
                   <Catalog>BIPmsgs</Catalog>
                   <Severity>3</Severity>
                   <Number>2230</Number>
                   <Text>Caught exception and rethrowing</Text>
                   <RecoverableException>
                      <File>/build/slot2/S000_P/src/DataFlowEngine/ImbRdl/ImbRdlStatementGroup.cpp</File>
                      <Line>666</Line>
                      <Function>SqlStatementGroup::execute</Function>
                      <Type/>
                      <Name/>
                      <Label/>
                      <Catalog>BIPmsgs</Catalog>
                      <Severity>3</Severity>
                      <Number>2488</Number>
                      <Text>Error detected, rethrowing</Text>
                      <Insert>
                         <Type>5</Type>
                         <Text>gen.catch_Request_Response_Compute.Main</Text>
                      </Insert>
                      <Insert>
                         <Type>5</Type>
                         <Text>3.3</Text>
                      </Insert>
                      <Insert>
                         <Type>5</Type>
                         <Text>THROW EXCEPTION  SEVERITY 3 MESSAGE 3001 VALUES( 1, 2, 3, 4, 5, 6, 8);</Text>
                      </Insert>
                      <UserException>
                         <File>/build/slot2/S000_P/src/DataFlowEngine/ImbRdl/ImbRdlThrowExceptionStatements.cpp</File>
                         <Line>238</Line>
                         <Function>SqlThrowExceptionStatement::execute</Function>
                         <Type>ComIbmComputeNode</Type>
                         <Name>gen/SadService#FCMComposite_1_12.gen/catch_Request_Response#FCMComposite_1_1</Name>
                         <Label>gen.SadService.gen/catch_Request_Response.Throw</Label>
                         <Catalog>BIPmsgs</Catalog>
                         <Severity>3</Severity>
                         <Number>3001</Number>
                         <Text>User generated exception</Text>
                         <Insert>
                            <Type>2</Type>
                            <Text>1</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>2</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>3</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>4</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>5</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>6</Text>
                         </Insert>
                         <Insert>
                            <Type>2</Type>
                            <Text>8</Text>
                         </Insert>
                      </UserException>
                   </RecoverableException>
                </RecoverableException>
             </ExceptionList>
          </detail>
       </soapenv:Fault>
    </soapenv:Body>
 </soapenv:Envelope>

Failure Flow:

Running the Sample

The failure operation returns a soap fault, in case an invalid XML (not complaint with the schema) is sent as the input.
This is done by turning on schema validation on the SOAPInput node and wiring up the Failure terminal.

Takeaways

  1. Messages are routed to the failure terminal of the node that generates the exception.
  2. Once an exception reaches the Input node, it will rollback and will send a reply to the client, but only if a reply has not already been sent.
  3. To send a custom message (like for example in case of a REST API, or if you want to hide the detailed soap fault message from the client), you will need to create a failure flow. In this failure flow you need to include a Throw Node to ensure that a rollback is issued, and you should do that AFTER you have sent the reply via a Reply node.
  4. The default flow structure generated by the Integration Service does not make it easy to include custom messages for Soap Faults, as the failure sub flow attaches to a common SoapReply, you need to change this to include an additional SoapReply node for the catch flow, and Throw after the SoapReply node to ensure a rollback.

For the curious one

Following the default SOAP fault, notice the detailed information included in the fault.

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
       <soapenv:Fault>
          <faultcode>Client</faultcode>
          <faultactor>Intentional Failure</faultactor>
          <faultstring>Demostration Purposes Only</faultstring>
          <detail>
             <Message>This is a custom failure to demostrate a transaction rollback with a custom message.</Message>
             <Exceptions>
                <Exception>
                   <Number>3749</Number>
                   <Text>Error occurred in ImbSOAPInputHelper::validateSOAPInput()</Text>
                </Exception>
                <Exception>
                   <Number>5902</Number>
                   <Text>Exception whilst parsing</Text>
                </Exception>
                <Exception>
                   <Number>3614</Number>
                   <Text>problem creating SOAP tree from bitstream</Text>
                </Exception>
                <Exception>
                   <Number>5009</Number>
                   <Text>XML Parsing Errors have occurred</Text>
                </Exception>
                <Exception>
                   <Number>5025</Number>
                   <Text>A schema validation error has occurred while parsing the XML document</Text>
                </Exception>
             </Exceptions>
             <ExceptionList>
                <RecoverableException>
                   <File>/build/slot2/S000_P/src/DataFlowEngine/MessageServices/ImbDataFlowNode.cpp</File>
                   <Line>1153</Line>
                   <Function>ImbDataFlowNode::createExceptionList</Function>
                   <Type>ComIbmSOAPInputNode</Type>
                   <Name>gen/SadService#FCMComposite_1_1</Name>
                   <Label>gen.SadService.SOAP Input</Label>
                   <Catalog>BIPmsgs</Catalog>
                   <Severity>3</Severity>
                   <Number>2230</Number>
                   <Text>Node throwing exception</Text>
                   <RecoverableException>
                      <File>/build/slot2/S000_P/src/WebServices/WSLibrary/ImbSOAPInputNode.cpp</File>
                      <Line>1107</Line>
                      <Function>ImbSOAPInputNode::validateData</Function>
                      <Type>ComIbmSOAPInputNode</Type>
                      <Name>gen/SadService#FCMComposite_1_1</Name>
                      <Label>gen.SadService.SOAP Input</Label>
                      <Catalog>BIPmsgs</Catalog>
                      <Severity>3</Severity>
                      <Number>3749</Number>
                      <Text>Error occurred in ImbSOAPInputHelper::validateSOAPInput()</Text>
                      <ParserException>
                         <File>/build/slot2/S000_P/src/DataFlowEngine/MessageServices/ImbRootParser.cpp</File>
                         <Line>815</Line>
                         <Function>ImbRootParser::parseNextItem</Function>
                         <Type>ComIbmSOAPInputNode</Type>
                         <Name>gen/SadService#FCMComposite_1_1</Name>
                         <Label>gen.SadService.SOAP Input</Label>
                         <Catalog>BIPmsgs</Catalog>
                         <Severity>2</Severity>
                         <Number>5902</Number>
                         <Text>Exception whilst parsing</Text>
                         <Insert>
                            <Type>5</Type>
                            <Text>Root</Text>
                         </Insert>
                         <Insert>
                            <Type>5</Type>
                            <Text>SOAP</Text>
                         </Insert>
                         <Insert>
                            <Type>5</Type>
                            <Text>3c736f6170656e763a456e76656c6f706520786d6c6e733a736f6170656e763d22687474703a2f2f736368656d61732e786d6c736f61702e6f72672f736f61702f656e76656c6f70652f2220786d6c6e733a7361643d22687474703a2f2f53616453657276696365223e0a2020203c736f6170656e763a4865616465722f3e0a2020203c736f6170656e763a426f64793e0a2020202020203c7361643a6661696c3e0a2020202020202020203c696e707574313e3c2f696e707574313e0a2020202020203c2f7361643a6661696c3e0a2020203c2f736f6170656e763a426f64793e0a3c2f736f6170656e763a456e76656c6f70653e</Text>
                         </Insert>
                         <ParserException>
                            <File>/build/slot2/S000_P/src/WebServices/WSLibrary/ImbSOAPParser.cpp</File>
                            <Line>2136</Line>
                            <Function>ImbSOAPParser::createSoapShapedTree</Function>
                            <Type/>
                            <Name/>
                            <Label/>
                            <Catalog>BIPmsgs</Catalog>
                            <Severity>3</Severity>
                            <Number>3614</Number>
                            <Text>problem creating SOAP tree from bitstream</Text>
                            <ParserException>
                               <File>/build/slot2/S000_P/src/MTI/MTIforBroker/GenXmlParser4/ImbXMLNSCParser.cpp</File>
                               <Line>1066</Line>
                               <Function>ImbXMLNSCParser::parseLastChild</Function>
                               <Type/>
                               <Name/>
                               <Label/>
                               <Catalog>BIPmsgs</Catalog>
                               <Severity>3</Severity>
                               <Number>5009</Number>
                               <Text>XML Parsing Errors have occurred</Text>
                               <ParserException>
                                  <File>/build/slot2/S000_P/src/MTI/MTIforBroker/GenXmlParser4/ImbXMLNSCDocHandler.cpp</File>
                                  <Line>702</Line>
                                  <Function>ImbXMLNSCDocHandler::handleParseErrors</Function>
                                  <Type>ComIbmSOAPInputNode</Type>
                                  <Name>gen/SadService#FCMComposite_1_1</Name>
                                  <Label>gen.SadService.SOAP Input</Label>
                                  <Catalog>BIPmsgs</Catalog>
                                  <Severity>3</Severity>
                                  <Number>5025</Number>
                                  <Text>A schema validation error has occurred while parsing the XML document</Text>
                                  <Insert>
                                     <Type>2</Type>
                                     <Text>5004</Text>
                                  </Insert>
                                  <Insert>
                                     <Type>2</Type>
                                     <Text>2</Text>
                                  </Insert>
                                  <Insert>
                                     <Type>2</Type>
                                     <Text>5</Text>
                                  </Insert>
                                  <Insert>
                                     <Type>2</Type>
                                     <Text>18</Text>
                                  </Insert>
                                  <Insert>
                                     <Type>5</Type>
                                     <Text>cvc-complex-type.2.4.a: Expecting element with local name "input" but saw "input1".</Text>
                                  </Insert>
                                  <Insert>
                                     <Type>5</Type>
                                     <Text>/XMLNSC/http://schemas.xmlsoap.org/soap/envelope/:Envelope/http://schemas.xmlsoap.org/soap/envelope/:Body/http://SadService:fail</Text>
                                  </Insert>
                               </ParserException>
                            </ParserException>
                         </ParserException>
                      </ParserException>
                   </RecoverableException>
                </RecoverableException>
             </ExceptionList>
          </detail>
       </soapenv:Fault>
    </soapenv:Body>
 </soapenv:Envelope>
Add a custom footer
Add a custom sidebar

Clone this wiki locally