SOAP and REST communication tools in PS CLEMENTINE PRO 3.0

Amongst the multiple functionalities available in PS CLEMENTINE PRO are PS-developed nodes that enable prompt and easy communication with network services. Based on REST and SOAP, they are widely applied both when creating internal processes and when integrating with external services. They are equipped with a graphic interface allowing them to be used without having command of new programming languages. 

In this article you will learn how to communicate with network services with the help of PS REST and PS SOAP nodes, and you will see several sample applications in a business environment. In subsequent articles we will also cover: 

• Building scoring processes in the API era - when, which and how much 'mobility' we need
• How to smartly run an SMS text message campaign with the use of PS CLEMENTINE PRO 3.0
• How to smartly build a report portal using PS IMAGO PRO and PS CLEMENTINE PRO

For both technologies three types of nodes have been designed: source, transition and export. As a result, complete communication with web services equipped with APIs based on SOAP or REST is possible. Let’s start with the source nodes.

Figure 1. Source, Process and Export Nodes for SOAP and REST communication

Collecting data from web services: How to use PS REST and PS SOAP source nodes

REST and SOAP are amongst the most popular approaches to collecting data from web services. The choice of the right one depends on how the network service is designed. These services can be free or for fee offering data on diverse topics from financial, stock exchange, and government data, to weather information, language translations, data about flights, hotels… even random jokes about Chuck Norris! 

Amongst the free data sources are current data on the COVID-19 pandemic in different countries worldwide. In this pandemic period we find ourselves in, current data are very important for many companies and institutions and can be critical to an organization. The sample node in Fig. 2 collects the epidemic’s development history in Poland . The data are collected from https://covid19-api.org/ which provides a free REST API. As you can see from Fig. 2, the PS REST node configuration can be very easy and often requires only very basic knowledge about the REST architectural style. When it is necessary to add more information to the node tabs, this is often provided in the documentation of the network service you want to use. The table in Fig. 3 displays the data resulting from the PS REST node.  

Figure 2.  Data resulting from PS REST source node: COVID19 rates

Figure 3. Data resulting from PS REST source node: COVID19 rates 

Node configuration is slightly more complicated when we want to collect data with the help of a message in the SOAP protocol. SOAP messages (envelopes) are created using the XML tag language which requires a certain knowledge beforehand. Many services such as those provided by public administration, are based solely on the SOAP protocol, so it is worthwhile learning more about it. More details are available, among others, on the w3c site (https://www.w3.org/).

The sample PS SOAP source node in Fig. 4 collects data about the average exchange rate of the US Dollar in March 2014 from the service of the Bank of Lithuania.

Figure 4. Sample PS SOAP source node:  Currency rates

Using dynamically created queries in processes

The source nodes discussed above offer the possibility to collect data using specific fixed parameters such as site address or content of the SOAP envelope. But what if you want the value of the parameters to be dependent upon the data you receive in the process? 

The possibility to dynamically modify queries sent to network services is offered by PS REST and PS SOAP transition nodes. With them, you can download e.g. data for today's date, or for countries selected in the earlier data processing process. In this way, we can modify service addresses and many other node parameters that need to be determined. 

Within the process we can create setting values that we previously entered as "fixed" to the source nodes – and in this way restore the results received earlier. However, these nodes have an additional mode under which they allow sending a separate request for each received record. This means that in each record, we can provide to the node a set of parameters with which we will configure the SOAP or REST request being sent. For each of these requests we will receive return values which will also be returned in particular records. One request – one response. This provides significantly greater flexibility in the approach to the procedures of data exchange with web services being created. For instance, returning to the service that provides data on COVID-19, using one node we can collect data for Poland, Germany and Czechia for three selected days. To achieve this we create multiple URL addresses from which the resources will be collected (Fig. 5). In the case of using this node it should be remembered to determine the output data model on the respective tab.

Figure 5.  Sample input list for PS REST transition node settings

The effect of the actions is visible in Fig. 6 – number of reported cases, deaths and recoveries for Poland, Germany and Czechia between 1st and 3rd November 2020 – data which is now ready for further processing within the process.

Figure 6.  Data resulting from PS REST transition node: COVID19 rates

In Fig. 5, apart from the country and date variables, there is column "Method", containing "GET" in each record. This setting determines the type of request (method) that we send to the particular service. In addition to "GET", we can use "POST" (sending new data to the service), "PUT" (usually in order to change/update already present data) and "DELETE" (removing data contained in the service). As you can see, a PS REST transition node can not only collect data from external services, but also share, change or remove from the sites any data that we create ourselves. This enables complete cooperation with network services.  Similarly, the PS SOAP transition node can be used if the service you want to cooperate with is designed with the SOAP protocol. This will require adaptation to the particular service and proper configuration of the SOAP envelope.

Output PS REST/PS SOAP nodes - application

PS REST and PS SOAP nodes that we refer to as "output" or "export" to some extent operate similarly to transition nodes: 

1. They allow operations in two available modes – sending a single request or separate requests for each provided record. 
2. The content of the messages sent can depend on the data received (variables are responsible for setting particular node properties), but does not have to – the message can be set statically and be sent in the same form, regardless of the data received from the previous data node. 

The main difference compared to transition nodes is that export nodes are terminal nodes. This means that they are used only to send data/messages, and not to receive them. It is impossible to collect data using an output node, meaning in the response we receive there is no information we can process further. For this reason, we usually use these nodes to: 

• send various kinds of orders to internal/external systems (e.g. to activate certain processes in such systems), 
• send data to the aforementioned systems,
• modify the resources we have made available in the network service (add new resources, edit or delete the present ones).

The applications of these actions are very broad and we will come back to them in subsequent articles on our blog. The one that we will present today is sending an activation request for a job created in PS CLEMENTINE Manager. At the same time, it is an example that can be used to activate other external processes, based on REST or SOAP API. 

Let's assume that we have created a task that collects data about the COVID-19 epidemic from the previously mentioned service for the selected country and a range of dates, and then enters these data to a database. The country and the range of dates for data collection are defined by the job parameters. We can activate and manually configure such a task, but we can also run it with the help of a SOAP or REST message. In this way, we can use such a task in a longer process by creating a stream which first determines the job activation parameters and then actually runs it. 

For instance: the stream that precedes the task  may be used to verify data correctness – check every day whether the data about daily cases in the database do not look "suspicious". If they meet the rules that suggest that selected records can be incorrect (e.g. we are observing negative growth in daily cases or we are observing the same growths a couple of times) then such records are selected, the date and the country are determined on their basis, and then the earlier mentioned task is activated, with the aim being to collect the currently available data again and enter them to the database. The task is started for the date and country parameters determined in the earlier process. Such action can be performed e.g. using PS SOAP output node by configuring the content of the envelope, and taking account of any data obtained from the previous node (Figure 7). 

Figure 6.  Sample SOAP output node: COVID19 rates

In subsequent entries we will discuss in more detail the launching of tasks in PS CLEMENTINE PRO 3.0 using web services.