Wednesday, July 12, 2017

The Messaging Engine failed to add a receive location

Receive Locations remained disabled, even after starting the application, even manual Enabling wasn't having any effect. This happened with newly deployed solution with couple orchestration having File based receive location. 

receive location disabled

When looked at event viewer- no errors, now this was puzzling - owing to this I rechecked the configurations,restarted the applications, host instances etc. After a while it clicked to me to have look at another machine event viewer (There are two BizTalk server as a part of Group)

And there I saw errors pointing about the receive locations not getting enabled.

receive location errors

Above errors were for second receive location as well.

Why it happened


As can be seen from error, it was clear that the file system was not accessible- but when I checked the folder, there was no restriction- so now it gets even more puzzling.

Answer to Puzzle - The dev env has two BizTalk server(Say A and B) in a group and the Host which I am using has instances on both the server whereas the folder which is configured as receive location exists on Server A. So when host instances on server B tries to reach out to the folder on local system i.e. Server B, it couldn't find it as it doesn't exist (It is on server A)
two host instances


What to do


As of now to continue with development I have stopped host instance of server B, thus only host instance on server A would be processing and it has reach to local file system.
one host instance disabled

If you encounter such error, following should be taken into considerations
1. The file system is accessible (uncheck the Read only )
2. If there are multiple BizTalk server then go for shared network folder as a choice for receive location.

Tuesday, July 4, 2017

BizTalk Developer getting started with Logic App

With the rapid adoption of cloud based services and with the advent of various types of cloud services serving/solving a need/problem there was/is a need to have a way to let the individual services be able to communicate to each other. Also to orchestrate a business flow with integrating various services, and this is where Microsoft introduced Logic Apps (Other IT leaders have too came up with their offerings)

Logic App is nothing but an Integration engine(tool) which resides on cloud, so now question might arise - yet another Integration tool. Already there is BizTalk Server which has been there for over a decade solving the need around integrating various applications, helping in automating the business flow and assisting B2B function properly in adherence to industry standards(EDI,SWIFT,HIPPA, HL7 etc).

True, both the offering from Microsoft at high level address the same area i.e., Integration, but BizTalk been on premises solution and Logic apps primarily to cater the integration needs of SAAS services, now getting more evolved with connectors getting added related to on premises applications, LOB etc.

The most fundamental difference is BizTalk is a product which you need to buy, procure a machine/s for it , install, build applications on it and you have to manage both applications and BizTalk server, whereas Logic App is a cloud based service for which you pay as you use (Platform as a service) and is managed for you by Microsoft.

Being a BizTalk Developer it is but obvious to correlate many things with Logic apps or compare the features available. It is my perception that Logic app is greatly influenced by BizTalk Orchestration, off course it is also a workflow so there will be:).



1. Native message format


BizTalk is XML based whereas Logic apps is JSON based, although both are capable of handling other formats as well like EDI, HIPPA, HL7, SWIFT etc (Logic App needs Enterprise Integration Pack)


2.Types of solution 



In BizTalk we have Messaging Solution(use only schema, maps and ports) and Orchestration(workflow) based solution whereas in Logic app we have only workflow.


3. Connecting to other systems/applications



In BizTalk it is via Adapters whereas in Logic apps it is via connectors. Connectors is encapsulation of authentication, data validation in combination of Triggers and actions. Technically all connectors are API apps that uses a metadata format named swagger, REST as pluggable interfaces and JSON as the data interchange format.


4. Instantiation of workflow



In BizTalk Message creates instance of Orchestration likewise trigger creates instance of Logic app.
Following are Trigger type available as of now
i. Request(endpoint) ii. Recurrence(schedule) iii. Http iv. ApiConnection v.HttpWebhook vi. ApiConnectionWebhook


5. Development tool 



BizTalk development is done only with Visual Studio whereas Logic app development can be done in Browser based Logic app designer and also support for development using Visual studio is present.


6. Flow controls


BizTalk Orchestration have following shapes which can be used to create workflow
BizTalk flow control


Whereas logic app has following as of now (I am sure more will be added in coming days)

logic app flow control


7. Promoted/Distinguished properties


In BizTalk some system properties are promoted by Adapters/pipelines and if anything from content is to be promoted/distinguished then explicitly has to be marked as so in Schema whereas in Logic App if you define schema the properties are auto promoted, and are available in workflow wherever appropriate.


8. Exception handling


In BizTalk exceptions can be handled at port level by enabling Failed Message routing and in orchestration by adding Exception Handling shape whereas in logic app you can check for error conditions of single Action and take required steps or you can use Scope to group multiple Action and apply check on its result using @result workflow function. Also we can Catch failures with the RunAfter property.



9. Long running transactions


BizTalk has out of box support for orchestration which can be marked as Long running whereas in Logic app it has to be architected at design time accordingly


10. Nested Workflow


BizTalk has support for calling or starting an orchestration from within another orchestration likewise Logic app support invoking another logic app from within Logic app


11. Deployment


BizTalk deployment on development environments is done via visual studio and on other environments with the use of MSI whereas in Logic app you save a workflow in browser designer – it gets deployed in the procured logic app instance, also deployment from visual studio is supported. 


12. Tracking and Monitoring


There is provision in BizTalk to track the artifacts and Group Hub page to monitor the events going on in BizTalk, also support for process monitoring is available with the help of Business Activity Monitoring(BAM). In Logic App Azure diagnostics is used to track workflow events .


13. BRE


One of the very strong capability/feature in BizTalk is Business Rule engine, logic app doesn’t have it as of now but would be good to have such feature.


14.  Retry Mechanism



Retry functionality is provided out of box in BizTalk and can be set statically over send port or dynamically in Orchestration. Logic App too have retry policies, as of now default retry mechanism is 4 attempts, once every 20 seconds.


15. Scalability



We can be scale out or scale out BizTalk - depends on edition whereas Logic app can be scale out or scale up – by using appropriate tier, but the time taken for Logic App is too less



16. SQL Dependency



BizTalk depends on SQL Server, as it is used to house databases and stored procedures which are needed at BizTalk design and runtime whereas you don’t have to worry about the backend of Logic App. It is actually abstracted and is totally managed by Microsoft, thus the term Serverless is associated with it.



17. Utilizing custom code in workflow



In a scenario where we need custom code to be used, it can be done with help of Helper class in BizTalk, likewise in Logic App we can use Azure functions.


18. B2B


BizTalk has out of box support for B2B supporting various industry standards like ANSI X12, EDIFACT,SWIFT,HIPPA, HL7,AS2 etc. Logic has a support for it too, but with the help of Enterprise Integration Pack which requires to have an Integration account.


18.Cost



Last but not the least, we have to invest before in case of BizTalk to procure the hardware, license etc whereas for logic app you need to have a Azure subscription first and you pay per action in Logic App.

After going through above points,there might arise a question - is Logic App a replacement to BizTalk? The answer is No - As mentioned earlier in case of integrating on Premises applications BizTalk is the option and in case of integrating SAAS services logic app is the option provided by Microsoft.

Although the capabilities of both may be overlapping but BizTalk and Logic App are not alternative but instead BizTalk and Logic App are better together. Reason being, not all industries would move everything in cloud owing to various factors like
1. Company Policies
2. Country Policies
3. Industry domain specific policies
4. Not all functionalities/applications available on cloud or are not that robust (which are available on prem) etc etc.

But to keep up with the pace of revolution happening in cloud and to extend the Business capabilities, companies would be utilizing the SAAS services along with what they have on prem. Many of the business process would need to connect to cloud based service e.g. ERP on prem data sync with CRM which is on cloud. 

Considering all this points following options are made available by Microsoft in order to enable the Hybrid Integration.
1. On Premise Data gateway – to access on prem resources from Cloud (Logic App, Power BI, Microsoft Flow etc)
2. BizTalk Connector in Logic App- to connect to BizTalk from Logic App
3. Logic App adapter – to connect to Logic App from BizTalk
4. Service Bus Adapter - to connect to Service Bus on Azure from BizTalk

hybrid integration

Following statistics was published by Gartner 


Gartner prediction






Final thought, something I think should be considered while making decision- Will the cost of having Logic App will be more than setting up BizTalk infrastructure on Prem. Say if logic app has 50/100 actions and millions of transactions processed? 

Hybrid Integration is future :) 


Tuesday, June 27, 2017

Inserting Multiple Records In On Prem SQL Using Logic App

In last post we saw how to Insert a single Record in SQL Table which is on Prem using Logic App Inserting data in On Premises SQL Database using Logic Apps


Here I intend to walk through the process of inserting multiple records using single Logic App, to do this we have two options we can choose from 
1. Split On 
2. For Each

So either we debatch/split the Request message and insert in table or loop through the array of records and insert in table, here I am using the later method.


Input

In earlier post we had single record  thus input was -
{
        "Description": "Edifice",
        "ID": "123",
        "Price": "3424"
    }

Here we want multiple records thus we need to create an array of records as following-
{
"Products":[
    {
        "Description": "Edifice",
        "ID": "123",
        "Price": "3424"
    },
    {
        "Description": "Edifice1",
        "ID": "1231",
        "Price": "2424"
    },
    {
        "Description": "Edifice2",
        "ID": "1232",
        "Price": "2344"
    },
    {
        "Description": "Edifice3",
        "ID": "1233",
        "Price": "2324"
    }
]
}


Now let's create Logic App 


I have covered creating instance of Logic App  in last post so will directly go to creating Logic App,first step is to add the trigger to start the LA, a HTTP endpoint 
Request Trigger
The trigger is also supposed to accept Request message and for that there is provision to provide a schema for it (it is optional), in last post I haven't provided schema, here I intend to show the use of providing it. Click on Use Sample payload to generate schema, in the popped up editor provide the sample payload and click ok.
provide sample for schema

I’ve used the same payload as a sample message to generate the JSON schema.The JSON schema validates the incoming request and is useful for helping subsequent workflow steps know which properties to reference. Just like disassembler in BizTalk promotes properties here Trigger does that based on schema defined.
for each action

Add next action as For each, and as you can see above , Products is available as parameter to be used in For Each. Glad to see that apart from  Body(which is available to all ), Products which is an array is only available(which is valid parameter)

for each action setting

Select the Products and click on Add an action and add SQL connector . As am using same table used in previous post I don't have to specify connection details again rather would use same connection. 
select Sql action

Select Insert Row, you will see Product table gets available in dropdown list. Here if you see the properties which are valid for columns are available to choose from (Apart from Body) and this is because we have defined schema.
Insert row action

Assign appropriate properties against column and below is how it should look
for each settings

That's it, Logic app is ready for the task

Complete Logic App


Testing


For testing I have copied the URL (which you get on Request Trigger after save) and used ARC tool to send request- data is same which I used to create schema
Arc tool
On checking the LA run history, trigger was successful and LA too.

Logic App run history

If you go one level deep , you would see For Each step has four Insert Row within it(in sample four records were used) all successful(click on Next to see status of next Insert row action) 

for each run history

 Finally we need to verify the entries made in DB


multiple records entered in SQL


 As can be seen above all four records have made entry in table.



Related Post