A big portion of any software engineering revolves around integrating multiple, disparate applications into a cohesive and functional whole. These apps may be built in house or third-party, or they may run on your network or distributed geographically, or they may be microservices designed to integrate. In any of these cases, you have several different options for integration, each with pros and cons.
The integration patterns listed here are ordered by least to most sophisticated, but also by least to most complex.
Transferring data using files provides the simplest, minimal approach to integrating two applications. With this pattern, you have each application produce a file that other applications can consume. File transfer can be done on a periodic basis to sync data between applications. If you need to sync data more quickly, you can use a Shared Database.
With a shared database, two applications use the same database and schema to keep data in sync. The databases transaction management system handles any timing conflicts better than a solution using File Transfer could manage. Unfortunately, using a shared database tightly couples each integrating application to the same data schema, leading to strong resistance to data and schema updates. The shared database also leads to a shared performance bottleneck.
Remote Procedure Call
If you need to invoke application functionality as well as share data, use remote procedure calls (RPCs). By wrapping each application in an encapsulating interface, you can guard against tightly coupled applications using the same shared database schema — each application is free to choose their own database and schema as they see fit.
Remote procedure calls are enabled using a number of supporting technologies, such as Apache Thrift, gRPC, or through JSON and RESTful APIs. Software developers are used to making function calls, so adding RPC calls fits in nicely with what developers are already used to. However, this is also a big disadvantage because remote procedure calls have vastly different performance and reliability characteristics than local procedure calls. RPCs also make it easy to tightly couple systems together.
If you wish to integrate in a more loosely coupled fashion, consider messaging.
Asynchronous Messaging allows you to integrate systems without coupling them so tightly together that they become unreliable or difficult to change. Messaging is a reaction to the problems inherent in programming distributed systems. With messaging, the sender and receiver do not need to coordinate ahead of time, or even be aware of one another. By forcing application developers to think about the asynchronous nature of message flow requires that components are designed to work well in a distributed environment, unlike RPC, which assumes system availability and masks data locality. In sum, when done well, Messaging provides the ease of use benefits of File Transfer with the immediacy needs required by modern businesses.
Unfortunately, messaging solutions often come with additional infrastructure requirements — a system that handles message routing and delivery, and the required operations and management of that system. However, when deployed and used well, Messaging provides the most flexible and robust method for system to system integration.
Images are from the book (and website) Enterprise Integration Patterns.