ODC architecture: Shared Data Patterns while migrating to ODC

When migrating from OutSystems O11 to ODC, we have to make choices how to implement the architecture of shared data patterns. This is one of the patterns we have to make choices about. In this article I describe five practical solutions for the architecture of shared data patterns and list the advantages and disadvantages.

• One Monolith
• An app for each domain, reference by ID
• An app for each domain, reference by Code
• An app for each domain, get the referenced data via Service Action
• An app for each domain while the data between the domains is synchronized

Imagine there an OutSystems 11 system, which is already in production containing lots of data. The system is fairly stable, we expect changes but not very big ones. The business prefers minimal deployments to production, ideally fewer than once a month. Now we should migrate this O11 system to ODC. Conform the domain-driven architecture, each domain will be converted into an APP, so we must choose the domains carefully. In this document I describe the practical solutions for the architecture of shared data patterns, with in my opinion the best solution. The considerations are illustrated via a common use case involving Orders, Orderlines and Product. In the conclusion, there will be an overview what is the best shared data pattern for migration.

Situation in OutSystems 11

In OutSystems 11, we have created a design for Orders, Orderlines and Products. In Order_CS we have the entities Order and Orderline and other related entities like OrderStatus, OrderType, OrderSource etc. In Product_CS we have entity Product and its related entities like ProductType, ProductGroup etc.

In the Product_CS we make the Product entity public, so that we can reference it in Order_CS in the Orderline entity. This creates foreign key with status protect

The Data model would look like this:

There are also screens for maintaining Products and Orders/Orderlines. In O11 those screens are in EndUser modules. We make the entities public read-only and make public server actions for the CRUD actions. There is no need for service actions, as all updates are inside one process.

Let’s discuss the five possible solutions for migrating this functionality to ODC

1.One monolith

Solution 1 is to put both the Order_CS and the Product_CS modules into one monolithic OrderProduct ODC app. This app can even include the screens with web blocks to maintain the Products and Orders/Orderlines. This will be a noticeably big monolithic app.

Advantages

• The entities Order, Orderline and Product with their respective static entities don’t need to be public
• The Delete Rule of the ProductId in Orderline can still be protect, therefor the consistency of the database remains intact
• It is possible to change the entities without causing breaking changes
• No problems with referential integrity when deploying this app to another stage
• Easy deployment, no distributed applications
• This solution is similar to the O11 solution

Disadvantages

• There is a lot of functionality inside one monolithic app, making the maintenance difficult when multiple developers work on it
• Minor changes require an update of the entire monolith

2.Two apps, Order and Product, reference by ID

Solution 2 involves moving entities of Order_CS into an Order app, possibly combined with the Order screens and actions. Similarly, move the entities of Product_CS into a Product app, along with the maintenance screens of the Products.

There are only weak references between apps for entities, but we need to refer the Product in the Orderline entity. Product will be a public entity, which is referenced in the Order app. This would require a foreign key in the Orderlines entity to ProductId with Delete Rule ignore, protect is not allowed. However, there would be no updates of Products in the Order app, so there is only a weak reference with a public entity.

If we model the relation like this, we can easily create an aggregate to fetch the Orders and Orderlines with its Products:

Advantages

• It is still possible to reference entities outside the app with their Id’s
• Aggregates from O11 do not have to change
• Aggregates are easy maintainable
• There is now a domain-driven architecture, keeping the Order-related functionality in one Order app and the Product functionality in a Product app
• Both apps are smaller

Disadvantages

• There is a weak reference between Order and Product app. If the Product entity changes, the Order app will need to as well to avoid breaking
• There is no referential integrity between Orders and Product in the database, when creating an Orderline the developer must be sure that the correct ProductId is place in the Orderline
• When deleting a Product, a check must be done to see if this Product is used in any Orderline

3.Two apps, Order and Product, reference by Code

Just like solution 2, we move the entities of Order_CS into an Order app, and the entities of Product_CS into a Product app, benefiting the domain-driven architecture. For this solution, the reference between Orderline and Product will not be the ProductId, but the ProductCode. This ProductCode is a unique code given to the product, enforced by a unique key on ProductCode.

In the Orderline, the ProductCode is added, but there is no foreign key anymore. However, we can still create aggregates in this situation:

Advantages

• There is foreign key anymore outside the App. There is a perfect domain-driven architecture, keeping the Order related functionality in one Order app and the Product functionality in a Product app
• Simple, fast, and easily maintainable aggregates can still to fetch the Orderlines with their Product
• This change can be implemented well before the actual migration to ODC

Disadvantages

• There is a weak reference between Order and Product app. If the Product entity changes, the Order app will need to update as well to avoid breaking
• There is no referential integrity anymore if the ProductCode in Orderline exists in Products. When deleting products, it must be checked if this Product still exists in Orderlines, otherwise orphans are created
• Migrating a OutSystems 11 system with data to this configuration, requires updating in the existing data, and changing all aggregates
• All entities, which will be used in this way, should have an attribute that is unique. If an entity lacks a unique attribute, it must be defined and created
• The solution is not according to the best practices of a relational database

4.Two apps, Order and Product, get data via ServiceActions

 As in solution 2 and 3, we move the entities of Order_CS into an Order app and the entities of Product_CS into a Product app, maintaining the domain-driven architecture. In this situation there is no relationship at all. We do not create aggregates to fetch the data from the Product for the Orderlines, but the data is fetched via a ServiceAction A ServiceAction in the Product app returns the desired data of Product (called GetProductByCode). The return cannot be a record of the type Product entity, so a static must be defined.

In the Order app, instead of the aggregate to the Orderlines with Product, we have to make a Data Action to get the Orderlines with the Product. First the aggregate of solution 1, 2 and 3 is done without the Product entity. The result of the Data Action is a list of the Orderline with the Product name, we have to make a special Struct for that.

We can immediately see that we need to use a loop to fetch for each Orderline its ProductName via the ServiceAction. Keep in mind that we need more ServiceActions when we need to fetch more data outside the domain.

Advantages

• There is no foreign key outside the app, ensuring a perfect domain-driven architecture, keeping Order-related functionality in one Order app and Product functionality in a Product app
• Changes in the Product app does not necessitate updates in the Order app, as long as the ServiceAction remains unchanged. New ServiceActions can be created alongside the old ones to maintain existing functionality

Disadvantages

• There is no referential integrity anymore if the ProductCode in Orderline exists in Products. When deleting products, it must be a check to see if this Product still exists in Orderlines, otherwise orphans are created
• Migrating an O11 system with data to this configuration, there needs to be an update in the existing data
• All the entities that will be used in this way, should have a unique attribute. If an entity lacks a unique attribute, it must be defined and created
• All the aggregates need to be changed for this solution into Data Actions. Which may lead to a range of ServiceActions called inside a loop
• ServiceActions need to be created. The output of this ServiceAction must be a static
• The solution is not according to the best practices of a relational database
• Performance will decrease

5.Two apps, Order and Product, but data is synchronized

As in solution 2 and 3, we move the entities of Order_CS into an Order app and the entities of Product_CS into a Product app, maintaining the domain-driven architecture.

In this solution we also create a Product entity with the necessary attributes in the Order_CS app. In that case we can use that new Product entity quite easy. This new Product entity can be used easily, but synchronization of data between the Product entity in the Product app and the new Product entity in the Order app is required. Synchronization will occur at predefined intervals, such as every 10 minutes or daily.

Advantages

• There is no foreign key outside the App, there is a perfect a domain-driven architecture, keeping the Order related functionality in one Order app and the Product functionality in a Product app
• Changes in the Product app only require updates to the synchronization process, if applicable

Disadvantages

• Product data is available in more than one place
• Data must be synchronized, so it may be outdated
• There is still a relation between both apps because the synchronization must use the Product entity in Product to fill the Product entity in Order

Conclusion

Given that we want to migrate a fairly stable existing application with data, we need a solution with minimal impact, while still benefiting from ODC’s domain-driven architecture. In this document, I defined 5 solutions.

• One Monolith
• An app for each domain, reference by ID
• An app for each domain, reference by Code
• An app for each domain, get the referenced data via ServiceAction
• An app for each domain while the data is synchronized

For maintenance issues, solution 1 is not advisable. However, the migration will be quite easy, maintaining this solution with more than one developer in this Monolithic app is not advisable.

Solutions 3 and 4 require the use of “Code-attributes” for each foreign key (such as the ProductCode attribute in the Orderline entity) and scripts to populate these “Code-attributes” into the entities. Creating and filling these new attributes and updating aggregates involves significant changes, potentially introducing errors.

Solution 4, although it fits best with the domain-driven architecture, requires major changes, including creating and populating new Code-attributes and converting all aggregates to Data Actions. Performance will also decrease. This solution is suitable for apps that are not stable and need frequent deployments, which is not our case.

So, in this case, solution 2 would be the best solution. There is no need to add extra attributes and update the data and aggregates for this. We still have domain-driven architecture. The definitions of the entities are stable and there is no need for frequent deployments.