ContracTABLEs: Tables as Domain Contracts
In the middle of dealing with your services and domains, have you ever been baffled when a team that you even hadn’t known existed, reaching out to you and stating a change you made broke their dashboards or workflows making use of a data source that you have possibly never heard of? And even more baffled (like your eyeballs popping out from their sockets 😳) as you see your data on a platform that you are unaware of or other people talking about your internal data representations?
Chances are that if you are using an enterprise data store, there could be some platform teams who are responsible in managing the database of your services or building a mirroring mechanism to create counterparts of your data automatically to enable downstream teams like business intelligence or analytics who need various data in tabular format work independently.
One implicit challenge, when you are dealing with a service, might occur as you either intentionally or unintentionally make your data publicly available outside your domain boundary, especially some internal data that possibly should make no sense to the outside environment. How could this happen? Well, some cases that come to my mind:
- You are quickly transitioning from monolith to microservices without thinking about the internal and outbound data representation.
- You are asked to deliver as fast as possible.
- An enterprise-level CDC (change data capture) mechanism is in-place (that syncs whatever you produce to an enterprise data store).
- You don’t have time to prepare an outbound representation and you add whatever you produce to the synchronization mechanism thinking “better everything than nothing”. 😶
- You don’t know which part of your data is a byproduct of your internal logic that needs to be isolated from the outside environment.
What was the problem here again? Ah yes… The domain data, especially the internals, becomes publicly available outside your domain boundary and when you want to change an internal representation, the very downstream workflows and dashboards get affected.
There are many solutions to overcome this problem (and possibly other satellite problems around) like contracts management, event-driven approach with schema registries, etc. However most of them do not allow you to abstract your domain data representation from a particular technology or approach and when acknowledged in a hurry, they can eventually become a technology lock-in.
As an alternative, I would like to suggest something more practical that supports incremental expansion or transformation as and when needed, even some of the fancy approaches I mentioned in the above paragraph.
Okay… What’s that I’m thinking about and suggesting?
Create a relational database table (why relational database table? Because it’s our advantageous comfort zone 🙂), which I call a contracTABLE, that sits at the very edge of your service and domain boundary and only allow the data on this table be shipped outside your domain in a tabular format.
You might say… “It’s 2023… Heard about APIs? Heard about event publishing and consumption?”
If you (or it turns out that you) need to support downstream teams that can only work with a tabular representation, you have to produce that data in tabular format, and as a service or domain owner, it’s your responsibility.
ContracTABLEs as I suggest, are actually an aid to keep inbound domain dependencies clean. You will not have your domain internals in that contracTABLE design. It will only contain data that is a byproduct of your service or domain. If you are asked (or you’re interested in) to fire a domain event, generate it from an entry on this contracTABLE. Why not? Or even you might consider feeding your outbound API with the data on your contracTABLE. 🙂
See… What I suggest here also follows the KISS principle that we all love:
- This approach focuses on a medium that you can use as a contract, which is as agnostic as possible — A relational database table, we all know how it operates, what it guarantees (ACID) and is widely supported.
- If you really need to support domain events or leverage an event-driven approach, or even an API, build your events or APIs around this contracTABLE.
- When the enterprise decisions, technological options are mature enough and everybody is pretty much confident and knowledgeable around the technologies and concepts, you can easily prepare a migration plan — I still suggest keeping this contracTABLE approach though. 🙂
And this approach also supports:
- Transactional outbox pattern when considered in guaranteed event delivery context.
- Other downstream teams (like analytics and business intelligence) that need tabular data to work on (don’t forget — there are teams that will not be able to consume an API).
- Decoupling of inner-domain or inner-service data models and changes from the outside environment.
- Prohibiting people from outside the domain building a dashboard or a workflow that is based on the replica of any data that should not be shipped outside your domain boundary.
- Compliance and isolation concerns. Although you can always keep the data content on your contracTABLE safe, if you need to sanitize or mask some of this data, instead of having each team handling this separately, I suggest asking a platform team to come up with an on-the-fly mechanism to sanitize such data as it flows naturally to build up the enterprise data store counterparts.
Here is an overall diagram that reflects my suggestion:
To recap:
- As a service and domain owner, you are responsible for your data. If some internal representations go outside your boundary, that’s your calling!
- Challenge yourself and come up with a single distilled contracTABLE that sits at the boundary and allow mirroring of or replication for this contracTABLE only (through a CDC mechanism, source-sink approaches, etc.) — If you can’t come up with a single contracTABLE for your domain, think twice, challenge your domain design, this could well be a smell.
- If your enterprise is bound to some means of compliance and if a data sanitization process needs to be implemented, ask for a generalized mechanism that runs on-the-fly as data is synchronized from contracTABLE to the enterprise data store or implement it yourself for all enterprise to use — This way, auditing and accountability will be simpler and monocephalous as the solution is a generic one.
- If you need to fire a domain event or additional events, try to use the structure you used in your contracTABLE design.
- Do not forget that not all downstream teams can make use of an API or build an event consumer, so it is not realistic to expect all downstream teams to conform to your way of handling things. There will always be some teams that need tabular data and as the domain owner… Yes, it’s your responsibility.
In the upcoming days (or weeks 🙈) I am also planning to provide a working PoC (hopefully along with source code and stuff) 🤞🏼 that would accompany this post so that we could all brainstorm around it better. Until then, your ideas and feedback are more than welcome. 🙇♂️
PS: I provided a link to KISS principle from an interaction design perspective on purpose. Looking from the user’s view as much as possible really makes a difference.
Build Apps with reusable components, just like Lego
Bit’s open-source tool help 250,000+ devs to build apps with components.
Turn any UI, feature, or page into a reusable component — and share it across your applications. It’s easier to collaborate and build faster.
Split apps into components to make app development easier, and enjoy the best experience for the workflows you want:
