I totally agree with you on your situation the order system doesn't need to care about how the message/statistic service handles the order data, but how about the down stream services need to be co-related? Would you provide us any suggestion?
Let say your order service need to trigger payment service and invoice service in one of the business flow, and we need to pass the payment id and order id to the invoice service so as to co-relate the invoice and payment? Most likely, the payment id can only obtain after the payment record pass to payment service. Suppose that we need to give the user pdf invoice once the order made in UI.
Hi CodeMaster, actually it really depend on how you design your service. If making order and payment are supposed to be tightly coupled which mean you want a synchronous operation, then you have no choice. Because you need to wait for payment to be success and use the payment id to link the invoice and return the result immediately to users. One of the direction that you can think about is to scale the instance and write the data into cache first and batch write to the database every few seconds. Using write-back approach, it makes the data written faster and avoid the large traffic hitting the database directly. Also, you can think about database sharding to avoid large table in the database.
But let's think about a decoupled solution. Do you really need to make order and payment tightly coupled? In a real world scenario, when your service invoke third party payment provider, they cannot tell you the payment is successfully made. Because they might integrate with different payment providers and wait for bank or lowest tier provider to respond the payment status. So, the external payment provider might not give you the success status at the first call, they might send a callback to your system and tell you the payment is successfully made after a few seconds. With this behavior, actually, we can still think about a decouple solution. Instead of letting order service to be an orchestrator of payment service and invoice service, can we use event driven approach to let the payment service notify the invoice service that a payment is successfully made? I think it is possible. Imagine you made an order in ecommerce platform, you won't receive the email invoice immediately. Instead, it has a delay between your receive the email invoice and finishing your payment.
This is what I want to mention: Accidental Complexity vs Essential Complexity. It really depend on the actual requirement. There is always a trade-off in software engineering. If your PM really want to make it coupled, maybe you can suggest your PM to do it in another way to see if he/she accept your advice. It is about how you communicate with your PM and design the product together.
Thanks for your suggestion. I think either the synchronous write-back approach or the event driven approach is good enough.
I admit payment and invoice may not be a good example. I think what I want to discuss in deep is when the existing system architecture have already built up with the orchestrator. One day, the new requirement comes and we have no choice but to add a new service which is tightly coupled between the existing services with some synchronous operation e.g. the id case above. Under this situation, any suggestion we do better generally? Let say if there any hybrid patterns to implement event driven on top of the orchestrator. You know, reusability plays important role in software engineering. I think it is one kind of the pain in microservice architecture i.e. we try our best but It is hard to foresee the upcoming features/requirement in the design phase and we are hard to discard/revamp existing architecture frequently.
Actually, this is also a product design problem. If engineering team feels too many things coupled together, you should suggest your PM to think about removing some unnecessary dependencies from the whole call chain in order to increase the response efficient. Also, let the PM know the pain in your current architecture. We always need to balance the architecture complexity and product value. This is the trade-off that we need to think about during architecture design. If it really affect user experience, then we can trade off a bit in architecture. If not, why not keep the architecture simple and try discuss with the PM. If you can explain the idea to the PM and the suggested idea is acceptable, I think most of the PMs are willing to adopt the solution.
I totally agree with you on your situation the order system doesn't need to care about how the message/statistic service handles the order data, but how about the down stream services need to be co-related? Would you provide us any suggestion?
Let say your order service need to trigger payment service and invoice service in one of the business flow, and we need to pass the payment id and order id to the invoice service so as to co-relate the invoice and payment? Most likely, the payment id can only obtain after the payment record pass to payment service. Suppose that we need to give the user pdf invoice once the order made in UI.
Hi CodeMaster, actually it really depend on how you design your service. If making order and payment are supposed to be tightly coupled which mean you want a synchronous operation, then you have no choice. Because you need to wait for payment to be success and use the payment id to link the invoice and return the result immediately to users. One of the direction that you can think about is to scale the instance and write the data into cache first and batch write to the database every few seconds. Using write-back approach, it makes the data written faster and avoid the large traffic hitting the database directly. Also, you can think about database sharding to avoid large table in the database.
But let's think about a decoupled solution. Do you really need to make order and payment tightly coupled? In a real world scenario, when your service invoke third party payment provider, they cannot tell you the payment is successfully made. Because they might integrate with different payment providers and wait for bank or lowest tier provider to respond the payment status. So, the external payment provider might not give you the success status at the first call, they might send a callback to your system and tell you the payment is successfully made after a few seconds. With this behavior, actually, we can still think about a decouple solution. Instead of letting order service to be an orchestrator of payment service and invoice service, can we use event driven approach to let the payment service notify the invoice service that a payment is successfully made? I think it is possible. Imagine you made an order in ecommerce platform, you won't receive the email invoice immediately. Instead, it has a delay between your receive the email invoice and finishing your payment.
This is what I want to mention: Accidental Complexity vs Essential Complexity. It really depend on the actual requirement. There is always a trade-off in software engineering. If your PM really want to make it coupled, maybe you can suggest your PM to do it in another way to see if he/she accept your advice. It is about how you communicate with your PM and design the product together.
Thanks for your suggestion. I think either the synchronous write-back approach or the event driven approach is good enough.
I admit payment and invoice may not be a good example. I think what I want to discuss in deep is when the existing system architecture have already built up with the orchestrator. One day, the new requirement comes and we have no choice but to add a new service which is tightly coupled between the existing services with some synchronous operation e.g. the id case above. Under this situation, any suggestion we do better generally? Let say if there any hybrid patterns to implement event driven on top of the orchestrator. You know, reusability plays important role in software engineering. I think it is one kind of the pain in microservice architecture i.e. we try our best but It is hard to foresee the upcoming features/requirement in the design phase and we are hard to discard/revamp existing architecture frequently.
Actually, this is also a product design problem. If engineering team feels too many things coupled together, you should suggest your PM to think about removing some unnecessary dependencies from the whole call chain in order to increase the response efficient. Also, let the PM know the pain in your current architecture. We always need to balance the architecture complexity and product value. This is the trade-off that we need to think about during architecture design. If it really affect user experience, then we can trade off a bit in architecture. If not, why not keep the architecture simple and try discuss with the PM. If you can explain the idea to the PM and the suggested idea is acceptable, I think most of the PMs are willing to adopt the solution.