The other advantage of the network boundary is that you can use different languages / technologies for each of your modules / services.
Don't get me wrong, sometimes it's worth it (I particularly like Spark's facilities for distributed statistical modelling), but I really don't get (and have never gotten) why you would want to inflict that pain upon yourself if you don't have to.
I’ve been developing for more years than dime if you have lived, and the best thing I’ve heard in years was that Google interviews were requiring developers to understand the overhead of requests.
In addition, they should require understanding of design complexity of asynchronous queues, needing and suffering from management overhead of dead letter, scaling by sharding queues if it makes more sense vs decentralizing and having to have non-transactionality unless it’s absolutely needed.
But not just Google- everyone. Thanks, Mr. Fowler for bringing this into the open.
Adding network is not a limitation. And frankly, I don't understand why you say things like understanding network. Like reliability is taken care of, routing is taken care of. The remaining problems of unboundedness and causal ordering are taken care of (by various frameworks and protocols).
For dlq management, you can simply use a persistent dead letter queue. I mean it's a good thing to have dlq because failures will always happen. About which order to procese queue etc. These are trivial questions.
You say things as if you have been doing software development for ages, but you're missing out on some very simple things.
Getting engineers who don't intuitively understand or maybe even care how to avoid coupling in monoliths to work on a distributed application can result in all the same class of problems plus chains of network calls and all the extra overhead you should be avoiding.
It seems like you tell people to respect the boundaries, and if that fails you can make the wall difficult to climb. The group of people that respect the boundaries whether virtual or not, will continue to respect the boundaries. The others will spend huge amounts of effort and energy getting really good at finding gaps in the wall and/or really good at climbing.
If you take a look at dependency management at open source software, you'll see a mostly unified procedure, that scales to an "entirety of mankind" sized team without working too badly for single developers, so it can handle your team size too.
That problem that the "microservices bring better architectures" people are trying to solve isn't open by any measure. It was patently solved, decades ago, with stuff that work much better than microservices, in a way that is known to a large chunk of the developers and openly published all over the internet for anybody that wants to read about.
Microservices still have their use. It's just that "it makes people write better code" isn't true.
I've often wondered if this is a pattern sitting underneath our noses. I.e., Starting with a monolith with strong boundaries, and giving architects/developers a way to more gracefully break apart the monolith. Today it feels very manual, but it doesn't need to be.
What if we had frameworks that more gracefully scaled from monoliths to distributed systems? If we baked something like GRPC into the system from the beginning, we could more gracefully break the monolith apart. And the "seams" would be more apparent inside the monolith because the GRPC-style calls would be explicit.
(Please don't get too hung up on GRPC, I'm thinking it could be any number of methods; it's more about the pattern than the tooling).
The advantages to this style would be:
* Seeing the explicit boundaries, or potential boundaries, sooner.
* Faster refactoring: it's MUCH easier to refactor a monolith than refactor a distributed architecture.
* Simulating network overhead. For production, the intra-boundary calls would just feel like function calls, but in a develop or testing environment, could you simulate network conditions: lag, failures, etc.
I'm wondering if anything like this exists today?
If you've only got a basic IPC system (say, Unix domain sockets), then you could stream a standard seriaization format across them (MessagePack, Protobuf, etc.).
To your idea of gracefully moving to network-distributed system: If nothing else, couldn't you just actually start with gRPC and connect to localhost?
Is there something I'm missing?
When you start with gRPC and connect to localhost, usually the worst that can happen with a RPC call is that the process crashes, and your RPC call eventually times out.
But other than that everything else seems to work as a local function call.
Now when you move the server into another computer, maybe it didn't crash, it was just a network hiccup and now you are getting a message back that the calling process is no longer waiting, or you do two asynchronous calls, but due to the network latency and packet distribution, they get processed out of order.
Or eventually one server is not enough for the load, and you decide to add another one, so you get some kind of load mechanism in place, but also need to take care for unprocessed messages that one of the nodes took responsibility over, and so forth.
There is a reason why there are so many CS books and papers on distributed systems.
Using them as mitigation for teams that don't understand how to write modular code, only escalates the problem, you move from spaghetti calls in process, to spaghetti RPC calls and having to handle network failures in the process.
(I side with the monolith, FWIW...I love Carl Hewitt's work and all, it just brings in a whole set of stuff a single actor doesn't need... I loved the comment on binding and RPC above, also the one in which an RPC call's failure modes were compared to the (smaller profile) method call's)
Most languages provide a way to separate the declaration of an interface from its implementation. And a common language makes it much easier to ensure that changes that break that interface are caught at compile time.
