The Thing No One Tells You About Microservices

2 місяці тому

Are your microservices even microservices? I, and many other thought leaders in software, think microservice architecture is widely misunderstood and as a result, not implemented correctly.
In this episode, Dave Farley explores how to design microservices to avoid sharing data stores between them.
-
🖇 LINKS:
🔗 “What are microservices 1” microservices.io/
🔗 “What are microservices 2” smartbear.com/learn/api-desig...
🔗 “The Original microservice Article” martinfowler.com/articles/mic...
-
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content! ➡️ bit.ly/ContinuousDeliveryPatreon
-
👕 T-SHIRTS:
A fan of the T-shirts I wear in my videos? Grab your own, at reduced prices EXCLUSIVE TO CONTINUOUS DELIVERY FOLLOWERS! Get money off the already reasonably priced t-shirts!
🔗 Check out their collection HERE: ➡️ bit.ly/3vTkWy3
🚨 DON'T FORGET TO USE THIS DISCOUNT CODE: ContinuousDelivery
-
BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com
#softwareengineer #developer #microservicesarchitecture #microservices

КОМЕНТАРІ: 257

@JustLikeBuildingThings 2 місяці тому

So much of this hurts. On a large government programme I turned up for greenfield project, and were dictated by the client we must use microservices and graphQL. We were then presented with a huge data model of a monolithic data structure with dozens of tables, all complete with attributes... before any work being done to actually decide what was being built. It made the project massively complex, to the point where graphQL was abandoned and project converted to a monorepo. No one understood the complexities of what they were asking for in terms of complex business logic and failure handling. The culture of muppets designing systems from ivory towers using buzzwords is the biggest IT con in our time, pissing up millions of public funds. If people knew the full extent of it they would riot.

@jasondbaker 2 місяці тому

A microservices architecture is a reasonable architecture to consider for modern greenfield projects. In this case, the problem sounds like the client didn't have the experience necessary to define the requirements for this sort of architecture. And that's a totally valid concern that teams need to consider.

@JustLikeBuildingThings 2 місяці тому

Teams cannot consider it when the government client says "this or else" you just have to comply. This happens all over Government; people with egos wasting tens of millions.@@jasondbaker

@GackFinder 2 місяці тому

As a consultant I see this all the time. You're completely right. 90%+ of what the IT industry produces is waste.

@JustLikeBuildingThings 2 місяці тому

Likewise where my experience is coming from. I've personally witnessed recently a Government programme with almost no oversight get canned after a year of work... who loses out? The consultancy? Nope they benefitted from no oversight to the tune of millions, the Government then kept paying ~£7000+ a day for a month to hand it over. This is RIFE and no one is talking about it because of the sheer volumes of cash it generates.@@GackFinder

@AerialWaviator 2 місяці тому

"We were then presented with a huge data model of a monolithic data structure" This should have been questioned, as it's a concrete model to a design yet to be implemented. All too often in appropriate design constraints are dictated, without valid reason by well meaning individuals, or organizations. This no-longer was a greenfield project. It's like a building architect/contractor being given a stack of napkins with sketches in crayon. (perhaps over exaggerated, but makes point of how much attention should be given to early constraints)

@GackFinder 2 місяці тому

Microservices are great if you're willing to deal with the the ridiculous extra costs to manage the ridiculous extra layers of stuff you need, such as monitoring, alerting, retry logic, versioning, authentication and authorization, deployment, dead letter queues, idempotency requirements... for every single microservice.

@peppybocan 2 місяці тому

imagine that people are willing to get into microservices regardless of all that. I work for a company that has very little amount of data or processing, but we are doing microservices, because it's trendy.

@GackFinder 2 місяці тому

@@peppybocan My current client is doing low-code microservices because it's trendy. It's an absolute horror show.

@sebastiannyberg3008 2 місяці тому

Cross cutting concerns should be largely managed by the platform the applications are deployed to. Implementing them for each microservice is death by a thousand cuts

@GackFinder 2 місяці тому

@@sebastiannyberg3008 That very much depends on what the definition of a cross-cutting concern is, and the more one includes in that definition, the more need there is for a "platform team", at which point most of the benefits of a microservices architecture is thrown out the window, and you might as well just build a monolith instead This is the exact issue my current client has. The platform team owns "the platform" but has no real way of affectuating good platform standards into the microservices of other teams. Other teams will just deploy whatever works on the platform by default regardless of any guidelines set in place by the platform team, and good teams with good suggestions on how to improve the overall state of platform (with e.g. correlation logging) are hindered by other teams that are "not there yet" because the suggested changes will break their services. This in turn means that no team really owns or cares about the platform even though it is a critical asset to all platforms it runs on top of. In other words, the perceived ROI you get for infusing lots of cross-cutting concerns into the platform is dwarfed by the many many issues of doing so.

@foobar8894 2 місяці тому

@@sebastiannyberg3008 Having the 'platform' solve all this is dead by monthly cloud bills, so yeah... But if you drop the assumption you need multiple applications (you don't) lots of those concerns go away and the ones that are left are no longer cross-cutting.

@skylineuk1485 2 місяці тому

I’ve been in the systems development game since the mid 1980’s. Sticking to any one methodology is crazy, different ones work better in some cases than others. Never start out with a methodology, decide on the methodology after you have specified the problem and allowed for growth, scaling and future direction as much as possible. The number of “university type” projects based on the so called holy grail of methodologies I have seen fail is just plain crazy 😢.

@TheMorges1 2 місяці тому

the overuse of these buzzwords is horrendous. Ive not long started work at a new company and today cheekily struck-through the words "microservices" and "service orientated architecture" in the company's technical documentation and replaced it with "distributed monolith". very tongue in cheek of course and people can always switch it back if they like, but lets see if it starts a conversation. The problem from my point of view with just splitting things into separate repos and calling it a day is you then have all the difficulties of working with microservices ( of which there are plenty ) and absolutely none of the benefits ( of which there are also many when its done right ).

@picleus 2 місяці тому

When I joined my current company as my first job 2 years back, the IT boss had already declared that we would "break our legacy monolith into microservices" for a number of unclear reasons. So me and two other fresh juniors went off and started rewriting the code as a distributed monolith with no idea what we were doing. And would you know it, now we have two unmaintainable codebases.

@Fanmade1b 2 місяці тому

I just quit mit second job in a row where a large part of my decision was based upon upper- and/or middle management making stupid technical and architectural decisions. In both of these jobs were software "architects" who were sure that they know better how to define microservices and how to use them. Not "when" to use them, because basically both of them are certain that monoliths are something that can only be bad and need to be avoided. In their world, no experienced developer would decide on a monolithic approach for anything. Apart from that, they both designed "miroservice architectures" that basically contradict everything mentioned in this video. Like ~30 "microservices" in the one company, which are all requiring the same package, which defines the data structure that all of them are supposed to be using. Of course, those 30 microservices are all developed and maintained by exactly one team with three members. Updating anything there is a always a big pain, very often resulting in bugs and errors that make it to production and in turn to the customers. Oh, neither one of them could manage to integrate anything that would allow proper logging. So just to find out where exactly an error in that mess happened was very often a hard task that could take half a day while involving half a dozen of people in itself. Not to mention the debugging process afterwards. I quit the first job because I thought that this was a problem of a small and understaffed company. So I switched to a larger corporation with billions in turnover, only to see that they did the exact same thing, just with more people involved. While I had my talk with HR about my reasons, I did a bit of self-reflection and I found out that I have learned a lot of good and bad things in each company I worked for, but my last job was the first one where I can't think of one good thing I learned. Of course this is only considering the professional context, I still got new contacts and met very nice people from which I also learned good things, but I've contributed some things to the code of that company's system that I've learned previously without learning anything beneficial from their code except how not to do it (most of which I've seen before). This is really frustrating and I have no motivation to start at any other company right now.

@itskittyme 2 місяці тому

go solo, be solo, stay solo, it can make you much more productive when you can easily decide your own architectural decisions, but what I find even better, is you can make an instant architectural change within a day, without having to convince others over countless meetings spanning over countless weeks. And now with the arrival of AI, you can even be as productive as an entire team.

@Fanmade1b 2 місяці тому

@@itskittyme To be honest, I don't believe that. In my opinion, it is always best to find someone who is better than me and then to work together (like a mentor). Sure, you can and should learn yourself and make your own mistakes. But you can reduce the time and effort required to improve by doing that work together. Also, mentoring yourself and sharing your own knowledge is a good way to create a deeper understanding and to further improve. Having someone who has another perspective can very often help and even prevent one from taking a wrong path which might be hard to return from later. Seeing this from the other side and looking at my examples above, it can be harmful to stay in a company where you can't properly learn and improve your skills. I've seen that happen often and I even know people who are now suffering from having been in that situation way too long.

@itskittyme 2 місяці тому

I'm learning so much from sharing my code with ChatGPT and vice versa, much faster than with a human team, but that's just my experience@@Fanmade1b

@pookiepats 2 місяці тому

Stop being a brat, this sounds ridiculous.

@jimiscott 2 місяці тому

Here are the hard truths about microservices... 1. Your organisation probably doesn't need them - do you have the scale? 2. Your first attempt at building microservices is probably wrong and you end up with a distributed monolith (or a distributed ball of crap) - I am going to say this is akin to first designing an OO system with loads of hierarchies - these will be wrong. Just do nicely independent services - they may be large - they may be small - as long as they're independent you should be OK.

@viniciusgarcia3452 2 місяці тому

"Independent service" might actually be a very good name to describe what I usually end up with. I might start outside it instead of micro services 😄 Or maybe "independent deployable service", it's long but coveys the actual meaning I need, being small is not important but it's usually a consequence of existing inside a well defined bounded context, and the term microservice makes it sound like being small is the most important aspect

@InconspicuousChap 2 місяці тому

Microservices are a way to split the work between developers, how the corporate management sees it, and that's the primary reason of their popularity. Just like OOP was 20-30 years earlier. The corporate approach to software development is hiring mediocre easily replaceable coders, as cheap as possible, split the work between them and expect them to build something working without designing it as a whole. Since no working software can be built without a design stage, they just pick up a "one size fits all" trendy design, whether it's applicable or not. The whole point of splitting the work is an attempt to evade exponential dependency of development and maintenance costs on the size of the product, resulting from poor design and mediocre coders' decisions along the way. Managers use scholar math to estimate that exp(N) is significantly higher than e.g. exp(N/k) * k (for k microservices and N code lines in the product), totally ignoring the fact that the multiplier here would not be k, but some kind of exp(k^2) because there would be k^2 interactions, and the complexity of mediocre programmers' work always grows exponentially with the size of the domain they try to model. So that actually results in the costs still being exponential, with even higher multiplier (exp(Ck^2 + N/k) instead of exp(N), where C is the average number of code lines per microservice interaction, and that's an optimistic approach assuming services behave consistently, which never happens in mediocre programmers' implementations), and no chance for developers to lower them even if they wanted to and knew how to do it.

@benisrood 2 місяці тому

@@InconspicuousChap Your analysis of the real costs involved and the false estimations (including by developers! We are often as guilty!) matches my experience. They completely misunderstand what things are and what their benefits and trade-offs are, all they do is try to translate it to resource management-ese. And this channel makes the same arguments and rationale in this very video! It's not a valid reason!

@InconspicuousChap 2 місяці тому

@@benisroodThey hire mediocrity who constantly screw things up, due to lack of qualification and motivation, that's how tying their hands becomes a self-justified measure. The managers are completely right within their paradigm of operation. Only the paradigm itself is deficient. And an average manager doesn't give a shit for what happes beyond their annual KPI, so their decisions often contradict the company's long term benefit. The best achievements my colleagues and I ever made were when non-technical managers didn't interfere much (either volunteerly or if we had a possibility to keep them at a safe distance).

@dameanvil 2 місяці тому

00:00 🎯 Microservices are powerful but often misunderstood in practice, leading to teams claiming to use them when they might not be. 02:34 📚 Agreeing on clear definitions and terminology is crucial for effective communication and learning within the industry. 05:15 🔍 Microservices must adhere to specific attributes like being small, focused, autonomous, and independently deployable to be considered as such. 09:32 🚀 Independently deployable microservices are essential for enabling organizational autonomy, scalability, and faster software delivery. 13:10 🔄 Sharing data between microservices via messages instead of a shared database reduces coupling and enhances scalability and maintainability.

@jmrah Місяць тому

This seems like a great job for AI. UKposts devs should add this as a feature.

@dameanvil Місяць тому

@@jmrah What would be humanity without parasites who see themselves smart telling others what to do?

@amurgcodru Місяць тому

Sounds like erlang and elixir but with way too much overhead

@jimhumelsine9187 2 місяці тому

Building a microservice is easy. Building a system of microservices is hard. -- Paraphrasing Sam Newman. I'm glad that Dave listed "Bounded Context." This is a Domain-Driven Design concept, and I'd like to follow up with another DDD concept: Context Map, which defines the communication channels between Microservices. Bounded Contexts and Microservices get most of the buzz, but the real work resides in the Context Map. And I'm of the opinion that for the most part, the Context Map is not the responsibility of the average developer. The average developer is responsible for the implementation of the microservice given the communication channels and protocols within which it operates. The average developer is very much interested in the Context Map, but mostly within the context of how his/her microservices interact with it. It is the architects and possibly the lead software developers who are (or should be) responsible for the Context Map. How often do they own this responsibility or even realize that they should? If no one is responsible for the Context Map and manages it, then entropy will take over. The system will become a huge distributed big ball of mud.

@jonashansen2512 2 місяці тому

Context Map sounds like what I often call a Message Broker or a Router. It can be a service on its own or a shared internal dependency between services.

@jimhumelsine9187 2 місяці тому

@@jonashansen2512 The Context Map is more abstract than a specific Broker or Router. The Context Map is about the relationship between the Bounded Contexts and not the specific communication mechanisms. It's like business communications before telecommunications. Businesses worked together via contracts. The distances between the businesses may be so great that meeting face-to-face would not be an option. Therefore, the contracts would be sent back and forth via the postal service. The business contract is a map that defines the relationship between the two businesses within their context. The postal service is the mechanism for delivering the signed contracts between the two. I know this analogy is a bit of a stretch and not perfect. The Context Map would tend to define how Bounded Contexts (i.e., microservices in implementation) interact. Is it via a command? Is if via a domain event? What information is within the command or event? What type of response is expected, if any. Etc. The Broker/Router is the implementation mechanism upon which the Context Map information is transmitted.

@user-zk5ym9ut1j 2 місяці тому

Why people forgot that distributed computing is hard and keep talking about microservices as silver bullet?

@aufderartwick Місяць тому

Microserices are good because they are loosely coupled 😂

@bobbycrosby9765 2 місяці тому

I think the problem is people and organizations that can't benefit from microservices are trying to use them because it's what Google/Netflix/Whatever uses. So they cut corners.

@scottnash70 Місяць тому

Remember how badly the term "cloud" was corrupted by legacy on-premise software providers. It got to the point where I needed to specify the key question "Are all customers running on the same version of the software?" whenever I was evaluating vendors.

@sammcj2000 Місяць тому

Well, this just got added to my list of things to share with clients when they fundamentally misunderstand and represent core concepts. While this personally wasn’t new to me, it is delivered and explicated in a very clear and calm fashion. Thank you .

@Vendavalez 2 місяці тому

If the problem you are trying to solve with microservices is tight-coupling, please do yourself a favor and don’t move to microservices until the underlying code base is already loosely coupled. If you don’t know how to make a monolith loosely coupled, what makes you think you will be able to keep microservices loosely coupled? You will end up with a distributed monolith and every change will require 7 PRs and 7 simultaneous deployments. And lord help you if you have to roll anything back.

@Rope257 2 місяці тому

I really appreciate how in this video you've taken my previous comment, regarding the title and the content's misalignment, into account. As always the content itself is superb of course.

@lilivier 2 місяці тому

I've always found that 'One task' is too vague. I've noticed that nobody has the same definition of what a 'task' is.

@thekornerr Місяць тому

What's your definition?

@DrunkenUFOPilot Місяць тому

Just on small personal projects, $0 budget, zero stake holders, I can get caught up in a web of vagueness trying to define "one purpose", "one task", "one responsibility". I suppose in a team, it still remains a rubbery concept but a team lead or very senior engineer can set a standard so progress can be made. That doesn't mean "task" or related concept has been defined, only set for now for this project.

@IronDoctorChris 2 місяці тому

I think part of this is because microservices are the correct choice for such a small percentage of development teams. 90%+ can likely get by just fine with a modular monolith. So they build like that and mistakenly refer to it as "microservices".

@ContinuousDelivery 2 місяці тому

The big mistake that I see all the time, is not recognising that what they have is a modular monolith, which is fine, and so, for example, still keeping each "service" in its own repo and having a separate "deployment pipeline" and then having an integration stage sometime later. If you just admitted it was a monolith, you could use the version control system for what it is good at, controlling the versions that work together. Modular monoliths are in many ways MUCH simpler than microservices, ultimately they are organisationally less-scalable, but otherwise perfectly fine. I have seen 10 person teams with 30 microservices, each in its own repo, and all facing integration-hell. We know how to solve these problems!!!

@VincentJenks 2 місяці тому

Extremely well articulated, thank you. As an architect and mentor, it’s very hard to convey these concepts, even to technical staff. It’s even harder to convey the value because many want the easy, naive approach that’s obvious on day one of a project. It’s experience with the hell that most large projects devolve into that have led us to new approaches, like microservices.

@roundtheloopandback Місяць тому

Around late 2012 I became aware of the microservices approach and definitions of what one was, my understanding then is as you have posted now, yet this video is still required in 2024 (and it really is required) because people simply cannot builld microservice architectures as intended. They are not easy and introduce issues of their own, this is an excellent post as normal on the subject and should be shown to business people and developers alike.

@Dangerousdaze 2 місяці тому

Clear and concise. Valuable information, as always!

@adrianpaulwynne 2 місяці тому

Not all coders are rock stars (yet). In any team, many are still in the early stages on their career. Some things are easy, like coding and testing a synchronous function that represents some business logic. Everyone in the team can do it. Some things are much harder, like dealing with synchronisation and eventual consistency and failure modes and deployment and security. Not everyone in the team can to do it well (yet). The microservices model pushes responsibility for the difficult problems down into every small (by definition) microservice. It becomes everyones responsibility Furthermore, the patterns implemented in each microservice have to be mutually-compatible Is this realistic, or a recipe for chaos, fragile systems and endless meetings? how do you solve this?

@ContinuousDelivery 2 місяці тому

I think that we should stop pretending that development is easy and focus more on learning as part of the job, one way to solve this problem is like surgeons or airline pilots, you train people on the job!

@adrianpaulwynne 2 місяці тому

@ContinuousDelivery a video on effective in house training for devs would be a good topic if you're interested in making it. Keep up the interesting content

@TidusUltimate Місяць тому

I worked on a project that had this same exact messaging pattern. We used to have a extra table that holds ids from other parts of the system. Also, there were two teams in this project: backend and frontend. When we started that project, everything was nice and small. Once it began to grow, frontend team started to ask backend to return more and more data from just one backend call. Data from 3, 4 or even 5 different microservices. We didn't want to but PM didn't really understand why because "it shouldn't be that hard". As time went by, our small id tables started to hold a lot more data (name, phone numbers, addresses and so on). Project grew fast and mantaining it became a huge problem. Looking back, I think it's really easy to blame the frontend team (I honestly don't know why they didn't want to make more than one database call) but that stubborness was a huge pain in the ass.

@rafaelbrugnollo 17 днів тому

From what I've studied (still just a beginner on MS) it's totally OK to denormalize and duplicate important data over the contexts. Using this Orders - Customers example, it's possible to understand that Name, Address, Phone Number belongs to Orders as well, in the sense that a Customer Address update from Customer context should not update an Address from an already processed Order for example.

@MikeTypes Місяць тому

Thanks for the explanation. Main issue I face is trying to sell the idea to other engineering stakeholders, who are very used to blocking calls, such an event driven system.

@deSemmel 2 місяці тому

Really struggle to see benefits of using separate services for order processing and customer details. It adds another layer of abstraction and error sources, the need of a strict contract, configuration overhead, versioning coordination, change negotiations. Microservices don't add anything of value here that will make development and change easier in the future. This approach will just result in horrible micro(service) management. These two models are part of the same domain and necessary to solve the same problem, processing an order for a customer. Where I see potential for Microservices is authentication, datalake ingestion, email notification or payment processing. I have seen too many services just containing a DB with one table and a few CRUD endpoints with 3 times more LoC of IaC. 80% of changes are policy corrections or dependency updates. Now imagine you have 100s of these flying around. Excuse my rant please.

@ContinuousDelivery 2 місяці тому

OK, I was trying for a simple example, but I would disagree that they are part of the same "Domain" if by that you really mean "Same Bounded Context" they maybe part of the same domain, but are certainly NOT part of the same BC. and as I say in the video, that is part of the definition of microservice. I guarantee you that Amazon don't keep their order management and customer details in the same place. For small, simple, systems, you are probably right, but I would argues that microservice is a poor choice for small simple systems.

@deSemmel 2 місяці тому

Apologies, I might have come across a bit too negative and I do agree that the scale of the domain has a big impact on the boundary design. My concern is that the adoption of microservices becoming the indisputable default - “We need a new invoice model? Let’s create a service for it” - driven by the false promise that they promote decoupling. They rather require it. In my experience, they come with a huge cost (as outlined earlier), which is not well communicated and they are hard to master (as you state yourself in the introduction). The vast majority of teams out there don’t have the resources or the problems that Amazon has to solve. Qualities like small, focused on one task, aligned with bounded context, loosely coupled can be to a high degree covered by code. Due to the agile “pressure”, these things are often neglected during development and are difficult to ingest into a service mesh later due to expensive refactoring of APIs and resource reallocation. I think microservices have their well-earned place but require experienced developers and DevOps skills. I would argue starting with modular monoliths and introducing microservices when required and the costs are justified. The decision should less depend on the size of the company and more on the complexity of the problem, which will grow of course ;) Btw, I really like your channel! Keep up the amazing work. I do learn a lot from it.

@ContinuousDelivery 2 місяці тому

@@deSemmel "Start with a monolith" is exactly the approach that I recommend! ukposts.info/have/v-deo/i2hynaeuqml_lqM.htmlsi=MPiOxWvo9rXjBuxd

@benisrood 2 місяці тому

@@ContinuousDelivery Using Amazon as an example is a total canard, sir. How few companies and products are at such a scale??? It's a very inauthentic example to cite.

@ContinuousDelivery 2 місяці тому

@@benisroodYou said you couldn't see the benefits, I was pointing out the benefits, they are essential for companies at Amazon scale, I agree that most companies aren't Amazon which is why I say that microservices are a bad choice for some companies. You don't need to be at Amazon scale for microservices to be useful, but they are a dumb choice for small teams building simple systems.

@alfbarbolani 2 місяці тому

Big flaw in this segregation strategy, and one that invalidates it in a lot of contexts, is synchronization. If the intra service communication is synchronous, then your customer service needs to wait for confirmation from all dependent services that they hace processed the “customer added” message, making your service architecture in fact a pile of RPC calls. But it guarantees that once the customer service confirms that a customer has been added, it can be used by dependent services. If you make the notification to dependents asynchronous, you effectively decouple services, but then users will complain that they cannot place an order for a customer even if the customer service has tell them that the customer has been added. Or that they blocked a customer from placing orders but orders can still be placed… A lot of synchronization issues pop around that are simply solved by not trying to replicate copies of the same data model across different independent entities and thus end up using a centralized database. Not saying that loosely coupled services are always useless, but certainly less applicable than the current hype wave says they are. That’s why lots of these “services” designs fail

@jasondbaker 2 місяці тому

Lots of these service designs fail because engineers don't implement the patterns correctly or they fail to account for exceptions. Let's take your example above. A customer service produces an async message which dependent services need to update their customer ID, but one of the dependent services doesn't receive the message in time and a customer's order fails. Respectfully, you make it sound like this is a critical and unaddressable failure mode. Message queues do fail at times and there are patterns we can implement to mitigate this failure mode. Yes, it increases technical complexity but that's what is required to make the platform easier to maintain and change.

@pnmcosta 2 місяці тому

Eventual consistentency asynchronous message communication with properly implemented replayability of messages will solve this issue, the order service can "wait" for a customer model for a predetermined time before it discards an order for invalid customer.

@ContinuousDelivery 2 місяці тому

This is always true in any distributed system though, the message based, ideal async, approach is a more realistic representation of real distributed systems. You can't build systems at scale that are synchronous, they collapse under their own weight, and the failure modes get more and more complex. At least a service oriented approach better surfaces the problems, and as others have pointed out, there are well known patterns for dealing with them. The truth is that distributed systems are always highly complex, the failure modes explode. You can NEVER guarantee that any communication will get through, so you always need to design for failure, and I'd argue that sync systems make that a lot more complex rather than less. I describe some of that here: ukposts.info/have/v-deo/gZGHhHeBiG2a2HU.html

@alfbarbolani 2 місяці тому

@@pnmcosta ah the lovely timeout idea. There is always someone that thinks that everything can be solved by a timer. I leave as an exerciser to the reader why it does not work. Clue: CAP theorem.

@alfbarbolani 2 місяці тому

@@jasondbaker so in order to make something “easier to manage and change” you increase its technical complexity? Surely I’m not the only one that sees the contradiction. Microservices is not an architecture to facilitate the partition of the problem domain. Or structure your code. If you don’t know how to do that, no architecture is going to save you. It is an architecture used where scalability is important and it is possible to isolate problem domains easily. When both conditions are not met, using it is wasting resources and time in vanity artifacts that do not add customer value.

@scottnash70 Місяць тому

Love the example of the Orders; it's a common issue in the enterprise systems I deal with as a PM. While denorming the account ID seems strange at first, it's MUCH easier to justify if you assume that the Accounts are in a third-party SAAS CRM system that has its own uptime policies (not uncommon). By definition you have to design with denorming so that it can operate independently. The more third-party systems, the more important this gets.

@chrisjones5046 2 місяці тому

my issue (which is not a valid issue) is that way back in the mists of time when I learnt databases. The person who taught me was really really really fanatical about normalisation. So doing things this way feels like ordering red wine with fish.

@christophjahn6678 Місяць тому

Did that person ever work on a real project?

@chrisjones5046 Місяць тому

@@christophjahn6678 he was very old, from a time before the internet, and in fairness most of the issues with our undergrad code came from not enough normalisation rather than too much.

@AerialWaviator 2 місяці тому

Great content, very concisely explained. That the common 'non-microservice' pattern (5:14) does not really have sharable name is a bit concerning. Perhaps the category of misnamed services should be referred to as 'interdependent services'.

@cameronosborne7405 2 місяці тому

We started the NestJS quest for our microservice solution. So far so good.

@foobar8894 2 місяці тому

I just build a monolith. So far so good.

@hfspace 2 місяці тому

nice vid. I came to the same conclusions when learning what microservices are upon stumbling on a distributed monolith at work

@stephendgreen1502 2 місяці тому

Autonomy truly is a big ask. It requires expert developers. If they leave, it requires that they be replaced with developers able to acquire expertise quicker than the previous devs could develop it, so work can continue autonomously. For example, an event gets raised by a microservice. No one team knows all the handlers for this event. Each only knows whether their microservice will handle it. So nobody sees the sequence of things that happen, nobody sees the big picture. So the devs need to be capable of understanding where their events and handlers fit in but without even knowing what the big picture is that are fitting in to. If they never get their handler completed, the event might never be handled, but nobody knows this. If a team needs a new dev to replace an expert who leaves, the new dev will not know if an event raised is ever handled. How can they find out? Go through all the codebases of all the microservices in various languages? What if these are moving targets and nobody knows all the codebases and which versions are in production? It all gets very taxing. Too taxing for an average dev. Autonomy is a really big ask.

@ContinuousDelivery 2 місяці тому

..Yes autonomy is difficult, but it is the whole game when it comes to microservices, otherwise what is the point? The people that wrote the original microservice article, James & Martin., included a diagram of a signpost like at a fairground ride "You need to be this Tall to take this ride" meaning that you needed the design sophistication to achieve the autonomy,.

@stephendgreen1502 2 місяці тому

@@ContinuousDelivery Did Martin even consider ACID and databases? Or is it a distributed implementation of pure function thinking, devoid of state, persistence, time, ‘side effects’ like that.

@ContinuousDelivery 2 місяці тому

@@stephendgreen1502 Yes, they were always building real systems with microsevices from the start, Microservices is really a big-systems approach. The Relational model is a powerful and useful one, but it isn't the only way to build systems. One of the things that doing things at Web-scale exposed was the limits of the relational model. Relational databases simply don't scale well much beyond 10s of thousands of users, and as soon as you dump that model, you have to take on all of the problems that it solves to make things work, and address it's limitations. ACID transactions in particular don't really scale well at all in distributed systems.

@stephendgreen1502 2 місяці тому

@@ContinuousDelivery Thank you. Much appreciated. I feel a corollary here might be that databases break the microservices architecture. If your domain is not for a huge scale global userbase and if NOSQL is not strongly preferred over RDBMS in your system, steering clear of microservices is probably going to be a good choice.

@ContinuousDelivery 2 місяці тому

@@stephendgreen1502 💯There is no "one-size-fits-all" approach, you have to choose the architecture that suits your problem. I think that the Relational DB model is an excellent and powerful abstraction that hides lots of difficult problems, as long as your problem fits that model. I think that way too many teams pick microservices, when they'd be MUCH better off with a simple relational DB backed system. Equally don't pick RDBMS if you are Amazon, they tried it, and it didn't work for them.

@DerDoMeN Місяць тому

The encapsulation/clear division of logic is from where I sit the only useful part of micro service design from a technical point of view (the rest is much better solved by just having those pieces either compiled into the same binary for the development purposes or spread out for production purposes but still a single app). The reason why/when microservices start to make sense is having far more developers on the project than it's sensible for a single team size and/or trouble finding enough programmers (again far too many for a single team is pre-requirement) with the will/experience for the same tech. So all in all microservices are a management tool with a side effect of influencing the code structure (pushing it into microservices mold) and not the other way around.

@anupam1 2 місяці тому

Agree with every statement said in the video. I have often seen that many teams use terms like agile, microservice architecture, ci-cd etc for totally something else

@TheBoing2001 Місяць тому

Again, it is very nice to define precisely the terminology. Sadly I fail to understand the example. I would argue that "select gmail from customer details where id" is the SAME "abstraction" that "Get /CustomerDetails/gmail?id=". The second one is idiosyncratic, not "micro". There is no less "implementation detail leaking", nor decoupling present in the example. Like so many people mentioned here, the problem is with middle management, who associate "micro" with simple, and scalable, team/development wise, not deployment/production/maintenance wise. The illusion of decoupling an scalability is strong in the industry, as well as testing. The understanding of scaling under load is weak, and could as well stay like this, because those edge case should be reserved to grown up. Adding latencies and lowering throughput is not wise in general, and a some point efficiencies should be valued with realistic metrics, like Watt per (useful)byte delivered to the user, not DORA ones.

@cameronosborne7405 2 місяці тому

Good video! Where can I get your awesome t-shirt?!

@ContinuousDelivery 2 місяці тому

I buy most of my T-shirts from HERE: ➡ bit.ly/3vTkWy3 Qwerty give CD viewers a discount the, so don't forget to use the DISCOUNT CODE: ContinuousDelivery if you buy one.

@underdog578 2 місяці тому

Very good presentation, as usual. I liked everything except when Dave said that independently deployable is the most important attribute of a microservice. I think this attribute is essential, but so is the fact that they follow the single responsibility pattern, should be loosely coupled, self-contained. You need all of these to deliver on the significant benefits of a microservices architecture.

@conundrum2u 2 місяці тому

in a previous job of mine I witnessed my "superiors" claim we were doing microservices whereby we were sending domain specific data all over the place. the services were not small, nor were they architecturally contained. it was the kind of environment where I would bring up, in a very respectful way, that something could be done better by doing x and y, and here's the proof. then I was ignored, no tasks were created to review the improvements at a later date. meanwhile the startup was hemorrhaging cash with our unnecessarily chatty infrastructure (because of the not-properly-contained services). there was no actual architect, btw. infrastructure was not discussed as a group, either. but they sold the company! the shell game never ends

@rbauer961 Місяць тому

Would it be useful to think of microservices as functional programming at the top level? Do you think choosing functional over OOP at src code would influence architecture positively?

@podell111 2 місяці тому

Would it make more sense to start with a typical monolithic design using a normalized Database? Then you can examine if the application needs a more complex design utilizing micro services

@alexrusin 2 місяці тому

Placing an order is just one aspect of e-commerce. What about getting the orders list? If we include customer information on an order, like Shopify does, and we have 250 orders per page, do we have to call CustomerDetails service each time we list orders? Wouldn't it be easier to do a joint in order service? Wouldn't that be somewhat slow?

@FiniteSteakMachine 2 місяці тому

You clarify that they're not microservices if they're not small, loosely coupled, and independently deployable. Not everything is an e-shop; many real-world problems have an irreducible complexity that shapes any correct solution. Now if you insist on splitting it up into small pieces they won't be loosely coupled, and if you insist on only adding network boundaries at loose coupling points, what's on either side of those network boundaries is no longer small. You may still end up with more separate services and looser coupling than an outright monolith, but now you don't have a name for it because it's neither a monolith nor microservices. Many other commenters seem to have come to the same conclusions in their own words. Additionally, you can't say that it's a problem for services to share a database schema (an unavoidable compatibility cost) then say the solution is to share additional network schemas (a completely avoidable compatibility cost). Every additional schema that has to be coordinated between services is a point of friction and risk indefinitely, whether it's a database or a network protocol is a relatively minor factor. These costs are not essential to the problem space, and the best you can hope for is that people working more independently more than makes up for it; but that is more often assumed than proven, especially with the extra bugs, edge cases, integration tests, and operational costs and risks of any additional network edge, especially at scale. Most of all I think "small" is the biggest problem here. Loose coupling and independent updates are very valuable to have, but "small" actively works against both of those goals, to a much greater detriment than just having an appropriate amount of code in an appropriately scoped service.

@willd1mindmind639 2 місяці тому

It requires a comprehensive system design and separation of concerns between system components with clear boundaries based on functional and performance requirements. Then you can determine which parts of the system can be implemented as microservices and which parts of the system do not. And yes, transactional databases and schemas exist for a reason and they are not a "problem" in larger solution domains, just like traditional web applications also exist for a reason and can also scale and are not a "problem" either. Everything is not facebook, twitter or even netflix where there is a very loose set of domain objects that can be managed as separate micro services.

@CosasCotidianas 2 місяці тому

1:17 _what you call microservice is different to what I call microservice_ : as well as in happens with DDD, sadly. Btw thanks for removing the boxing bell noise when you ask for subscribe, it was annoying.

@seagullmouse 2 місяці тому

This approach just shifts complexity to unpicking the messages flying around. Also requesting data via REST/RPC is slow.

@nezbrun872 2 місяці тому

Bingo.

@adrianpaulwynne 2 місяці тому

The data journey doesn't stop in the transactional system, but continues through data analytics and (increasingly) AI. The large normalised data model persists in part because it supports SQL, and data analytics tooling and people use SQL (not code). You can of course build an ETL (or ELT) pipeline to construct a data warehouse from microservices calls but this creates a complex dependency between two moving parts - the microservices architecture and the analytics requirement. Data Analytics VPs would be (rightly) wary of building a beaurocratic dependency on the (already stretched) development team to build and maintain this integration, and can't offer a good career path to people with this skill set in their own department.

@ContinuousDelivery 2 місяці тому

but you REALLY don't want to confuse the analytics problem with the transactional behaviour of the system, at least if you are build systems at any kind of scale. the performance profiles of these two VERY distinct bounded contexts are radically different. You are making me shiver remembering the bad old days of dealing with systems that stopped being able to process orders, when someone was trying to see what the sales profile was last month. For bigger systems, at least, you are forced to separate things like these, and so yes, the complexity goes up, but the systems keep working as they grow.

@adrianpaulwynne 2 місяці тому

@ContinuousDelivery the issue I'm raising is that the microservices architecture DOES force some deep integration (i won't use the word "confuse") between the two worlds (i.e. transactional and analytics) in a way that using large normalised schemas does not. Large normalised schemas provide a clean integration point that is technically simple, clear, well understood and stable. I'm not saying (here) which approach is better for particular contexts

@nezbrun872 2 місяці тому

@@ContinuousDelivery If you're interested in performance, constructing everything into smaller and smaller microservices for your dogmatic commandments and sacred cows, almost all of the processing will be in the distributed calling overhead. Maybe you sell cloud ;-) And you haven't explained how you achieve determinism and repeatability in your system. While that's fine for social media applications, it isn't for transactional systems. Unless you work for the Post Office.

@TauvicRitter Місяць тому

Will there be an extreme architecture with a single smart service powered by a LLM that can transform any input into any output. Would that solve all our problems?

@pierre-antoineguillaume98 Місяць тому

In this exemple, it means we can't process new orders if the CustomerDetails microservice is down. Why not replicating the data OrderProcessing needs into OrderProcessing ?

@pnmcosta 2 місяці тому

This is a great video as always. There's only one thing that I don't agree with how you said it. The order service should not get customer email from the customer service when it needs it, this is brittle, what if the customer service is down? The former should instead be fed the customer model from all create/update messages from the latter and store only the customer attributes it requires before processing orders instead. When focusing on the messages like you clearly pointed out, this is trivial to achieve imo.

@alexclark6777 2 місяці тому

The problem with that approach is what happens when the business adds a new requirement, such as a credit limit or a discount based on age of account, that is enforced at the point of ordering? If you have a million customers are you going to have the customer service also push one million attributes over to the order service to sync up and eliminate that "brittle" coupling to satisfy this new business requirement? Or are you going to poll & cache on-demand as each customer places an order?

@minor12828 2 місяці тому

The most relevant detail is that the team can manage their budgets. Microservices is a team thing. It is not related to tech or architecture.

@Cerebradoa 2 місяці тому

Does this mean that I cannot process orders if the customers service is down?

@ContinuousDelivery 2 місяці тому

No, but it means that if you need to do that, you will need to implement some form of reliable delivery in the messaging between services. But that is true for any style of distributed system.

@Cerebradoa 2 місяці тому

@@ContinuousDelivery Messaging Systems are about async communications, and a retry option could be implemented. I agree. However, we are talking in here about a real time operation, and if these two services are not aligned and treated as one, this misscommunication can happen (since they can be modified on their own). To me this approach is like encapsulating objects in projects, making the communication among them much more complex... For nothing). A real order processing service would not need a customer service. The customer data will be in it, and yes, repeatin data. That could be synchronized through messsges, as you said. Ideally, a team should be able to work in a service independently of other services. That's the real power.

@nezbrun872 2 місяці тому

@@ContinuousDeliveryAh yes, an enterprise messaging system... typically using a good old fashioned transactional RDBMS. These "small" microservices are going to be doing a lot of heavy lifting to make up for what they've lost due to recovering from loose coupling failures.

@EadwinTomlinson Місяць тому

I think autonomy is the real key requirement. If it's autonomous it's independently deployable however it can't be autonomous without bounded context.

@csvegso 2 місяці тому

The most annoying, misleading term for me in the microservices world is "autonomous". For me an autonomous service means it does not rely on any other services. After receiving a request it can perform that request by own. For me a service which needs to invoke 5 other services just to peform a request it received is simply not autonomous. Like a team is not autonomous if it relies on other teams to perform its job.

@krajekdev 2 місяці тому

Maybe the term is not misleading. I do real "autonomuous" services and within the teams we all are aware to not introduce any direct dependencies and if necessary, track and maintain them responsibly. I think it is just the usage of the word by some people that is misleading.

@jntaca 2 місяці тому

I prefer monoliths horizontally scalalable with load balancing, isolated service modules that processes no bussiness logic ( PDF rendering, notifications, credentials management, etc..) and a master to master replicated DB for writting and N slaves for reading. Microservices architecture is a pain for dealing with data consistency because you have to do complicated tricks to perform what database transaction rollbacks do in a simple manner.

@TauvicRitter Місяць тому

Add word document rendering and I'm your buddy

@podell111 2 місяці тому

Most clients don’t understand the complexity of what they are asking for when they ask for a micro services based design

@mirzamerdovic 2 місяці тому

In very crude term all of it boils down to your database schema design. If you build a database schema with unclear bounded context, or a schema that is hard to extend and evolve, you are bound to fail no matter the architecture you go for. What I am trying to say is I agree with the notion you should start with a monolith and as the time goes see if you need more distributes architecture or not, but none of this can happen without good data design. A good database design can withstand anything, new architecture, re-writes, and so on. Sorry for getting a bit off-topic here, and thanks for a great video!

@ambroise7972 2 місяці тому

According to me, microservices are most of the time used because most developers have poor software design skills. They become quickly overwhelmed as a codebase they work on grows. This is because they have a limited toolkit : every problem is solved using an entity centric CRUD way of thinking with some kind of "services" were all logic is pushed into. They know this will quickly transform their codebase into a big ball of mud. So before it become a mess, they create a new clean software project (they call a microservice) to put the new logic into. In the end, they have lots of poorly designed codebases, but they can manage them using basic design skills they are familiar with. Of course, the cost of this is catastrophic on the long run and they would better improve their design skills. The problem is that they don't know how much time they will need to improve enough, they don't know if they will succeed to improve enough, and above all business have a very short time view and press them to deliver as quick as possible.

@dana-pw3us 2 місяці тому

original way of thinking

@Shocker99 2 місяці тому

1:37 "Just a Labe" 😉

@lost-prototype 2 місяці тому

Ugh, dealing with others' mess over this right now.

@zgeorgem 2 місяці тому

can each service have it's own database but each communicate with each other's database via a redis cache?

@piff57paff 2 місяці тому

If you integrate on DB level you take away the ability of the services to evolve their internal data representation independent of their public API. You wouldn't want to get into the position that you can't add or change an attribute of your internal representation, because somebody depends on it. Been there; no joy. What you can do, is keeping local copies of a subset of the data handled by another service. Already structured in a way that allows efficiently working with it for your use case. You could get that data by listening to events and potentially pulling some data from its API if the event is of interest for your own service. But this will then require you to deal with keeping that local copy accurate, which comes with its own set of challenges.

@TauvicRitter Місяць тому

Looks to me it's a discussion between using a blunt and simple tool that works pretty ok and a sharp knife that cuts precise but requires a skilled and experienced worker. We have to think about the people that will design, develop, maintain and use the system. Can they fully understand it's implications.

@CristianYones 2 місяці тому

I still not get it why a common message interface is considered less coupled than a common good normalize database schema. In your example, if I need to start considering the postal code, or age, or anything else in the order processing, I'll have to modify both services and the OrderProcessing database. The deploys will have to be coordinated also. Having two services with one database is just much simpler, I modify only the OrderProcessing service and that's all.

@piff57paff 2 місяці тому

If you integrate on the DB you have a hard coupling between the services and changes on the "internal" data model now require a coordinated rollout across services. You also can't easily evolve to different DB types, say from relational to key-value. With separate services you have more choices in how you model data and also how you expose which data and if you create audit logs for sensitive data access. Sure, if you want data not previously exposed by the other service, you need to reach out and wait till they deploy that change. But you can go about your business until you pull that data, as their change should be backwards compatible, meaning there is no need for coordinated roll outs.

@karlstenator 2 місяці тому

If a microservice fails in the cloud with no user base to notice, does it cause an outage? It matters not, for the microservice has failed.

@coldblaze100 2 місяці тому

W comment

@ContinuousDelivery 2 місяці тому

If any software with no user base fails anywhere, it doesn't matter!

@LudvikM Місяць тому

The version of "micorservices" that years ago I had to deal with, and learned to deeply hate, was a perversion of the original concept. For instance, very often I had to investigate issues with updates consisting of a sequence of calls to 2 or 3 microservices, only one of which (the 2nd or 3rd one, of course) failed and left data inconsistencies behind, as of course there was no "transaction" involving the whole sequence that you could simply rollback. Not knowing then that those were not true microservices, my conclusion was that the concept itself sucked. But as it almost always the case in this business, concepts are usually fine, and it's the people trying to shoehorn them where they don't fit that suck. In my case, the "guru" that designed those "microservices" could not be convinced, no matter how I hard I tried, that someting was 100% wrong with such design if data inconsistency was a daily result of it. But everyone (else) thought he was some kind of genius. Go figure.

@robmyers8948 2 місяці тому

The magic that is present in SQL query design seems to take a back seat in your example of the orders and clients micro services. A simple SQL join would provide the needed data, but in your example each of the services would need to manage this join logic and have the underlying architecture to support inter service messaging, all requiring additional design and maintenance. Performance would also be impacted.Sometimes simple concepts as you have discussed miss all that is required in what you haven’t discussed.

@nezbrun872 2 місяці тому

Precisely. Not to mention the lack of transactional integrity and problems for DR, resilience, HA etc.

@christophjahn6678 Місяць тому

What was the biggest system you ever worked on? What you say is right, as long as certain assumptions go with it. And scale is the elephant in the room here. If we are talking about 200k transactions per day with a relatively linear distribution that is all fine. But there comes a point when row-locks in the database, as just one example, become a real issue. There is a reason, after all, that XA transactions never became very popular.

@nemea6698 Місяць тому

Nice

@zshn 2 місяці тому

Microservices is really hard. Most projects that think they do microservices are making exceptions/tradeoffs which makes the system NOT microservices architecture.

@Ureallydontknow Місяць тому

This approach turns an easy problem into a distributed concurruncy problem. You will also need more microservices to do the house keeping. Its better to abstract away the database as a monolith so the db team has freedom and control over the db system architecture and data consistency. That way load balancing and hardware maintenance is easy to perform. I also don't see how the monolithic repo is better.

@DeWhiskeys 2 місяці тому

Ok, but what does autonomous mean? Services may require functionalities offered by other services in order to carry out a task. Services performing Sagas / Orchestration can't be autonomous

@ContinuousDelivery 2 місяці тому

Autonomous means "independently deployable", and that means that if you build a service that depends on another, your job is to decide what your service will do when the other is not available, because in a distributed system there are ALWAYS times when the other service won't be available!

@DeWhiskeys 2 місяці тому

@@ContinuousDelivery I appreciate you taking the time to reply, but it feels like the conversation just shifted from "autonomous" to "independently deployable". So service A needs service B to perform task X, perhaps you decide to return an error if service B is unavailable (just an example). This makes service A and B "independently deployable" according to your definition, but a missing service B makes A partially working... Sure, it's a partial failure over a complete failure, but doesn't look like independence. Maybe this is my background in system engineering talking, but I would probably try to find a different way to express that feature.

@hb-man 2 місяці тому

In order for B to stay independently deployable, service A must not care about a specific version being deployed. The effort clearly seems to be on the consumer side. However it also constrains B, as it cannot be changed at will without considering the consumers, or it will turn useless-as-a-service quite quickly.

@georgebeierberkeley 2 місяці тому

For our app, it's the data. The customer and user table determine access, calculation settings, etc. It's hard to think about how we would separate that into siloed tables. That makes microservices difficult to implement for us.

@ecodoge 2 місяці тому

Perhaps microservice architecture is not the right tool.

@ContinuousDelivery 2 місяці тому

It's certainly possible that microservices aren't a good fit, they aren't right for everything. But I think that the problem you mention is less about microservices, and more about normalised data. I'd say that normalised data is a useful tool for small scale systems, and a poor choice to larger-scale and more distributed systems. I think you have to let that go as the scale and complexity go up, microservices is one way to do that, there are others. Normalised data is nice, if it fits your problem, because it hides lots of nasty complexity, but that complexity is always there, just beneath the surface and it leaks out if you need your systems to be fast or scalable. Then you need to find other solutions, and face the more fundamental realities of "eventual consistency" - Software isn't simple!

@cthoadmin7458 2 місяці тому

Can someone tell me what "aligned with a bounded context" means...

@nezbrun872 2 місяці тому

I read it to mean well-defined & limited input and functional criteria. I also took it to be some fashionable IT manager buzz jargon that's doing the rounds to try to make others look stupid.

@cthoadmin7458 2 місяці тому

@@nezbrun872 I do find terms like this quite difficult to understand. It would make learning microservices based architecture (from a testing point of view) easier if plain english were used.

@SianaGearz 2 місяці тому

Replication faults are stinky. Compact monolithic design is so much simpler and more robust at the start of the journey or if you're targeting a field with very limited saturation potential. Being the kings of a very niche little field is a comfortable place to be.

@diononeofyourbusiness8634 2 місяці тому

I am honestly still appalled at what I am seeing in my day-to-day in regards to loose coupling of systems. I think that it is a symptom of small minds thinking that everything needs to be assured and strongly linked, instead of trusting the unknown other applications. One neat example that I still appals me is that in this day new applications and services are developed that do not use UUIDs or ULIDs.

@alanonym8972 2 місяці тому

I disagree at least partially with the first take. Definitions are made up and WILL BE different from one individual to another. If only just because the definitions of the words used in those definitions are themselves based on made up definition. If you want to "talk" about the same thing when talking about microservices, you need to agree with the person to which you are talking to about: what "small" is, what a "task" is, what "Bounded Context" means precisely, how "autonomy" is defined (because truly autonomous services are what we usually call "monolith"), etc... Because when you say that my services need to be small to be called microservices, I would argue that 50K lines of code is either big or small depending on the person or the team you are talking to. Some might argue that the number of lines of code is irrelevant, we should talk about small complexity, but then how do you put that into something that everyone will agree on ? Of course we need some kind of common ground to talk to each other, but there is no point in trying to be rigorous about definitions (especially the "original" definition), because you won't.

@matswessling6600 2 місяці тому

but agreeibg with the person you talk about these words is exactly what definitions are there for.

@Tkdestroyer1 2 місяці тому

Just because something isn't perfect, doesn't mean it shouldn't be done. Using definitions improves communication, despite being imperfect.

@alanonym8972 2 місяці тому

@@Tkdestroyer1 Yeah I completely agree, but we will never talk about the same thing when talking about microservices. You should never assume that the person you are talking to has the exact same definition as the original definition because definitions are not precise and they mute by nature. So if you want a conversation with a person that will not be sterile, you should always (vaguely) agree on the terms you use in that situation, and not try to enforce one based on an arbitrary decision like "the original creator meant that...", because no one actually cares about that. At this point, "microservice" is mainly used to vaguely describe a type of architecture. It is the benefits/drawbacks of this type of architecture in the specific context in which your discussion takes place that matters, not what the definition is.

@andrewbrown8463 2 місяці тому

Maybe the biggest problem here is how you define what you are splitting into Microservices. It makes sense to split logging, storage, email sending, payment processing etc into microservices but the idea of splitting up core business logic into Microservices and then accepting the terrible compromises suggested in this video make no sense at all. The idea of any technical solution should be to solve problems, not make new ones. There are very few use cases on the entire planet where an order processing system is so big it should ever be split in the way this video describes. According to the Microservices guide book that came out of Uber, the entire idea of Microservices is to share infrastructure and I can't see any reason why a shared database is outside of the general design principles of Microservices, or at least the way it was defined by Susan Fowler.

@jsonkody 2 місяці тому

I prefer nano or pico-services .. Erlang/Elixir :D

@TauvicRitter Місяць тому

Do these really exist? Amazing

@HoD999x 2 місяці тому

i think the problem can't really be solved. the total complexity goes up with each microservice because you *will* add some redundancy.

@rossbagley9015 2 місяці тому

Total complexity does increase, but complexity per-team should be more manageable.

@username7763 2 місяці тому

@@rossbagley9015 The longer I am into my software career, the more I am convinced that this doesn't happen. You pay for complexity no matter what. Yes some code structure and information hiding can help in some areas. But if your system architecture results in something that is overall more complicated, you will pay for that complexity several times over.

@bobbycrosby9765 2 місяці тому

@@rossbagley9015 I would add: if this doesn't look like a net win for your organization, then microservices probably aren't for you.

@dennisribeiro5262 Місяць тому

Is google ecosystem a case of microservices?

@buffyf2680 2 місяці тому

I've always been curious: could databases like Cassandra be called a shared database? :)

@jasondbaker 2 місяці тому

I think any database could be considered a shared database in this context if two distinct services performed operations against the same database schema or keys. So yes, Cassandra or Redis or DynamoDB could all become shared databases given the service implementation.

@ContinuousDelivery 2 місяці тому

I don't think that this is about the technology, so much as the patterns of use. If you share data between two apps, or two services then yes, it is a shared database. Sharing data via a database is risky because of the degree of coupling. So if my app talks to yours via Cassandra how do we avoid me breaking your app, particularly if I don't know that you are using my data? That is the problem here. This is a design problem. I think that in general "Messaging" is a better model for exchange of information, because it can be more explicit about structure and context, but that is only down to design too. There is no tech that solves this problem for you!

@PaulSebastianM 2 місяці тому

I think some so called microservices are in fact large SoA systems.

@jasondbaker 2 місяці тому

That's my experience as well with most orgs claiming they have a microservice architecture. Oftentimes each service is supporting an entire service domain with numerous endpoints. I don't get too hung up on this though because it's still a net win if they have clear service boundaries and no shared databases.

@GackFinder 2 місяці тому

Yep. And SOA architecture is just a CORBA architecture, invented in the early 90s.

@PaulSebastianM 2 місяці тому

@@jasondbaker yeah well there is the flipside of having one microservice per endpoint and that is extreme as well. Ultimately it's about consistency AND context boundaries. Cohesion. What changes together stays together.

@stupid-handle 2 місяці тому

While I could personally agree on the fact most "systems'" "architects" out there are absolute trash, tagging a microservice as fully "independently deployable" and calling it *the accepted definition* by the *academia* seems to me a bit like saying you should get into your car by the other door because it always faces the side walk. It is up to you to determine what a microservice actually is for a given project and how it behaves and interacts with it's environment, and that only means to a newcomer the inner architecture has something to do with microservices. This is exactly the same as your definition at the end of the video, where you propose a scheme that shouldn't either be taken as a microservice per sé, because each of these would doing more than literally just one task. While I agree on these being independently deployable as a general rule to restrict garbage, there are many things highly scalable and good perfomant that can result of systems without that particular requirement if you know what you are doing. It's not all black or white. Besides these border cases you showed at the beginning of the video, and besides many others much worse I've had to suffer over the years, I also agree this is where most of "developers" fail. I've come down to the conclusion that no matter how they prefer to be called, a java, python, or you name it "programmer", is actually a scripter. They don't write software, they write scripts, and therefore they shouldn't be given any other tasks that writing for the web or the likes. Don't ever dare to let them define any sort of architecture. It's even the just for the web and they can't get it right. Or maybe it's just my experience, but for some reason, programmers that are not bound to the metal have proved to be so completely useless for me that I simply don't use them anymore, and much less if they are from generation y and onwards. Absolutely hopeless.

@aslkdjfzxcv9779 2 місяці тому

@jsdavo 2 місяці тому

I would drink a pint with this bloke.

@MykhayloS 2 місяці тому

There are good microservice platforms and good monolithic platforms. Neither is better by definition.

@ErnaSolbergXXX 2 місяці тому

Micro services are only truely possible if it doesnt accept any incomming requests and doesnt do any external communication. If it do any of it, there have to be some sync between the applications on how they should share data/communicate and it doesnt really make any different if this is done through rest api or database schemas.

@phoenix-walker 2 місяці тому

Just because microservice have to sync some communication doesn't mean they are not possible. The contract design is still a thing that needs to be adhered to. So long as service A continues to honor the contract with any dependent consumers of its events, then how it generated those events can be changed 100s of times. Sophisticated software companies, have had teams rewrite whole microservices without issue. The issue is that too many people think they are doing microservices, but they really aren't. They are distributing a monolith.

@Rcls01 2 місяці тому

I would say most of the organizations I've worked with struggle with two things: independently deployable and being SMALL. While the first is simply a matter of adopting some things like versioning if you introduce breaking changes, these services end up being massive in size. What I've always done is write one function that does one thing well and that's the whole service.

@arion_vulgaris 2 місяці тому

I suspect you must be using shared DB schemes then

@sheep2909 2 місяці тому

Videos like these always make software development, especially architecture and devops look like communism: "That's not real microservices!", "That's not real agile!", "It works, you're just doing it wrong!"

@michaelrstover 2 місяці тому

"Focused on one task". Does this actually mean anything? Everything is supposed to be focused on one task. Every function, every class, every package, now every microservice. I don't think there's a non-subjective way to define "one", and I find this particular phrase worse than useless, because it seems like it's saying something concrete but isn't.

@ContinuousDelivery 2 місяці тому

I suppose that to some extent it depends on your level of pedantry, but my pragmatic approach is: can you accurately describe what it does without using the word "AND"? "It calculates the VAT for an Order" NOT "It calculates VAT for an Order and dispatches the order"

@michaelrstover 2 місяці тому

"It handles Orders". There, one thing. "It publishes stuff". One thing. I would just leave that phrase ("Focused on one task") out and describe it as being about a given domain boundary, where the boundaries are arrived at in a pragmatic process that finds where the communication needs are smallest and most amenable to async, event, or message passing styles of communication (and not synchronous functional style). I think my main point I would press is that every different business is likely to find where their most effective boundaries lie to be different from other businesses, as the communication needs will be driven by the business needs. I don't believe there's a universal ontology for everyone to follow. Does one thing? Meaningless. Divided in such a way that communication and dependencies are minimized? Useful.

@cj8481 2 місяці тому

In this example; if it is a requirement to be able to look up your order history including account details it would be incorrect in case an account changed (address etc) As such we would need to have more account details beyond accountID stored in the OrderProcessing when an order is placed. In this case it would be a definition of an account in the bounded context of ordering.

@ForgottenKnight1 2 місяці тому

Depends on what data the business needs. Your approach might also be incorrect if an account can send multiple orders to multiple addresses different from its main one. In that case, an order would have to contain delivery details: address, contact phone number or contact email, in case it has to be picked up in person, etc.

@cj8481 2 місяці тому

@@ForgottenKnight1 yes good point. Depends on requirements :)

@swedishpsychopath8795 2 місяці тому

FAKE: Object Oriented Programming (OOP) with classes and inheritance was invented in Norway and implemented in the SIMULA-67 programming language.

@itskittyme 2 місяці тому

I use µservices

@dorcascross7115 Місяць тому

'promosm'

@stephendgreen1502 2 місяці тому

Unattainable for many domains. What if multiple ‘microservices’ need to share a common login system? If the login system is a microservice too then how do you avoid coupling and maintain autonomy? It might seem possible, but for most developers it is not possible to do it well enough. One microservice might need a tweak to the way logins work to provide security for a new feature. They therefore need the login microservice to be changed. But how does a team change it without affecting other microservices? It is probably not attainable. Not in the real world. Too likely that a change will undermine autonomy and loose coupling. If microservices had to be strictly developed to retain that title, hardly anyone would be able to use them, except if their business is mostly about streaming like Netflix.

@ArthurBugorski 2 місяці тому

Sorry, but why are your microservices logging in anything? The client shouldn't be interacting with each individual microservice. There should be services that clients interact and it validates requests, and the rest respond appropriately.

@Tomaszevics 2 місяці тому

Loosely coupled doesn't mean it's not coupled at all, it means that it's designed the way that there's as little coupling as possible. Login (authentication) service is a good example for the necessity of some coupling. The way you can achieve the loose coupling even if you need a tweak to the way logins work for one of the microservices is that you keep the compatibility with the other microservices as well. Once you have an interface defined and in place that other services rely on, you must avoid making breaking changes on it. This doesn't mean though that you can not add features to it. I agree though that avoiding changes that undermine autonomy and loose coupling is not always easy and it can be tempting to break these rules...which isn't necessarily bad, it's just not microservice anymore.

@stephendgreen1502 2 місяці тому

@@Tomaszevics Then you need to know about the other microservices in order to change the login service. That means the teams are not autonomous.

@jasondbaker 2 місяці тому

We use versioning, both at the API-level and message level, in a producer or upstream dependency to mitigate these sorts of changes. That way we can add or change the functionality in our login service without blowing up services that may depend on the login service.

@adambickford8720 2 місяці тому

Usually that's where some kind of standard comes in. For example, all services will have to support the same JWT IDP (or something federated) to authenticate so each service can handle authorization.

@IronCandyNotes 2 місяці тому

Microservices are services that are stripped down... What is that not stripped down service? Something that throws a whole website at you. Microservices are the things you call with XHR or fetch in your browser and what allows SPAs to do IO. All the other stuff n checklists are just arbitry made up after the fact BS-lists by consultans n know it alls. A cgi/php script that echos some JSON qualifies as a microservice... as it just like some unix tool can do one thing very well on demand and makes the MVC-framework-monolith unnecessary.

@natescode Місяць тому

All microservices problems are solved by creating another microservice

@renanmonteirobarbosa8129 2 місяці тому

microsservice comes from an old concept of parallel computing and there is decades of studies on the topic which get hidden by a bunch of hype and useless informationa and misconeption online.

@GackFinder 2 місяці тому

Wait... are you saying that microservices is basically just SOA, and that SOA is basically just CORBA from the early 90s? If so, I agree with you.

@renanmonteirobarbosa8129 2 місяці тому

@@GackFinder What ppl call now k8s microsservices or similar is just asynchronous swarm parallel computing

@illegalsmirf 2 місяці тому

That they’re shite? Could have told you that

@benisrood 2 місяці тому

You forget the more crucial requirement: does the product have such scaling requirements that it necessitates this architecture? The answer for rhe majority of systems is absolutely not! It is most definitely *not* simpler to decouple the entire system and data, and unless you are Amazon-scale, you don't need to separate out everything to implement order fulfillment for an online store. The more logical requirement would be to accept that the majority of the steps in order processing and subsequent fulfilment are indeed asynchronous and need to be modelled as events. This does not intrinsically require a microservices architecture. You are putting the architecture before the real model, just like everyone else did when *they* went around selling microservices, and now you have the gall to blame us for being too stupid to implement it properly. The problem is, just like all the other salesmen of IT architecture, you too elide the true concerns and requirements! You act like you aren't selling anything, and are trying to help, but if thats true I'm afraid its *you* that are misunderstanding why so many so-called "microservice" systems are wrong and why people are doing it wrong. It was sold under false pretenses to the industry the first time, and you aren't actually doing anything to truly ameliorate it. Saying "I am just using the correct definitions of terminology", and "we need to understand these core concepts" doesn't get to the root of the problem, especially when for starters you think the problem is that products *do* genuinely need this type of architectural pattern. Especially from the ground-up, most businesses dont have the requirements thay necessitate it and they never will. That a microservices architecture supposedly allows management to have "smaller, agile teams" is not an actual business necessity for creating systems around microservices, and even mentioning it is deeply suspect.

@matc8085 Місяць тому

The industry suffers of BDD, buzzword driven development

@TheHellishFrog Місяць тому

Again! A few guys on internet monopolized meaning of "microservice" word - just like microsoft monopolized word "window". I see little value in the loose coupling. I have seen a lot of bad designs that were totally loosly coupled - and I won't use it as something mandatory. Still - I call my architectures microsercices. Sorry boss.

@nezbrun872 2 місяці тому

This misses the point of transactional integrity, and will have a profound effect on DR, resilience, high availability. and point-in-time recovery. How are you going to test and reproduce problems to fix them when you're so loosely coupled? Your dogmatic model is just kicking the can down the road to make it a problem for someone else to hack a data patch in production when things go wrong (just like the Post Office), and make it very difficult for you to diagnose & fix longer term, no doubt by which time you'll be off somewhere else ;-)

@ContinuousDelivery Місяць тому

It's not my "dogmatic model" it is how microservices are defined!

@ANONAAAAAAAAA Місяць тому

For the 99% of the cases, relational databases are by far much reliable and performant than your ass architected microservices which require joins outside of the databases. Don't build microservices, start building monoliths which harness the power of RDBs as much as possible.