Event Driven Architectures vs Workflows (with AWS Services!)

Рік тому

Event based applications and workflows are two ways to coordinate actions across a distributed system. Both have pros and cons and are appropriate in different circumstances. In this video, learn about Event Based architecture vs Workflows and how they could be built using AWS services.
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
📚 My Courses 📚
AWS Lambda - A Practical Guide - www.udemy.com/course/aws-lamb...
🎉SUPPORT BE A BETTER DEV🎉
Become a Patron: / beabetterdev
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driven Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#aws
#SoftwareDeveloper

КОМЕНТАРІ: 71

@omrigoldberger9395 Рік тому

There's a very good article about it called "Choreography vs Orchestration in the land of serverless", TL;DR: Use both in the same project, using orchestrators inside a bounded context and choreography between bounded contexts.

@BeABetterDev Рік тому

Good article!

@tracy_gao Рік тому

Did the author said how could define the boundary? AKA when to use orchestrator and when to use choreography

@aliveagain Рік тому

I was wondering if he would end on that note. Seems like together they would counter each other's cons.

@getrasa1 Рік тому

Love the way you explain things. Very clear and concise. Great work!

@europana7 3 місяці тому

Be nice if channel owner just mentioned a quick appreciation note for your generosity …

@julianfranco7689 Рік тому

Fantastic video! Extremely clear and thorough on both patterns

@mahdyyk Рік тому

As always, this video covers everything in a comprehensive manner and makes it easy to apply practically

@JohnS-er7jh 9 місяців тому

so glad I found this channel, much appreciate your effort. I am just learning AWS and this mthod of teaching find is so much easier to understand. Whereas others will just mention Services in silos and then start drilling down into all the granual info on each Service (without even explaining how it relates/connects to other services (making it difficult to get your head around it all).

@BeABetterDev 9 місяців тому

You're very welcome and welcome!

@flashflexpro 6 днів тому

This is a great video, and can be greater when adding the monolith client retrying approach because tons of people are still stuck in there.

@deedoi Рік тому

Thank you very much for your explanation, which is nice and easy to understand. 👍

@chetan3283 Рік тому

Thanks for sharing this information, useful 👍

@et4493 Рік тому

Great job man. Thanks

@LawZist Рік тому

Great explanation!

@turboboostcash3876 Рік тому

Thank you so much for the explanation.. pleas can you do a video more on Aws IOT service and also the more on database

@Guzguz28 Рік тому

Thank you for a great presentation

@BeABetterDev Рік тому

Glad you enjoyed it!

@pengdu7751 Рік тому

great video. it reminds me of the difference between OOP (mapped to EDA) vs non-OOP (mapped to workflow). Solving the same problem but modeling the problem differently which leads to various tradeoffs

@TotallyNotAuroras2ndChannel Рік тому

Wow. This is what I just said.

@explorer26 Рік тому

Thanks for the video and example, learned new kinds of stuff on EDA and workflow. I was thinking, in the case of the EDA approach again for monitoring, maybe we should log or publish the state and the success/failure of every operation. Triggering a lambda on failed operation/step could also help us in unwinding initial steps and marking all of them as invalid. Does it make sense?

@benisrood 3 місяці тому

There are so many more sub-events in the logic and execution in handling the transaction steps in a distributed system just for placing the order based on the user's shopping cart that you gloss over. That would be enough for a single video alone. The credit card service would be inside the order placement, just for starters! Before doing any subsequent fulfillment steps let alone packaging, a hold would need to be placed for the required funds on the user's account with the card issuer 😂 You need something like Kafka/Kinesis as an event log to distribute the events to appropriate consumers, etc etc. I do agree that the Workflow approach has significant benefits, and you are right about the costs and tradeoffs of orchestration with this approach. I just think that you need to go into a bit more detail with the event-based architecture side, because your diagram is extremely oversimplified. And I don't think IoT is an essential aspect of the system at all, these sorts of systems functioned before such devices even existed.

@shubhamdhingra6089 Рік тому

Very helpful!

@viewerone 4 місяці тому

With event driven you could use traces for observability to visualize what step you’re at.

@ViktorSchmidt 6 місяців тому

Looks like absolute overkill architecture for 99,9999 % of all apps and businesses 🤔. But, good explanation and overview of AWS madness. 😊

@deepsharma9056 Рік тому

Informative! What s/w have you used to show the workflow?

@arkster00 Рік тому

We heavily use Step Functions at work. We don't require a rollback in the event a problem happens but we do need to know that a failure happened. For that we use Cloudwatch integration with slack that monitors each lambda in the workflow. Note that in each step function task that executes a lambda, if mandated you could essentially add the retries, exponential backoff and trigger details as well to reprocess the lambda and perhaps move it into a DLQ for redriving it at a later time. BTW, thank you for all the AWS content and easy explanation that you provide. I've benefited a lot from it.

@cristiannechita2032 Рік тому

There is another option if you are using Standard step functions. You could create an Event Bus rule that reacts to Step Function transition to Failed and as an action it can publish the message to SNS and from SNS you can send it everywhere. But this is an option only if you use Standard. For Express I usually use the Async invocation and let the caller deal with the potential failure of the Step function.

@arkster00 Рік тому

@@cristiannechita2032 Is there a benefit to doing that using event bus instead of having the step function routing the error message and input payload to say another step for SNS in the step function itself? I've tested this and it works well when a lambda fails and the retries are exhausted since step functions are async operations.

@cristiannechita2032 Рік тому

@@arkster00 The benefit that I see is that you keep your step function cleaner (without putting those transition from various nodes towards a common error handling task). Also, there are Tasks without Lambdas (e.g. Pass) that can apply transformation and those can fail as well. This should not replace the retry operations of your lambda step (it makes sense to keep it to deal with any transient errors).

@arkster00 Рік тому

@@cristiannechita2032 Thanks. Something to think about.

@josephattabenninjr7317 Рік тому

another great video as always!

@BeABetterDev Рік тому

Glad you enjoyed!

@kipsangjacob270 Рік тому

Awesome video

@dinushkam2444 Рік тому

very helpful video

@fooflateka Рік тому

@Be A Better Dev can't you use Distributed Tracing to see status of orders in SOA?

@mayukhee1 Рік тому

what will be the best template in lucid chart to replicate the step function workflow part?

@pythonbrothersandfamily 7 місяців тому

you are a super hero. thanks a looot

@priyankaojha1 Рік тому

Is Prefect a good tool for production ready , asynchronous microservices based workflow orchestrion?

@hectorharris4051 22 дні тому

Hi what do you think about using SQS + Dead letter queue in event-driven architecture to help monitor

@isohumulone Рік тому

I am not sure what is happening between the two lambda functions that can determine the success/timeout/default of the first Choice node. An iot device? Like a sensor? Is it forwarding data? What reads that data?

@outofbody4788 Рік тому

can you do workflow withut step functions?

@edikzlotnik1 Рік тому

Great lecture!! Which tool do you use to draw a visual representation of architecture ?

@BeABetterDev Рік тому

Thanks Eduard! I used Google slides for the first part, and the Step Functions Studio Editor for the second half. Cheers.

@pollypravir5378 13 днів тому

Thanks

@dEXtER2ks Рік тому

wouldn't you have to revert changes made in a step in case of failure when using step function approach, or you would simply correct the failing part i.e ask customer for another credit/debit card and retry? In my experience only durable functions on azure have the functionality to re-submit the request, i am sure there would be something similar on AWS end.

@easycloudcomputing6901 Рік тому

Well it depends on where you redirect the customer. If you want your customers to pay with another channel then you can build your logic in triggering another lambda and at the same time route the user on the frontend of payments channels list, so that user can select the payment channel of his choice. Auto resubmission will make user stuck in the loop.

@sankara-sabapathy Рік тому

Hi, Thanks for this video. I would like to know why we cannot maintain microservice architecture that are handled by individual small team while using step functions? Still the step functions can be created or maintained by the architect and we can have different teams to maintain the services right? Thanks.

@cristiannechita2032 Рік тому

Look at the step-function as being part of a micro-service. That team owns everything regarding that workflow (from it's definition in IaC till deployment).

@BeABetterDev Рік тому

Christian's point is spot-on.

@mr_possible6197 Рік тому

So basically if my application has 3 services working independently, i can assume that those services can be monitored separately using step functions right ? Its to say, micro monitoring sortof a way, eh ?

@sankara-sabapathy Рік тому

Got it. Thanks Cristian.

@TotallyNotAuroras2ndChannel Рік тому

This really brings back memories of when I used to use OOP in C++ to create an event driven application. Thanks! Now, let me implement this in OOP Python first.

@xpkareem Рік тому

What are you doing step-function?

@jasonsimmons3959 Рік тому

Step functions are great, but I’ve found monitoring to be a weak spot. The state display on the diagrams is very rudimentary, and you’ve got to use other tools and work with the logs to actually get a real monitoring dashboard.

@austinlowther4156 Рік тому

Great content, I've been watching you for a few months. I wanted to give feedback on the timing at the end of your videos with suggestions. Within a few seconds of suggesting other videos and overlaying them to be clicked, the video ends and your suggestions are no longer available. It would be nice to have a bit more time to react: read the video titles, determine the relevancy to what I am trying to learn, and then clicking.

@akritworanithiphong Рік тому

What do you use to draw the illustration? 9:20

@egomezr Рік тому

I'm almost sure this is the editor for StateMachine (a.k.a StepFunctions) in AWS.

@thealys Рік тому

Nice

@rahulvarma2100 Рік тому

Good video. Reality is fancy bla bla architects see the first diagram on event driven choreography as a short running stateful work flow just because there are arrows in diagram & then bar of argumentation starts with integration SMEs 😂😂

@mihai6564 Рік тому

good video

@BeABetterDev Рік тому

Glad you enjoyed!

@rishiraj2548 Рік тому

👍💯

@Euquila Рік тому

Workflow is Event-driven with global state.

@MindMusicKushMittal Рік тому

all workеd

@yansongguo8354 8 місяців тому

Not totally true😊 In the Eda design you are using SNS and it will generate chatty conversation between teams And if we focus on microservice only following the essence of DDD Each step in Workflow could be a service so just handling service with well defined bounded context will fit team responsibilities very well

@honjiaful Рік тому

well, good video but seems this is not event-driven.

@sarc007 Рік тому

Customer Charging happens first and rest other events follow

@BeABetterDev Рік тому

Not always. Cards are usually pre-authorized for a charge and charged only upon shipping.

@philtherealdeal2435 8 місяців тому

While I love this vid, I think @sarc007 has a point. This is almost never the case in reality. Companies want your money ASAP and will confirm pre-auths in a nightly batch, or up to 3 days later. If your shipping flow gets stalled, you don't want your pre-auths timing out. It feels like you used artistic license to switch them around so you could make your point about observability.@@BeABetterDev