AZ-900 Episode 15 | Azure Big Data & Analytics Services

AZ-900 Episode 15 | Azure Big Data & Analytics Services | Synapse, HDInsight, Databricks

День тому

Microsoft Azure Fundamental full course.
Skills Learned
- Describe Azure Big Data & Analytics Services such as
- Azure Synapse Analytics
- Azure HDInsight
- Azure Databricks
Study Materials: marczak.io/az-900/#ep15
* Episode Resources
* 📚 Study cheat sheet marczak.io/az-900/episode-15/...
* 🧠 Practice Test marczak.io/az-900/episode-15/...
* Study Guide
* Microsoft Azure: Azure Synapse Analytics azure.microsoft.com/en-us/ser...
* Microsoft Azure: Azure HDInsight azure.microsoft.com/en-us/ser...
* Microsoft Azure: Azure Databricks azure.microsoft.com/en-us/ser...
* 🌐 Wikipedia: Big Data en.wikipedia.org/wiki/Big_data
* Expand your knowledge - extra resources
* Microsoft Documentation: Azure Synapse Analytics docs.microsoft.com/en-us/azur...
* Microsoft Documentation: Azure HDInsight docs.microsoft.com/en-us/azur...
* Microsoft Documentation: Azure Databricks docs.microsoft.com/en-us/azur...
* 🌐 Apache Spark Homepage: Overview spark.apache.org/
* Azure Databricks Tutorial | Data transformations at scale • Azure Databricks Tutor...
Agenda
00:00 Episode introduction
00:30 What is big data?
01:42 Azure Synapse Analytics
04:15 Azure HDInsight
05:27 Azure Databricks
09:24 Summary of Big Data services
Want to connect?
- Blog marczak.io/
- Twitter / marczakio
- Facebook / marczakio
- LinkedIn / adam-marczak
- Site azure4everyone.com

КОМЕНТАРІ: 182

@BijouBakson 2 роки тому

Here I am, 2 years later and now an Admin associate, preparing for my Solutions Architect Expert AZ-305 exam... and still not entirely clear with Big Data. I am having to do the Data Engineering module to deal with it once and for all. These videos help clear things out. Thank you Adam.

@markcaneda7764 3 роки тому

The channel name says it all, very intuitive visualizations and explanation. This is much better than udemy paid courses. Thank you for creating such contents!

@AdamMarczakYT 3 роки тому

Wow, thanks! Appreciate your kind words Mark :)

@idougy18091809 Рік тому

It's even better that Ucertify which a course leader was teaching from.

@koduruchandu 3 роки тому

@MarczakIO Hi, this is Chandra from Bangalore, India. Today i have successfully cleared AZ-900 exam by just going through the 37 youtube vedios you prepared. In spite of having prior fundamental knowledge on cloud computing, i could pass the exam with the help of your vedios on Azure. Hence, would like to thank you a lot 🙏 for making available such a great vedios which anyone can easily understand the Azure basics needed for exam.. Is a great social service..so nice of you😊

@AdamMarczakYT 3 роки тому

Thank you, highly appreciated :) Glad you passed, keep it up!

@PawanKumar-tu6ti 3 роки тому

Here we meet again, Adam! I passed my exam yesterday of az-900, thanks to Microsoft documentation and especially YOU, and your teachings.

@AdamMarczakYT 3 роки тому

Fantastic! congrats, keep up the good work :)

@PawanKumar-tu6ti 3 роки тому

@@AdamMarczakYT thanks, can you start making azure series on az-204? At least make a roadmap and prerequisites for it.

@oguzhankrcal357 3 роки тому

Today, I've joined to Microsoft Azure Virtual Training Day: Fundamentals meeting. It has been finished yet and I missed you. ☺️ I came to here by running. Thanks man. Your English is understandable, your blackboard is awesome. They should learn how to present AZ-900 from you.

@AdamMarczakYT 3 роки тому

Wonderful! Welcome back to where all the action is! :D

@abhisheksoni6012 9 місяців тому

Thank you so much Adam for making these difficult-to-grasp concepts very easy to understand and interesting to learn. There are no words for your teaching skills. You are awesome. Thanks a lot Adam !

@sumitprakash6840 2 роки тому

You are simply fantastic. I really love the way you describe each topic. Thanks for support and looking ahead to see you more to take advantage of new technology.

@ilyasontube 3 роки тому

all of the Adams videos are not only very clear and informative but also very inspirational, real MVP!

@AdamMarczakYT 3 роки тому

Great to hear! thanks!

@benediktemydlandsolend8515 3 роки тому

Thank you for your explanations. Love the way you illustrate and show us examples. Looking forward to your next video.

@AdamMarczakYT 3 роки тому

More to come! thanks!

@simonkelso3447 3 роки тому

Great Job Adam, this is forming the basis for my Azure learning. Keep up the good work !!

@AdamMarczakYT 3 роки тому

Great to hear! Will do!

@ngophuthanh 3 роки тому

Thanks a lot, Adam. As always, this episode is very well organized and presented. Diagrams and visualizations are excellent. Your deep knowledge and teaching skills are amazing.

@AdamMarczakYT 3 роки тому

Many thanks! :)

@SanatanYadav108 2 роки тому

Hi Adam, you are fantastic mentor. Your presentation is extremely interesting. Thanks alot for your hard work behind these presentations.

@mmohan1234 11 місяців тому

Excellent video on Azure's Big Data and Analytics products.

@aurosish1 2 роки тому

Thanks to you Adam, now I got a quick overview of Azure Big Data & Analytics Services.

@AdamMarczakYT 2 роки тому

Fantastic! Thanks!

@binhngo12336 2 роки тому

Adam. u are the king. best teacher I ever seen man. all the best for u man...

@thesloable 2 роки тому

Thanks for creating concise and easy to understand materials. Hope to see more courses covered in this channel.

@AdamMarczakYT 2 роки тому

Thanks, will do!

@jcb3883 2 роки тому

I'm getting my certificate this friday. Wish me luck. Thank you for doing this series.

@prititiwari5183 3 роки тому

You explained every topic thoroughly.

@AdamMarczakYT 3 роки тому

Thank you! :D

@vitovito123 2 роки тому

Adam, thank you so much. I passed my exam today.

@alexluke_again Рік тому

the least i can do i like each video and recommend to friends. thank you for doing this!

@spencerthomas9377 Рік тому

Great stuff as always!

@timmusaka9577 Рік тому

The best content is on UKposts!

@KDOERAK 2 роки тому

another great overview!

@csabafeher2632 3 роки тому

I cannot describe how useful, professional and engaging your content is! Best of its kind! Btw, after couple of episodes I have realized you have cool T-Shirts! ;-)

@AdamMarczakYT 3 роки тому

Wow, thanks! :)

@clintonchristopher6532 2 роки тому

Hi Adam, been following back to back on this video tutorial as i am about to take the azure fundamental exam, and i must say its been extremely helpful. Thanks so much ...

@AdamMarczakYT 2 роки тому

Fantastic!

@scotmarkieboy 3 роки тому

Hey Adam, thanks for putting this content together and making it available for free.

@AdamMarczakYT 3 роки тому

My pleasure!

@shaunakv100 2 роки тому

Thanks a ton for making this videos. You have not only simplified all concepts for better understanding but also kept it interesting till end. Your presentations, diagrams are also so clear. I learned a lot from your videos. I will be happy if I can support you in someway Thanks Again for helping us with these videos

@AdamMarczakYT 2 роки тому

Awesome, thanks!

@vinayinfo15 10 днів тому

Nice cover of az 900 study tipics

@franciscorossi5633 Рік тому

You are a genious, thanks mate!

@denkozlov4220 2 роки тому

thank you Adam

@davidgates5189 Рік тому

great as always

@farshidjamali 2 роки тому

Thank you, Adam!

@AdamMarczakYT 2 роки тому

My pleasure!

@Lordxfx 3 роки тому

Finally an insight to HDinsight :)

@AdamMarczakYT 3 роки тому

😄

@sanghvian 3 роки тому

I am learning so much apart from just Azure !!! Thanks a lot Adam !

@AdamMarczakYT 3 роки тому

My pleasure!

@swatikandarpatel1127 Рік тому

Thank you I cleared exam today.

@SpeoLori 2 роки тому

This course is amazing. Congratulations. And there are deep-dive videos about specific topics. How did you created the drawings? They are so neat.

@talentdlamini9326 3 роки тому

you a life saver thanks Adam

@AdamMarczakYT 3 роки тому

Thank you! :D

@jhosnatas 2 роки тому

I've been aproved in az-900 thank you

@AdamMarczakYT 2 роки тому

Congrats :) Keep it up!

@thekbee5736 3 роки тому

The explanations in your videos are the easiest to understand of everything else out there

@AdamMarczakYT 3 роки тому

Thanks! :D

@gauravkumar796 3 роки тому

Awesome.. I like the summarised comparison.

@AdamMarczakYT 3 роки тому

Thank you! Appreciated :)

@JagdishJ666 2 роки тому

Outstanding explanation and easy to learn

@AdamMarczakYT 2 роки тому

Thanks! :D

@jvstechtips Рік тому

Very helpful, thanks

@dhawrylkiw Рік тому

Thanks!

@toluagoro3232 2 роки тому

You are amazing. The way you explain key concepts is top notch. I wish you can teach me for my security + certification

@AdamMarczakYT 2 роки тому

Fantastic, thank you, best of luck! I wish I could teach everyone, that's why I created this channel!

@ruchijain2584 3 роки тому

Thanks for amazing videos !!

@AdamMarczakYT 3 роки тому

My pleasure!

@stephaniebuadu3474 3 роки тому

This was great! Thanks

@AdamMarczakYT 3 роки тому

Glad it was helpful!

@dharangsharma6863 2 роки тому

Hi Adam ! Just a quick question, Are your Practice Tests enough to clear the exam, or do I need to purchase some sample papers before appearing for the exam? Also want to thank you for the amazing content, will definitely recommend it to others.

@jurijssolovjovs3575 2 роки тому

great work thanks mate.

@AdamMarczakYT 2 роки тому

No problem 👍

@saltnlightmalaysia 3 роки тому

Wonderful material!!!

@AdamMarczakYT 3 роки тому

Thank you! Cheers!

@alamzaibfarooq 2 роки тому

The course is much better than Udemy courses. How is it possible that Adam is not charging anything for this course. He is amazing. The animation is slides are very easy to understand and eye catching.

@luism2754 10 місяців тому

Dude for real, I have been paying a curse that is hours of hours of a guy saying the same thing over and over and just making everything more complicated. But thanks to Adam now I am understanding everything

@mrindian7422 2 роки тому

Nice explanation.. thanks

@AdamMarczakYT 2 роки тому

Glad you liked it

@devanandapatro7889 3 роки тому

Hi Adam, ur way of teaching is really awesome!! I attempted the az900 exam today and cleared with good marks.. Thanks for all your effort.. 😊😊

@devanandapatro7889 3 роки тому

@@wiroee yes, I have watched all episodes and thereafter took a practice exam from Udemy..

@AdamMarczakYT 3 роки тому

Great job! Thanks! :D Keep it up! :)

@samruddhikalsekar1038 3 роки тому

@@devanandapatro7889 could you mention which practice exam exactly? There are multiple options on udemy

@tustinnrent8502 3 роки тому

another great video! waiting for DP-900! :-)

@AdamMarczakYT 3 роки тому

You and me both!

@tustinnrent8502 3 роки тому

@@AdamMarczakYT JK, no pressure!!! Started doing it the old-fashioned way this week by reading MS docs. Many of your videos have already been an excellent complement!

@debreczenytivadar1220 3 роки тому

Hello Adam. Now I watch all your videos and everything is very professional. Here’s a little random sound clash around 9:28.

@AdamMarczakYT 3 роки тому

Thanks. Yea some overlap happened there of two audio tracks, unfortunately youtube doesn't allow to reupload the videos. Would need to delete this.

@rqn2274 3 роки тому

great video as always

@AdamMarczakYT 3 роки тому

Thanks again!

@pankajkumar-wt2ro 3 роки тому

Good content Adam ,I am waiting for next video

@AdamMarczakYT 3 роки тому

More to come!

@sahajrajmalla 2 роки тому

I have been using Azure SQL and making API to query those databases but the query is taking very long time to execute. Now i want to add spark's distributed system and query through the database. Which azure service is better suite for me? Thank you for this wonderful free content Adam. I really appreciate it alot!

@PawanKumar-tu6ti 3 роки тому

UKposts knows how to suggest people, here we come again, thanks brother, your help means a lot, with every video I watch, I can't resist myself from leaving a comment!

@AdamMarczakYT 3 роки тому

I appreciate that, thank you!

@windmid39 3 роки тому

This is good for DP-900 too. The comparison make thing more clear but not just list out the services and characteristic.

@AdamMarczakYT 3 роки тому

Thanks! Yea some topics are mandatory for all fundamental exams like AZ-900, DP-900, AI-900 and even MS-900.

@shaileshsingh1445 3 роки тому

you are awesome :)

@AdamMarczakYT 3 роки тому

Thanks! :D

@rs-tarxvfz 2 роки тому

6:20 I think they have unified the creation of workspace within the Azure Portal itself. So no need to go to another platform

@StephenBoesch 4 місяці тому

Can these capabilities be scripted or configured via properties/json files? I prefer to use text editors to set things up *repeatably*, check in the config scripts to git, and be able to clone and tweak them.

@abdulkleit 3 роки тому

Hello Adam, eagerly waiting for your remaining episodes from 32-39.

@AdamMarczakYT 3 роки тому

Soon! :) I had some time of during x-mas and new year, back on schedule soon! :)

@abdulkleit 3 роки тому

@@AdamMarczakYT Thank you Adam. Happy New Year!

@SiyamandRashid 3 роки тому

Very good videos and the most beautiful part of it the way you prepared... what kind of tablet did you use please? Regards

@AdamMarczakYT 3 роки тому

It's just my surface pro laptop :)

@SiyamandRashid 3 роки тому

@@AdamMarczakYT thanks a lot for the reply, i just bought wacom small size but it is kind difficult to draw... Keep it up brother

@RandyRieger 2 роки тому

You need to stop being so awesome. Actually don't. Love how you explain everything

@AdamMarczakYT 2 роки тому

Can't stop being awesome!

@TheLangmusi 3 роки тому

Thank you for the summary. I wonder what if I want to save a data frame to Azure SQL DB in a Python Notebook like you demonstrate here? Moreover, can you explain which line on your Python block 1 register df as temp table in SQL? Thanks for your answer.

@AdamMarczakYT 3 роки тому

Note that I registered data frame as Spark SQL view, it's not SQL DB view. Check out my azure databricks tutorial to learn a bit more on how to operate using databricks. Thanks for stopping by.

@TheLangmusi 3 роки тому

@@AdamMarczakYT Thanks for the answer. Ok, I got it. I will watch your databricks tutorial for more info. Thanks agin.

@Abecedarianme 2 роки тому

Very informative video. Hoping you could do for DP-203 too.

@AdamMarczakYT 2 роки тому

Noted, thank you :)

@sergiocoutinho6133 3 роки тому

Congratulations for the videos .. your diagrams are excellent for assimilating the content and i am now confident to do AZ-900 soon ! But I confess that I get a little confused by the terms PIPELINE SYNAPSIS, DATA FACTORY or HDINSIGHTS .. sometimes they seem the same thing to me. I even thought that SYNAPSIS corresponded to the old SQL AZURE DATA WAREHOUSE, without the pipeline option. Best regards !

@AdamMarczakYT 3 роки тому

Best of luck on the exam! You can do it! Those services serve similar purposes but the details are different. Only once you dive into each one of them, then you will be able to distinct them better! :)

@Search4Knowledge 3 роки тому

Thanks for this awesome info! Just a question, is HDInsight still part of the DP200 certification? Also when to choose between HDInsight and Databricks as both are spark based?

@AdamMarczakYT 3 роки тому

Well as you've seen in the video HDinsight is not spark based, Spark is one of several available engines for HDInsight. But more detailed comparisons can be made when using Spark here and in Databricks. In general Databricks is better optimized version of the spark as it was created by Spark creators.

@sid0000009 3 роки тому

Hello Adam, how do we choose between Databricks and HD Insight if we need a Spark Cluster for our use case?

@AdamMarczakYT 3 роки тому

Thanks a tough question! But a very good one. I guess both can achieve pretty much the same so it's just matter of organizations/skillsets/etc. In my company we decided to use Databricks for all out data transformations projects. But I have few MVP colleagues which prefer using HD Insight. Some companies also use HD Insight because it's more secure.

@rs-tarxvfz 2 роки тому

Hi Adam, I din't get difference between *synapse pipeline vs data factory*. Pls..

@sidarthg2477 10 місяців тому

I think time for an update to the course. Coz I think this course was formed before DP-900 came into existence. So all these portions are now not under AZ-900 and instead it now emphasize more on networking and less on data analytics tolls

@limsky8012 3 роки тому

When I should choose the Synapse and when Databricks? What type of question I should answer to make a decision?

@AdamMarczakYT 3 роки тому

It's a difficult choice. Maybe I'll do a video on this in the future, there are too many factors to explain in a simple comment.

@imsteven3044 2 роки тому

What program do you use to make the notes of your videos?, such as the minute 1:23

@cebabu 3 роки тому

Generally we store enriched and transformed data sql dataware house, now callled synapse. Can I also store all raw data(structure ) in synapse.

@AdamMarczakYT 3 роки тому

Those are just tools, use them however you like/need :)

@haiyingwang5987 3 роки тому

Dear Adam, is it normal for free subscription to get "cloud provider launch failure" when trying to start a cluster in databricks? Is there anyway to get it starting somehow? Thank you!

@AdamMarczakYT 3 роки тому

Possibly, free subs have 4 vCore limits. try selecting single node checkbox and select smallest available cluster size, which I think was F4s.

@haivercarrillo8603 3 роки тому

Hi Adam, great content as usual, can I ask you a favor, if you can explain the azure data explorer. Thank you!

@AdamMarczakYT 3 роки тому

Thanks. ADX is definitely on the list :)

@abhijitboda Рік тому

THIS PART IS REMOVED FROM EXAM SINCE MAY 2022

@5siah530 Місяць тому

Is this the only topic covered in the course that has been removed from the exam?

@Mybigsissy 3 роки тому

question 5 on your practice test: the question was about migration, so how does an analytics service perform the task?

@AdamMarczakYT 3 роки тому

Migration is a process, not a tool :) Think of it from the perspective of a company.

@timoose3960 2 роки тому

Does anyone know what tools he uses to make the drawings/sketches on top of the video?

@jorgeregis5614 2 роки тому

Your are awsome , also give us some tips for learn english hahaha , your accents is understandable.

@AdamMarczakYT 2 роки тому

Thank you! 😃

@MrDanesorensen 2 роки тому

Struggling to understand the key differences between Azure Synapse Analytics & DH Insights. Anyone able to help clarify this for me?

@hochun836 2 роки тому

Hi Adam, is there a plan to make a video about synapse tutorial ? I like your tutorial !!

@AdamMarczakYT 2 роки тому

Possibly! Thank you! :)

@BijouBakson 3 роки тому

God, that was heavy! Oopf! fundamental huh!? :) I definitely didn't grasp this before I passed my exam, still don't understand it much. This course is a little rushed in my opinion. May it's because it's an entire complex field? I'm watching for the 3rd time now. Let's go. :)

@AdamMarczakYT 3 роки тому

Well the services are quite similar so I understand how they can be confusing. They are competing offers for big data. But don't give up! :)

@BijouBakson 3 роки тому

@@AdamMarczakYT Haha! my vocabulary is short of give-ups, luckily. I understand the overall idea better now! thank you :)

@shaunakv100 2 роки тому

Do you have also similar videos on DP-900 as well?

@AdamMarczakYT 2 роки тому

Not yet, maybe in the future :)

@thegreenroom5820 Місяць тому

I am assuming this type of collaboration is extremely expensive in production

@gocrow23 3 роки тому

Can't find code, can you pls put the Databrick code piece you used for illustration?

@AdamMarczakYT 3 роки тому

Hey, check my databricks tutorial. It’s much better if you want to learn it and it has samples attached. In this tutorial I just showcased databricks but I didn’t expect people to follow. I’m on holidays now but I can upload samples from this episode when I get back. Thanks for watching :)

@vr1908 2 роки тому

Hi Mark, Can i have a script for "Connect to Microsoft Opendatset for Sample Data"

@Mauszinator666 Рік тому

I don't know if i'm the only one, but the sounds got a little messed up around 9:28. Maybe its just me, or otherwise i hope it is usefull feedback

@BijouBakson 3 роки тому

You missed to mention that Azure Synapse Analytics is the new name for Azure Data Warehouse.

@AdamMarczakYT 3 роки тому

I didn't miss it. It was on purpose. Majority of the people who study Azure Fundamentals are not familiar with Azure so I rather avoid using already outdated terminology/names as it might introduce confusion. Of' course I will mention this in my upcoming Azure Synapse tutorials later this year.

@BijouBakson 3 роки тому

@@AdamMarczakYT I understand now, thank you

@thprodigy 3 роки тому

what's the difference between SNOWFLAKE and Azure HDInsight?

@AdamMarczakYT 3 роки тому

Closer comparison would be between Azure Synapse Analytics and Snowflake. They are both Massively Parallel Processing databases with unified workspace and ETL capabilities.

@artak9439 3 роки тому

What about Azure SQL Data Warehouse ?

@AdamMarczakYT 3 роки тому

It's been renamed to as SQL pools in Synapse.

@artak9439 3 роки тому

@@AdamMarczakYT Thanks

@massimo8672 3 роки тому

how do we get the slides

@AdamMarczakYT 3 роки тому

Sorry Massimo but I don't share my decks, thanks for the visit! :)

@juggergabro 2 роки тому

09:28

@Downstar 2 роки тому

9:28 I thought that i accedently opend a video in a new tab or something hahah

@AdamMarczakYT 2 роки тому

Hehe, small audio sync issue :)

@Downstar 2 роки тому

@@AdamMarczakYT hahah no problem, i really ejoyed your content and i passed the exam yesterday! U have learned me alot, thanks!

@CosmeJunior 3 роки тому

These episode for me, the most difficult to memorize any tips from anyone ?

@AdamMarczakYT 3 роки тому

Try my reference study materials and read Microsoft Learn, maybe a different perspective on the topic will help!

@uskapyaar8852 3 роки тому

i am wondering who would've disliked this video

@AdamMarczakYT 3 роки тому

Me too :)

@uskapyaar8852 3 роки тому

@@AdamMarczakYT you're just too nice at explaining all these concepts that too for free.. i am gonna stick with you.. 🙏 thanks for help i am feeling confident for my exam now ❤️

@prateekbhardwaj9943 Рік тому

1 lac people watched this video i am wondering how many cloud engineers are in the world. and how many companies who require these engineers ! its a competitive world isnt it.