Secret To Optimizing SQL Queries - Understand The SQL Execution Order

  Переглядів 355,971

ByteByteGo

ByteByteGo

Рік тому

Get a Free System Design PDF with 158 pages by subscribing to our weekly newsletter: bytebytego.ck.page/subscribe
Animation tools: Adobe Illustrator and After Effects.
Checkout our bestselling System Design Interview books:
Volume 1: amzn.to/3Ou7gkd
Volume 2: amzn.to/3HqGozy
The digital version of System Design Interview books: bit.ly/3mlDSk9
ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

КОМЕНТАРІ: 146
@cumbi-mongo
@cumbi-mongo Рік тому
Great video! One addition: The "EXPLAIN" command is an invaluable tool for optimizing SQL queries. It provides a detailed execution plan, allowing the developers to understand how the database engine processes a query. By analyzing the execution plan, you can address the performance bottlenecks with proper optimizations, e.g. proper indexes.
@Omar-ic3wc
@Omar-ic3wc 11 місяців тому
Thanks for sharing this.
@luis5d6b
@luis5d6b 10 місяців тому
Thanks a lot for the addition, really good :)
@ksm1847
@ksm1847 9 місяців тому
@cmertayak - I second you. It's an awesome command I use many times at my work to optimise. My go to command to improve queries execution.
@zeelthumar
@zeelthumar 6 місяців тому
Thanks for sharing
@DavisTibbz
@DavisTibbz 3 місяці тому
Oh yes, if you run EXPLAIN in some desktop client like Mysql Workbench, shows you detailed chart diagram of your Query, quite useful
@uzair004
@uzair004 11 місяців тому
Opt for indexes with SELECT, WHERE, JOIN clauses. Use full column comparison to get data instead of half or computed comparison (i.e startsWith) Avoid ORDER_BY on large data retreval Use limit of smaller number with pagination for more data.
@sampri22
@sampri22 Місяць тому
Could you explain how? What if i need large data retrieved with order by. How would i use limit and pagination in this case? Thanks
@ayazahamed8254
@ayazahamed8254 8 місяців тому
The way you explained with the animations are Awesome. Great Job. Very Well Explained.
@user-yi4cp1uh1f
@user-yi4cp1uh1f 9 місяців тому
bro this way of teaching is really really make sense. thanks a lot for these visuals.
@JohnS-er7jh
@JohnS-er7jh 9 місяців тому
One of the best SQL videos I have come across, just the way it is put together and the infographics. If you are learning SQL, you really should understand the mechanics behind optimizing queries, how databases work. Just adding more hardware or VM resources will not fix the issue if your queries are not optimized properly.
@rembautimes8808
@rembautimes8808 2 місяці тому
Very well presented, thanks for explaining SARGAble concept
@lucyk7292
@lucyk7292 Рік тому
Thank you for a fantastic visualization of the SQL queries execution order. That's exactly what I have been missing in the other materials. I really appreciate your style of teaching
@AliBensoukehal
@AliBensoukehal 11 місяців тому
Simple and to the point explanation. Love it. Thanks 👍
@CyberMew
@CyberMew Рік тому
Very good intro. Would like a more detailed explanation on more complex queries.
@adicide9070
@adicide9070 Рік тому
they don't do detailed explanations. it's basically "use indexes". don't sort lots of data. well, thanks.
@davidlee588
@davidlee588 11 місяців тому
@@jonbaird9718agreed, UKposts is made for juniors
@twistedace6365
@twistedace6365 7 місяців тому
thanks, helped clear up some issues I had.
@user-si5ue8dv7y
@user-si5ue8dv7y 4 місяці тому
This is the best explanation I've ever seen. Big thumbs for you!
@jagmanderbazzad8425
@jagmanderbazzad8425 9 місяців тому
Nice and simple explanation.Thanks
@developersharif
@developersharif Рік тому
*Explanation level is so beautiful!*
@JeremyChone
@JeremyChone Рік тому
wow, what an awesome introduction to SQL optimization.
@martingrof1685
@martingrof1685 7 місяців тому
Amazing. Thank you!
@caseyspaulding
@caseyspaulding Рік тому
Wow. To the point with knowledge I can use today. Thank you.
@avijeethati5323
@avijeethati5323 10 місяців тому
Excellent video explaining basic concepts in very short time..❤ Impressive graphic animation, could you please share how the execution plan animation was done
@ThinhLe-eh9re
@ThinhLe-eh9re 9 місяців тому
Thanks for your sharing Bro's.
@abhinav10x
@abhinav10x 29 днів тому
Very profound, please share more on SQL like windows and CTE, your explanation is very approachable.
@antonboiko788
@antonboiko788 9 місяців тому
cool, didn't think it's possible to include all these concepts in 6 min video. One thing, it's great to watch it when you want to summarise already existing knowledge
@tallalmoshrif6643
@tallalmoshrif6643 10 місяців тому
Great video, very informative and well explained bravo!
@99aabbccddeeff
@99aabbccddeeff Рік тому
Excellent explanation, thanks!
@vasiliynet3425
@vasiliynet3425 Рік тому
Awesome as usual! Thanks a lot!
@nixjavi7220
@nixjavi7220 2 місяці тому
these videos are amazing!!!! thanks!!!
@robbybankston4238
@robbybankston4238 11 місяців тому
Love your channel. Your videos are great.
@darrenklein6090
@darrenklein6090 Місяць тому
Thank you, this was really helpful.
@karunakaranr2473
@karunakaranr2473 Місяць тому
Thank you for your time and effort to explain any of the subjects. Really like it and more over able to register the concept in mind easily. Thanks again,.
@MrSuriyam
@MrSuriyam Місяць тому
Hi Sir thank you 🙏 for taking the time to explain the SQL. Sorry Iam new and very helpful.
@gliderguld
@gliderguld 7 місяців тому
Well explained. However I do miss 1) the generation of more query-plans and selection amongs them (cost estimations) and (as an element herein) 2) different table access tactics (sequential scan, index access or index only).
@stpaquet
@stpaquet Рік тому
Understanding how the DB engine works with indexes is key. you may assume that a WHERE purchase_date >= 2022 AND purchase > 100 would be the same if you have indexes on purchase_date and purchase, but it might be required to have a composite index... Order in the WHERE clause may also be important as it helps reducing the dataset before applying the second condition.
@MiningForPies
@MiningForPies 7 місяців тому
WHERE order has no effect on most sql systems. The only way you can force SQL to filter data first is to use a derived query.
@mer2550
@mer2550 11 місяців тому
good things to practice for the interview. Thanks
@niketu99
@niketu99 7 місяців тому
Superb video! Simple explanation on query optimisation.
@viktorsaranchuk2932
@viktorsaranchuk2932 11 місяців тому
Thanks. Good to know! Useful!
@jamesayento1321
@jamesayento1321 Рік тому
Thanks for this! Will there be a transcription soon?
@moneycrab
@moneycrab Рік тому
I heard it called "predicate pushdown" when you move a condition earlier in the plan
@bigbrother1211
@bigbrother1211 11 місяців тому
Thank you so much!
@Sudharshaunvilla
@Sudharshaunvilla Рік тому
Fantastic explanation.
@sengs.4838
@sengs.4838 10 місяців тому
So good explanations
@antonanton22743
@antonanton22743 Рік тому
thanks a lot for your content
@mikeshin77
@mikeshin77 Рік тому
Very good video. It is really helpful.
@yashwanthbedre8220
@yashwanthbedre8220 11 місяців тому
Best explanation ever
@user-yq3si3so5n
@user-yq3si3so5n 2 місяці тому
Awesome visualization, I've been loving all the short videos on this channel! Clarifying Q. The execution order has SELECT happening after HAVING, so this should mean that the calculated column total_spent doesn't exist at the time the HAVING clause is evaluated?
@naveenverma2951
@naveenverma2951 Місяць тому
oh my goodness, this is too good for non IT background jumping ship to see where AI will land. Thx. You are my 3blue1brown for IT
@mohan1958
@mohan1958 2 місяці тому
Great. Thanks for sharing..
@christopherli5828
@christopherli5828 3 місяці тому
thanks so much!
@zackwong1000
@zackwong1000 Рік тому
You should select from the orders table then join the customers since your where clause is a column in orders table! Your SQL is joining on unnecessary rows from orders & customers!
@user-it7ur3bc9n
@user-it7ur3bc9n 6 місяців тому
Additionally, for the optimizer to "make up" a reasonably good plan (from the various alternatives), it needs to know a bit about the data (value) distribution. This is where STATISTICS / ANALYZE (depends on the DB vendor) come handy. It helps the optimizer do estimates for the various steps (rows, size of data, etc.) of each plan, and figure out which of the different plans is the best candidate to execute. Therefore it is important to collect this information on critical columns (usually join, where clause columns). It is also important to keep this information regularly refreshed so that the optimizer does not make bad decisions based on stale statistics. Very bad things can happen with stale statistics.
@DerBarde2012
@DerBarde2012 3 місяці тому
Your presentation is so pleasant to watch, is it manually key-framed in the video editor or are there tools to do that naturally?
@jerichaux9219
@jerichaux9219 Місяць тому
You guys are awesome!
@resistentemente
@resistentemente 9 місяців тому
What program is this used in the presentation?
@pieter5466
@pieter5466 Рік тому
As usual, excellent and to the point video!
@ashutoshtripathi3817
@ashutoshtripathi3817 3 місяці тому
muchas gracias!
@bhooshan25
@bhooshan25 9 місяців тому
good explaination
@helal.ismail
@helal.ismail Рік тому
Very simple and to the point, love the visualization too
@shmuelsigler2918
@shmuelsigler2918 11 місяців тому
מדהים!
@huywarrior
@huywarrior 10 місяців тому
Lord Buddha. I'm looking for an active data flow visualization that can shorten data query response times! A great video, it saved me today. Leaving with 1 subscription as a fan! 🔍⚡
@anirudh7463
@anirudh7463 Рік тому
This stuff is gold. Thank you for making this available for free. Really appreciate it!
@anitacusick7658
@anitacusick7658 6 місяців тому
1:26
@gabrielb.962
@gabrielb.962 Рік тому
Index usage tip: When using params in your query (e.g., select .... where year > ?), databases may not utilize an index if it is unbalanced. For instance, if you have approximately 1 million rows with year = 2022 and only 1000 rows with year = 2023, the database cannot predict whether the parameter will be useful for filtering. To resolve this issue, pass the value directly in the query itself, allowing the execution plan to determine if the index is suitable for the intended purpose.
@stpaquet
@stpaquet Рік тому
As I wrote in my comment, good understanding on how you db engine works is key. And they are all different. So never assume that a good query on a MySQL will be a good query on Postgres, Oracle or any SQL engine.
@maf_aka
@maf_aka Рік тому
this opens the gate for SQL injection, don't do this
@maksymbaranovskyi8362
@maksymbaranovskyi8362 Рік тому
@@maf_aka I think the idea was not to use prepared statements *where you don't need them.* E.g. if you already have validation in place that ensures your received value is enum (number, null, etc.) - you can be sure no SQL injection is possible there - so no need to use prepared statements *there.*
@lethern2
@lethern2 10 місяців тому
Ok, but then you get a different query plan for each (different parameter / set of parameters) query
@stpaquet
@stpaquet 10 місяців тому
@@lethern2 yep. but that's why you need to understand how your db engine works
@NiamorH
@NiamorH Рік тому
Nice bird's-eye view introduction. It is not clear how to 'use appropriate indexes' to optimize for sorting, and how to implement pagination. Especially in your example where the sort order is made on an aggregate.
@mwaikul
@mwaikul 11 місяців тому
What tool do you use to generate your animations?
@crossedpolars
@crossedpolars 9 місяців тому
Very good video
@d4devotion
@d4devotion Рік тому
This is pretty cool.
@ronenTheBarbarian
@ronenTheBarbarian Рік тому
Would building a cte table and then running a non-sargable query on it, should also be avoided?
@GabrielGasp
@GabrielGasp 8 місяців тому
I always thought that the SELECT happened before HAVING, considering that we can use SELECT aliases in the HAVING filter.
@freindimania11
@freindimania11 Рік тому
order_date is mentioned as indexed - is that implicit or explicitly defined?
@sriteja2510
@sriteja2510 10 місяців тому
Hi The actual plan should be derived from the explain and explain analyze right instead from the query?
@seraphfull
@seraphfull Рік тому
thank you for your video, i working on IT with 10 years experience, but I never know the order between JOIN and WHERE, utill I watch this video
@Moali11
@Moali11 6 місяців тому
Can you make a video explaining the difference between system design and software architecture?
@AbhishekVaid
@AbhishekVaid 8 місяців тому
Something doesn't add well here. If you notice HAVING clause refers to 'total_spent' which is defined in SELECT, so dependency wise HAVING should be after SELECT and not before it.
@sevsxes
@sevsxes 11 місяців тому
Will it be even faster if we always order where first and join after?
@apexyu164
@apexyu164 Рік тому
so in the above example, which place we should index ?
@fishinawaterbottle
@fishinawaterbottle Рік тому
I feel like this is a bit misleading because sometimes where and select influence the first stage. As you said, when there’s a covering index, the database won’t read the entire table. So the select and where influence what is read from the source. Order and limit can also come it at the source as well if the index can be used with the order. You refer to this when you talk about “sorting the whole table”. CTEs and sub queries are not mentioned but that’s okay i guess.
@guocity
@guocity Рік тому
很不错
@sahandjavid8755
@sahandjavid8755 4 місяці тому
Question: at the end of the video you mentioned do not sort the whole data and use pagination for optimizing ORDER BY and LIMIT. Those are the things I use for pagination! What do you mean by that? The other thing is from your video LIMIT happens after ORDER BY. How come it can help when ORDER BY has already happened?! Btw great videos and content, thank you for these
@blackisblack22
@blackisblack22 10 місяців тому
I have always thought that the Sql structure is poorly designed by not starting from FROM and placing the reference at the end of the statement, for example in a SELECT it should go just before ORDER BY, in an UPDATE the SET after WHERE, etc. Somehow they wanted to remedy the problem by introducing the WITH clause but I'm sure many regret that whoever designed the language should have worked a little harder at the time.
@MPIPolitics
@MPIPolitics Рік тому
my app didnt reached 40 queries per second yet but i will implement that just in case my app will be next amazon :D
@zixuanzhao6043
@zixuanzhao6043 16 днів тому
This query actually does not need to join customers table since all the fields are present in the orders table already. (unless there are invalid / dirty customer_id data in the orders table and you want to filter them out)
@theawesomeharris
@theawesomeharris 10 місяців тому
hi, can you enable captions/subtitle for this video? thank you!
@Mr.zafar_siddiqui
@Mr.zafar_siddiqui 10 місяців тому
Can you/anyone please explain execution of case when and window function with group by
@wjrasmussen666
@wjrasmussen666 Рік тому
Will this work with MySQL as well?
@MewMewCherry1990
@MewMewCherry1990 9 місяців тому
Why are we using HAVING total_spent >_ 1000, but not WHERE total_spent >_ 1000 ? Can you please explain?
@yash1152
@yash1152 10 місяців тому
0:12 _[JOIN comes before WHERE]_ is there any way to make the WHERE clause execute first to narrow the rows required to make the JOIN in the first place?? this is the only reason i still do this using a nested query rather than JOIN
@notbalding
@notbalding 7 місяців тому
a CTE can be benefitial in your use case.
@mahmudulmohtasim7612
@mahmudulmohtasim7612 11 місяців тому
In this example the 'total_spent' alias is already in use in the HAVING clause without defining. How is that possible?
@tianyuhu349
@tianyuhu349 9 місяців тому
yes, I have the same question, it doesnt make sense...
@MarredNDisenchanted
@MarredNDisenchanted Рік тому
Having uses total_spent from the SELECT, so how come HAVING is executed before the SELECT?
@urgorka
@urgorka 3 місяці тому
I'd say so too. This is error. First SELECT part is evaluated, then - HAVING part.
@vikrantgaikwad5529
@vikrantgaikwad5529 Рік тому
I think Order by is evaluated before select as order by might change selected rows...is it correct?
@michelvandermeiren8661
@michelvandermeiren8661 11 місяців тому
Yes this vid is full of mistakes
@some1rational
@some1rational Рік тому
Is that a typo in the first select clause, total spent should be total_spent?
@tianyuhu349
@tianyuhu349 9 місяців тому
yes, i think so, and I have another question, 'Having' uses total_spent from the SELECT, so how come HAVING is executed before the SELECT? Doesnt make sense...
@user-ff3gn4re4g
@user-ff3gn4re4g 2 місяці тому
is there a way to contact you? I have some specific questions on indexes?
@amrithpurandhar9882
@amrithpurandhar9882 7 місяців тому
Can anyone help me when does the function count or sum will be executed will it be after limit ?
@ghaidam9749
@ghaidam9749 5 місяців тому
can someone explain to me what's mutant query plans with a real life example?
@thuynv
@thuynv 11 місяців тому
you should have more subtitles
@Matthew8473
@Matthew8473 3 місяці тому
This is top-notch in every aspect. I read a book with similar content, and it was top-notch. "Better Sleep Better Life" by William Brook
@MuhammadFahreza
@MuhammadFahreza 6 місяців тому
I still don't understand the difference between first point noted on here 3:19 and second point noted on 3:23. Would you mind to re-explain it ? thank you!
@KuopassaTv
@KuopassaTv Рік тому
Ambiguous query
@andyvoice
@andyvoice 3 місяці тому
subtitles not available
@AliBensoukehal
@AliBensoukehal 11 місяців тому
What about mongodb ?
@pinusong
@pinusong 5 місяців тому
Why are there no subtitles? I need subtitles. Thank you very much!
@Yemen-Soft
@Yemen-Soft 6 місяців тому
Where is the translation of the CC?
@rahmatjo6837
@rahmatjo6837 5 місяців тому
bang nanya kenapa ya, ukuran size mysql cpanel sy sangat besar, padahal ukuran asli size phpmyadmin nya sangat kecil ?
@user-dg9by2ju2y
@user-dg9by2ju2y 11 місяців тому
👍
@batyr1166
@batyr1166 Рік тому
Please add subtitles
@haooooo
@haooooo 11 місяців тому
這集沒字幕..
ACID Properties in Databases With Examples
4:57
ByteByteGo
Переглядів 74 тис.
SQL performance tuning and query optimization using execution plan
49:23
techsapphire
Переглядів 238 тис.
Learn Database Normalization - 1NF, 2NF, 3NF, 4NF, 5NF
28:34
Decomplexify
Переглядів 1,6 млн
Good APIs Vs Bad APIs: 7 Tips for API Design
5:48
ByteByteGo
Переглядів 189 тис.
Database Indexing Explained (with PostgreSQL)
18:19
Hussein Nasser
Переглядів 286 тис.
T-SQL Skills: Loading Millions Of Rows Of Test Data In Seconds
11:19
Jeff Zuerlein
Переглядів 9 тис.
System Design: Why is Kafka fast?
5:02
ByteByteGo
Переглядів 1 млн
6 SQL Joins you MUST know! (Animated + Practice)
9:47
Anton Putra
Переглядів 70 тис.
Advanced SQL Tutorial | Subqueries
8:37
Alex The Analyst
Переглядів 348 тис.
Top 7 Ways to 10x Your API Performance
6:05
ByteByteGo
Переглядів 298 тис.
The five-step guide to SQL tuning | CloudWorld 2022
25:59
Oracle
Переглядів 15 тис.
10 Coding Principles Explained in 5 Minutes
5:44
ByteByteGo
Переглядів 72 тис.