How do indexes make databases read faster?

  Переглядів 44,724

Arpit Bhayani

Arpit Bhayani

2 роки тому

System Design for Beginners: arpitbhayani.me/sys-design
System Design for Experienced Engineers: arpitbhayani.me/masterclass
Redis Internals: arpitbhayani.me/redis
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafters.io/join?via=...
In this video, I explained how indexing speeds up databases by reducing disk I/O. A database stores records serialized onto disk, with attributes occupying specific bytes. Without indexes, reading rows sequentially can be time-consuming. I illustrated how blocks are read from disk and how indexing optimizes this process. By creating an index on a column like 'age,' queries become more efficient. With an example, I compared query times with and without indexes, showing an 8x performance improvement with proper indexing strategies. Indexing is crucial for database optimization and faster query execution.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.me/knowledge-base
Bookshelf: arpitbhayani.me/bookshelf
Papershelf: arpitbhayani.me/papershelf
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack.com
Thank you for watching and supporting! it means a ton.

КОМЕНТАРІ: 152
@ankitverma1790
@ankitverma1790 День тому
Great content...what I understood before is that if you have index on certain column, db will make the record s sequential on the basis of that column and it will access the record like an element is accessed from an array in o(1). That made me doubt as well sometimes when I thought to add indexes on multiple columns. This video made the concept crystal for me
@aayankhan5198
@aayankhan5198 2 роки тому
People run behind influencers and left with this GoldMine! Loved ur content.. following you when you joined Unacademy. 1 of the Most knowledgeable and inspiring person in the Tech Field.
@AsliEngineering
@AsliEngineering 2 роки тому
🙌🙌 areeee thanks 🤘
@pankajgaur3360
@pankajgaur3360 Рік тому
indeed
@SR-we1vl
@SR-we1vl Рік тому
Great content sir! 😄 Just a summary: Whenever a record is read from the DB, the entire block is fetched which contains that particular record. Since block reads are expensive(takes more time to read) it is better to index the records based on the property you want to search. So now when you search the record it is first checked into the index table which is stored in quite a few blocks, so block read time will be very low. Since we know in which of the blocks, the exact record(which can be computed from index table using id attribute) is present, we can directly compute those blocks and return the record from them.
@kashyapsharma8721
@kashyapsharma8721 Рік тому
Indices explained in the most simplest of terms. Excellent video , thank you Arpit.
@nimitkanani1691
@nimitkanani1691 Рік тому
Your videos are so simple to understand and the pace is also very easy to follow. Great work !!! Perfect for me, and you are also covering very relevant important topics. Keep up the work, man.
@RutwijGhadge
@RutwijGhadge 7 місяців тому
Nice and clear explanation 👍
@RTXXONn
@RTXXONn 3 місяці тому
wowww so simple yet so elegent.
@DemystifyFrontend
@DemystifyFrontend 7 місяців тому
I am lucky that I stumbled upon your channel it's really asli engineering in the true sense I really appreciate such content which touches the foundation concepts Thank you again, Asli me asli engineering 🙏🙏🙏
@bheemireddyvamsi7042
@bheemireddyvamsi7042 2 роки тому
Best video ever on the indexing concept. Thanks, Arpit, expecting many more videos from you on system designing. By the way, your neat notes are enough to clear all doubts. It will be good if you could add a video on the time complexity of operations like updating and delete on millions of data. How to increase the performance of DB like partition, and segregation techniques. Yes, I read this on many blogs but feel like they are half explanatory so need to understand them in depth.
@abhishekvishwakarma9045
@abhishekvishwakarma9045 Рік тому
Really liked how you explain things so easily with that much detail, totally awesome arpit sir 😁
@user-js3cx3qh9v
@user-js3cx3qh9v 9 місяців тому
Thanks bro!! Loved the content... great one!!
@harshildoshi3299
@harshildoshi3299 9 місяців тому
Loved the way you simplified the concept. Would you recommend any book(s) for the concepts includes Indexing, Cursor, Query Optimization etc?
@Sanjay-pu4sp
@Sanjay-pu4sp Рік тому
Fantastic! You have a real flair for teaching mate!
@97vaqasazeem
@97vaqasazeem Рік тому
Thanks alot i learn alot from your content it is clear and to the point !
@jaskiratwalia
@jaskiratwalia 3 місяці тому
Wonderful explanation of such an important topic. The way you explained it with practical examples made the concept quite clear to me. Thank you so much!
@ahirjoy
@ahirjoy Рік тому
Waiting for your book on System Design one day. There is real need for such book in the market. Keep inspiring.
@adamyatripathi2743
@adamyatripathi2743 Рік тому
Maza aa gaya! Behtareen! 🔥
@jspnser
@jspnser 6 місяців тому
I don't think even my professor in under grad or grad school has taught me like they way you're explaining the concepts so succinctly, thank you.
@kalpeshmali1476
@kalpeshmali1476 Рік тому
Wow aaj pata chala 😂 literally Gold 🔥🔥🔥🔥 keep it up sir
@Shivam-sl4sp
@Shivam-sl4sp 3 місяці тому
This is just too good! Thanks man.
@israaezzat2353
@israaezzat2353 2 місяці тому
you nailed it !! seriously its the best explanation I have ever seen for Index thank you 🙏
@deepakkothari2850
@deepakkothari2850 Рік тому
This was wonderfully explained 🙏
@vasiovasio
@vasiovasio 5 місяців тому
Great video. I just want to tell you that you have a typo on your thumbnail image - "indices" instead of indexes.
@raj_kundalia
@raj_kundalia 5 місяців тому
Thank you, Arpit!
@pratibhasharma4006
@pratibhasharma4006 Рік тому
Inspiring content. Thanks a lot sir
@LeoLeo-nx5gi
@LeoLeo-nx5gi 2 роки тому
Arpit bhaiya trust me this was really awesome (though I knew about indexes but still learnt many new things), Thanx ^_^
@vamsireddy5051
@vamsireddy5051 5 місяців тому
Really superb explanation, you not nailed normally but nailed it deeply 🙌
@parambharti7095
@parambharti7095 9 місяців тому
Very informative video. Thanks @Arpit Sir.
@varunkoranne6872
@varunkoranne6872 10 місяців тому
The content and explanation is just too awesome
@harshitanand7349
@harshitanand7349 5 місяців тому
Many people have superficial knowledge about indexing. All they know is that indexing makes querying faster. But they don’t know how. Glad I found this video through your twitter post. Will read about B+ trees soon.
@nytlyf2085
@nytlyf2085 Рік тому
21:49 perfect example of a range query optimization.
@biswaMastAadmi
@biswaMastAadmi 4 місяці тому
Beautiful explanation brother!
@jpnr8
@jpnr8 Рік тому
Thank you for lesson.. its awesome..
@420lomo
@420lomo Рік тому
Recently discovered your channel, very engaging content! Thanks
@abde2849
@abde2849 9 місяців тому
Hey Arpit - When db engine have to read multiple blocks, is that operation sequential or multi-threaded? Sequential can be too slow right? This is regarding example for reading 34 blocks taking 34 seconds...
@sarangbondre
@sarangbondre 2 роки тому
Hello, Arpit. It would be fantastic if you could include the following two points in the video or answer them here. 1. Can we index on the string (varchar/text), and if so, what happens? 2. What are the drawbacks of indexing (will it take up more memory?) Where should we not do it?
@HowTo-bk2jn
@HowTo-bk2jn Рік тому
Great explanation!
@singh.aadarsh
@singh.aadarsh Рік тому
Bro You probably in google 1cr package. Because you have a huse knowledge abouts system. I watch your almost 20+ video and every video has nice and details information. So you do amazing work, to teach Deep concept about engineering side. 👍👍👍
@adwaithssundar1620
@adwaithssundar1620 Рік тому
Superb explanation!!
@meeta890
@meeta890 4 місяці тому
Thanks for a great video! It helped me gain understanding of how an index works on a table while querying. It would be great if you could also make a video on how indexes are stored eg. B+ tree and what happens behind the scenes when a query with indexed column is fired.
@shrutimath3330
@shrutimath3330 Рік тому
Amazing content 🙌
@KK-kl7se
@KK-kl7se Рік тому
Good explaination , btw what tool are you using in your explanation @Arpit
@chandrachurmukherjeejucse5816
@chandrachurmukherjeejucse5816 Місяць тому
I don't know why channels uploading 100 videos on z-index/ basic js are more popular than this goldmine. As always great content sir
@AsliEngineering
@AsliEngineering Місяць тому
Thank you for resonating. As you found the stuff I put out helpful, it would be great if you could spread a word across socials. It would help people become more curious and overall better engineers. Thanks again!
@udayakumarsakthivel6285
@udayakumarsakthivel6285 4 місяці тому
You made index content as so simple 🤪. Great explanation 🎉
@shreyanshnanavati
@shreyanshnanavati Рік тому
Brilliant Video Arpit :) , Thank you so much, clears alot of concepts around Indexes :)
@AsliEngineering
@AsliEngineering Рік тому
Thanks man!
@situationinsights2957
@situationinsights2957 8 днів тому
Thanks brother for this video it helped me alot very nice explanation subscribed you
@soni.himansh
@soni.himansh Рік тому
Brilliant Video :)
@drunkcse7476
@drunkcse7476 Рік тому
Please make a video on types of index (sparse and dense) and real life use case
@naive-fleek7420
@naive-fleek7420 5 місяців тому
thanks bruh!
@nytlyf2085
@nytlyf2085 Рік тому
This guy is good!
@JardaniJovonovich192
@JardaniJovonovich192 2 роки тому
Great Video, Arpit !! It'd be interesting if you could make a video on BTree Indexes and clustered BTree Indexes.
@AsliEngineering
@AsliEngineering 2 роки тому
Soon :)
@kps4881
@kps4881 Рік тому
Till Arpit makes one you can check out this video. ukposts.info/have/v-deo/mYqbjahvbqhiw2w.html
@abhijitmitra4665
@abhijitmitra4665 5 місяців тому
Hi Arpit, Really thanks for creating such wonderful content and including how disk space getting managed for database. Curious to know what happens the blocks, when we add new column or change datatypes of column. Because in that case overall row size will change, and if data is stored in disk sequential order, then how databases internally manage that size change. If we consider a row which was earlier of size 200Byte and then later converted to 900Byte or so, I assume majority of the block reference which clustered index was holding will also change. That looks like huge write operation. Curious to know how that gets managed internally??
@mahimanbissa8308
@mahimanbissa8308 9 місяців тому
Hi Arpit, I didn't understand the part after we get the IDs after applying indexes. How will it search the IDs (buffered from the indexing) in the non-indexed database? in the example you provided while calculating query time after indexing, the two IDs you searched for were 7 and 2. And in your example, 7th ID occured in the first block while the 2nd ID was in the second block. What if the 2nd ID was in the 4th block? Would the database search in every memory block i.e. 2nd and 3rd and then 4th until it gets the 2nd ID?
@AsliEngineering
@AsliEngineering 9 місяців тому
Read about B+ trees and how they work. That will help you understand how rows are fetched.
@dhineshrathinam8756
@dhineshrathinam8756 4 місяці тому
Key points to take: 1. While reading the data it reads full blocks 2. Making an indices will create the separate table like structure with sorted order along with its primary key mapped
@vengateshgv8824
@vengateshgv8824 Рік тому
Hi bro. Can you please tell me. Is index take more ram when compared without index?
@gauravraj2604
@gauravraj2604 Рік тому
Hi Arpit, Amazing explanation. Thanks. I wa wondering how this block size (600B in your case) is decided?
@AsliEngineering
@AsliEngineering Рік тому
Typical size if 4kb. I took 600 as an example. Not too small not too large is the only criteria
@aditigupta6870
@aditigupta6870 3 місяці тому
Also in the worst case, it may happen that the age =23 records are present across all the 34 blocks, in that case the performance with indexing will even be poor I think: 2+34 =36
@santosh_bhat
@santosh_bhat 8 місяців тому
Very good explanation on index, learnt a ton !! Thanks One doubt, what if the resultant IDs from the index were part of each of the 37 blocks ? Eg: IDs - 1,4,7,10,13,16,..... Then we would read all 37 blocks so totally 37 sec + 2 sec( reading index ) > sequential read So when should we not use an index on a column ?
@venkateshnaga4550
@venkateshnaga4550 3 місяці тому
can u pls help here @AsliEngineering
@roshnigupta9296
@roshnigupta9296 2 роки тому
Hey Arpit, amazing explanation !!! I had a doubt that is it safe to assume that no matter what the Query is(considering only select queries with where conditions and not joins ), the time taken to execute the query will always be constant for a given table, in this case, 34 sec, without the indexes, or are there other parameters which can make the execution time to vary with significant differences, when queried different columns.
@AsliEngineering
@AsliEngineering 2 роки тому
The query execution times depends on a lot of other factors but we can certainly say that the pointed queries will require the same number of Disk IOs. The query execution time will depend on the number of rows being fetched, current load on the DB, IO tuning of the DB, Cache settings, etc. One thing that would impact the lookup time the most is caching policy and buffer size which would mean a large part of index can be kept in memory leading to quicker execution.
@ShubhamGupta3110
@ShubhamGupta3110 6 місяців тому
for alter table scripts when we add columns..does block allocation gets redone ?
@DurgaShiva7574
@DurgaShiva7574 7 місяців тому
One very basic doubt.... index is returning the ids of the rows..and NOT the pointers to the block(which i am not sure can even be stored and returned)...these ids may be present at the very LAST block of the storage, due to which we will need to traverse the WHOLE blocks.. which finally defeats the concept of indexing, and this video seems to be irrelevant.. Please guide me regarding my question
@random4573
@random4573 9 місяців тому
How will this work for lets say a string value or a text area
@prashantgupta387
@prashantgupta387 Рік тому
12:51 great content 😁
@yusufmirkar6508
@yusufmirkar6508 6 місяців тому
we used index for integer column so index fit in 2 blocks, if we have used string as index column (60 or 128 bytes), index would have been much larger, and hence take many blocks , right ? (60 + 4) * 100 = 6400 = 11 blocks
@DurgaShiva7574
@DurgaShiva7574 Рік тому
one of the best videos i have seen till date.. u r a gem of a person... But,. one doubt, here in RDBMS we were able to know the block from the id very easily because we knew that every block has a fixed and SAME number of rows stored in it... but, how will the indexing explained in this video work for NoSQL DB?.. as the amount of data / Number of rows stored PER BLOCK is Not constant .. thus ,after looking into the indexed table , we dont know , the given id is to be found in which block????
@maheshbalasubramanian605
@maheshbalasubramanian605 9 місяців тому
@DurgaShiva perhaps the index table in such a case must also be storing the length of record (in bytes) to help the DB fetch record for exact size. But am not sure, plz confirm how this is handled, @ AsliEngineering...
@kirankumargunturu8517
@kirankumargunturu8517 8 місяців тому
How do we know or guarantee ( based on id column in this example) that our table is stored sequentially ( 17:37 ) which helped us to read first two blocks ? I am assuming id column marked as PK. You can correct me please
@Avinashkk360
@Avinashkk360 7 місяців тому
I think the assumption is table stored in B+ tree structure.
@angnguyen-vn2on
@angnguyen-vn2on 8 місяців тому
I have a question, How do I know in reality size how a block is? Also the example in the video in the case of 1 record with age = 23 is in the last block. then the number of block reads is 2 (index block) + 34 (record block) = 36 is even more time consuming without indexing. Does the database have to read the blocks in turn to find the block containing the matching id
@AsliEngineering
@AsliEngineering 8 місяців тому
The size of the disk block is configured by OS typically 4KB. The DB does not have to read all blocks to find the block containing row with a certain ID. this is where B+ trees come in. It is O(log n) lookup.
@navinmittal4809
@navinmittal4809 2 роки тому
Do you have any suggestions on how to optimize writes/updates if there's an index column? You can assume you aren't updating index column value, but there are millions of rows to update.
@AsliEngineering
@AsliEngineering 2 роки тому
If the updates does not change the values on the index column there will not be any write happening on the index. This evaluation is taken care by the query optimization stage. In general when a massive update is run, there is a possibility of table (if not rows) are locked in order to protect and give a consistent view of the data to transaction happening at that moment. One way to ensure your DB performance is not hampered much because of massive update is by breaking it into micro transaction if possible. This way the db will be locking a few rows at a time. Let me know if you have any other special use-case which I did not cover. Happy to answer.
@santoshbhatnagar2155
@santoshbhatnagar2155 Рік тому
Can you share an example for the question being asked please.
@VinayKumarcs127
@VinayKumarcs127 2 роки тому
Hey 👋 Great content brother. Which software do you use for the notes? And do you use ipad pen ? Thanks
@AsliEngineering
@AsliEngineering 2 роки тому
Thanks. I do use iPad with Apple Pencil.
@VinayKumarcs127
@VinayKumarcs127 2 роки тому
@@AsliEngineering BTW, Which app is this
@kaafivikrant
@kaafivikrant Місяць тому
How do we know that the 4th index is in the second block only?
@soodankit1993
@soodankit1993 7 місяців тому
@AsliEngineering I have couple of questions here: 1. You mentioned that age index will be sorted in some order. Does that mean during insertion it'll try to move records to identify the correct place for the inserted record to maintain the sorting order. Will it have negative impact on performance. 2. In your example, ids with age 23 was exhausted by block 2 but what if there was one more record which is part of last block. In that case the response time will go much higher. Please help me with the answers in order to clarify all these doubts.
@AsliEngineering
@AsliEngineering 7 місяців тому
that's why you have B+ trees. I have a video about it on my channel
@abhishek3835
@abhishek3835 2 роки тому
Hi Arpit. When we used the age index table, we got row ids = {1,4}. Why did we only require 2 blocks search to find records with ids = 2 and 4? How did it know only those 2 blocks will contain rows with id 2 and 4, shouldn't all blocks be searched for this?
@AsliEngineering
@AsliEngineering 2 роки тому
Using the index we narrowed down the row ids we are interested in. When we go through the table to look for the rowwith those IDs we can leverage the storage structure used to store the table. This is where the magic of B+ trees come in for the relational databases. By default all the tables/databases are structured to make the lookup by primary key very efficient and this is something that powered by the core storage structure (B+ trees for relational databases). So when we are looking for the relevant rows with particular IDs the database engine would know which all block to reads and how to read them [typically done by managing and maintaining the disk offsets]. This makes are entire process super-efficient and not needing us to read entire table to find rows with specific IDs. Hope this helps.
@abhishek3835
@abhishek3835 2 роки тому
@@AsliEngineering Thanks for the reply. What you said makes absolute sense but I had another doubt after reading this. What if a table doesn't have a primary key, can indexing still work in this case? If yes, what will the index table point towards now as there is no primary key to point towards?
@suraj-upadhyay
@suraj-upadhyay 2 роки тому
Thanks for the explanation bro
@manchukondamanoj
@manchukondamanoj 2 роки тому
Thanks for asking this, I had the same doubt.
@karanbangia14
@karanbangia14 8 місяців тому
Can you tell how you choose indexes optimally? If i have 3 columns Id, fistname, lastname and i want to query by firstname , bylastname,byfirstand lastname how many indexes and what index should i make?
@AmarjeetYadav-of5qe
@AmarjeetYadav-of5qe Місяць тому
Index on firstname: If you frequently query by firstname, having an index on this column can significantly speed up those queries. This index will help the database quickly locate rows based on the firstname. Index on lastname: Similarly, if you frequently query by lastname, having an index on this column can improve the performance of such queries. Index on both firstname and lastname (composite index): If you often query by both firstname and lastname together, a composite index on these two columns can be beneficial. This index can speed up queries that filter based on both the firstname and lastname simultaneously. The number of indexes and which indexes to create depend on the query patterns of your application. Here's a summary: If you frequently query by firstname, lastname, and both firstname and lastname together, you might create three separate indexes: one on firstname, one on lastname, and one composite index on (firstname, lastname). If you only query by firstname and lastname individually (not together), you might create two separate indexes: one on firstname and one on lastname. If you rarely query by both firstname and lastname together, you might prioritize single-column indexes on firstname and lastname separately and skip the composite index. Remember, each index adds overhead during data modifications (inserts, updates, deletes), so you should only create indexes that are necessary for improving the performance of your queries.
@mukulr5171
@mukulr5171 Рік тому
❤️🔥🔥
@ShubhamGupta3110
@ShubhamGupta3110 6 місяців тому
for age = 23 we got record ids = [1,4] so we read 2 blocks which were adjacent. But lets if my record ids are [1,4,99] for age = 23 then we need to check all 34 blocks right since 99 record id will be present in 34th block?
@rahulkumarmishra1638
@rahulkumarmishra1638 Рік тому
Hi Arpit, Great Content Thanks, I Have This Doubt What If Each Block Has A Entry Where Age==23, So Our result buffer will hold all the IDs and each id will be stored in a different block, so in that case, our total will be 34 sec for reading all blocks and 2 sec for index scan so total time is 36 sec but in previous approach without an index it was 34 sec, so how to deal with such situations.
@AsliEngineering
@AsliEngineering Рік тому
During query evaluation, database runs an optimization and checks the best approach. It does this by maintaining the cardinality of each indexed attribute. Here as it would see a skew, it would not use index for evaluation.
@rahulkumarmishra1638
@rahulkumarmishra1638 Рік тому
@@AsliEngineering Got it Arpit Thank You 😊
@jayeshlokhande6367
@jayeshlokhande6367 Рік тому
Hey Arpit, thank you for the great explanation! Have one doubt, how does the database know that from [1,4] 1 exists in block 1 and 4 exists in block 2? Is there another data structures which stores this ?
@SumitSharma-bb5ik
@SumitSharma-bb5ik Рік тому
+1. Same question
@SumitSharma-bb5ik
@SumitSharma-bb5ik Рік тому
I guess id's are the address to the blocks, so that we can directly jump to that address and fetch the data.
@AsliEngineering
@AsliEngineering Рік тому
B+ trees.
@prachiagrawalcipher
@prachiagrawalcipher 2 роки тому
If id corresponding to age==23 were present in all the disk blocks,34 blocks then read query will become slow in this case. It would have taken 36 seconds. How to optimize that?
@AsliEngineering
@AsliEngineering 2 роки тому
Excellent question. A database engine can optimize this by not iterating over index when it finds the cardinality of resultset is almost equal to the number of table rows. This is why every database has the query optimization phase.
@prachiagrawalcipher
@prachiagrawalcipher 2 роки тому
@@AsliEngineering will spend some time reading about this query optimization phase
@sahil0094
@sahil0094 2 роки тому
@@AsliEngineering but even to find cardinality of result set on index search, it would still can process all blocks, right? So anyhow time is wasted
@rohansharma8618
@rohansharma8618 Рік тому
@@prachiagrawalcipher But what about time consumed in maintaining index in ordered form
@HimanshuKumar-xz5tk
@HimanshuKumar-xz5tk Рік тому
Lets say you got the ids from index - [1, 4]. You still need to iterate through all the blocks to check whether 1 or 4 is present in that row or not. Explanation seems incomplete to me. How do we get block from ids?
@AsliEngineering
@AsliEngineering Рік тому
B Trees
@HimanshuKumar-xz5tk
@HimanshuKumar-xz5tk Рік тому
@@AsliEngineering thanks
@navinmittal4809
@navinmittal4809 2 роки тому
Hi Arpit, I got little confused when you said index is on id column instead of age column at time 13:26. Later it got clarified that it's age column.
@AsliEngineering
@AsliEngineering 2 роки тому
My bad. Flow flow me ho gaya. Will be extra careful going forward.
@akashagarwal6390
@akashagarwal6390 Рік тому
after getting row id, how does the DB engine know which row to fetch exactly (assuming its not a PK or even if its a PK, in both cases, if it makes a diff) & from which block on disk?
@AsliEngineering
@AsliEngineering Рік тому
have a members-only video on that already.
@niteshrawat576
@niteshrawat576 Місяць тому
Question: What if one of the fetched index is 1000, now fetch the data block where this row is present a sequencial scan till that data block happens or is there some other way?
@niteshrawat576
@niteshrawat576 Місяць тому
Ahh, with a little researching I got to know that index also holds the "rowId" which is not the primary key but the internal id to identify which diskstore the row data might be present. This make sense to me but what confirmation if that is indeed the case.
@TheCosmique11
@TheCosmique11 2 роки тому
What software is used in this video to write notes, feels good to read from the notes.
@AsliEngineering
@AsliEngineering 2 роки тому
Description has the information 🙌
@KHUSHI-zd2hr
@KHUSHI-zd2hr 11 днів тому
does this mean that dbms creates an index for the column when a query is run on that column? i understood how indexing works. i just dont understand at what point an index table for each of the columns in a db is created.
@AsliEngineering
@AsliEngineering 11 днів тому
You are required to create indexes on columns that you will be querying. You typically do this while creating the table, or after that by firing CREATE INDEX query.
@theoptimaldeveloper
@theoptimaldeveloper Рік тому
one thing that is clear from this is indexing comes with the cost of making writes slower. But Can it make read slower also? Like In cases where we have to return almost entire table?
@Avinashkk360
@Avinashkk360 7 місяців тому
In case where we have to return almost entire table, query optimizer may choose not to use indexes
@pankajvermacr7
@pankajvermacr7 3 місяці тому
hey, i have a question let, suppose 1 row have 700 bytes of data, and cell block in disk space is of size 600 bytes, then how this row data is going to store?
@AsliEngineering
@AsliEngineering 3 місяці тому
That's a great question; glad you were thinking of edge cases. Read about TOAST in PostgreSQL.
@hookcrook795
@hookcrook795 Рік тому
what device or software do u use for writing content while u teach?
@AsliEngineering
@AsliEngineering Рік тому
GoodNotes on iPad
@harpalsinhjadeja5571
@harpalsinhjadeja5571 Рік тому
I don't know why no one pointed out but the time to compare is 34 seconds and 4 seconds not 32
@gauravraj2604
@gauravraj2604 Рік тому
Maybe I missed that but I didn't get how system is deciding that ids (1,4) are available in block 1 and block 2? Are we checking 1 by 1 in a linear-search fashion?
@gauravgupta5530
@gauravgupta5530 Рік тому
I have also this same question, how the processor will decide which block it needs to read to get the data it founds from indexes. @Asli Engineering by Arpit Bhayani.
@kps4881
@kps4881 Рік тому
If you look at B+ Trees you will get the idea. The B+Tree stores the location of the disk block(shown here as the id for easy illustration). So instead of 1 and 4 the address/offset to the disk is stored and then those blocks are read by the engine.
@gauravraj2604
@gauravraj2604 Рік тому
@@kps4881 thank you for reply. But can you please explain a bit more in terms of address/offset and block 1 / block 2?
@kps4881
@kps4881 Рік тому
@@gauravraj2604 The leaf node of the B-Tree or B+Tree is a pointer to the offset of the disk block which contains that actual data. So once you traverse the tree and reach the leaf, you get the pointer to the disk block that is used to read the data from the disk directly.
@sanjeevsingh-zz2hg
@sanjeevsingh-zz2hg Рік тому
5:07 why it so? why full block size is read instead of the one which we are interested in?
@AsliEngineering
@AsliEngineering Рік тому
Because that is how disk IO works. When you read anything from the disk, your file system reads the entire block (that contains the intended region).
@dipankarkumarsingh
@dipankarkumarsingh 11 місяців тому
@akshatgoel5971
@akshatgoel5971 2 роки тому
I discovered your channel today while going through your blog on replication. I like the explanation but I think video title should say how indices make database reads faster. Write performance decreases because of additional indices.
@AsliEngineering
@AsliEngineering 2 роки тому
Excellent suggestion. Altered the title. Cannot edit the video now, but I get the point. Thanks a ton for the input.
@akbarhussainpatel6663
@akbarhussainpatel6663 2 роки тому
I think this is not always the case if index update is async.
@svkrclg
@svkrclg 6 місяців тому
how will the index be stored if there is no primary id in the table ?
@AsliEngineering
@AsliEngineering 6 місяців тому
Order of insertion.
@9231Gan
@9231Gan 10 місяців тому
This is not a one day knowledge from apart from your course where should i learn this what books or paper to refer
@vivekbansal-bc9eg
@vivekbansal-bc9eg 6 місяців тому
isn't index stored as B Tree?
@AsliEngineering
@AsliEngineering 6 місяців тому
Yes. The Index is stored as Btree. The other popular option is Hash data structure.
@abhishekbose414
@abhishekbose414 6 місяців тому
is it intentional to blur the video?
@MO-fg2cm
@MO-fg2cm 6 місяців тому
uska tab ka resolution kam hoga
@bhagiyarajmahesh8972
@bhagiyarajmahesh8972 2 роки тому
What if the there are 33 records of age == 23 in those table of 100 users. And each record (age == 33) present in each block. That is one record(age ==33) in each block.so in this case we have to read all the 34 blocks + 33 index = 67 secs. Am i right? What will happen in this scenario.?
@leaptatva
@leaptatva 2 роки тому
For index it will anyways be 2 reads since all the 33 age records will still be in 2 blocks for index. So 34+2 = 36 secs?
@AsliEngineering
@AsliEngineering 2 роки тому
@@leaptatva Thanks for pointing it out. It will take only 2 block reads to all the index blocks and then 33 for each of the individual disk blocks. You are correct Sunil. My bad I should not have skimmed through it. Really sorry.
@AsliEngineering
@AsliEngineering 2 роки тому
I made a blunder in replying you. Please go through the messages below.
Handling timeouts in a microservice architecture
23:38
Arpit Bhayani
Переглядів 16 тис.
Why do databases store data in B+ trees?
29:43
Arpit Bhayani
Переглядів 25 тис.
SQL Index |¦| Indexes in SQL |¦| Database Index
9:57
Socratica
Переглядів 601 тис.
Database Sharding and Partitioning
23:53
Arpit Bhayani
Переглядів 56 тис.
SQL indexing best practices | How to make your database FASTER!
4:08
CockroachDB
Переглядів 47 тис.
How Dukaan moved out of Cloud and on to Bare Metal w/ Subhash | Ep 5
1:37:08
Database Indexing for Dumb Developers
15:59
Laith Academy
Переглядів 34 тис.
Learn Database Normalization - 1NF, 2NF, 3NF, 4NF, 5NF
28:34
Decomplexify
Переглядів 1,6 млн
Clustered vs. Nonclustered Index Structures in SQL Server
8:04
Voluntary DBA
Переглядів 638 тис.
How to handle database outages?
30:38
Arpit Bhayani
Переглядів 7 тис.
Database Indexing Explained (with PostgreSQL)
18:19
Hussein Nasser
Переглядів 285 тис.
Компьютерная мышь за 50 рублей
0:28
dizzi
Переглядів 2,2 млн
Why spend $10.000 on a flashlight when these are $200🗿
0:12
NIGHTOPERATOR
Переглядів 17 млн
Phone charger explosion
0:43
_vector_
Переглядів 46 млн
Рекламная уловка Apple 😏
0:59
Яблык
Переглядів 787 тис.