Ya, true man, @gauravSen, and this guy are providing this valuable information. If you know anyone like these guys on youtube reply back

Hussein my man you also share you knowledge Love you

Nice work... any thoughts on this review ? medium.com/double-pointer/review-of-grokking-the-system-design-course-c8613c28f3a1

@@ujjwalbansal1070 Check out Hussein Nasser's videos, they are very informative and fun to watch.

Oh my god!! Life has come full circle for me. Finding a Hussein Nasser comment on Narendra L's video. You two are my idols!!

Thank you! I was scratching my head about what that means... :)

Yeah, indian numbering doesnt go up in terms of 3 digits like (thousand, million, billion, etc..) It goes up in 3, 2, 2, ... eg . 1,000 -> Thousand 1,00,000 -> Lakh 1,00,00,000 -> crore ...

@ 20:10, he represents 1 million as 10,00,000 as 10 lakhs (see position of commas)

You should either stick to the metric system or use a global system that doesn't rely on localized counting systems. This would be a red flag for me.

Thanks a lot . I enabled subtitles, but no luck ☺️.

😂

Did you get the offer?

oh oh 🙏

Thanks a lot :)

Whoa im completely new to soft dev at this level but that seems like a really good idea to me. If someone disagrees please lmk why

This is basically the same as using the zookeper pregenerated 'ranges of keys' presented in the video.

The single slash means integer division in a lot of languages.

it's actually not a typo, because it's in Python 2 (see the print operator below) :)

So there are a couple of approaches as explained in Grokking the System Design interview and I am quoting them here : "If we chose to actively search for expired links to remove them, it would put a lot of pressure on our database. Instead, we can slowly remove expired links and do a lazy cleanup. Our service will make sure that only expired links will be deleted, although some expired links can live longer but will never be returned to users. Whenever a user tries to access an expired link, we can delete the link and return an error to the user. A separate Cleanup service can run periodically to remove expired links from our storage and cache. This service should be very lightweight and can be scheduled to run only when the user traffic is expected to be low. We can have a default expiration time for each link (e.g., two years). After removing an expired link, we can put the key back in the key-DB to be reused. Should we remove links that haven’t been visited in some length of time, say six months? This could be tricky. Since storage is getting cheap, we can decide to keep links forever."

Hi, If I reuse one of the expired link again then is it possible that a customer tries to access a very old URL and gets redirected to a wrong URL ( recently used post clean up) . This might cause an issue if , a system needs to provide short URLs and keep the mapping for 10 years . Any solution for this?

@@sombudhya That is genuinley good question and to be honest I think you may be right... according to this design I cant see imagine thaa a mapping will guarantee no repetiion of url "keys". But if you look at it pragmatically, to run out of 62^7 addresses your service needs to have to go through 3.5 TRILLION Writes which is alot 😂 but.... a busy service *could* use that up very quick..... a simple solution to that problem could be to transition to 8 character spaces on top of precious 7 character, that is over 22 TRILLION URLS! and Im pretty sure for any system to loop back from that counter, so much time would have passed that the distributor of old links would have realised themselves....

@Abhishek ChandelRate limiting can stop user for certain period of time only.

Like you said we will use base 62 on the counter. Because we're using base62 we will actually have 62 different combinations for each character. Because counting in base 62 would include all these characters [0-9 A-Z a-z]. instead of just [0-9] 2) If you go to the highest number in base 62 which would look like: zzzzzzz this will equal a little over 3.5 trillion. Which is an extremely high number of unique IDs and that would take a significant amount of time to reach. But, after reaching that number then I assume they would probably clear out old data that aren't being used anymore and restart the counter... but if for whatever reason they could not keep up then I think, and I could be wrong, the solution would be to increase from 7 character limit to an 8. Which would make the number of ID's insanely high, over 200 trillion.

@@metaversity3586 I think Sarafs 2nd question was more in line with how do we use the base 62 encoding to generate a 7 (or a different specific number) char encoding? Do we simply take the first 7 chars of the encoding we get by converting the long URL to base 62 using the auto incremented id? Wouldn't this result in the same potentially non-unique issue as the MD5 encoding approach? This is something I am confused about as well.

In btech level interview?

@@ssssahil yes....so it's very very easy, isn't it?

Did u get in?

yaa, there are many flaws in the soln discussed. I shared a list of problems in my comment.

absolutely right we only need to store the unique generated string 👌

@@yasserakbbach7342 You can have the primary the table link value as primary and unique in that case it won't happen voila

good question, I think the cut off might be the only solution, or we have to randomize until we reach 6 characters

ah, thank you! I was re-watching minutes trying to figure out what that was!

From what I understand of URL shorteners, what you described is actually a desired feature, especially in order to provide accurate metrics for the user (eg. if two users shorten the same long URL and want to know which shortened URL had the most visits, each shortened URL must be unique, not the long URL).

@@madeofhuman thanks

Sure, when time permits!! :)

@@TechDummiesNarendraL What happens when Zookeeper service goes down, doesn't that introduce a single point of failure or should we think of a backup?

@@mohitdiwan22 What happens when a server goes down? Same thing. Multiple copies.

Same URl wont be converted to Short URL again. It returns the existing one from DB. If the URL doesn't exists in DB, Then it generates a new SHORT url.

@@saikishorer So does it have to check from database?

@@saikishorer But what if two user try to shoten the same url at the same time , then in that case there will be two entry of the same URL , how can we avoid that.

i like this information...thank you for sharing it

we can base64 encode the counter numbers and as explained in the video all the numbers between 0 to 62^7 can be encoded to create a string of length

It's been 3 years and no reply... The counter solution does not use a hash. It's just a number represented in base 62 so there won't be a collision if the same counter values are not used

Im also wondering about this

how about taking it as a digit system with 62 characters, that meas we have abc....xyzABC...XYZ012...789 corresponding to 0...61, so an input 0 will have output aaaaaaa, and input 1 will be aaaaaab, so aaaaaba will be 62. For numbers from 0 ~ 62^7-1, we can have exactly 7 characters.

I believe we can remove the entries which are used or expired.

at that point you can truncate the hash to 7 characters

I think this can be controlled in the code of generating the unique code. generateUniqueCode(deci) { const a = [0...9a...zA...Z]; while(deci>0 && len

When you perform Base62 on the Timestamp and prefix of server ID, the resulting string will contain more than 7 characters. Since the tinyURL cannot have more than 7 , you will forced to choose only first 7 or last 7 (as per your choice). This will create collisions.

Zookeper is distributed system. If one fails you will have one more node to take care.

Zookeepr is deployed as a cluster - typically you would have 3, 5, 7 or more zookeeper nodes managing many instances of an app (say 20-200 instances). Zookeeper knows how to synchronize itself so if app instance 5 went down, all zookeeper nodes will eventually (and rather quickly) know about it.

@@TechDummiesNarendraL Thank you for the videos! Amazing job! One question - if we need to restart all Zookeeper nodes in the cluster, how Zookeeper would know the actual count of the counter ranges when it goes up? Do we need to store it somewhere?

I was also wandering the same thing until I look again at the code and played with and I got the answer. Notice that the base62 code that he showed is oblivious to the origin url, it is only output the counter in base 62. In this approach we ignore the original url while generating the key and only map counter in base62 as key to the original url

​@@user-jp5ck4xw2t Does that mean our Counter would need to start at a number high enough to generate 7 characters on its own since 1 for example = n which would not work in this case. Also what if the 7 digit prefix of 2 numbers is the same would this cause a collision?

It is OK to have the scenario you described. You could have two different shortURL point to the same longURL, but not the other way around. In the example he provided, two different longURL (said longURLA and longURLB) resulted in the same shortURL (us.com/GA29x51). As a result, when shortURL (us.com/GA29x51) was referred and translated back to its corresponding longURL, redirection could go either longURLA or longURLB, which is not deterministic.

Yes. Does it matter to have different short URLs? Depends on the requirements.

Even i am thinking of same.

zookeeper can be configured as single server or in quorum mode with multiple servers

Sure, Some time

IRCTC please

@@TechDummiesNarendraL irctc on demand

*Did he ask to concatenate? I think you can just store mapping of input long url to counter value in your DB that should be good enough and will guarantee zero collisions.*

@@studiousguy8138 but how will you map the hash in the URL with the counter. When the user uses the short URL, how would map it to the counter in the DB

@@rishabhjain2404 Hi! Rishabh it will not map to counter it will take it from database you can make the unique id as primary index and then while fetching it will be faster. SO when user click the shorten URL it will take the unique id fetch from database via primary index and as long url will be stored with short url you will get it.

@@shreysom2060 I think the question is, multiple integers can be mapped to the same hash using Base62. How to retrieve the long URL now if you only have the hash, as it may be mapped to multiple long URLs?

@@hoyiya i think he mentioned that every shortened url will be unique right,in that case it can act as primary index in rdbms and user can fetch it right? I will watch the video once again though

i think the solution here is to provide the already inserted short url? If it's the same long url, we wouldn't insert anything that already exists.

@@justSomeUserOnYT With the counter approach, we just get the counter and generate the short URL. We cannot check if the long URL is already available as it is costly. My approach to work around this is to allow having multiple short URLs for same long URL. But cache the long URL to Short URL pair. So that if there are multiple requests for the same URL, it will be rendered from cache. At least we can restrict the same user misuse the API. Use LRU cache eviction so that the most frrequently used long urls will not be duplicated. On top of this, there should be a rate limiter to avoid DDos.

We can consider ZK as a single service, but it is not a single server. It will remain available as long as majority of the under the hood machines stay alive. I'm pretty agree with you about "non randomness" of the resulting urls is a problem. Because your service is pretty DDoS-friendly :). But you can made this approach better almost for free. Say, you were given the range [2^16, 2^24]. You can generate some random number M from [2^16, 2^24]. And let your counter currently having value C. Then you can use (C xor M) instead of just C. This makes your urls not so much consecutive (you should not forget about the case when C equal to M :) ) Obviously, you can use more sophisticated hash functions as long as they are injective.

probably your conversion is incorrect, in my code b62(3000000000000) = QODkgKc which is in seven characters.

Yes, but once you start talking and they see you know your stuff, they will interrupt you.

Not for entry level positions. For positions like sde2 , sde3 these are the deciding factors

Correct! for that we need to design a system failover, and the same is also applicable on DB unavailability.

How'd it turn out? Offer?

Can you explain a bit more about this?

we should use queue