Thank you. That is very kind of you.

Thank you, for the kind comment.

Thank you. That is very kind of you to say.

Thank you, Avik.

Thanks Gouled

Sounds Awesome! There is so much cool stuff you can do with a stereo camera. I made my own because I had some extra esp32-cams. But, the good thing about getting a proper stereo camera is that you can sync the frames a bit more easily, which is important if you want to do stereo video. I had some difficulties with synchronisation when I tried to use the esp32-cams for stereo video.

@@jonathanr4242 Oh? Surprised. With short wires the latency should be short enough to synch the first frame.

@@OMNI_INFINITY I was using wifi with frame buffering. I didn't realise it at the time, but one solution would be to grab the most recent frame in the buffer, and drop the others.

@@jonathanr4242 Ah. Can connect rx and tx of those seemingly.

Thank you. I appreciate it.

Thank you for the kind words.

Thank you, Mahesh.

Thank you. There are a number of projects, tweaks and modifications I'd like to make. The structured-light scanner is a very good idea.

I wish @@jonathanr4242 that I had came across your channel/video before I bought the kinectv2 adapter!!!

Thank you, Collin. Yes, openCV is powerful stuff. I am working on another project with the stereo camera which I hope to post in the next couple of weeks.

Thank you

Thanks James. I will look at the laser line trick. I am planning to add a LIDAR to the esp32 camera at some stage. Hopefully, this will happen in the near future sometime.

@@jonathanr4242 LiDAR are the cool toy everyone wants to play with. And they are wonderful, not doubt. A little spendy, but I'm seeing that the prices are getting down into the below $100 range? Hmmm... looks like a lot of those are NOT really LiDAR. LOL. The real problem with LiDAR and most 3D cameras is that they bury you in data. In the same way that the NN takes several seconds to lock in on the objects in this project, sorting through the LiDAR data and making use of it takes time. The laser line thing is nearly instant. It's a great first shot at ranging, which then helps things like image recognition because instead of handing the NN an entire frame, you can hand it just the one area where the object is known to be. That really speeds up recognition. And the total cost is like $5... or less actually, you can get cheap modules for a $1 from china.

Thank you, James. I will definitely look into it. Is the module you're referring to a time of flight sensor? Do you have any references for projects that have done this?

@@jonathanr4242 I did it multiple times back in the 80's and it's been done many times since then but usually in slightly different form. It's nothing more than a laser line connected to an io pin to turn it on and off, and a camera. You capture a frame with the laser line off, then turn it on and capture another frame. Subtract the two frames, and the only thing left will be any reflections of the laser line against objects in front of the camera. Because of parallax between the line and the camera (the line is placed a bit below the camera), the distance from the bottom of the picture, up to the pixel where the line shows up, will be the range to the object. To clarify, the image subtraction is pixel by pixel. The first pixel (upper left corner) of the first image has a value (or values in RGB) and you subtract that value from the first pixel of the second image, making a new image with a pixel value which is the difference. One common use is with a turntable to make a low cost 3D scanner. like this from my site: techref.massmind.org/techref/new/letter/news0310.htm I'm suggesting that you don't use the turntable and you turn the laser and camera on their side. The laser projects out a plane under the camera, and only detects things that are resting on the surface. But then... what doesn't rest on the surface?

@@JamesNewton Thanks James! That sounds awesome. It gives me so many ideas. I had a project where I tried to use a laser to guide an electric water gun to the target. But I couldn't get this to work because the red light from the laser saturated the camera (showed up as all white), so it didn't work very well. I think you've just told me how I can solve this.

Thank you, Arnab. I'm working on something at the moment with a depth camera that might help. Hoping to have it out by the end of this month or middle next month at the latest. R-CNN is a bit slow, might be better to use YOLO if you want to do it in real time. In the video after this (esp32cam for autonomous task completion) , I do an implementation with YOLO, which is much faster than the R-CNN implementation.

@@jonathanr4242 Yes, I have been seeing you are doing a lot of work with esp32 cams and implementing it with various case. I have seen the video on your yolov8 implementation. I have one concern though, As i am working on people counting, i might use it at night too. In that case, can stereovision help with IR footage?

@@arnabbarua1670 I don't see there would be an issue with using two IR cameras, as long as you can get a decent image from each camera and do feature mapping. As well you can estimate the size of the person, so can estimate how far a way they are from a single camera, which might help a bit. I mean it might be an advantage if you have both cameras with IR LEDs because you'd have more IR light, but on the other hand there might be too much IR light (I'd say the more the better, given those IR LEDs are not very bright). Sounds very interesting. I was looking at one of those the other day for raspberry pi, but it was a bit more expensive than I wanted to spend.

Hi. Thanks for your comment. Yes the project needs a bit of tweaking. There are probably a fair few heuristics one can employ. In fact, every AI project will have some tweaks applied that will greatly improve them. Also, I want to say that I discovered in another project that when I bring in the image from the esp32 cam, I'm taking it from a queue, so that I don't get the most recent image, I can get a bit of a delay. To fix this, I found this wrapper for the image capture stuff which avoids this bug. I've used the wrapper in another video with the time of flight camera. The github repo should be referenced in the description of that project. So basically, it uses a thread to keep reading eliminating frames in the queue so that when you want to grab a frame, it grabs the most recent, which is essential if you have moving objects.

Hi Ans. Thank you for your kind words. I am using an old-ish lenovo yoga slim laptop. It takes around 5 seconds per image. In my next video I use YOLOV8 (model m), which is about 10 times faster. I haven't tried on a jetson nano, but I would like to. Please let me know how it goes.

Did you calibrate your camera?

Hi Kevin. Thanks for your comment. In my experience, ov2640 from different sellers (buying from ali express) can have varying degrees of quality. So, I would say you need to calibrate every one of them. I haven't really spent much time on calibration. I usually just plug in the sensor and hope for the best.

Thanks. Most of the calibration information should be in the python notebooks on the github.

Also, I should probably mention that my approach is fairly simple. I'm only using the distance of the object from the camera, and the distance apart of the two cameras. My assumption is that both cameras have the same focal length.

Thank you but the parameters when taking the photo or video which concerns the values ​​width, height, fx, fy, cx, cy that I quoted how are they found?

@@welsonfy5246 I think my approach is a little different to what you find in the text book. I have two parameters, which are tan(theta) and focal length. And then, I take the corner point of the object as the matching point in each image to calculate the distance. I go through the calculations in the video, and they should be included in my code. There is room for improvement. For example, one improvement could be once you find the matching objects you can correct for parallax.

It will be much faster with a gpu. Also YOLOv8 will be faster than mask r-cnn.

Hi. I’m not sure I understand the question.

Thanks for your comment. It's actually very simple to connect the hardware. All I do is make sure the two esp32 cams are on a grid, so that they are fixed in position, and then upload the sketch to them. In all honesty, the kind of stereo cam I've used is a bit messy. I would probably recommend using a proper usb type stereo camera.

Thanks for your thought inspiring comment. Using 640x480 resolution, I think I was able to accurately measure up to around 120cm (if I remember correctly ?). As the resolution increases, so does the maximum distance. You can calculate the theoretical maximum by taking the left/right distance apart as 1 pixel. But I would probably divide this by at least three, taking into account practical considerations. But then I guess the distance resolution would also decrease as the distance away increased.

Hi. I think it would be possible, but would require decent labeled data for training.

The pixels making up distance dc. I'm trying to replace all distance measurements with pixel counts, so that it makes sense in digital image processing.

Yes. The only issue is you will need lots of training data.

Hi. Thanks for your comment. Yes. It’s not difficult to measure height and width. You can do it from the geometry by taking the ratio of pixels across to the pixel width.

Thanks for your question. You’d be better off looking at key point detection in something like yolov8 or v9.

You can run it on a local network, but it needs a computer that can run open cv and pytorch.

Thanks for your message. There are a couple of things you can do. 1. If you want to speed up the object recognition part, you can use YOLOv5 or YOLOv8 instead of r-cnn I am using. This is much faster. Also running it on a GPU will make it faster. I have implemented this in the next video. The code is on this github repo: github.com/jonathanrandall/autonomous_task_stereo_cam 2. Secondly, if you just want to speed up the video capture part, you can try running on a separate thread. See the solution to this question on stackoverflow: stackoverflow.com/questions/43665208/how-to-get-the-latest-frame-from-capture-device-camera-in-opencv You might want to try moving the imshow inside the videocapture class if you implement the above thread.

That sounds very cool, indeed. It would be amazing to build something like that.

Yes. You would need to run it on a gpu, and also YOLO would be a bit faster.

You need to calibrate the camera. Basically, you need to measure the same object at a few different distances and adjust the focal length and tantheta.

My go to pla es are dronebot workshop, random nerd tutorials and Paul mcwhorter from toptechboy.com, the best way to learn is to do projects.

@@jonathanr4242 hey man ...thanks for the reply u gain my respect as a teacher 🛐

You need to use two esp32.

​@@jonathanr4242 Sir, Should I train model for object detection? I had used two esp32 like you but it was not detecting any object like person etc. Can you please help?

Thanks for your comment. You can ask any questions here or email me.

@@jonathanr4242 ciao. YT is non publishing my answer. What is your email address?

I don’t think so. It’s specifically designed to be used with two cameras.

If you know the target is stationary you can move the camera to get multiple shots to do the trigonometry

Thanks for you comment. I am quite far from the gold coast. I am in Sydney. Happy, if you have any ideas you want to bounce off me.

It's difficult to tell without know what part of the code is causing the exception. Usually, I would try to isolate the error.

what is the error that you are getting?

@@jonathanr4242 pytorchstream readerfailed reading zip archive failed finding central directory

@@a7medrafat31 Oh! I've never seen that error. In the next video, I do the stereo pair matching with yolov8. I've put the notebooks on the github for those as well. It's much faster than pytorch, but not as reliable. And you can tweak it, I suppose. The git repository is called autonomous_task_stereo_cam

@@jonathanr4242 i resolved the error but i want to know how to make inference continously ?

@@a7medrafat31 You will probably need a stereo camera that can synchronise the left and right images. If you're using the esp32-cam, there will be a lag between left and right images. Also, you will most likely need to implement the object segmentation and detection on a GPU to get it to keep up with the frame rate.