How to use IPAdapter models in ComfyUI
Переглядів 83,340
День томуEverything you need to know about using the IPAdapter models in ComfyUI directly from the developer of the IPAdapter ComfyUI extension.
👉 You can find the extension "ComfyUI_IPAdapter_plus" on github here: github.com/cubiq/ComfyUI_IPAd...
00:00 Introduction
01:33 Basic Workflow
05:55 IPAdapter Plus Model
07:32 Prepping Images
09:16 Sending Multiple Images
14:07 Face Model
15:18 SDXL
17:42 Img2Img
19:10 Inpainting
20:00 ControlNet
21:40 Upscaling
23:17 Saving embeds
26:33 Conclusions
🎵 Background Music
-- "Part A" by Alexander Nakarada (www.serpentsoundstudios.com)
Licensed under Creative Commons BY Attribution 4.0 License
-- Last Stop Synthwave by Karl Casey @ White Bat Audio (whitebataudio.com/)
-- CyberPunk City by Peritune (peritune.com/blog/2020/05/22/...)
@AB-wf8ek 6 місяців тому
Just want to say Thank You Thank You Thank You, from the bottom of my heart. There are very few developers that take the time to actually explain their tools, let alone include additional options such as the saving embeddings, that offer huge potential for sharing, and extending the workflow in regards to resource management. You are a huge asset and very much appreciated.
@ronnykhalil 6 місяців тому
Your comfy node (and this video) are invaluable resources! Thanks so much for helping me wrap my head around IP!
@pedrogorilla483 6 місяців тому
Thank you for creating this implementation! Very clever solutions for handling workflows! I saw you added weighting options for the images as someone requested it, I was doing it through repeating the same images a few times to increase their weight, it was very messy 😅
@user-kw3ch2xr6h 5 місяців тому
this is so absolutely bonkers I can't beleive how much things have changed in 3 months. I just found your channel and binged your last months content and holy whattheheck this is brilliant.
@aayushporwal6281 2 місяці тому
Just want to say Thank you. For 2 days, I have been searching the way to inpaint using image and in the video, you have explained it. in a very easy way to understand. Thank you very much.
@matthewkolb9950 3 місяці тому
Excellent video! You have such a pleasant style of communicating that it really was a pleasure absorbing all of this information. Well done and thanks!
@dasomen 7 місяців тому
What a useful tutorial, absolutelly fantastic, thanks a lot!
@SearchingForSounds 5 місяців тому
Oh man, this is changing my world. So much we can do with this. And... you explained your tools. Thank you so much!
@carstenli 7 місяців тому
Incredibly valuable tutorial. Keep up the good work.
@Tj4ckinator 6 місяців тому
Great video! So informative and straight to the point 👍 I would love to see your video on the training
@SsicKsSicK 5 місяців тому
Thank you so much for this AMAZING feature and also the detailed readme plus VIDEOS! We need more people like you! You are an enrichment to the AI / Open Source community
@VincentRie 3 місяці тому
double down on that. thank u so much for this amazing piece of work.
@johnmcaleer6917 7 місяців тому
Wonderful tutorial...very clear and easy to follow...👍🏻
@dflfd 4 місяці тому
thank you, this is fantastic -- very well explained. The saving embeds is brilliant!
@kleber1983 4 місяці тому
So much content in a single video, this is amazing... thanks so much!
@resh6701 7 місяців тому
It worth 30 min without hesitation
@manunoyonify 7 місяців тому
Awesome. Adding different image ratio inputs and outputs and the ability to give custom weights to batched input images would be a blessing !!! Thx for ur work !
@brandonflores4 2 місяці тому
i think scott detweiler made a video on weighted inputs. one of his comfyui episodes. unsure if it had to do with ipadapter.
@EH21UTB 7 місяців тому
Wow! Fantastic video, I learned so much, thank you!
@WhySoBroke 7 місяців тому
Since you have so much expertise and knowledge in this topic, I really look forward to the training model tutorial 😊
@latentvision 6 місяців тому
I'll work on that but it's really for kinda edge scenarios and optimization (I guess can be useful for some art styles)
@i.venture 3 місяці тому
New to ComfyUI. Thanks for this. It was very helpful for someone like me who has heard about IPAdaptor, but had no clue what it really does.
@lucretius1111 5 місяців тому
Just came across your amazing tool. Congratulations and thank you! Amazing applications for this in the future, I think.
@musty5551 7 місяців тому
Amazing work bud!
@jameslanman 3 місяці тому
Thank you for your time and energy on this. This was a great introduction to comfyui.
@adelechelmany 7 місяців тому
Man, I can't thank you enough for this, Bravo. 👏👏👏
@WhySoBroke 7 місяців тому
Amazing tutorial!! Much respect ❤️🇲🇽❤️
@EliSpizzichino 7 місяців тому
Bravo! Fantastic work with IPAdapterPlus, I would be very interested to see also a video on the training process you mentioned. I'm trying to train a style that is quite unique, so I can't just use one image. I'm getting poor result with Lora standard network training, and standard dreambooth training
@steventapia_motiondesigner 7 місяців тому
Thanks so much for IP Adapter its been working nicely in Automatic 1111. I still have to learn how to use it thoroughly. More tutorials would be appreciated!
@drosselmir Місяць тому
I'd classify this as the top 5 informational comfy/SD video to watch. Thank you Mato sir! Also looking forward to the training tutorial.
@fedoborod8890 6 місяців тому
This is AMAZING! Your explanation and trick, omg! I learned alot!
@stephantual 4 місяці тому
This a game changer and continue to a be game changer. Not to mention you are kind enough to provide not just a video, but A GREAT video on how to use your tool! Thank you a million times.
@obscuremusictabs5927 5 місяців тому
Amazing. Thank you for all your hard work.
@swannschilling474 7 місяців тому
Thanks a lot for this one!! Great tutorial!!
@simonrobson615 3 місяці тому
Brilliant, thanks so much. Your system and explanation is awesome, I've learnt so much!
@euggraph1048 4 місяці тому
Great Job explaining everything , Thank You!!!
@yiluwididreaming6732 7 місяців тому
One of the best nodes I've seen for COMFY. Using it to lead renders with my current workflows and results show increased accuracy and detail. SUBBED!!
@CPDCT 7 місяців тому
thanks for creating this! Game changer on comfyui!
@aliyilmaz852 Місяць тому
great explanation, and very good benefits you add into ipadapter. Thank you so much
@parlabaneisback 5 місяців тому
A fantastic presentation, thanks so much.
@Inner-Reflections-AI 7 місяців тому
Thanks for the great guide!
@LTE18 6 місяців тому
Thank you. This is huge to the community
@ALatentPlace 7 місяців тому
Great extension and great tutorial! Awesome! Thanks for this It was posted on my Discord yesterday after I released a video about a more basic noise injection technique just using nodes. I will defintely try this out and, if it's ok for you, introduce it to my german speaking audience.
@latentvision 7 місяців тому
hey thanks! I checked your videos, I don't speak German but they are really well done and easy to follow. Absolutely take whatever you want from my video and the content on my repository.
@enriqueicm7341 4 місяці тому
This was very useful and very well explained! thanks you a lot!
@creativojavier 5 місяців тому
Grazie mille for sharing this tool and explaining it so clearly!
@enthuesd 6 місяців тому
spectacular presentation, thank you
@ryanontheinside 3 місяці тому
thank you so much yo! you are, literally, incredible.
@joonienyc 5 місяців тому
just beginning with it and already seem this great nodes
@godpunisher 7 місяців тому
nice one, i'm going to use it. thank you.
@alpaykasal2902 3 місяці тому
WOWWW! This looks amazing... I'm going to try this out tonight (I may not get any sleep this weekend :) )
@M0K0K0 3 місяці тому
Thank you very much for this video and nodes !
@Ekopop 4 місяці тому
thanks a lot for your very clear explanations and this awesome tool.
@Sebastian-cn8lh 5 місяців тому
incredible detailed!... thank you !.
@techeman369 4 місяці тому
Great Explaination simple crystal clear thank you so much
@reapicus557 Місяць тому
Thank you very much for this tutorial! This tool is very powerful, and it is going to make my workflows so much easier to construct.
@AnnisNaeemOfficial 2 місяці тому
I love this tutorial. But at 14:21, you have a load clp vision: ipAdapter_image_encoder.sd15.safetensors. I have been looking everywhere for this image encoder but cannot find it. I can only find the clipvisionG or the ViT-H clip tensors. Any tips?
@arshartcloud3374 2 місяці тому
Amazing! so many useful tools in one video
@angelotsk3173 7 місяців тому
woow very nice, i wish you make more videos and tutorials .. Thanks thanks
@piratediffusion 6 місяців тому
Amazing work! Very inspirational
@psionicxxx 4 місяці тому
Marvelous, keep up the good work.
@SamBeera 5 місяців тому
Excellent, you are awesome, and thanks very much for the explanation and video.
@yashpatidar.8506 7 місяців тому
Great Job
@TheBestgoku 2 місяці тому
thank you soo much, clear and to the point.
@xorq 6 місяців тому
More videos like this please !!!
@juanmariaolaizola1858 Місяць тому
really really nice, many thanks!
@aleksanderk5695 6 місяців тому
Many thanks for Your work!
@pk.9436 6 місяців тому
Great Work please keep going!
@hakandurgut 7 місяців тому
Very good tutorial, thanks
@Jan-jf4th 5 місяців тому
Great video, thank you!
@rainerzufall1868 7 місяців тому
So useful! Thanks!!
@hamedsadeghizadeh6660 4 місяці тому
Nice work, thanks
@svenhinrichs4072 6 місяців тому
This is magic! Thank you very much...
@ac3d657 5 місяців тому
Thankyou, You do a much better job at explaining compared to a person paywalling content behind patreon. Thanks for your work on ipadapter its another indespensible tool and hopefully others will help and work toward further amazing improvements to the whole Stable Diffusion scene, and less paywalling content on youtube and such and more open source and less stagnation.
@latentvision 5 місяців тому
thanks for you message, I feel the same way. Mind you I don't think there's anything wrong in asking compensation for quality content, but since I developed an Open Source tool I find it's only fair that I also share the know-how on how to use it. I guess it's the only way we can actually evolve.
@ac3d657 5 місяців тому
@@latentvision thankyou for your response. Correct, people wouldn't be able to learn and only watching a couple of your videos and you have made understanding comfy in general a lot easier - support chains and donations are fine but when they hide what is essentially open and free infomation found in the same videos like yours is very concerning in this community and only invites stagnatioin in the ai/stablediff space.
@alecubudulecu 7 місяців тому
Very helpful. Is IPadapter similar to A4’s “reference” controlnet?
@falconbmstutorials6496 5 місяців тому
Your voice (I don't know if it is your own) sounds a lot like "my name is Giovanni Giorgio" 😊 Thanks so much for your very calm way of explaining and naturally also for your time & energy invested into the development of IPAdapter!!!
@MyWhyAI 5 місяців тому
Great tutorial! How were you able to get such high denoise on the upscale? Anything over .2 for me starts to change the look.
@igorkotov8937 Місяць тому
Thank you!
@haishengchi3333 5 місяців тому
this tutorial is best
@pschoro 7 місяців тому
COngrats !!!
@alamondefield2823 7 місяців тому
TY me and my ComfyUI loves you
@aerofrost1 6 місяців тому
How much does the model affect the outcome? Also, how do you resize the latent image in the img2img section? I'm on a 4GB GPU so I can't run high resolution images, it will take me ages. I need to resize the images that are latent. Also the latent image does not maintain the lines like yours, yours is almost like using controlnet canny, it keeps everything the same and just changes the style, mine doesn't. It changes the entire look of the image, pose, etc.
@bkdjart 3 місяці тому
Thanks for the tools and tutorials! One question though is where can I find the clip encoders to download? I am looking everywhere and I can't seem to find the IPAdapter SD15 clip encoder anywhere. Also once I download it where do I put it so it shows up correctly? Additionally is there a way to find out the underlying path of where each node is looking for?(It would make pathing so much easier when manually adding files)
@cocopops733 2 місяці тому
great stuff
@frazuppi4897 4 місяці тому
Ciao matteo, grazie
@rohitanshu 7 місяців тому
@latentvision Is there a way to know how the image is being described internally? I mean, can that text be extracted somehow?
@Pat-1000 4 місяці тому
Thank You
@jasemali1987 4 місяці тому
Ladies and gentlemen, we have found THE GUY ! Thank you so much for everything you are doing to the Ai community, this is great work and a great explanation. May I ask what is the limitation of the IP adaptor in terms of the fed images, like on what bases it determines the tokens? For example, if i used a line art image with a realistic checkpoint trained for producing mostly photographic images, will the IP adaptor get the tokens from the trained checkpoint or from the IP adaptor model ?
@latentvision 4 місяці тому
thanks for the kind words. IPAdapter is a very strong conditioning but the main checkpoint will always show its character. It's better to pick the right model for the image you wish to generate
@imtaha964 4 місяці тому
thank u man
@valter987 7 місяців тому
Man you make confyui makes so easy to use, thanks for the amazing video, I want to try it but confyui still scares me a bit
@latentvision 7 місяців тому
thanks! well maybe I'll make an intro video to comfyui next
@valter987 6 місяців тому
@@latentvision your video helped a lot, it manage it to work it perfectly, only one thing is not working for me is upscaling, it worked using esrgan but id like something like hirez fix in the A1111, thanks for everything
@valter987 6 місяців тому
@@latentvision Nvm, i was afraid to try confyui and now im amazed, its not easy but after you get used its not that harder either
@carstenli 7 місяців тому
Matteo, what is a good practice to emphasize the influence of a particular input image? - Use two 'IPAdapter Apply' nodes with different weights and merge them somehow before sending a mixed MODEL output to the KSampler? Merging the images via the Batch Images node sometimes favours the 'stronger' image.
@latentvision 7 місяців тому
if one image has A LOT more relevance you can use the prep node and toggle the "add weight" option. That image will have twice the strength. If you need it more fine tuned unfortunately at the moment you have to daisy chain IPAdapter nodes. I'm exploring the possibility to add image weights
@angelotsk3173 7 місяців тому
It will be a helpful add
@hilbrandbos 6 місяців тому
where do you get the clipvision models that you use?
@ChristianIce Місяць тому
T'ho sgamato Matteo, sei uno di noi :) A note. I installed Ipadapter just yesterday, just so you know I had to create an Ipadapter folder into the model folder. Placing the model in the custom nodes section didn't work for me.
@ysy69 6 місяців тому
Hi, thanks for making and sharing this extension. I am looking for the node you mention about Prepare Image for ClipVision. Would you mind telling us where to find? thanks
@ysy69 6 місяців тому
I found it :)
@rsunghun 7 місяців тому
this is great! I'm going to play with it. Thanks. By the way, the link in the description is broken :)
@latentvision 7 місяців тому
you are right, thanks for the headsup!
@jordanvanderwesthuizen4520 22 дні тому
Hey, great video! Really curious how you got your generation's to generate so fast! I run through google colab on a MacBook Pro M3 Pro 18GB RAM using a T4 GPU in Colab to run ComfyUI and my generations take up to 20 minutes. I've tried using a V100 GPU to speed things up and I feel like they take roughly the same time for me.
@latentvision 20 днів тому
that's a 4090!
@user-hg4hg5ix7f 5 місяців тому
bro sei un grande
@elowine 6 місяців тому
I'm having a hard time finding the clipvision model, did you use the 'h94/IP-Adapter' one on huggingface? Thx!
@latentvision 6 місяців тому
yes they are both in the h94 repository on huggingface. the "sdxl" one is good only for one model (the base sdxl). for all others you use the vit-h
@elowine 6 місяців тому
@@latentvision Thanks for developing this tool! So much easier to work with IPAdapter when its not just a commandline and a prompt
@AlexDiMella 2 місяці тому
Matt3o! Keep these vids coming! Excellent instructions. If I may ask here- After loading extensions, the "Load IPAdapter Model" has only "undefined" in its slot. No dropdown menu to choose anything. Any idea whats up? Everything else is working. Thanks!
@latentvision 2 місяці тому
put the models under ComfyUI/models/ipadapter
@AlexDiMella 2 місяці тому
Worked! Thank you!@@latentvision
@AlexDiMella 2 місяці тому
That worked, thank you!@@latentvision
@nodewizard 4 місяці тому
Truly amazing. I would give you a Noble Prize if I could. Thank you Matteo. For upscaling with IPAdapter, why not send it through a ControlNet Tile along with an Ultimate Upscaler? You could create an image in an SDXL checkpoint, then IMG2IMG with ControlNet tile in a SD1.5 checkpoint. The denoise of the tile model should be 0.3 or less to give the closest results to the image.
@latentvision 4 місяці тому
hey thanks the point of that segment is to show you the strength of the IPAdapter model. Of course you'll have to mix it with other nodes in a real life scenario. If you use tile controlnet already though you probably don't need to add IPAdapter
@NHCH 4 місяці тому
daje matteoo!
@latentvision 4 місяці тому
😅
@hoodooguru1450 7 місяців тому
Hi, nice work on the IPAdapter, what about the add_weight widget on the "Prepare Image for Clip Vision" Node? How much weight is added to the image if it is activated?
@latentvision 7 місяців тому
the image has twice as much weight, it's just a patch until I find a way to add fine grained weight
@hoodooguru1450 7 місяців тому
@@latentvision Thanks for the quick reply. That should come in handy.
@bandinopla 5 місяців тому
i get the error "RuntimeError: mat1 and mat2 shapes cannot be multiplied (77x2048 and 768x320)" at 20:49 when trying to tun the control net. Any hint on what it could be? (Im using a SDXL model)
@bandinopla 5 місяців тому
SOLVED: i was using an SDXL model so i downloaded a SD1.5 model instead and it worked.
@latentvision 5 місяців тому
you need to change all the models to SDXL (controlnet and IPAdapter)
@latent-broadcasting 5 місяців тому
Thanks so much! I applied this combined with Kohya DeepShrink and the results are amazing. Just a question, If I want to take the face using the face adaptor and I want it to combine it with an adapter that takes the style of the photo, what should I do? I can't find a workflow for that
@latentvision 5 місяців тому
you can daisy chain IPAdapters, check my attention masking video, you can see how it works there
@latent-broadcasting 5 місяців тому
@@latentvision Thanks for your answer! I'll check that video
Scott Detweiler
Переглядів 54 тис.
Апостроф TV
Переглядів 951 тис.
Abe aTech
Переглядів 8 тис.
Data Leveling
Переглядів 42 тис.
Andrea Baioni
Переглядів 7 тис.
Scott Detweiler
Переглядів 52 тис.