Dear all, due to the breaking changes in IPAdapter, some of the nodes used here are no longer available. I have updated the workflow to include the changes accordingly and also made some small changes to the workflow. Please download 'V2_ipadapter_face_clothes_controlnet.json' from the workflow link. Thank you!

So many good tips I couldn't keep up writing them all down ill be rewatching this for sure.

I'm glad I found this video, very well explained on the IPadapter workflow. Subscribed to the channel.. btw, love the Singaporean accent :)

I was stuck on how to use IPAdappter.. your video really helps me alot! keep up the good work. can't wait for your next drop

Just wanted to comment on how much I appreciate your step by step process and explanations through the video and your help in the comment section! It seems with a number of the other creators in the space they create walk throughs which assume you already know a lot of the necessary setup and steps that novices simply don't know. Keep up the great work!

Dear all, I have made a mistake in understanding the attention mask element in the IPAdapter node. I thought it was used to increase the face weight but it was actually used for the positioning of the final output in the image. You could omit the step of masking out the head of the model in the 'Image Preprocessing' step. As for the clothes, you should still keep it only if you want the final output region to be the same as that of the input image.

Came from AI Jason, glad I did! Thanks for the great explanations.

Thank you so much!! I have applied and it works perfectly!

just gonna say it here, you know you spend to much time in comfyui when you keep trying to click drag the screen up love your content this stuff is a massive help to get things working

This is so helpful! thank you so much for preparing!!

Thank You. Thank You. Thank You. Latent Vision brought me here.

Keep up the good work! Really nice videos! Would love to see a video about InstantID as well.

Perfect video :) Great pace!

Thank you so much, this is a great video tutorial

Thank you so much for leveling up my game

Amazing content! Keep it up 💪🏻

I created a little workspace save state add-on for comfy UI. You can find it in the comfy UI manager. I called it multi-workspaces because that's what the original intent was, but as it sort of carried on, eventually just became save states. Still working on a concept for multiple work spaces, the idea would be maybe to make them work together and save in the same JSON blob. You should go check it out.

Excellent video! Thank you

this guy is the G.O.A.T!

Thanks for this man . how to have exact clothing? also the face from the first checkpoint will completely change when changing to new check point in step 2

Little bit upset i spent all day installing everything needed just to have errors thrown at me "assertion error this and assertion that.." Also have scrolled through comments and realise that it won't replicate a face i have already created which is a bummer.. Great video dude, you've done well explaining everything and explaining it well. Very patient and precise tut. Hopefully i can get this mess sorted out but for now i'll have to stick with fooocus. Incredible job you've do for us. Thank you!

great video! thanks

thanks for the tutorial. 1 problem I encoutered was controlnet. IpAdaptor works fine to keep the face consistant. But once i connected controlnet, the face changed into a very different generic one. I tried to use the same workflow you uploaded. Does it have anything to do with the aspect ratio of the controlnet source image? or the controlnet model?

Hi, do i just import my own clothes image into your new ipv2 workflow and dont require to mask like you do in the video? and the settings and presets are already selected by you right? thanks !

Thanks for that! Nice and working workflow with easy explanation :) Do you know, is there a way how to use IPAdapter and create image with not a direct looking face? Something like "looking over her shoulder" or from sice or something like that. When I tried add it to the prompt it always put the direct face there, so it looks like the character has a broken neck completely :(

This tutorial is excellent! The instructions provided are crystal clear, and the thorough explanations render it effortlessly comprehensible. Your efforts are truly commendable! 👍

Han, thank you so much for your videos. They are very helpful and your workflow have help me immensely. I'm trying to see how to improve the upscaling for skin details at 4k and 8k. Could you help me?

I would love to see you add animatediff to your workflow too. Keep up good work !

hello i need help i cant find Prepare Image For InsightFace on confy interface

great video, great content, subscribed and liked! easy to follow along, thanks for the awesome content! one question: everything was working fine until i added the pose and now the face is hard to keep consistent. any tips?

Thanks for sharing this! Really interesting stuff. How do you keep the body shape consistent?

amazing, can you make a comfy UI guide? i love your voice and style of explanation. PLEASE!-like how to install, how to make flows, how to install plugins, best General settings,etc. PLEASE!

Hello I installed ComfyUI InsightFace but, it doesn't show '' Prepare Image for InsightFace'' in the options how to fix?

Can't seem to get the clothes portion working but seems like everything else is working perfectly

Thanks a lot, really helpful! Just a question... why to use a sd15 LoRA on a SDXL workflow? There is ip-adapter-faceid-plusv2_sdxl_lora.safetensors available... (instead using ip-adapter-faceid-plus_sd15_lora.safetensors)?

Hi Han, thanks for the video! I received the following message at IPAdapter FaceID, Error occurred when executing IPAdapterFaceID: expected scalar type Float but found Half File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) May I know how to resolve this?

Hello! After following the tutorials from your videos, i can't seem to find the ipadapter in the add node part. Should i paste the checkpoints, loras, ipadapter files on the different model folder you indicated on your past video (separate from the original models folder)?

hello there, first thank you for your work. In your video description you posted alink to 'vit_H' model at 'h94> IP-Adapter>models>image_encoder>model.safetensors - my question is, where to put and apply this model? the only models with 'vit_H' in your workflow I see 1 - ip-adapter-plus-face_sdxl_vit-h and 2 - CLIP-ViT-H-14-laion2B-s32B-b79K -> which both are different from it, so I am a little bit confused. Can you please help me figure out this mess?

I love this workflow, but I'm now trying it with a picture of myself, so I have bypassed the get face, and just uploaded a picture of myself into the "image preprocessing" area, also bypassed the clothing and control net, and can't get the result to look like me, it's not matching my face very well, at all. I know sometimes AI faces are much easier to duplicate, is that just the case here? or any hints otherwise?

Your tutorial is awesome - could you tell us what specs you are running this on? like cpu, gpu and ram? Thanks a lot!!

why do i get NO MODULE GOOGLE error on the ipadapter step. Can you help me?

I am getting "No module named insightface" error with the Load InsightFace node. Please help.

Hi all, (IMPORT FAILED) UltimateSDUpscale im facing this issue and tried , installing it again , reinstalling , tried fix . Its not installing still. do let me know how to fix this. thanks in advance

Perhaps training a diffusion model in the same way that Amazon did to change the entire outfit without altering the details could help with this process. Are you able to training a model like that just by reading the paper they made available?

Thanks. Is there any way to copy the facial expression to a specific photo?

This video is great! I'm having a problem though. I cant get a full body shot and there is always some kind of object blocking her at the base of the generation, sort of like shes hiding behind it. any thoughts? I'm using sdxl.

Thanks. how can I change the background to a photo background file of my choice instead of changing the background with a prompt?

It’s super cool thank you!! But im still facing an issue tho, i can't see " Load InsightFace " node from the insightface module of the " Apply IPadapter FaceID " and im running on python 3.10.11. If anyone got an update or solution i'd HIGHLY appreciate, thanks a lot!

can anyone help me I got this error : Sampler : have the same dtype, but got query.dtype: float key.dtype: struct c10::Half and value.dtype: struct c10::Half instead.

Hi Data Leveling. I work on colab, have fixed the insightface-issue, but i am struggeling on that: Error occurred when executing IPAdapterApplyFaceID: mat1 and mat2 shapes cannot be multiplied (257x1664 and 1280x2048). Ive changed the dimensions several times and also the checkpoint-models, but with no results. Do you have any suggestion on that? Thanks Bruno

This is Gold! I have a doubt, I have seen a lot of consistent face tutorials, but what about a consistent background? is it possible? like, the same background with the model in different poses.

how much of a image generation cost in terms of s/it or it/s should one expect to see from this? I am taking minutes to generate images with this flow when I can do "normal" generations in seconds.

Is gpu 16gb ram like 4080 enough for this kind of work or sdxl?

Thanks for the video. However, the results are far from desirable for commercial uses mainly because the generated cloth deviate too much from the original. I think a ControlNet Tile inpainting workflow might yield more stable results. But I could be wrong. Thanks again.

Hello, I really like the content, so I subscribed and am watching. However, the following error message appears in the "Sampling KSsmpler step" in the workflow. How can I resolve it? RuntimeError: Expected query, key, and value to have the same dtype, but got query.dtype: struct c10::Half key.dtype: float and value.dtype: float instead.

Can this be combined with your newest video load images from batch so , I have a variety of posses or a varity of outfits or a variety of faces or possibly I can do 2 or maybe more of above in one workflow

When loading the graph, the following node types were not found: InsightFaceLoader IPAdapterApply PrepImageForInsightFace IPAdapterApplyFaceID Nodes that have failed to load will show as red on the graph.

can you do a video about sparse control using IPadapter face ID pls. im trying that now but not sure why my vids generated are orange in color 😭

How well does this do with cartoons and anime?

hello sir, i am not able to find 'prep for insightface' menu in ipadapter menu of add node ? whereas i installed insightface from your previous video which you said to follow before starting this. PLZ HELP . 🙏

Which GPU are you using ? I have an RTX 2060 ( 6GB VRAM) and I'm getting an Out of Memory issue once I add the IpAdapter Clothes and I run it. Any ideas on how to solve this ? Thanks

How can we use the workflow to segment the face and the clothed body of another and combine the 2?

I cant find note name "prepare image for insightface"? why? i just see "prep image for clipvison"

Hi. When I try to import your workflow it just does nothing. Not to sure if the workflow is working as intended

Thanks for the video. Is it possible to put the exact same dress and not "redream" it?

Hey man, i run to this error when i try to use ip adapter with sdxl ( Error occurred when executing IPAdapterApply: Error(s) in loading state_dict for ImageProjModel: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([5120, 1024]) ) i have tried with both sd1.5 and sdxl encoder, same error. Love your videos btw

Error occurred when executing KSampler: mat1 and mat2 shapes cannot be multiplied (462x2048 and 768x320)~~~ I ran exactly the same as you, and got this message from the sampling - ksampler side.

Hm this looks really useful... i downloaded the json file from github, dragdrop it into comfyui and nothing happens? What am I doing wrong?

6:23 prepare image for insight face doesnt appear for me

i downlaod your workflow for example and it not working, You know what i'm doing wrong? I can't open it in window comfyUI SD 1.5 A1111

Hard to tell from your comfyi UI video, but IPAdapter based face transfer looks a lot more like a head cut/paste job in photoshop (wrong size head, lighting not transfer correctly) than it does an actual photo, or even a generative image?

can this be done in a1111

I am so confused. When you cover something with the mask I was under the impression that means you do NOT want that to be used. Example for your jacket. If you masked the jacket in black that means you do not want the jacket. (I am asking). That lost me. To make it worse I was doing the opposite and it was working.

As someone who works daily with AI and SD image generation processes like you, would you say this method is like the current state of the art in creating consistent AI characters, while controlling their pose and clothing? Is there also a method to get a consistent location/ environment for the character? Thanks btw, this is the best comfyui tutorial video I have watched for the topic addressed, keep up the good work!

r u from sg brother? 😂

INFO: InsightFace detection resolution lowered to (512, 512). ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 636, in apply_ipadapter self.ipadapter = IPAdapter( ^^^^^^^^^^ File "F:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 272, in __init__ self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "F:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}: \t{}'.format( RuntimeError: Error(s) in loading state_dict for ProjModelFaceIdPlus: size mismatch for proj.2.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([5120, 1024]). size mismatch for proj.2.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([5120]). size mismatch for norm.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for norm.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.proj_in.weight: copying a param with shape torch.Size([2048, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280]). size mismatch for perceiver_resampler.proj_in.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.proj_out.weight: copying a param with shape torch.Size([2048, 2048]) from checkpoint, the shape in current model is torch.Size([1280, 1280]). size mismatch for perceiver_resampler.proj_out.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.norm_out.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.norm_out.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.0.norm1.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.0.norm1.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.0.norm2.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.0.norm2.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.0.to_q.weight: copying a param with shape torch.Size([2048, 2048]) from checkpoint, the shape in current model is torch.Size([1280, 1280]). size mismatch for perceiver_resampler.layers.0.0.to_kv.weight: copying a param with shape torch.Size([4096, 2048]) from checkpoint, the shape in current model is torch.Size([2560, 1280]). size mismatch for perceiver_resampler.layers.0.0.to_out.weight: copying a param with shape torch.Size([2048, 2048]) from checkpoint, the shape in current model is torch.Size([1280, 1280]). size mismatch for perceiver_resampler.layers.0.1.0.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.1.0.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.layers.0.1.1.weight: copying a param with shape torch.Size([8192, 2048]) from checkpoint, the shape in current model is torch.Size([5120, 1280]). size mismatch for perceiver_resampler.layers.0.1.3.weight: copying a param with shape torch.Size([2048, 8192]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). Hello, the above error occurred. I changed the Load lPAdapter Model to ip-adapter-faceid-plusv2_sd15.bin. It worked. What is ip-adapter-faceid-plusv2_sdxl.bin Cannot run？

can anyone please help me with this error in comfyui Error occurred when executing InsightFaceLoader: module 'cv2.gapi.wip.draw' has no attribute 'Text' i tried reinstalling opencv-python and opencv-contribe but still get same error

Hi there! I`m trying to follow your guide here but I can't find anywhere the following nodes: - InsightFaceLoader - IPAdapterApply - PrepImageForInsightFace - IPAdapterApplyFaceID What am I missing?

So I did everything to install, however I get this error message, even though "pip list" shows that InsightFace is installed. Error occurred when executing InsightFaceLoader: No module named 'insightface' File "C:\dev\StableDiffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\dev\StableDiffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\dev\StableDiffusion\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\dev\StableDiffusion\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 627, in load_insight_face raise Exception(e)

an error:Error occurred when executing IPAdapterApplyFaceID: Error(s) in loading state_dict for ProjModelFaceIdPlus: size mismatch for proj.2.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([5120, 1024]). size mismatch for proj.2.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([5120]). size mismatch for norm.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for norm.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1280]). size mismatch for perceiver_resampler.proj_in.weight: copying a param with shape torch.Size([2048, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1280]).

Error occurred when executing IPAdapterApply: InsightFace must be provided for FaceID models. File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 698, in apply_ipadapter raise Exception('InsightFace must be provided for FaceID models.')