Preview sampled images with TAESD

[space-nuko]

Lets KSampler show previews with taesd

Download the .pth models from https://github.com/madebyollin/taesd and stick them in models/taesd first

Previews are encoded to JPEG bytes first then sent over websockets, they're resized to 512 pixels first so the network load shouldn't be too much (~50kb per frame for 2048x2048 resolution)

Also adds taesd-related nodes, they're functionally equivalent to VAE loader/encoder/decoder nodes

output2.webm

[WASasquatch]

This is great. Definitely a great core feature. Though, why is the sampler preview version so much richer in color?

[ltdrdata]

This feature will be a powerful impression for users looking at ComfyUI.

[space-nuko]

Though, why is the sampler preview version so much richer in color?

Not terribly certain, it could be an artifact of the PIL conversion to JPEG. Here is another comparison using the WD VAE for decoding instead

[WASasquatch]

Though, why is the sampler preview version so much richer in color?

Not terribly certain, it could be an artifact of the PIL conversion to JPEG. Here is another comparison using the WD VAE for decoding instead

Is it normalizing to 0-255 before Image.fromarray()? I'd think if it wasn't it'd be more noticeable in inverted areas but maybe

[space-nuko]

Yeah it's clamped beforehand, I think some of the code I adapted from A1111 is suspect...

Some of it unscales latents to a range within [-2, 2] which are then scaled again by 0.5, if I scale by 0.25 instead it looks slightly better but has a more washed out look

[space-nuko]

Update: Now wraps taesd nodes in a new node to convert them to a LATENT_PREVIEWER, making the preview method plug-and-play

[comfyanonymous]

After testing this a bit I don't think having a previewer node connection is a good idea. Since previews are not something that affect the output image they should not be a part of the workflow itself.

I think to start previews should just be a command line option: --enable-previews method and we can add a way to control it from the interface afterwards.

[WASasquatch]

I think to start previews should just be a command line option: --enable-previews method and we can add a way to control it from the interface afterwards.

I agree with the first part, but not this. You may want a preview for something like a taxing HR image, but not something like a small 512x512 hand fix, or face fix. Simple preview boolean toggle would be fine.

[space-nuko]

Is there a use case for some nodes having previews and others not/having less expensive preview methods? Or being able to individually change settings for each node's preview?

Also this still doesn't help with the case of nodes that want custom preview methods besides taesd like Disco Diffusion

[WASasquatch]

Is there a use case for some nodes having previews and others not/having less expensive preview methods? Or being able to individually change settings for each node's preview?

I literally just gave one. You may not want to slow down a process via callback previews if it's not necessary, especially if you're not caring about the result as something you may want to cancel. And if isn't about whether or not the result is worth continuing, it's not really a useful function (in general) at all besides "ooh,look, the diffusion process"

[mflux]

I followed the instructions but still don't understand. Where's the TAESD Decode node from your screenshot?