How to use local keyframe pictures to generate animation without using prompts?

[ismakegame]

keyframes:
{
"0": "video/1.png",
"10": "video/2.png"
}
prompts:
{
"0": ""
}
But it didn't work

[kabachuha]

For now, try using the 'frame interpolation' section in Video output. It allows inserting multiple pictures via drag and drop and then interpolating between them. The best model for this is 'FILM' which is selected and autodownloaded in the subtab.

If you're interested in regular Deforum animations (with 2d and 3d movement effects), adding CLIP interrogator to the Guided images mode may be worthwhile. Open a feature request in the 'Issues' here. @MatissesProjects will definitely be interested in it

[MatissesProjects]

Nice feature idea! Ill take a look into the CLIP interrogator today, I havent done anything with it yet.

[MatissesProjects]

@ismakegame does this look like what you wanted to achieve or is it something else?

[MatissesProjects]

it could be down to the distance of the keyframes + the tweening schedule if the above looked good to you

[ismakegame]

I want like this:
https://www.tiktok.com/@fefa.aiart2/video/7200873530743983365

or

https://www.tiktok.com/@capusoda/video/7200780477811166470

I guess they didn't do it via prompts.

[MatissesProjects]

interesting, that looks like its using the already existing RIFE model, and some large interp X

[kabachuha]

There is now the FILM model as well, and it's a bit more advanced

[MatissesProjects]

Just got another person asking about this @kabachuha do you think adding a schedule for something like an interpolation from their current prompts to the ones generated by clip would be interesting?

I see this like

prompt

{
"0":"prompt1",
"100":"prompt2"
}

images

{
"0":"image1",
"50":"image2",
"80":"image1"
}

with a schedule of 1 or no extra or w.e we decide this would be
new prompts

{
"0":"prompt, and clip_of(image1): .5",
"50":"clip_of(image2)",
"80":"clip_of(image1)"
"100":"prompt2"
}

[MatissesProjects]

maybe even allow for prompt1 to be appended or not as an option

[kabachuha]

clip_of may not say anything to a noob user, how about something like clip_descr?

Also, it would be nice to add the same thing for 'DeepDanbooru' which is already present in the WebUI just like the CLIP interrogator

[MatissesProjects]

yeah this makes sense, what I mean by clip_of( was that was injected and would be the resulting clip data.

I agree that we should do deepDanBooru as a toggle or optional setting