A picture is a lie, and a very nice one, too. It is a time immemorial. That's the call that AI image-to-video calls. Tools such as Photo-to-Video.ai are making it possible to transform regular photographs into video clips with minimal (or no) video editing skills.
This technology is to make a short animation clip from a single still picture. No manual keyframes. No timeline scrubbing. No plugin which causes your software to crash every 3 saves. Just an image in and motion out — and there's something really weird and helpful going on in between.
The model doesn't follow the edges and move them around. It infers what is going on in the scene. Clouds should drift. Flames should flicker. Someone standing close to a fan likely is moving his or her hair. The AI bases its decisions on what it has observed in the real world and makes mistakes, but in a way that's not infrequent and can be startling.
Results vary. Wildly. No, it's not a complaint, it's the feel of working with a tool still in its infancy. A good composition of a portrait in a neutral background could bring out a beautiful animation, with the skin remaining intact and the scarf swirling in the wind. Put in a busy market and you could end up with a picture of 17 people each holding 4 arms. You soon find out what works and what doesn't. Clean inputs. Clear subjects. Strong contrast.
Motion prompting is the beginning of skill distinguishing good from accidental outputs. Giving the direction of the movement: "slow dolly forward", "flickering candlelight", "leaves falling diagonally" will give the generator direction. Without the prompt, that's an opportunity for an AI to be creative and take no risks.
This is being integrated into actual work processes by creative people. Portfolio shots are animated so that they do not scroll. Teachers add dimension to archival pictures that books can't. It was once said that a historian would feed a portrait a century old, and watch the changes in the expression of the face, only slightly perceptible and like the expression of a sleeping face. It was not about technology, but more of a retrieval, she said.