Artists are going to be chaining networks together to do some interesting stuff…
My hope was to write some initial findings on my machine learning experiments this week. But, I generated so many thoughts and sketches (30 pages worth) that it became a mass (and mess) of information. In short, I am in a bit of a shock.
It’s hard for me to process the reality of what I am seeing as I study machine learning.
I am excited to parse some of the information, but first I felt it necessary to define my excitement about it. I see machine learning models as the opportunity to create the ultimate graphics hack.
Previs Hackers
Previsualization is a bit different from the rest of animation production. I have a previs friend who likes to say that we are “the first into combat”.
When a big budget movie fires up, a previs artist has to figure out what the hell the damn thing is. A good storyboard can fix story problems, but discussions about “how to build it,” happens during previsualization. These days, major budgetary and creative conversations are dependent on previsualization.
Therefore, since it’s throw away for problem solving, it’s ok to “cheat” in previs. When I mean cheat, I mean change the scale, use zoe-trope cards for effects, fake the camera rack focus with a gaussian blur in After Effects. Large production pipelines blow up, when they are dependent on badly scaled, messy assets. Perfection hates rampant creativity, so previs is often pushed to the sidelines.
One of my favorite things about being a previs aritst is creative “cheating” or non-standardized problem solving. The job really gets good when you stumble on these little hacks. I love to take a break with a fellow artist and go for coffee, or a beer, or a walk around the block. If you tell them what you are working on, the good ones will tell you how they would solve the problem.
“You know what you should do?” they might say in the Starbucks line, “I’d render the character on green, and then make an offset layer in comp for their position.”
“Dude, why worry about the ramp while you are blocking the action?” they say while deciding on what kind of beans in the Chipotle line “You should animate normally and time remap it.”
I call this “graphics hacking.”
It’s a mind set where it’s ok to cheat, to reuse things, to break them. Anything goes, to get the shot. And the graphics hacking conversations are the ones I live for.
Previs with Networks?
I’ve started to imagine the graphics hacking conversations that artists will be having a few years from now when they use the things I am now discovering. I think the conversations will be incredibly different.
“You know what you should try?” they might say while checking their bitcoin account on their phone “maybe train a network to classify and remove all the leaves.”
“You should try scraping the color sets from from that aerial photography set” they might say as they enter the automated uber “and then I’d use that new style GAN to make it look washed out.”
I am about half way through an online class on machine learning. I am about a quarter of the way through a book on the subject. This past weekend, I created first neural network hack. In a matter of minutes, I generated a cat on Runway ML!
If these networks can do what I think they can do, if we can get the data right, the ability to make unimaginable things a reality will be pretty insane. An art form that is based on smashing large portions of data together to yield a fake reality.
Again, I’m reeling. They will be the ultimate graphics hack.
I am falling down the rabbit hole. Anyone not taking this technology seriously is in for a shock. I hope to start aggregating my thoughts into work flows in future posts.
As always, please feel free to comment or reach out with thoughts. Thanks for reading, see you next week.
Reference:
I’m having an absolute blast learning from @genekogan: http://genekogan.com/
ITP@NYU / Machine Learning for Artists: https://ml4a.github.io/classes/
I‘m primarily in RunWay ML: https://runwayml.com/
Towards Data Science – Creating Art with GANs: https://towardsdatascience.com/gangogh-creating-art-with-gans-8d087d8f74a1
