How Creating Art with Machine Learning / AI Feels in 2022
By trade I’m a software developer with decades of experience.
I’ve also spent several years of my life painting to pay the bills.
I’m a mixed breed. A bit of a mutt like most of us. :)
I’ve been toying around with machine based art generation for a few years now. Check out my Instagram for reference.
I don’t claim genius or to be an expert. I have some experience and history that gives me a unique perspective to share.
Tools of the Trade
Artists for centuries have explored various mediums. As they explore, new tools are blended with the old.
Likely, it all started with somebody making a mark with a stick in the mud and finding it pleasing.
Perhaps the stick was an innovation. A tool. A cheat. I bet there were even critics who considered the use of sticks somehow blasphemous and improper.
Today the stick and mud has evolved into paintbrushes and pigments and canvas. However the old tool of drawing by hand has remained.
Even today, there are still people who enjoy, and have refined the art of creating art with sticks and mud. Somewhere, some child is drawing a picture on the beach with a stick this very moment. Old technologies never die, they are merely built upon.
What is Art?
As long as there has been language there has been the arguing over what words mean and how to use them.
Arguing over word meaning is a delightful sport enjoyed by both kings and commoners.
What is a Ukrainian? What is a man? Hot topics indeed in my own time.
My goal here isn’t to offend or plea. However, I do have an idea of what art is, and I’m not afraid to use it! :)
To begin I’ll use the tool of dichotomy and split fine (useless) art from base (useful) art.
I consider myself a bit of a code-artist, and I do consider code a form of art. However, this code-art stuff it is useful, and so I find it helpful to distinguish it from ‘fine art’.
I also consider myself a bit of a fine artist. My instagram is full of useless images that some enjoy viewing enough to ‘give a like’.
In the past I sold hundreds of paintings and lived as a full-time artist. I tend to switch back and forth. The code I admit pays significantly better money, but the art is in many ways its own reward. These days I engage in ‘fine art’ for the pleasure it brings me. Coding, my other love, pays the bills. I am a lucky man to have two loves.
What is Good Art?
I’m going to say a true thing that nobody wants you to hear: Morality and meaning are 100% subjective. The world outside your head is your own dream and creation. Pray don’t tell the children lest they start to disobey.
Personally I find fine art to be ‘good’ when it speaks to me. Great art communicates something to me that I feel deep within my soul. It ‘resonates’, perhaps even with something that I have no name for but is within me all the same.
I note that a human hand is not needed for such art to move me but it helps.
If you’ve never visited I highly recommend going to the Grand Canyon in the United States. If you don’t have access to that then perhaps a sunset over an ocean. To me this is fine art created entirely by nature.
I can have a similar experience by observing a painting. The images it depicts can be of any sort, many raw nature would be incapable of producing. Or perhaps it is better said that nature uses our hands to create such works. Nature created us, so in the end it is all natural creation via various means.
Bad art, in contrast, is art that I find boring. It doesn’t grab me, speak to me, or engage with me.
Playing with Fire
For past several years I’ve been playing round in the digital art space. In particular I’ve created my own ‘art engine’ for generating images similar to models like Dall-E or Stable Diffusion, but instead of using neural networks, I’ve been coding ‘by hand’.
My results with my hand-coded ‘art engine’ have been nowhere near as impressive as the ML models, but the effort has given me insight into how one works with ‘generative art’.
The process I’ve adopted is to dance and iterate with the generated results.
The machine generates, the human adjusts the knobs. Much like how when one is painting the brush itself often does much of the ‘work’, but the human guides the brush.
The ‘machine brush’ behaves in a similar fashion, but of course is MUCH more flexible and powerful.
There are two main ‘knobs’ for ML models that I’ve discovered:
- Prompts
- Domain specific Constraints
In my ‘art engine’ prompts are more like playing with raw assembly, where working with ML models feel much more like working in a ‘high level language’.
IMHO using natural language as the ‘interface’ for ML models is what gives them their power from a human-usage perspective.
One can of course provide other prompts in terms of example images, and masks for in/out painting. I consider these more constraints than prompts.
Prompts are more playful and dynamic. Constraints are more like guards and limitations. Both are needed but play different roles.
Unlocking the Joy of Creation
For me the main takeaway when dealing with ML in an artistic setting is the effortless ease at which one can explore and refine.
It is like working with a powerful abstraction. I no longer think in terms of a concrete single work I now work at the level of the series of works I can produce and explore. A stream of art rather than a single item.
I find myself spending hours playing with this new tool and being amazed by the results. I highly recommend giving it a go. A bit difficult and ‘techy’ at the moment, but I have faith the costs and capabilities will only improve with time. It is just so damn fun. :)
What is in a Name?
I’m presently not aware of a name yet for this new form of art that I like.
‘Generative art’ is close, but I have a feeling better names will arise that more closely resemble the nature of the human/machine partnership. Perhaps ‘Artificial Art’ or ‘Imaginary Art’ will be the name. Much like ‘imaginary numbers’ were so derided upon their discovery, that a rather derisive name was chosen which stuck.
‘Artificial Art’ makes me smile, I think I’ll use that. An inside joke on those who will no doubt be outraged by the innovation without understanding.
Summary and Prediction
No tool is perfect, but using ML models for image generation is so powerful and so intuitive that it is hard to imagine it won’t come to overshadow many of the others. It won’t supplant them, but it will become popular. Much as ‘digital painting’ became popular vs ‘traditional painting’.
There is joy to be had in producing and consuming art. These new ML tools will find their place quickly in the toolbox of many ‘amateur’ and ‘professional’ artists alike. We are in very early days of this ‘revolution’ (and I think revolution is not too strong a term).
ML won’t ‘kill’ the more traditional arts, though I’m sure many will claim it will. However I think it will lead us to a deeper understanding of those arts, and perhaps a deeper understanding of what art itself is.
Originally published at https://github.com.
