The Data Science Studio and the Mozarteum University Salzburg demonstrated the potential of curated image-generating AI using methods from software development.
Harmless misunderstandings at best, and sexist or racist stereotypes at worst, still come with the use of image-generating AI. A satisfactory result usually requires several “prompts”, i.e. instructions. A project team from RSA FG Studio Data-Science, in close collaboration with the Mozarteum University Salzburg, has considered how this process can work more efficiently and sustainably. Their solution: “Curatorial Companionship”.
Domain experts evaluate the resulting artistic artifact step by step, the feedback flows into the designs and helps to improve the result. As part of the project, a paper was also published in the renowned ERCIM News. In it, the authors from the RSA and the Mozarteum criticize the need for high-quality digital artistic content. The Curatorial Companionship also deals with archiving and categorization, which is what digital curation is all about. According to the authors, this method can also help to supplement and create incomplete or alternative works of art and objects, and its use also strengthens cultural diversity.
Synthetic fiction from science fiction classics
Thomas Ballhausen from the Mozarteum and Stefan Gindl and Markus Tauber from the RSA FG provide an example of this in their publication “Curatorial Companionship: A New Framework for Managing High-quality Digital Cultural Content and Data”. For them, one way of using Curatorial Companionship and ArtScrum is so-called “synthetic fiction”. The science fiction series “Dune” by Frank Herbert achieved fame on the big screen through current film adaptations and one from the 1980s. However, the avant-garde director Alejandro Jodorowsky was already planning a film adaptation in the 1970s. With sketches and other information about the planned project, AI can now get to work and create a synthetic fiction.
Stefan Gindl und Thomas Ballhausen stellten das Projekt zusammen bei den Austrian Music Theater Days 2023 unter dem Motto „Fluid Spaces – New Music Theatre in the Era of Machine Learning” vor.
For the implementation, the team drew on the toolbox of software development. Scrum is an approach in which successive prototypes are rigorously tested and adapted before the final goal, the product, is achieved. This is in contrast to traditional, hierarchical approaches where the product is defined in detail at the beginning. As the creation of art is a non-linear process, the authors believe that the “ArtScrum” framework is ideal for the creation of art through generative AI. It combines the practices of curatorial guidance, from the vision of (data) collection, through preservation, creation (by generative AI) to the final selection of an artifact in a series of iterations, where the output of each previous stage becomes the starting point for the subsequent iteration. The final selection stage marks the end of the process, where the final digital artifact is the result.
In principle, the idea was developed for artists. However, the concept can also be applied to the creation of other, non-artistic artifacts. Examples of this would be the design of prototypes of sensor and IoT devices, or the imitation of sensor data.
Stefan Gindl and Thomas Ballhausen presented the project together at the Austrian Music Theater Days 2023 under the motto „Fluid Spaces – New Music Theatre in the Era of Machine Learning“.