Creativity in Science and Art

*Franc Solina*

### **Abstract**

This article compares creative work in science and the arts based on the author's own experience. In the field of science, the author works in the field of computer vision and is most interested in modelling 3D shapes from depth images. He started to collaborate with new-media artists almost three decades ago to produce interactive art installations that used also computer vision technology. Gradually, he developed also his own art installations. Ten years ago, he took up sculpting in wood and stone using the direct carving method. He has sought to enrich his sculptures in a virtual dimension by means of video projection. The scientist/artist describes how his experience in computer vision modelling 3D shapes has influenced his sculptures and compares how creativity is expressed in both fields. Although there is usually a wide gap between science and art, creative activity in both fields has surprisingly many common features.

**Keywords:** computer vision, sculpture, creativity, 3D models, superquadrics

### **1. Introduction**

Creativity is difficult to define, as the evolution of the concept of creativity throughout history since antiquity shows. The word creativity is derived from the Latin word *creare*, meaning to make or create [1]. The meaning of the word 'create' has increasingly shifted throughout history from the idea of simply doing something to the idea of creating, where it is important to first have a new idea and then try to bring that idea to life [2]. As to what the original source of creativity is, there are of course different interpretations. After all, the Christian tradition says that God created our world and that this is an expression of God's will. Today, creativity is usually interpreted as the ability to perceive the physical world or the world of ideas around you in completely new ways, to discover previously hidden patterns and to combine them into new solutions. For this reason, we speak of creativity primarily in the arts and sciences, for the work of artists and scientists is said to be characterised primarily by new works of art or new scientific knowledge.

Nevertheless, in the Renaissance, creativity in art and in science was closely related. Many people, not least Leonardo Da Vinci [3], combined scientific and artistic approaches in their work. However, the further development of science increasingly encouraged the specialisation and division of individual scientific disciplines, not to mention the disintegration of science and art. Various studies have reinforced this division, for example, studies of the differences between the right and left halves of the human brain [4]. It is thought that the right side of the brain is primarily responsible for emotions, intuition and thus creativity, while the left side supports more

analytical skills such as learning, memorising and processing information. Based on the individual differences between the right and left hemispheres of the brain, a person should therefore be more gifted in either the arts or sciences or in a more holistic or analytical view of the world. This dichotomy was put forward by C.P. Snow in his public discussion *The Two Cultures* [5] and has even been reflected in academic disputes between different fields (humanities or social sciences versus natural sciences or engineering) in the so-called Sokal affair [6] in which natural scientists accused social scientists of violating scientific principles.

However, contemporary creativity research, especially that of Mihaly Csikszentmihalyi [7], reveals common psychological processes that occur during creative activity. The psychological state of a person working creatively is characterised by optimal attention and involvement in the process, a state described by the word flow [7]. In order to enter a flow state, the individual can be supported by a suitable environment, music and discussion of the planned work and goal, which have been explored within art therapy [8, 9]. A suitable environment is important to the creative process so that the individual can remain in the creative process for as long as possible, not be distracted and be fully engaged and immersed in the creative process.

The principle of *mise-en-place* [10], which originated in the culinary arts, is often used to prepare a suitable working environment. For the preparation of a dish, which usually requires a process that must not be interrupted, it is desirable to have all the ingredients and utensils ready—on the table in front of you—before you start cooking. This principle, of course, applies in many other areas of work, not just the kitchen. We do not want to interrupt our concentration and focus on the creative process by looking for tools or components we need in the middle of the work.

The modern rapid development, especially of information technology or computer science, requires many developers and sophisticated users to have a thorough knowledge of the technological side, which in practice means at least knowledge of programming as well as creative use of this technology. In fact, the technology is evolving so fast that one cannot expect that the necessary knowledge to use this technology can be formed into specific tools that potential users can use without a deeper understanding of the technology. Therefore, only those who know how to develop the technology can understand how to use it creatively in other, sometimes entirely new ways. A typical example is computer game developers, the vast majority of whom are programmers by profession. For this reason, 20 years ago, interdisciplinary study programmes emerged around the world that combine both technological knowledge in a particular field and the creative use of that knowledge, often to create artistic products. This is the case, for example, in the field of new media art. One of the first degree programmes of this kind was the *Digital Media Design* [11] programme at the University of Pennsylvania in Philadelphia, which consists of about half computer science and engineering courses and half art courses. In Slovenia, the *Video and New Media* degree programme comes closest to this. At this study programme at the Academy of Fine Arts and Design in Ljubljana [12], I have a secondary teaching appointment.

### **2. My research experience**

I want to write about creativity in science and art from the perspective of my own experience. My main profession is computer scientist. After graduating from the University of Ljubljana in electrical engineering, I earned a Ph.D. in computer science [13] at the University of Pennsylvania in the United States. I did my Ph.D. in the GRASP Lab, where I specialised in computer interpretation of images or videos—a research area we call computer vision. This means that we use various computer methods to figure out what or what kind of objects are in an image, what shape they are, where they are in physical space, to try to determine their identity, recognise people, etc. Already during my Ph.D. studies, I was mainly concerned with threedimensional interpretation of image information and started to use a special kind of geometric models, namely superquadrics [14]. Superquadrics are a generalisation of Lamé curves in three dimensions. They were introduced into computer graphics by Barr [15] and into computer vision by Pentland [16] to model rectangular and curved shapes. In my Ph.D. thesis [17], I developed a method for their reconstruction from depth images.

One of the advantages of superquadrics is that we can use just one equation to describe a wide variety of basic geometric objects, e.g. spheres, cubes, cylinders, etc.:

$$
\left( \left( \frac{x}{a\_1} \right)^{\frac{2}{\epsilon\_2}} + \left( \frac{y}{a\_2} \right)^{\frac{2}{\epsilon\_2}} \right)^{\frac{2}{\epsilon\_1}} + \left( \frac{x}{a\_3} \right)^{\frac{2}{\epsilon\_1}} = \mathbf{1}. \tag{1}
$$

Eq. (1) is the implicit superquadric equation in object space. The size parameters ð Þ *a*1, *a*2, *a*<sup>3</sup> represent the size of the superquadric along the axes *xo*, *yo* and *zo* of the object centred coordinate system, while the shape parameters ð Þ *e*1,*e*<sup>2</sup> represent the roundness of the vertical and horizontal edges.

With superquadrics, we want to model the shape of an object in a kind of holistic and abstracted way, without the irrelevant details that might otherwise be important for identifying the object. **Figure 1** shows stone sarcophagi whose shape was first

#### **Figure 1.**

*Sarcophagi on the remains of a sunken Roman ship off the island of Brač in the Adriatic Sea, modelled with superquadrics [18]. The 3D point cloud was acquired using multi-image photogrammetry by a diver with a handheld camera. Modelling 3D point clouds with more compact, part-based geometric models is a current challenge not only in archaeology and heritage science, but also in robotics.*

captured underwater using multi-image photogrammetry, and the resulting 3D point cloud was then modelled using superquadrics [18]. With new and improved means of acquiring 3D data, 3D documentation of individual artefacts and entire environments is becoming increasingly important in heritage science. However, large 3D point clouds suitable for display and presentation must be segmented and modelled with appropriate geometric models to allow further analysis and understanding of the imaged scene.

My original method of superquadric recovery was based on iterative least squares minimisation of a fitting function that was too slow for a real-time application. Nevertheless, the method has been used in many very different applications, and my publications on superquadrics reached almost 2000 citations on Google Scholar. Interestingly, after a hiatus of almost 20 years when our lab stopped intensive work on superquadrics reconstruction and segmentation [14], we are working on it again as we try to speed up superquadrics reconstruction and segmentation by using deep neural networks [19].

### **3. My artistic experience**

I started teaching at the University of Ljubljana in 1988, and when the World Wide Web came along, I realised that it could be a good tool for presenting visual art, which I always liked very much. In 1995, together with my students, I built the Slovenian Virtual Gallery, a virtual space with paintings by famous Slovenian painters that could be explored by clicking in the direction of the desired movement or on the paintings themselves [20]. This first-generation virtual gallery was a success and received the highest rating—four stars in the Magellan Internet Guide, based on the depth, ease of exploration and Net appeal. At the same time, in 1995, my colleague Ken Goldberg of GRASP Lab, who was at the time at the University of Southern California, developed the influential art installation *Telegarden*, where users could instruct an industrial robot over the Internet to plant seeds and tend plants in a circular garden around the robot [21]. Goldberg's successful combination of a scientific and artistic career was a source of inspiration for me. Through the Slovenian Virtual Gallery project, I started collaborating with Srečo Dragan, the first Slovenian video artist and professor at the Academy of Fine Arts and Design in Ljubljana. Over the years, we have produced many interactive art installations involving both computer science students and art students [20, 22].

Under Dragan's influence, I soon began to create my own art installations. My most successful installation, inspired by Andy Warhol's portraits of famous people, was the interactive installation *15 seconds of fame* [23], which was first exhibited in 2002. The installation uses automatic face detection to create pop art portraits from randomly selected faces of gallery visitors standing in front of it, which are displayed on the computer monitor for 15 seconds. The installation was created before the era of selfies even began, but it has already satisfied people's need for self-discovery and self-assertion.

Ten years ago, in 2012, I started sculpting in stone and wood rather accidentally and out of a need to do more with my hands than just type and sit behind a computer screen. Perhaps my experience with 3D documentation of physical objects in the context of heritage science also had some influence on my desire to touch and feel real objects. After a few workshops under the guidance of academic sculptors Alenka Vidrgar and Dragica Čadež Lapajne, I began to work independently. My sculptural

#### *Creativity in Science and Art DOI: http://dx.doi.org/10.5772/intechopen.101955*

work so far was recently presented in my solo exhibition, which took place in DLUL Gallery in Ljubljana in autumn 2020 [24].

Like computer programming, sculpting requires concentration and thought, especially in direct carving, the technique I primarily use. While computer programming and research allow for easy experimentation and lots of trial and error, it's impossible to glue a stone back together once it's chipped. But my computer vision research has given me an experience that makes me see the objects around me mostly volumetrically—I can easily imagine how I would model them with superquadric blocks. Interestingly, superquadrics or superellipses were already used in furniture design and architecture by the Danish mathematician, designer, writer and poet Piet Hein [25]. Piet Hein designed a large public square *Sergels Torg* in Stockholm, Sweden, in the 1960s in the shape of a superellipse (**Figure 2**).

I make my sculptures from regular blocks of stone, but more often from irregular rocks or large pebbles. In my sculptures, I often look for abstract and pure geometric forms that remind me of superquadrics, like the sculptures in **Figures 3** and **4**.

In the block of Carrara marble in **Figure 3**, I have made a large and shallow indentation in the form of an ellipse. Inside the depression is a round hole that goes all the way through the block of marble. Both subtractions of material could be modelled as superquadrics. The title of the sculpture is *Big Eye*.

I found the rock for the *Taschenleerer/pocket emptier* in the Lesno Brdo quarry (**Figure 4**). I noticed the distinctly dark, almost black colour of the rock, because black limestone is not typical for the Lesno Brdo quarry. In short, when I have a rock in front of me, I try to find the abstract form hidden within it that would require the least amount of material removal. I can describe this process as a kind of discovery and

#### **Figure 2.**

*The superellipse-shaped fountain in the middle of Sergels Torg, Stockholm, Sweden [26]. Original photographer: Anders Bengtsson; CC BY-SA 2.0 via Wikimedia Commons [27].*

#### **Figure 3.**

*The sculpture titled big eye was created from the slab of Carrara marble on the top. The finished sculpture on the bottom has an elliptic indentation and a round hole, both examples of superquadrics.*

exploration of the possibilities that a particular piece of material offers. The original rock from which the *Taschenleerer* was made already had a depression in the centre (**Figure 4**). I highlighted this indentation in the sculpture and rounded the overall shape into a strikingly symmetrical form. The overall shape could be modelled with a superquadric (**Figure 4**).

I have also finished the stone differently. The outer surface is polished to bring out the texture of the stone, and the vertical side of the concave central part of the sculpture is chiselled with a tooth chisel.

I usually find a suitable name for my sculptures only after I have finished them, or during the work, when the final form has already taken shape in my mind. I was not

#### **Figure 4.**

*The sculpture entitled Taschenleerer was created from the rock seen on the top. The outer shell of Taschenleerer is in the shape of a superquadric (bottom).*

#### **Figure 5.**

*TOP: The virtually augmented sculptures sun and galaxy from the light fountain series. A Kinect depth sensor and video projector are mounted above each sculpture. The projected light dots move as water drops across the sculptural surface. BOTTOM: The virtual water drops merge on the Gallaxy sculpture to form a spiral gutter, eventually flowing into the hole in the Centre. The short lines of light are the result of the longer exposure of the moving points of light.*

familiar with the German word *Taschenleerer* before—pocket emptier in English, but there are also synonyms in Italian (svuotatasche) and French (vide-poche). I found the German word *Taschenleerer* interesting, a bit puzzling, but it was very descriptive for my sculpture, because the indentation in the middle can also be used to put away the small stuff we usually carry in our pockets. Lidija Golc, who wrote about my sculpture exhibition [28], with the help of Fran [2] defined the appropriate Slovenian term for T*aschenleerer*, which did not exist until then—i.e. žepni praznilnik.

I am trying to combine my knowledge of computer science with sculpture. I am investigating how a sculpture can be enriched with virtual content [29]. In the past, artists have often placed stone sculptures in a watery environment— either with standing water in which the sculpture was reflected or with flowing water in the form of various fountains that introduced a dynamic element. For the *Light Fountain* sculpture series—I have done two sculptures in this series so far, *Sun* and *Galaxy*—I used the Kinect depth sensor to capture the 3D shape of the sculpture. This 3D shape information can then be used to calculate how the raindrops falling on the sculpture should move. Since these are only virtual water drops, they are represented by points of light projected onto the sculpture via video projection. These points of light actually behave like water drops as they slide across the surface of the sculpture in the direction of maximum slope (**Figure 5**).

The installation is also interactive, as the Kinect sensor continuously captures the 3D shape. When someone touches the sculpture, the 3D shape changes and the light dots move across the new 3D configuration. Hand movements can easily trigger a 'splash' of the projected light dots. In the video [30, 31], one can observe the virtual dynamic enrichment of the sculptures. In the sculpture *Sun*, the drops falling evenly over the entire surface combine to form deepening rays of sunlight and then flow over the edge of the sculpture. In the sculpture *Galaxy*, the drops coalesce into a spiral and eventually flow off through the vortex in the centre of the spiral. This is an example of spatial augmented reality (SAR) achieved by video projection onto real physical objects. However, one must have appropriate 3D models of these objects to achieve this effect. Such augmented reality can be observed from any direction and by several people at the same time without any special equipment.
