(For English, please scroll down.)
資訊社會的影像生產與觀看，已經使過往的類比與數位之爭顯得微不足道。在文章開頭的空拍機例子中，我們最多只能下下指令，實際上空拍機是自己在運作。這完全不同於膠卷的可觸碰性，空拍機背後是各種技術的相互運作，我們其實難以直觀的經驗這個過程。針對數位物的本體論，許煜提出一個很具啟發性的思考，「在數位編程的層面上，它們是文本文件；在操作系統中，它們是二進制代碼；最後，在電路板的水平上，它們不過是電壓值和邏輯閘操作所產生的信號。那難道我們要把電壓差異看作是數位對象的實質嗎？與其從物質的辯證關係出發，更應該把目光放在數位物作為一種管理數據的技術上。」演算法就是一種組織事物的程序，跟解決問題有關，也可以說是一種科學上的典範轉移。這個脈絡我們可追溯到十八世紀的殖民時代，當時許多航海歸來的科學家帶回世界各地的動植物，並且用自己的喜好為之命名，造成一物多名，或異物同名的亂象。直到科學家林奈發現花的雄蕊和雌蕊可以被作為植物分類的基礎，發表了以植物生殖器官進行分類的方法，在《植物種誌》（Species Plantarum）中提出了雙名法，後來也用同樣的方式為動物命名，此種命名法一直沿用至今。這種組織事物的思考更是近代計算機領域的核心，例如羅伯特・佛洛伊德（Robert W. Floyd）所發明的算法，是解決任意兩點間的最短路徑的一種算法，之後他更提出了「程序設計的範式」。這種編程包括了定義演算法以及資料結構，在計算機領域目前已發明了上千種不同的編程語言，而且每年仍有新的編程語言誕生。就這層意義上，我們就可以把十多年來的網際網路建設，想像成一種對世界進行既深且廣的數位化模擬，雲端平台就是一個巨大且不斷改變資料檔案的主體。
新加坡藝術家何子彥在《The Critical Dictionary of Southeast Asia》這件作品中，即是將資料庫與敘事結合成新的形式。自2016年開始，何子彥與程式設計師一起建構了一個線上平台，針對東南亞這個充滿爭議的概念，何子彥先以26個字母分別衍生出不同的詞彙，這些詞彙可能是觀念、主題或人物，接著他們又收集大量網路上匿名者的影音素材，並加以註釋，最後該平台會將這些詞彙與影音素材自動編輯成一個即時重組與生成的影片。東南亞這個概念一直在殖民歷史、國族政權與冷戰架構等各種力量的拉扯之中，既破碎又模糊。而何子彥透過演算法自動編輯資料庫素材，在素材與詞彙之間，恰恰能夠解放既定的意識形態框架。這就像是百科全書的編撰，透過各種方式列出條目，為的是要創造世界的意義，所以我們很難找到真正不含敘事的百科全書。今天演算法所產生的是一種關係邏輯，而非固定的型態，更是讓資料庫與敘事產生無止盡的混雜關係。
編程語言透過演算法把資料結構做出編輯，但我們卻無法直觀的經驗到這個過程，必須透過介面才能轉化成獨特的使用者經驗。例如我們在瀏覽網頁時，無法一次消化大量的資料，而是透過搜索引擎點取一個接一個的連結，也就產生了使用者認知與編程語言運作的差異。最外顯的例子就是電腦遊戲，雖然本質上是資料庫邏輯，但我們所能感受到的只是介面，介面提供了進入資料庫的通道。三位藝術家陸浩明、Alexis Mailles與Peter Nelson共同在第一人稱射擊遊戲（FPS）Counter Strike: Global Offensive伺服器上開發了一件作品《AUTOSAVE REDOUBT》，即是在進行這樣的探索。在太平洋戰爭初期，英軍在醉酒灣興建了一系列的防禦設施以抵抗日軍的攻擊，他們便以此防線上的地堡結構為原型，按照真實比例製作FPS地圖。他們做了檔案研究與實地考察，並且和香港大學建築系的測量師和歷史學家共同合作，利用遊戲程式設計出一個仿照歷史實景的地圖，透過這樣的方式，歷史文獻與地理資訊轉化成了使用者直觀的身體經驗。
另外一位德國藝術家Harun Farocki的作品《Serious Game》系列則是反向的對於遊戲與戰爭作出提問，這系列作品的素材來自於加州29棕櫚城的美軍訓練基地，以及一個幫助歸國軍人從創傷後壓力症候群中復原的治療機構。在第一個影片《沃森倒下了》中，我們可以看到士兵們用鍵盤與滑鼠進行一場虛擬的軍事訓練，教官則不時的在遊戲虛擬空間中添加爆炸裝置、敵方戰鬥人員等等，這是一個針對阿富汗地區的地理環境與戰爭元素所建構的資料庫。第二個影片《三名亡者》則是拍攝一個仿照阿富汗鄉村的實體訓練區，並且有三百名群眾演員讓全副武裝的士兵與真人接觸，但就Farocki自己的介紹，「演習給人的感覺卻像是現實世界正在模仿遊戲的世界。」原先編程語言把來自現實世界的資料結構轉化成可直觀的虛擬介面，弔詭的是，現在實體的訓練區居然再現的是這個虛擬介面。第三個影片《沈浸》則是拍攝治療機構如何使用頭戴式顯示器（Virtual reality）替退伍軍人進行治療，找回受到壓抑的破壞性記憶。這一系列的作品透過現實與虛擬的對質，使編程語言的問題浮現，編程語言所建構的世界，是否會反向的作用於現實世界？甚至影響軍人或大眾對於戰爭與衝突的理解。
在英國藝術家Ruth Jarman和Joe Gerhardt組成的團體「Semiconductor」的作品《形態如何生成？》當中，便提供了一種超越人類尺度的視角，展現了礦物、原子與演算法之間的相互關係。他們與礦物實驗室合作，將原子的世界觀進行數位化模擬，呈現了原子相互鍵結、群聚、構成晶格再成為晶體的動態過程，並以一段在馬里亞納海溝採集到的地震聲音訊號，作為電腦的參數去產生動畫。原本可能需耗費數萬年的礦物成形過程，透過數位化的編程語言，再現了超越人類尺度的時空觀。
Screen as the Docking Port
Taking out a drone weighed only 450 grams from the pocket, spreading the wings on the palm, and letting it start filming at a height of 50 meters and in a parameter of 7 km at a speed of 60 km/hr., everyone can manipulate one through an app on cellphone without certain training. Different from the conventional remotely controlled aircraft, this requires technologies in at least 4 different fields. The first is the Global Positioning System (GPS) designed by the US Department of Defense. There are 21 operational satellites and 3 spares deployed on 6 orbits with a 30-degree angular difference between satellites in each orbit, so as to ensure that 4 satellites are visible at any given point on Earth. At first, out of United States national security’s concern, an error value of 100 meters was added artificially for civilian use. After year 2,000, the jamming to civilian use has been removed, so the accuracy of positioning for civilian use is increased. As long as there are at least three satellites connected, the drone not only acquires a precise location horizontally, but the elevation vertically. The second technology is Inertial Measurement Unit (IMU). It is used for the measurement of speed, bearing, and acceleration of aircraft, which is further combined with a clustering algorithm based on bird flocking behavior to avoid collision. The third technology is compact, lightweight, hi-fi CMOS image sensor and lens that are waterproof and shock-resistant, able to travel through any terrains and environment. The camera physically replaces the body of photographer. The fourth technology is computer and wireless communication. Besides being remotely controlled, the drone can also be used for file transfer or even to upload a live feed in real-time. This is a fusion of multiple technologies. The remote control in our hands is not to control, but just to give precise orders. After all, the drone has always been controlling itself at the back of the remote control.
From the instance of drone, we can see the image production technology nowadays is in fact a convergence of a series of different fields. Apart from the new technologies employed in the process of filming, in what circumstance are these images today transmitted? If we comb through the previous image-watching mechanisms, we will see clearer the transformation within. Through a remote communication technology, TV started to allow the audience around the world could watch the same time object. It replaced the ancient flowing image technology of movies and become part of the daily life. If we adopts the division method of the communication field, “synchronous” and “asynchronous”, television and telephone are of synchronous mediums, whereas movies and answering machine are of asynchronous mediums with time lag in between the transmitter and receiver. As the mass media, television possesses the feature of transmitting content all at once. Nevertheless, the emerging Internet shatters this synchronicity of experience, replacing it with “semi-synchronous” effect instead. This effect is like the newspaper. Even though the information is delayed every day, it is as if we are receiving information in real-time.
On the Internet, we can browse at will, without the sense of loss as you turn off the TV when FIFA game has ended, for it is always hustle and bustle on social websites. Thanks to the all sorts of services of web 2.0 working behind, although each website operates on different servers, they maintain some cooperative relationship with each other, nonetheless. A website has functions like citation, RSS, and Ping. Various platforms such as Google or YouTube even provides embedded function beyond their own websites. Therefore, we can watch the first video on Facebook, and link to YouTube for the second, followed by the third video popping up automatically. As we visit a website, dozens of advertising companies will match information based on our hobbies, return rate, regions, and ages, in order to engage a bidding of advertisement. The advertisement that wins the bid will pop out on the website we are visiting. It takes only 0.1 second to complete the whole process. The flowing of men and information, the collaboration among Internet services, and the flowing of money altogether constitute a complete ecosystem. We walked from cinema to television, and get on the Internet as the mechanism of image watching has gone a long way.
The convergence of multiple technologies from the production to the watching of images today is a mutual influence among a huge number of mediators and participants that gives birth to a dynamic order as a whole. It is like a neural network with local interactions weaving out the entire cloud platform. It is not just a technological architecture, but a form of a system. It is like the controls the National Communications Commission endeavors to attain, centralized like the state institution does. It is like a decentralized free market as well, modulated by diverse platforms and agreements and blended with the various interfaces in daily life. We live in it, most of the time as users, not as a citizen or an economic man. An astonishing order is realized before anyone noticed it, whereas no one sees clearly the whole picture. Yet, the cloud platform can be good and evil at the same time. How an insignificant user distances himself/herself from this colossal monster, so as to identify some details to take action, is the primary problematic Offline Browser tackles.
Internet becomes an ecosystem, in which we have to contribute information in exchange for services on the global infrastructure, giving rise to a wide range of new forms of control. The numerous sources of image production nowadays lay out an environment with tighter and tighter surveillance. Under such circumstances, the German artist Hito Steyerl proposed an absurd mean to be invisible, presenting How Not To Be Seen: A Fucking Didactic Educational .Mov File in a sarcastic tone, such as transforming into a unit smaller than a pixel to submerge oneself in the world constructed with images. One of the shooting site in reality is a marked concrete in the middle of a dessert in California, USA, where the American Air Force used to calibrate their surveillance cameras. The place is now totally deserted. However, Hito in the work points out an even more unsettling outcome. The place is not abandoned because the surveillance has ceased. As a matter of fact, it is because a more advanced system has been launched without us being informed of what it is. In an age of online all the time and in the face of the new form of capitalism and nationalism, we have to try to engage with different technological systems to develop possible reactions. As biennale themed with creative practices of images, we pay especial attention to the action connotations of image artworks. Various screens could be seen in the previous video art exhibitions. On one hand, these screens are the vessels of images; on the other, underneath the background of information society, these screens are terminal devices as well. They are the docking ports for individual to the cloud platform in their daily lives, and the interfaces for these cloud platforms to be detected. On this layer of meaning, a screen no longer plays the role in the traditional video art context, but a springboard to actions, an element to the actions of the artists in this exhibition.
Encyclopedia-like Image Narrative
The production and watching of images in the information society has dwindled the competition of analogue vs. digital in the past. In the example cited at the beginning of the essay, all we can do is to give orders, while the drone takes the control of itself. Utterly different from the touchability of films, what lies beneath the drone is the interaction of diverse technologies. It is actually hard for us to experience this process intuitively. Regarding the ontology of digital objects, Yuk Hui proposed an inspirational thinking: “on the level of digital programming, they are texts; in an operation system, they are binary codes; lastly, on the level of circuit board, they are nothing more than the signals generated from voltage and logic gate operation. Hence, do we have to see the difference in voltage as some physical digital entity? Instead of departing from the dialectical relationship of matter, we should focus on digital objects as a technology that manages data.” Algorithm is the program to organize things, pertaining to problem-solving, which is also a paradigm shift in science in a matter of speaking. This context can be dated back to the colonial age in the 18th century, when many scientists returned from voyages with fauna and flora from all corners of the world, which were named to their likings. As a result, a chaos of one object with many names or different objects with same name was thus born, until Carl Linnaeus, the scientist who came up with the idea of using the stamens and pistils of flowers for the basis of taxonomy of plants. Thus, Carl Linnaeus published his taxonomy based on the reproductive organs of plants, and proposed binomial nomenclature (two-term naming system) in Species Plantarum, which were used in the naming of animals as well. The system of binomial nomenclature has been adopted ever since till this day. This thinking of organizing things is even more to the core of in the area of computer in modern times. For instance, the algorithm Robert W Floyd designed is to find the shortest path in between two given points. He proposed later on The Paradigms of Programming. This paradigm defines algorithm and data structure. In the field of computer, thousands of programming languages have been invented, and still many more emerge every single year. On this level of meaning, we can picture the Internet construction through dozens of years an extensive in-depth digital simulation of the world with cloud platform as a gigantic subject that constantly modifies data.
From the perspective of image creation, the relationship between database and narrative seemingly in contradiction to each other are in fact of coopetition. If we examine the working process of filmmakers, we may find the materials accumulated in the shooting period will form a database first. Besides, the shooting schedule is not always in accordance with the narrative sequence of the film, but with the logic of production. After the database is established, the editor will then develop the narrative path therein. Consequently, issues with database and narrative will emerge more or less in each film. Yet, the development of algorithm nowadays intensifies further the role of database in the process of production.
The artwork by Singaporean artist HO Tzu Nyen, The Critical Dictionary of Southeast Asia, is a novel form that combines database and narrative. Starting from 2016, HO Tzu Nyen constructed an online platform about the controversial concept of Southeast Asia together with programmers. Ho derives different vocabulary from 26 alphabets, which can be concepts, themes, or figures. Next, they collect huge quantity of audiovisual materials by the anonymous online, and add notations to them. Finally, the platform will automatically edit these vocabulary and audiovisual materials, assembling and generating video in real-time. The concept of Southeast Asia has been trapped in the struggle among the powers of colonial history, national regime, and the cold war structure, fragmented and blurred as such. Through the algorithm that automatically edits materials in the database, HO Tzu Nyen precisely liberates the existing ideological framework in between materials and vocabulary. It is like the compilation of an encyclopedia, listing entries through various means, so as to create the meaning of the world. Hence, it is hard for us to find an encyclopedia that truly contains no narrative. What the algorithm generates today is a relative logic, not a fixed form, which further propels database and narrative to generate a ceaseless hybrid relationship.
Programming language edit the data structure via algorithm. Yet, we are unable to experience this process intuitively, requiring instead an interface to transform it into unique user experience. For example, when we are browsing webpages, we fail to digest huge amount of data, so we click links via search engine one after another, from which comes the difference between the user perception and the programming language operation. The most evident instance lies in computer game. It is essentially the logic of database, but what we can experience is only the interface, and the interface provides the channel into the database. Three artists Andrew Luk, Alexis Mailles, and Peter Nelson, developed the work, AUTOSAVE RDOUBT, on the server of the First-Person-Shooter (FPS) game “Counter Strike: Global Offensive” to engage in such exploration. In the early period of the Pacific War, the British troops constructed a series of fortifications at Gin Drinkers Bay to defend from the attacks from the Japanese army. Taking the bunker structure on this line of defense as the prototype, they created a FPS map based on the ratios in reality. They conducted archival researches as well as field researches, and worked together with the surveyor from the Department of Architecture of the Hong Kong University and historians. Using the game program to design a map true to the historical site in reality, they transformed the historical documents and geographical information into intuitive bodily experience of users.
Whereas the artwork of German artist Harun Farocki, Serious Game, inquires from the opposite direction about game and war. The materials in this work series came from the 29 Palms Marine Corps Base of US Army as well as a medical facility that helps returned soldiers to recover from PTSD. In the first video, Watson is Down, we can see soldiers go through a virtual military training using keyboard and mouse while the instructors adding explosive devices and enemy combatants here and there. It is a database constructed with the geographical environment and combat elements in Afghanistan region. In the second video, Three Dead, a physical training area is set up based on an Afghanistan village, together with three hundred extras, for fully-armed soldiers to make contact with real people. Nevertheless, “the exercise feels like it is the real world mimicking the game world,” said in Farocki’s introduction. Originally, the programming language transform the data structure from the real world into an intuitive virtual interface. Yet, what makes it strange is that now the physical training area is representing this virtual interface. The third film, Immersion, filmed how a medical facility treats a veteran with virtual reality in order to recall the suppressed traumatic memory. A series of works resurfaces the issues with programming language via the confrontation of reality and virtual reality. Can the world fabricated with programming language reversely affects the real world, or even the understanding from soldiers or the public toward war and conflict?
Symbiosis with Non-human Users
On the cloud platform, we are nothing but users, an IPv6 address defined by a series of codes. The other users may not be persons. They can be machines as well. For instance, the aerial photography cloud platform developed over the two years allows us to connect to some drone at a tourist attraction remotely at home. The drone is another user. When it get close to a restricted airspace, it will communicate with yet another user automatically. The latter may be just a program in the restricted area. Therefore, a user can be just a program like all kinds of User Agents like a web crawler or a financial algorithm. At times, when we need to bypass a firewall in some restricted domain, we also need to match with a User Agent like a proxy located in different domain, and become yet another virtual User. Users can also be all sorts of sensors. These sensors are equipped with different senses. It may be sensing something physical like light, heat, magnet, etc. or detecting chemical reactions or something biological like enzyme, antibody, or hormone. Through these sensors, the true identity of User can be a machine or a program, or any physical or non-physical entity. That is to say, on the cloud platform, what we are facing are in fact a huge number of non-human Users. It could be microorganism invisible to the eyes, or it could be the traffic system of the entire city, the climate environment, or even a planet light years away. In addition, the platform will keep growing, allocating more things to the positions of User. It is a communication platform where things are interrelated, whereas we are merely supporting roles.
In Where Shapes Come From, an artwork by “Semiconductor” formed by Ruth Jarman and Joe Gerhardt, a perspective beyond the limit of humans is provided to demonstrate the relationship among mineral, atom, and algorithm. They worked with a mineral laboratory and create a digital simulation of the world of atom, showcasing the dynamic process from the bonding and clustering to forming the lattice and finally crystal. Also, the seismic sound signals collected from the Mariana Trench are taken as parameters in the computer to generate animations. The process of mineral formation that may take tens of thousands of years is represented in a space-time beyond Mankind’s scale via digital programming language.
The contemporary technology structure is forcing us to abdicate the leading role in the Anthropocene and become parts of the overall structure. What technology ushered in, be that nanometer-scale microcosm or the grand cosmic exploration, highlights rather the insignificance of each individual. Pondering over this issue, perhaps we can revisit the ancient origin. Taiwanese artist WU Sih-Chin created a work this year, My name? I have a lot of names. He dove into the myths of the indigenous peoples in Taiwan, particularly the legends about“Ta'ay” told among different tribes. Taking reference to the collected data, he created an augmented reality device, transforming legends into entities through programing language. Among different indigenous peoples, they have similar descriptions about Ta’ay. They were short in size and dark in complexion, liked to attack the indigenous people and prurient, and eventually got wiped out. In addition, Ta’ay in many myths are described as the owner of crop seeds, and humans were thieves stealing seeds from them. These technologies of alien culture exceeded human’s imagination. Thus, the thieves became heroes of the culture that taught people how to farm, serving as the communicators of the two world. Such a role can still be seen in the rituals practiced in tribes even to this day. WU Sih-Chin engages the second-stage extension in this exhibition, unfolding two communication modes at the same time. One is the language used to communicate with deities in rituals; the other is the contemporary programming language we see through exposing the production process of augmented reality. The two are both communication modes to tackle the world, while rituals ultimately become ports. Within just a decade or so, we have been thrown back from the Anthropocene. In the face of the ever-proliferating cloud platform as a whole, we are as ignorant as when we encountered the tribe of some alien realm for the first time. Apart from the bewilderment and disturbance toward the alien race, we need to conduct digital field researches, and find the mean to symbiosis in practices.