MR project, Group 1: Häcka

Post by: Joris Weijdom Comments Off on MR project, Group 1: Häcka
Mixed Reality project
  • Iwein Reimerink (DVTG 3)
  • Joyce Lieshout (TVG 2)
  • Kimberly van Eijk (TVG 2)
  • Merel van der Lande (TVG 2)
  • Nadja Turlings (TVG 2)
  • Wilbert Schuurman Hess (DVTG 3)

Häcka is an experiment with video projection on translucent screens. The performance is designed to be a loop that could potentially run forever.

A queen is floating in abstract space absorbing energy of dead moths lying on the floor. When critically energized she gives birth to a new generation of moths. They fly around the space and eventually die and drop to the floor. Their life energy is floating upward and is being absorbed by the queen closing the perfect cycle of life and death.

When the the flock of new moths die and fall to the floor a large quantity of paper moths fall from the theatre ceiling in front of the audience in an attempt to give the moths a physical presence.

Because of the light situation the video camera’s have captured only a fraction of the actual piece. In the very first 2 minutes of the full video presentation of the Performance Lecture Mixed Reality an impression of the  actual performance can be seen. Or have a look at the photos below for better quality images.

Foto’s: Lucinda Bouten, Stefanie Bonte, Kimberly van Eijk,  Joris Weijdom.

Slide 6

•Iwein Reimerink DVTG 3
•Joyce Lieshout TVG 2
•Kimberly van Eijk TVG 2
•Merel van der Lande TVG 2
•Nadja Turlings TVG 2
•Wilbert Schuurman Hess DVTG 3

MR project, group 2: Tabula rasa

Post by: Joris Weijdom Comments Off on MR project, group 2: Tabula rasa
Mixed Reality project
  • Dirk Sonneveld (TVG 2)
  • Lisa Louwers (TVG 2)
  • Lucinda Bouten (TVG 2)
  • Ramir P. von Bannisseht (DVTG 3)
  • Rik de Rooij (DVTG 3)
  • Alex Giesche ATW (Giessen, Duitsland)
  • Maika Knoblich ATW (Giessen, Duitsland)

The second student project group has been researching the potential of live drawing with light on 3D objects. Although they had many ideas the end result focuses on the simple aesthetics of live drawing itself. The white character head has been designed with a 3D computer program, printed out on paper and folded into an analogue 3D head. A procedure briefly explained in video chapter three of the full video registration of the Performance Lecture Mixed Reality

Below you can see the chapter four video showing the Tabula rasa performance. In the second half of this video Marcel Alberts gives his reflection on the piece. Scroll further down to see the photos for a quick impression.

Foto’s: Lucinda Bouten, Stefanie Bonte, Kimberly van Eijk,  Joris Weijdom.

MR project, group 3: Return to sender

Post by: Joris Weijdom Comments Off on MR project, group 3: Return to sender
Mixed Reality project
  • Anneloes van Assem (TVG 2)
  • Hanneke Schriever (TVG 2)
  • Job Jutten (DVTG 3)
  • Rajiv Krijnen (DVTG 3)
  • Stefanie Bonte (TVG 2)

The third student project group got fascinated by a shadow having its own life. In order to project a shadow that could be animated and at the same time have a front projection for the performer and objects was quite a challenge. Also they managed to do a simple 3D projection map on the physical cube.

A girl appears on stage and sees a large box. When touching the box a strange thing happens; her shadow gets a life of her own. The girl tries to gain control over the shadow in a struggle with light and darkness. Eventually she destroys the box and restores order. However this is not quite the end…

Below you can see the chapter seven video from the the Performance Lecture Mixed Reality showing the Return to sender performance. In the second half of this video Marcel Alberts gives his reflection on the piece. Scroll further down to see the photos for a quick impression.

Foto’s: Lucinda Bouten, Stefanie Bonte, Kimberly van Eijk,  Joris Weijdom.

MR project, group 4: Schildeurij

Post by: Joris Weijdom Comments Off on MR project, group 4: Schildeurij
Mixed Reality project
  • Aeshta van Dam (DVTG 3)
  • Hugo van Dun (TVG 2)
  • Ina Herps (TVG 2)
  • Marije de Wit (DVTG 3)
  • Tamar Doedens (TVG 2)

This last student project group took the Droste effect, the endless repetition of an image in an image, as the starting point of their research. This ancient effect still has the power to mesmerize the audience. What will happen if somebody loseWeight Exercises their ball in a painting that is depicting the space where the painting hangs? Although the effect seems to be a simple live video feedback, suddenly the effect becomes much more complex, both technically and conceptually…

Below you can see the chapter twelve video from the the Performance Lecture Mixed Reality showing the Schildeurij performance. In the second half of this video Marcel Alberts gives his reflection on the piece. Scroll further down to see the photos for a quick impression.

Foto’s: Lucinda Bouten, Stefanie Bonte, Kimberly van Eijk,  Joris Weijdom.

Projection mapping project of

Post by: Joris Weijdom Comments Off on Projection mapping project of
News and agenda

02feb_11uur_joris01On the 28th of January the CBKU (centre for visual arts Utrecht) has opened its doors in a new renovated location in the centre of Utrecht. Cross media artist Maurice Bogaert has designed a whole concept around this opening.

Part of this concept is the installation of six scale models of the space given to six Art institutes in Utrecht. Each scale model can be seen at the six art institutes locations through a live webcam-feed. One of them has been placed in the CBKU space. On this particular model projects a sequence of several animations by several artist in such a way that the perspective of the projected images is precisely the same as the physical model.

Have a look at the six webcam-feeds here:
The model is the second feed from the top.
Or read more about the project at the website of

One of the animations has been made by joris Weijdom as part of his research on Mixed Reality applications. See the movie below.

The other Artists are:

  • Martin Boverhof
  • Machiel Veltkamp
  • Pyhai Jin
  • Jan Willem Deiman
  • Wim van Eck
  • Arnaud Loonstra

Here is the part made by Joris Weijdom. This is a HD movie. If your connection is slow please select a lower resolution (lowest = 360P) in the play-toolbar on the right side.

Added video Morfose project

Post by: Joris Weijdom Comments Off on Added video Morfose project
News and agenda

In june 2009 we did a project testing the Performance Engine v0.2 in a theatrical context. Have a look at the 5 minute video impression on the Morfose – projectpage.

Flyer Performance Lecture: Mixed Reality

Post by: Joris Weijdom Comments Off on Flyer Performance Lecture: Mixed Reality
News and agenda, PL Mixed Reality


17th of December 2009 – Akademie theater (Grote Zaal) – Janskerkhof 17/18, Utrecht – 19:00 till 21:15

At the 17th of december the research group will give a lecture with theatrical moments, a so called Performance Lecture, on the subject of Mixed Reality. In cooperation with the educations of Design for Virtual Theatre and Games (DVTG) and Theatre Disign ( TVG)

What is physical and virtual space? How can you combine, or mix, them in a theatrical context? What possibilities give new technologies?
These questions are subject in the lecture given by Joris Weijdom, head of the Research Group Virtual Theatre. The students have produced theatrical installations inspired by the subject of Mixed Reality and the above mentioned questions.

The lecture starts at 19:00 till 21:15. Entrance is free. Please make reservations through the HKU Agenda.

Download the digi-flyer!


A viola, dance and a machine

Post by: Joris Weijdom Comments Off on A viola, dance and a machine
LAB Falk Hubner

What remains of a performer, if he or she may may only give the impulses for auditive and visual processes of a performance, but when the actual performance is done by audiovisual electronics?

This is the central question Falk Hubner asked himself in the Shadows-project.

P1030270To try out the possibilities of live interactive recording and manipulation of sound and video Falk came to do a LAB. He would bring the viola player Örse Ádám and dancer Gabriela Tarcha with him as performers. In his briefing he formulated the following aim for the LAB:

At the end I want to know what I precisely want with the electronic system, what it should be able to to. At the end of the lab I want to take the patches home, so that I can continue working on them on my own. I want to know what the system will be doing.

Also he wrote a list of experiments he wanted to try out:

  • Delay for audio and video, and combining the electronic material with the performance-fragments of the performers
  • Processing and manipulation of audio and video – filters, saturation, modulation
  • Relationships and possibilities of manipulation between audio and video (video-signal controlling audio, vice versa)
  • Recorded (e.g. buffer~) vs. live-processing
  • Experimenting with projections on the screens standing in front of the performers, and how the live and projected pictures could merge (or not)
  • Experimenting with the microphones: What can they record besides music and voices (movement noises)?

P1030336At that stage the Performance Engine and especially the Sound Engine was not yet capable to do real-time recording and processing of sound. Also the 2D engine was not yet capable of using live-video input. So we had to negotiate what we could do in the short time we had in preparation to the LAB. We focused mostly on the Sound Engine and prioritized real-time recording and playback of audio as the main feature that needed to be developed.

P1030296The first day we tried out several features. Using the WII-mote, motion tracker and region patch we could let the dancer decide to record audio with the WII-mote that the Viola player was playing and playback the last sample by moving into several area’s on the floor.  At some stage additional pre-recorded audio fragments where added to the playback capabilities. These prerecorded tracks where manipulated by Falk in advance, thus compensating for the lack of real-time sound manipulation in the Audio Engine.

P1030425The dancer had now most of the control of what was being recorded and played back. To balance this out we gave the Viola player also a separate WII-mote. Now she was in control of what would be recorded and which sample would be played back when the dancer would move into the playback region. The dancer however could still ‘respond’ with her pre-recorded sounds. An interesting ‘dialogue’ of sound and movement was starting to unfold.

At the end of day one video was introduced. Some video was shot on-stage and some was recorded earlier. Video close-ups of both players where put at separate places on the back wall. Motion tracking was now used to fade in and out both faces in opposite direction. A third full wall video was put over both close-ups and connected to a marker of the Reactivision system. The dancer could manipulate both the playrate and fading of this third movie. Both performers needed now to move to control the video’s.




The second day Falk introduced a little patch he made in MaxMSP himself. This piece of software was counting down a random amount of seconds. When hitting zero it would display a number on either the left or right side. These numbers corresponded with actions the dancer or viola player needed to execute. This way they would be ‘controled’ by the machine. The patch was actually connected to the Statemachine, enabling it to give cues to the Sound Engine.

P1030518In the last part of the day all media where combined in an effort to make a small presentation workflow. A day later the experimental scenes where presented to an audience. While Falk was very happy with the opportunity to be able to test his ideas he also realized that his approach in the LAB didn’t deliver material he wanted to use for his final piece. The LAB helped him, through quick-prototyping, to realize that a new approach to his research and performance ideas was necessary. Which saved him a lot of time in developing technological systems that would in the end not result in the experience he was looking for. Both the viola player and dancer where very enthusiastic  about the LAB and the technological possibilities.

Photos Falk Hubner

Post by: Joris Weijdom Comments Off on Photos Falk Hubner
LAB Falk Hubner

Exploring a poem

Post by: Joris Weijdom Comments Off on Exploring a poem
LAB Nirav Christophe

P1030180Nirav Christophe wanted to interactively explore the Dutch poem ‘De Visser’ by Judith Herzberg with the dramaturgical guidance of Mart-Jan Zegers. Being a writer himself we felt that his performative possibilities should not include the usage of a keyboard or any other writing tools but rather involve his whole body and thinking process. To interact with the poem itself two systems where used; the text engine and the 3D engine. To be able to become witness of his thinking process Nirav had to talk out loud we decided to use the Sound engine for live audio-sampling. Motion tracking and a Wii-mote where used to enable Nirav to control the sampling process and playback.

P1030197The first day live sampling of spoken text and playback through interaction with the WII-mote was explored. The lines of the poem where loaded in the Text Engine and could be projected one by one through WII-mote interaction. Nirav would speak out loud his associations while browsing through the lines of the poem. Recording his voice and playing them back. After some free association sessions a selection of spoken texts where made and put in a playlist. Having added his own thoughts to the system it was time to improvise more on-stage with his body.

P1030193The motion tracker was used in combination with the Region Patch to create three active fields on the floor. Nirav could now with his WII-mote record live speech and place his live recording in one of these three regions. When he would move in that region the sample would play back. The pre-recorded playlist of thoughts where accessible through another set of buttons on the WII-mote. The whole system now enabled Nirav to trigger 2D Text lines form the original poem, play his pre-recorded spoken thoughts. Sample live spoken thoughts, place them in one of three regions and play them back by moving physically through them.

Altogether this configuration gave Nirav an immediate and intuitive spatial interface to compose his reactions to and with the poem. For a few hours he improvised and played freely to explore its possibilities and performative potential. We noticed that the Region patch was putting too much raw data into the system so its performance slowed down making the responsiveness of the motion tracking less immediate. It was visibly disrupting Nirav’s creative and performance flow.

P1030211At the end of day one we introduced the third system: a realtime 3D digital space in which the Poem was build as separate sentences. Each 2D text sentence is an image put on a plane that is floating in 3D digital space. The planes can be moved and rotated relatively to each other. A virtual camera is ‘seeing’ this space from a certain position and angle. Its perspective is projected within the physical space on a back wall. Nirav could manipulate the position and angle of the virtual camera with a keyboard lying on the floor. The 2D text sentences could be moved and rotated in digital 3D space by manipulating markers called Fideucials that we printed on cardboard and where scattered on the floor.

P1030223Nirav needed quite some time to be able to orient himself in the physical space in relation to the virtual camera in the digital space. Additionally the physical position and rotation of the cardboard markers in relation to the relative position and rotation of the 2D text planes in the digital 3D space was a lot to handle. We decided to at least connect the virtual camera also to the WII-mote so Nirav would not be constrained in his physical position. Also we made a physical logical structure for the markers for the poem to be in the right order.

In day two we had only four hours to play. Nirav now played with all systems at the same time. Triggering sentences from the Poem as projected 2D text, responding with either pre-recorded or live recorded spoken thoughts and reactions. Triggering this audio with the WII-mote or by moving through the regions. Meanwhile trying to also manipulate the position of the virtual camera in digital 3D space and positioning the cardboard markers to show the 2D sentences of the original poem in a certain order and distance. Through the lack of hands-on experience with digital 3D space and the mental intense task to relate this to the physical space and his position Nirav clearly got lost. At some point surrendering himself to the system shouting ‘I am a marker!’.


P1030226On the last day Nirav presented his finding to an audience. Although the tracking and region system was dramatically slowing down in performance the result of the two day experiment was considered a great success. Both Nirav and Mart-Jan gathered a lot of ideas potentially suitable for a performance. Lacking a deadline for an actual performance enabled both makers to freely experiment and explore possibilities.

WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in