Working with Isadora

Isadora is a visual programming environment which is mainly used for prototyping interactive set-ups used for theatre, performance, visuals and interactive installations.

Installation instruction Isadora 4 USB Key Version with USB key license
Knowledge subjects

Video Effects general
Layering and Masking
Generative Visuals
Particles
Text
Logics
Projectionmapping
User Actors
Control panel
Serial OSC NDI
Midi
Sound
Motion Tracking
Working with 3D
Shaders

General Resources
Receive data from Arduino in Isadora
Send information from Isadora to Arduino

Installation instruction Isadora 4 USB Key Version with USB key license

To be able to save your work in Isadora, the HKU lending and the Blackboxes have USB keys. Below is an instruction:

Download the “Isadora 4 USB key Version” for Windows or MacOS at

1. https://troikatronix.com/get-it/ under the Downloads heading.

2. Under the heading “Related Downloads” click on “USB key Version” for Windows or MacOS. Another page appears.

3. Scroll to the bottom of the page and click on the correct USB key driver download link for your platform (Windows or MacOS). You will be redirected to the THALES website.

4. Click on the “Sentinel® LDK and Sentinel HASP® Run-time Environment Installer GUI” download link.

5. A pop-up screen appears with the “End User License Agreement”. Scroll down and click “I accept”

6. Install the Sentinel USB key for Windows or MacOS.

7. Install “Isadora 4 USB Key Version."

8. Insert the physical Isadora USB Key into the USB port and boot up Isadora.

9. In the Isadora menu bar, go to Help -> Register

10. Copy the code that you receive from a teacher or workshop employee or can find in the lending environment and paste it in the empty registration field.

11. You can now save Isadora patches with the USB key. Have fun!

Note!: If you use a USB (C) adapter, make sure that it stays in place properly. Connection to isadora and the dongle will be disconnected if the adapter moves or becomes loose. Then you cannot save your patch. So save your work regularly.

Knowledge subjects

In this chapter you will find pages diving deeper in a number of different subjects you will come across when learning and working with Isadora.

Knowledge subjects

Video Effects general

Tutorial

About Video Effects

Isadora has many already buit-in video effect with which you can experiment. Do not be afraid to play with the different parameters and experiment as much as possible. At the Isadora site in the "Addons" section you can find many moore effects. Also the forum of the Isadora site is a good source for tips&tricks.

Tips&Tricks

Play with the different parameters and combine different video effects for unexpected results.
Try to use effects in different sequences to create other effects
Every video effects actor has an "bypass" input that you can turn ON to not use the effect this way you cna better see what does what.
Make yourself familiar with what all the actors do to better get an overview of what is possible

Assignment

Make a tryptich using your own chosen theme. Use a single movieclip and create a composition using different video effects and movie players. Choose one clip form Pexels as inspiration.

Knowledge subjects

Layering and Masking

Tutorials

What is this?

Just like in image editing programs such as Photoshop you can work in Isadora with layering and blend modes. Depending on where the actors are on the scene they will be renderd in different layers. You can also set the layers manually. And in th eprojector you can set the blending options.

Practice Assignment

Create a composition of different movies using the layers and blending modes. Use the underwater world as theme.

Knowledge subjects

Generative Visuals

Tutorials

Generative Visuals

Whithing Isadora are already a lot of actors that can help you to generate visuals without using images or movies. Some of these actors are: Shapes actor, Lines actor, 3D ropes, Particles, Live drawing Actor. The generator actors also play an important role to generate numbers in diferent sequences and intervals.

Tips & Tricks

Do not underestimete the power en versatility of the shapes actor. It can take any shape or form
For the more advanced users the combination with the Javascript actor opens up another world of generative possibilities.
Do not forget to experiment with Video Effects. Motion blur can also often be your friend.
Using live sounds input with the sound frequency bands actor can help make sound reactive visuals

Practice Assignment

Make a small VJ patch that reacts to a favourite piece of music. Try to capture the atmosphere of the music in the visuals you create (disco, techno, lounge, country, blues).

Knowledge subjects

Particles

Tutorials

Tips & Tricks

There is a 3D particles actor and a 3D model particles actor. The first is for 2D particles and the second is for 3D particles
When working with particles you are working in 3D space so begin with adding the 3D stage orientation and set the "z translate" to -10 or something similar otherwise you will not see the particles because you are own top of the particles.
Do not forget to play with the particle count and location/velocity to really see them move.

Practice Assignment

Make it snow with an appropriate soundtrack. Bonus points for interactivity or special effects.

Knowledge subjects

Text

Tutorials

Using te Text/ure actor with multiple words

Kinectic type

Tips&Tricks:

There are many actors who do "something" with text. the "Text Draw" actor is the one to draw text on screen.
There is a trigger text actor you can use to triger different pieces of text after eachother
The "value to text" actor is very handy if you want to dispaly some values
With the Get/POST URL text you can use live sources form the internet.

Practice Assignment

Make a text animation based on the song Wonderwall. Don't forget the possibilities of the Tex/ure actor.

Knowledge subjects

Logics

Tutorials

Using Logics

When you start understanding Isadora you will get more ideas of what you can do and how you can use Isadora in a interactive dynamic way. Then it is important to get to know the different actors that can better help you to "program" Isdora to do what you want. A good start is tale a look at this list of actor for creating logics:

comperator
selector /router
javascript
toggle
trigger value
limit-scale
generators

Practice Assignment

Try to make a patch where the movie only plays when you move infront of the webcam. Try to make a patch where a movie or animation only moves when the mouse does NOT move.

Knowledge subjects

Projectionmapping

Tutorials

What is projectionmapping?

Projection Mapping uses everyday video projectors, but instead of projecting on a flat screen (e.g. to display a PowerPoint), light is mapped onto any surface, turning common objects of any 3D shape into interactive displays. More formally, projection mapping is "the display of an image on a non-flat or non-white surface"

Tips&Tricks:

Set the output resulotion in izzymap to the resolution of your projecter to see the limit of the projection
You can manipulate the Input and Output be aware of the difference. Input is the cropping of the source material, Output is how the input is projected.
in Izzymap you have differen layer optiuons with different shapes dependig on the object that you are mapping.
You can publishing mapping properties so you can control them outside of izzymap.
Once you have used izzymap the blend cna only be set inside of the izzymap window
Izzymap is so versatile you can easily create nice video effects within one projector

Practice Assignment

Choose a season as theme and:
Map a cube with a single movie
Map an object with a hole
Map a piece of cloth

Knowledge subjects

User Actors

Tutorials

What are User Actors?

User actors are important to bring order to your Isadora patch. They help organize your patch and think about your program in order of modules. You can compare it with function/object based programming. You can put user actors inside useract6ors but be careful.

Tips&Tricks:

Build up slowly and add inputs/outputs as needed
You can label your inputs/outputs to know hat they are
You can lable you useractor so you know what it does
Inputs/Outputs are Mutable and you san set the type
You can change the sequence of your userActor inputs/outputs
There is a difference between UserActors and Macro's

Practie Assignment

Step 1:
Create a small visual interesting looping animation
Step 2:
Put this animation into a user actor.
Think about which inputs/outputs you need with the user actor
Step 3:
Create a scene with a couple instances of the user actor.

Knowledge subjects

Control panel

Tutorials

What is Control Panel?

Isadora's Control Panel feature allows you to create a custom user interface, with buttons, sliders, knobs and more. Yhis enables you to build a patch with an interface that can be operatied by anyone.

Tips&Tricks:

click with right mouse button to choose actor/control split to work on it
toggle between "edit mode" to see how control panel will look
be aware thet in controlpanel view (or split view) you will only see the contropanel actors on the left
Double clik on control panel item to view properties
You can connect control panel items to inputs and outputs
double click on a input/output to set or remove a controlpanel link

Practice Assignment

Make a small VJ application with an interface using the controlpanel. You get Bonus points for design (thing backgroudn image, background colors and design of buttons). Make sure teh folowwing functionality is present:

The ability to choose a video (Bin Picker)
The use of two sources
A slider o corssfade between the two sources
abbility to add effects.

Knowledge subjects

Serial OSC NDI

Tutorials

Serial data / Arduino

OSC

NDI

Wat is Serial / OSC / NDI?

One of the many features of Isadora is that it can communicate using many protocols. Communication over the Serial port is handy for a wide range of devices (even DMX), bnut mostly it is used to communicate with Arduino or other micro controller boards. OSC (Open Sound Control) is a high bandwith foillouw protocol form MIDI. With this protocol you can send data between different devices and programs. this give you the possibility to use many different software together. (OSC background info, Wikipedia) NDI is a communication protocol for sending video over the network. Like OSC you can us eNDI to send a video stream form one computer to another computer or one program to another program (What is NDI?).

For serial/Arduino: 9 in 1 multi function expansion board

Tips & Tricks

For OSC and NDI make sure the computers are on the same network and whithin the same IP range. Be on the lookout for networks that have closed ports as a security measure. If possible create your onw network with fixed IP addresses.
For data intesive applications a wired network is always better and more stable then Wifi.
An OSC port can be opened only ONCE. So you can not listen on the sam port on the dsame computer with different programs!
When starting with Isadora an Arduino start with the most simple excersise! Check the Serial Monitor in the Arduino IDE first to see if the data is send correctly before figuring out how to recieve the data in Isadora.
A serial Port can be opened only ONCE! If the port is open In Arduino Serial Monitor) then it cannot be opened in Isadora

Practice Assignment

Create visuals using images/movies and other elements that you control with your phone. Use the theme: "Walking A Storm"

Knowledge subjects

Midi

Tutorials

What is MIDI?

MIDI Musical Instrument Digital Interface) is a technical standard that describes a communications protocol, digital interface, and electrical connectors that connect a wide variety of electronic musical instruments, computers, and related audio devices for playing, editing, and recording music. wikipedia

Tips&Tricks

Actors with the word "watcher" are used to watch or listen for hardware inputs. Use the Note On wathcer and Control Watcher to see which button does what on your midi device.
To set-up your midi-input/output go to "Communications"and then midi-setup
Each MIDI port can transmait or receive up to 16 channels each channel you can send various messages. The three most common ones are note off/on and the control change.
Each note has a pitch between 0-127 and a velocity between 0-127

Assignment

Create a drummahine where you cmbine sound and visuals. Choose which sample to paly with which button and make a small animation for each button/sample. For inspiration look here

Knowledge subjects

Sound

Tutorials

Tips & Tricks

You can use both WAV/AIFF or MP3, but MP3's will be played using the movieplayer which works different then the sound player
With the sound player you can play with speed to get a lot of interesting effects
With the MP# player you can play sound backwards
On macOS you have a lot of extra audio actors that are part of core Audio that you do not have on Windows
You can use the tone generator actor to generate a tone with different frequncies and create effects.
Do not forget to play with uysing sound as an input! Try the Sound Level Watcher actor, the Sound Frequency watcher actor and the Sound Frequency Bands actor.

Assignment

Make a simple story with sounds only. Hint, use timed triggers and audio speed. Make an audio sequencer? Make Horror story

Knowledge subjects

Motion Tracking

Tutorials

Motion Tracking

Using a (web) camera as live video input in Isadora gives you a lot of possiblities for interaction. You can make reactive visuals that "listen" to what happens in front of the camera in many different ways. It is quite a complex and often unstable topic, but very interesting.

Tips & Tricks

Light is very important! If you change you lightsettings you need to re-callibrate.
If you do not want to be dependend on natural light you can use an camera with a visible light filter and infrared lights.
Experiment and try! Motion tracking can be instable so embrace the chaos and keep trying.
You can use camera tracking as a motion detector or as a "is there someone present" sensor. Meaning you can make it so simple or complicated as you want.

Practic Assignment

Make viusuals that responds to a live video source.

Knowledge subjects

Working with 3D

Tutorials

3D guru session
[Isadora Guru Session #7: Generative Visuals with the 3D Ropes Actor](Isadora Guru Session #7: Generative Visuals with the 3D Ropes Actor)
Thoughts & Examples of using 3D in Isadora
[First person 3D navigation in Isadora](https://troikatroni x.com/add-ons/3d-game-style-navigation/) Background in 3d rotations

Working with 3D

In isadora you are not limited to working in 2D but can just as easily work in 3D. Either witn 3D particles or 3D models that can be displayed using the 3D player. You can also position 2D materials in 3D space using the 3D Projector actor.

Tips & Tricks

You can only use .3DS models in Isadora
The 3D stage orientation is an important actor to add to your scene when working with 3D, because with it you position the camera (when the camera and the 3D object are on the same spot you will not see the object)
Do not forget your light settings when working with 3D. They are set using the 3D light orientation actor.
You can also work with 2D material in the 3D world using the 3D proejctor actor or the 3D mode of the regular projector (click on the eye icon of the projector to enable the required inputs)

Practice Assignment

Either use the 3D model particles actor to create an 3D scene or download a 3D model and import this in Iadora.

Knowledge subjects

Shaders

Tutorials

What are Shaders?

Shaders are computer programs programmed in the OpenGL Shader Language (GLSL). GLSL is a programming language, similar in style to the C programing language, that tells your graphics card (GPU) how to manipulate or generate images. The power of GLSL shader programs is that, unlike programs that run in your computer's main processor (CPU) they run in parallel for every pixel of the image. This means they are incredibly fast, even with very high resolution images. As of version 2.3, Isadora offers an actor called GLSL Shader that allows you to compile and run GLSL code. The GLSL Shader actor allows you to integrate stunning computer-generated imagery as well as new video effects into your projects

Practice Assignment

Create a new Isadora file with a couple of images and videos. Then try-out some of the shaders that can be found here or here. Then create a visual composition inspried by the words: Gold, Dawn, Human, Minority

General Resources

Here you will find a general list links to different Isadora resources.

Knowledge

Isadora Main site (for general info)
Isadora Help (many knowledge articles and tutorials)
Isadora Forum (ask your question and find answers)
Isadora Add-ons (for useractors & plugins)
Isadora manual (good for reference)
Isadora Instagram (a couple of random tutorials)

Tutorials

Isadora Workshop/Resources of other people

Materials

Video codec information

Isadora performance tips
Isadora Knowledgebase article on HAP codec
Encode to HAP:

(Video files with an H.264 codec might freeze when played in Isadora. You can convert your file to an AVI with a HAP codec to solve this issue by following the instruction below)

1: download video convertion software: Shutterencoder

2: download the codec on this Github page " HapQuickTimeCodec.Mac.v12.pkg " or " HapQuickTimeCodec.Windows.v12.msi " depending on what system you work with.

3: Install the codec by opening the file you just downloaded. This is a plugin that works within your video software. After installing this you can find the HAP codec in the list of video codecs in your video convertion software (Like Shutterencoder)

4: Tutorial encode to HAP

Receive data from Arduino in Isadora

(These are notes, still need to be converted into a nice report)

Arduino code:

#define BUTTON_PIN 2

void setup()
{
 Serial.begin(9600);
 pinMode(BUTTON_PIN, INPUT_PULLUP);
}

void loop()
{
 Serial.println(digitalRead(BUTTON_PIN));
 delay(100);
}

source: https://roboticsbackend.com/arduino-push-button-tutorial/

Isadora settings

Communications:> Serial port setup
Port 1
Device: select -> usbmodem

Communications>
Enable serial ports

Use serial in watcher actor
open the actor by double klikking it and fill in this code : value:int=eol
in actor put eom char on 10

NB! If you have this on, close the serial monitor in Arduino,
If the serial monitor is on, Isadora cannot connect to the Arduino.
This also works the other way around. If you want to adjust your code in Arduino and you want to test the code, you must turn off the serial monitor in Isadora.

Open serial monitor in isadora:
windows > show monitor
Here you can see that the data is coming in from the Arduino. On the far right you can see that the value changes from 0 to 1 when you press the button. The serial monitor is for monitoring purposes only. You can turn it off when you see that the connection works.

In the Serial in watcher actor you see that the value changes from 0 to 1. You can use that output for your patch.

Send information from Isadora to Arduino

Think of it as if you were building the circuit in Arduino and you use the message from Isadora to Arduino to turn the circuit on or off.

Work files for Arduino & Isadora are added to this book, screenshots just for show ;)

Files not made by me, for more info contact me (Simone van Dordrecht) or Machiel Veltkamp <machiel.veltkamp@hku.nl>