Working with Isadora
Isadora is a visual programming environment which is mainly used for prototyping interactive set-ups used for theatre, performance, visuals and interactive installations.
- Installation instruction Isadora 4 USB Key Version with USB key license
- Knowledge subjects
- Video Effects general
- Layering and Masking
- Generative Visuals
- Particles
- Text
- Logics
- Projectionmapping
- User Actors
- Control panel
- Serial OSC NDI
- Midi
- Sound
- Motion Tracking
- Working with 3D
- Shaders
- General Resources
- Receive data from Arduino in Isadora
- Send information from Isadora to Arduino
Installation instruction Isadora 4 USB Key Version with USB key license
To be able to save your work in Isadora, the HKU lending and the Blackboxes have USB keys. Below is an instruction:
Download the “Isadora 4 USB key Version” for Windows or MacOS at
1. https://troikatronix.com/get-it/ under the Downloads heading.
2. Under the heading “Related Downloads” click on “USB key Version” for Windows or MacOS. Another page appears.
3. Scroll to the bottom of the page and click on the correct USB key driver download link for your platform (Windows or MacOS). You will be redirected to the THALES website.
4. Click on the “Sentinel® LDK and Sentinel HASP® Run-time Environment Installer GUI” download link.
5. A pop-up screen appears with the “End User License Agreement”. Scroll down and click “I accept”
6. Install the Sentinel USB key for Windows or MacOS.
7. Install “Isadora 4 USB Key Version."
8. Insert the physical Isadora USB Key into the USB port and boot up Isadora.
9. In the Isadora menu bar, go to Help -> Register
10. Copy the code that you receive from a teacher or workshop employee or can find in the lending environment and paste it in the empty registration field.
11. You can now save Isadora patches with the USB key. Have fun!
Note!: If you use a USB (C) adapter, make sure that it stays in place properly. Connection to isadora and the dongle will be disconnected if the adapter moves or becomes loose. Then you cannot save your patch. So save your work regularly.
Knowledge subjects
In this chapter you will find pages diving deeper in a number of different subjects you will come across when learning and working with Isadora.
Video Effects general
Tutorial
About Video Effects
Isadora has many already buit-in video effect with which you can experiment. Do not be afraid to play with the different parameters and experiment as much as possible. At the Isadora site in the "Addons" section you can find many moore effects. Also the forum of the Isadora site is a good source for tips&tricks.
Tips&Tricks
- Play with the different parameters and combine different video effects for unexpected results.
- Try to use effects in different sequences to create other effects
- Every video effects actor has an "bypass" input that you can turn ON to not use the effect this way you cna better see what does what.
- Make yourself familiar with what all the actors do to better get an overview of what is possible
Assignment
Make a tryptich using your own chosen theme. Use a single movieclip and create a composition using different video effects and movie players. Choose one clip form Pexels as inspiration.
Layering and Masking
Tutorials
What is this?
Just like in image editing programs such as Photoshop you can work in Isadora with layering and blend modes. Depending on where the actors are on the scene they will be renderd in different layers. You can also set the layers manually. And in th eprojector you can set the blending options.
Practice Assignment
Create a composition of different movies using the layers and blending modes. Use the underwater world as theme.
Generative Visuals
Tutorials
- Wave Generators
- Scaling Values
- Example fromm colleague
- Isadora Guru #2 Generative
- Isadora Guru #7 Generative with 3D ropes
- Isadora 1001 #9 Using th eGenerators Actors
Generative Visuals
Whithing Isadora are already a lot of actors that can help you to generate visuals without using images or movies. Some of these actors are: Shapes actor, Lines actor, 3D ropes, Particles, Live drawing Actor. The generator actors also play an important role to generate numbers in diferent sequences and intervals.
Tips & Tricks
- Do not underestimete the power en versatility of the shapes actor. It can take any shape or form
- For the more advanced users the combination with the Javascript actor opens up another world of generative possibilities.
- Do not forget to experiment with Video Effects. Motion blur can also often be your friend.
- Using live sounds input with the sound frequency bands actor can help make sound reactive visuals
Practice Assignment
Make a small VJ patch that reacts to a favourite piece of music. Try to capture the atmosphere of the music in the visuals you create (disco, techno, lounge, country, blues).
Particles
Tutorials
- examples of 3D Particles
- 3D particles tutorial
- Intro into Particle system
- Particles rand tex map example
- Free Particles patch
- Particles Example File
- Particle with words Example File
- Particle Example file Repel/Attach
Tips & Tricks
- There is a 3D particles actor and a 3D model particles actor. The first is for 2D particles and the second is for 3D particles
- When working with particles you are working in 3D space so begin with adding the 3D stage orientation and set the "z translate" to -10 or something similar otherwise you will not see the particles because you are own top of the particles.
- Do not forget to play with the particle count and location/velocity to really see them move.
Practice Assignment
Make it snow with an appropriate soundtrack. Bonus points for interactivity or special effects.
Text
Tutorials
Kinectic type
Tips&Tricks:
- There are many actors who do "something" with text. the "Text Draw" actor is the one to draw text on screen.
- There is a trigger text actor you can use to triger different pieces of text after eachother
- The "value to text" actor is very handy if you want to dispaly some values
- With the Get/POST URL text you can use live sources form the internet.
Practice Assignment
Make a text animation based on the song Wonderwall. Don't forget the possibilities of the Tex/ure actor.
Logics
Tutorials
Using Logics
When you start understanding Isadora you will get more ideas of what you can do and how you can use Isadora in a interactive dynamic way. Then it is important to get to know the different actors that can better help you to "program" Isdora to do what you want. A good start is tale a look at this list of actor for creating logics:
- comperator
- selector /router
- javascript
- toggle
- trigger value
- limit-scale
- generators
Practice Assignment
Try to make a patch where the movie only plays when you move infront of the webcam. Try to make a patch where a movie or animation only moves when the mouse does NOT move.
Projectionmapping
Tutorials
What is projectionmapping?
Projection Mapping uses everyday video projectors, but instead of projecting on a flat screen (e.g. to display a PowerPoint), light is mapped onto any surface, turning common objects of any 3D shape into interactive displays. More formally, projection mapping is "the display of an image on a non-flat or non-white surface"
Tips&Tricks:
- Set the output resulotion in izzymap to the resolution of your projecter to see the limit of the projection
- You can manipulate the Input and Output be aware of the difference. Input is the cropping of the source material, Output is how the input is projected.
- in Izzymap you have differen layer optiuons with different shapes dependig on the object that you are mapping.
- You can publishing mapping properties so you can control them outside of izzymap.
- Once you have used izzymap the blend cna only be set inside of the izzymap window
- Izzymap is so versatile you can easily create nice video effects within one projector
Practice Assignment
Choose a season as theme and:
Map a cube with a single movie
Map an object with a hole
Map a piece of cloth
User Actors
Tutorials
What are User Actors?
User actors are important to bring order to your Isadora patch. They help organize your patch and think about your program in order of modules. You can compare it with function/object based programming. You can put user actors inside useract6ors but be careful.
Tips&Tricks:
- Build up slowly and add inputs/outputs as needed
- You can label your inputs/outputs to know hat they are
- You can lable you useractor so you know what it does
- Inputs/Outputs are Mutable and you san set the type
- You can change the sequence of your userActor inputs/outputs
- There is a difference between UserActors and Macro's
Practie Assignment
Step 1:
Create a small visual interesting looping animation
Step 2:
Put this animation into a user actor.
Think about which inputs/outputs you need with the user actor
Step 3:
Create a scene with a couple instances of the user actor.
Control panel
Tutorials
- Isadora 010 session
- Examples control panels
- Control Panel Basics Pt 1
- Control Panel Basics Pt 2
- Control Panel VJ
What is Control Panel?
Isadora's Control Panel feature allows you to create a custom user interface, with buttons, sliders, knobs and more. Yhis enables you to build a patch with an interface that can be operatied by anyone.
Tips&Tricks:
- click with right mouse button to choose actor/control split to work on it
- toggle between "edit mode" to see how control panel will look
- be aware thet in controlpanel view (or split view) you will only see the contropanel actors on the left
- Double clik on control panel item to view properties
- You can connect control panel items to inputs and outputs
- double click on a input/output to set or remove a controlpanel link
Practice Assignment
Make a small VJ application with an interface using the controlpanel. You get Bonus points for design (thing backgroudn image, background colors and design of buttons). Make sure teh folowwing functionality is present:
- The ability to choose a video (Bin Picker)
- The use of two sources
- A slider o corssfade between the two sources
- abbility to add effects.
Serial OSC NDI
Tutorials
Serial data / Arduino
- Arduino Frimata
- Very old Wiring tutorial
- Github using BareConductive with Isadora
- Simple Arduino-ISadora tutorial
OSC
- Isadora Guru session #9 Interaction using OSC
- Using Isadora with touch OSC
- Isadora Gurusession #9 Interaction using OSC
- Using accelorometer data with Isadora
NDI
- The Isadora NDI addon
- Isadora Guru session #10 Faciliting remote Performances
- Small NDI example to Unreal
Wat is Serial / OSC / NDI?
One of the many features of Isadora is that it can communicate using many protocols. Communication over the Serial port is handy for a wide range of devices (even DMX), bnut mostly it is used to communicate with Arduino or other micro controller boards. OSC (Open Sound Control) is a high bandwith foillouw protocol form MIDI. With this protocol you can send data between different devices and programs. this give you the possibility to use many different software together. (OSC background info, Wikipedia) NDI is a communication protocol for sending video over the network. Like OSC you can us eNDI to send a video stream form one computer to another computer or one program to another program (What is NDI?).
For serial/Arduino: 9 in 1 multi function expansion board
Tips & Tricks
- For OSC and NDI make sure the computers are on the same network and whithin the same IP range. Be on the lookout for networks that have closed ports as a security measure. If possible create your onw network with fixed IP addresses.
- For data intesive applications a wired network is always better and more stable then Wifi.
- An OSC port can be opened only ONCE. So you can not listen on the sam port on the dsame computer with different programs!
- When starting with Isadora an Arduino start with the most simple excersise! Check the Serial Monitor in the Arduino IDE first to see if the data is send correctly before figuring out how to recieve the data in Isadora.
- A serial Port can be opened only ONCE! If the port is open In Arduino Serial Monitor) then it cannot be opened in Isadora
Practice Assignment
Create visuals using images/movies and other elements that you control with your phone. Use the theme: "Walking A Storm"
Midi
Tutorials
What is MIDI?
MIDI Musical Instrument Digital Interface) is a technical standard that describes a communications protocol, digital interface, and electrical connectors that connect a wide variety of electronic musical instruments, computers, and related audio devices for playing, editing, and recording music. wikipedia
Tips&Tricks
- Actors with the word "watcher" are used to watch or listen for hardware inputs. Use the Note On wathcer and Control Watcher to see which button does what on your midi device.
- To set-up your midi-input/output go to "Communications"and then midi-setup
- Each MIDI port can transmait or receive up to 16 channels each channel you can send various messages. The three most common ones are note off/on and the control change.
- Each note has a pitch between 0-127 and a velocity between 0-127
Assignment
Create a drummahine where you cmbine sound and visuals. Choose which sample to paly with which button and make a small animation for each button/sample. For inspiration look here
Sound
Tutorials
Tips & Tricks
- You can use both WAV/AIFF or MP3, but MP3's will be played using the movieplayer which works different then the sound player
- With the sound player you can play with speed to get a lot of interesting effects
- With the MP# player you can play sound backwards
- On macOS you have a lot of extra audio actors that are part of core Audio that you do not have on Windows
- You can use the tone generator actor to generate a tone with different frequncies and create effects.
- Do not forget to play with uysing sound as an input! Try the Sound Level Watcher actor, the Sound Frequency watcher actor and the Sound Frequency Bands actor.
Assignment
Make a simple story with sounds only. Hint, use timed triggers and audio speed. Make an audio sequencer? Make Horror story
Motion Tracking
Tutorials
- Camera Tracking Part 1A - Difference Movement pt 1
- Camera Tracking Part 1B - Difference Movement pt 2
- Camera Tracking Part 2 - EYES Difference
- Camera Tracking Part 3 - EYES Freeze
- Camera Tracking part 4A - Chroma Tracking pt1
- Camera Tracking part 4B - Chroma Tracking pt2
- Basic Motion tracking Tutorial
- Isadora Guru session #17 Motion Tracking with Eyes++
- Infrared tracking tutorial
- Isadora Camara Tracking examples
Motion Tracking
Using a (web) camera as live video input in Isadora gives you a lot of possiblities for interaction. You can make reactive visuals that "listen" to what happens in front of the camera in many different ways. It is quite a complex and often unstable topic, but very interesting.
Tips & Tricks
- Light is very important! If you change you lightsettings you need to re-callibrate.
- If you do not want to be dependend on natural light you can use an camera with a visible light filter and infrared lights.
- Experiment and try! Motion tracking can be instable so embrace the chaos and keep trying.
- You can use camera tracking as a motion detector or as a "is there someone present" sensor. Meaning you can make it so simple or complicated as you want.
Practic Assignment
Make viusuals that responds to a live video source.
Working with 3D
Tutorials
- 3D guru session
- [Isadora Guru Session #7: Generative Visuals with the 3D Ropes Actor](Isadora Guru Session #7: Generative Visuals with the 3D Ropes Actor)
- Thoughts & Examples of using 3D in Isadora
- [First person 3D navigation in Isadora](https://troikatroni x.com/add-ons/3d-game-style-navigation/) Background in 3d rotations
Working with 3D
In isadora you are not limited to working in 2D but can just as easily work in 3D. Either witn 3D particles or 3D models that can be displayed using the 3D player. You can also position 2D materials in 3D space using the 3D Projector actor.
Tips & Tricks
- You can only use .3DS models in Isadora
- The 3D stage orientation is an important actor to add to your scene when working with 3D, because with it you position the camera (when the camera and the 3D object are on the same spot you will not see the object)
- Do not forget your light settings when working with 3D. They are set using the 3D light orientation actor.
- You can also work with 2D material in the 3D world using the 3D proejctor actor or the 3D mode of the regular projector (click on the eye icon of the projector to enable the required inputs)
Practice Assignment
Either use the 3D model particles actor to create an 3D scene or download a 3D model and import this in Iadora.
Shaders
Tutorials
- Isadora Guru #11: Generative Visuals and Shaders
- Isadora made Shaders
- Shader tutorial
- Shader Guru tutorial
- Shader repo
- Forum post about shaders
- The book of shaders
What are Shaders?
Shaders are computer programs programmed in the OpenGL Shader Language (GLSL). GLSL is a programming language, similar in style to the C programing language, that tells your graphics card (GPU) how to manipulate or generate images. The power of GLSL shader programs is that, unlike programs that run in your computer's main processor (CPU) they run in parallel for every pixel of the image. This means they are incredibly fast, even with very high resolution images. As of version 2.3, Isadora offers an actor called GLSL Shader that allows you to compile and run GLSL code. The GLSL Shader actor allows you to integrate stunning computer-generated imagery as well as new video effects into your projects
Practice Assignment
Create a new Isadora file with a couple of images and videos. Then try-out some of the shaders that can be found here or here. Then create a visual composition inspried by the words: Gold, Dawn, Human, Minority
General Resources
Here you will find a general list links to different Isadora resources.
Knowledge
- Isadora Main site (for general info)
- Isadora Help (many knowledge articles and tutorials)
- Isadora Forum (ask your question and find answers)
- Isadora Add-ons (for useractors & plugins)
- Isadora manual (good for reference)
- Isadora Instagram (a couple of random tutorials)
Tutorials
Isadora Workshop/Resources of other people
Materials
- Isadora Workshop files on Github
- Isadora Example Patches on Github
- Materials for Github repo Examples
- Nextcloud folder with Material
- Pexels free video/iamge
- Free stock footage 1
- Free stock footage 2
- VJ clips
- ProjectionMapping File
Video codec information
(Video files with an H.264 codec might freeze when played in Isadora. You can convert your file to an AVI with a HAP codec to solve this issue by following the instruction below)
1: download video convertion software: Shutterencoder
2: download the codec on this Github page " HapQuickTimeCodec.Mac.v12.pkg " or " HapQuickTimeCodec.Windows.v12.msi " depending on what system you work with.
3: Install the codec by opening the file you just downloaded. This is a plugin that works within your video software. After installing this you can find the HAP codec in the list of video codecs in your video convertion software (Like Shutterencoder)
Receive data from Arduino in Isadora
(These are notes, still need to be converted into a nice report)
Arduino code:
source: https://roboticsbackend.com/arduino-push-button-tutorial/
Isadora settings
Communications:> Serial port setup
Port 1
Device: select -> usbmodem
Communications>
Enable serial ports
Use serial in watcher actor
open the actor by double klikking it and fill in this code : value:int=eol
in actor put eom char on 10
NB! If you have this on, close the serial monitor in Arduino,
If the serial monitor is on, Isadora cannot connect to the Arduino.
This also works the other way around. If you want to adjust your code in Arduino and you want to test the code, you must turn off the serial monitor in Isadora.
Open serial monitor in isadora:
windows > show monitor
Here you can see that the data is coming in from the Arduino. On the far right you can see that the value changes from 0 to 1 when you press the button. The serial monitor is for monitoring purposes only. You can turn it off when you see that the connection works.
In the Serial in watcher actor you see that the value changes from 0 to 1. You can use that output for your patch.
Send information from Isadora to Arduino
Think of it as if you were building the circuit in Arduino and you use the message from Isadora to Arduino to turn the circuit on or off.
Work files for Arduino & Isadora are added to this book, screenshots just for show ;)
Files not made by me, for more info contact me (Simone van Dordrecht) or Machiel Veltkamp <machiel.veltkamp@hku.nl>