Mixed reality looks at the spectrum between reality and virtual reality:

Reality —>

Augmented Reality —>

Augmented Virtuality —>

Virtual Reality

Reality is the world we live in. Augmented is that reality extended and added to with virtual components overlaid or added to it. Augmented virtuality is a virtual reality with real world objects superimposed, with virtual reality often encompassing this. However virtual reality can simulate things beyond real world existence which would no longer be augmented virtuality.

Immersion is often seen as virtual. However you can simulate it in a very real way. Making people feel more connected with something is key to drawing their interest. Punch Dunk use Immersive Theatre, they allow viewers to experience a narrative in their own way. Mixed reality allows people a very similar opportunity.

Reality, as with virtual reality is an experience. You cannot expect people to be drawn into something without initial engagement. Therefore the way to peek a viewers interest should be regarded as significant as the experience itself.

A vr headset can be more than just a tool. It can be aesthetically changed to add to the immersion. Reality can we twisted with virtual elements and added to through software to warp how we view and perceive it. These don’t have to be dramatic physical changes in order to get a large reaction out of a viewer.

Beginning ideas:

Bypassing reality —> Expression… Gender… Aesthetics to remove prejudgment.

Hyper reality —> Adding to reality, making it more vibrant or overbearing.

Inverse reality —> Seeing only through moving objects or only seeing advertisements.

Looking at science fiction and other future inspired media for inspiration. Look at needs or wants. What could be augmented to improve every day life. AR Kit from apple used to add or cover up existing reality.

Shows such as altered carbon, black mirror and the video above display how augmented reality could be used to drastically alter life in quite dark cynical life. We could block out or sensor sections or reality or add other things to ‘enhance’ reality. But it is questionable weather or not this is a good idea for our own sanity.

Using Vuforia and Turbo Squid to access and upload assets for use in unity, I am learning how to create an augmented reality system in the application in order to replace imagery and overlay 3D onto it. Turbo allows for free access to 3D model assets while Vuforia is a location for storing assets to use within unity.

 

Here is some tests I have done In augmented reality using unity. They work very well when transferred onto my phone using Xcode. I want to experiment with using many different images on screen at once in order to create a layout of virtual objects in the real world.

I want to experiment with creating a set of pieces which are linked to 3D models in a similar manner to a chess set. These would show up their models when used alongside a unity based mobile app allowing for multiple games to be potentially played using them dependant on the desired configuration.

Here is a pawn from a set of chess pieces I made in the past. These pieces could be used to generate a virtual chess set from many identically shaped chips. However they could be configured in different ways to play other things such as draughts or other simple board based games.

I want to create a variety of models in obj format and attempt to display these on top of icons using the image target system along with Vuforia within the unity software.

Within the unity environment, buttons could be positioned along the bottom of the screen in a similar fashion to here allowing for the different presets to be revealed. This would work via the use of C sharp coding in the MonoDevelop section.

This was reasonably successful, however I wanted to try using tons of images at once which was less successful.

Here I am trying to recognise multiple of the same image for the use within the chess set idea. However from reading forums and experimenting myself, I have figured out I will need to create a different image target for every single piece even though they may represent the same playing piece and 3D obj model. This is due to the software which once it finds the image target that it was looking for, it will stop trying to find further instances of the same image. This reduces processing load, however also makes using many image targets at one more complicated.

I will generate another set of 3D images, aiming to make the more recognisable to the Vuforia software. I also need to account for the print out colour being much more subdued than the on screen ones. I think once I have printed the images out, I should scan them back in to allow the computer to understand how they are less saturated in reality.

Here you can see the unity project with a new set of image target implemented. These were 5 star image targets meaning they are recognised much more easily by the software.

Here you can see the original digital versions with high vibrancy. However bellow the same images are shown in more toned down version because thesis how they would appear on screen while viewing them through my phones camera. I had to desaturate the image targets to this level to make them more recognisable by the software. 

I now need to create models for each type of piece within the chess set. Also because of the pop in and out of the pieces, I feel it would only be appropriate for the pieces to potentially have a fade in and out effect so that the viewer would have a smoother experience with the high number of image targets:

 

 

Here are several pages from my sketchbook which lead me to generate the idea of making an augmented reality sketchbook. I like the potential idea of the models themselves being rather abstracted and sketch like, contrasting with what we expect the models of chess pieces to appear to be. A regular chess set is a very defined preset where colour doesn’t usually even come to play in any regard beyond just as a binary way to show opposition. However in my set,  I feel that incorporating colour could confuse and convolute play of the game for the viewer and potentially change the whole experience.

Here are several images of the board which I produced to be used with the augmented reality unity app. The pieces are also places within their standard placing in the final picture and jumbled up in the first, with the second picture showing how the pieces can be stacked for storage or transport purposes. I still need to add my image targets onto these pieces once I have made them as recognisable as possible.

Here you can see the board with the addition of the image targets onto the pieces. I feel that they liven up the whole thing and without knowledge of what the actual item is make it appear as a confusing, almost futuristic artefact. I like the fact that the colour co-ordination appears unknown, with only the white or black scribbles differentiating the sides. I believe that as an observer without the app, you may not even realise the players are enjoying a game of chess, which messes with the idea of augmenting reality as it is only altered or improves for the very person within the game. For everyone else, it appears as sheer madness. It could almost be seen as some form of private chess as the app would bring understanding to the whole experience.

The fact that the scribbled image targets are so recognisably different to the computer while they all seem very similar and insignificant to a human is very unusually to me also. I think it makes the augmented aspect of the piece even more unusual. There is very little way for a human to recognise what each piece without a lot of trial and error, along with the use of the app also.

Here you can se how the app recognises each target and places the characters upon it. To my eye, they all seemed relatively orientated, however the computer knows the exact way each piece faces and thus many of the characters face In different directions. From here, I need to create each of the pieces trying to add to the general aesthetic of the piece and add further intrigue to the player and onlookers experience.

 

Looking at chess sets I found online, I wanted to create a slightly unusual chess set. The first 2 above images are of a set I found which seemed very digital. The image at the bottom is of set I created using fusion360 from Autodesk all using the revolve tool. I then shaded them into black and white within unity itself and exported them using xCode so I can now test this in Augmented Reality.

I tweaked the whole system adding the function of extended view within Vuforia which allows for image targets to linger after the camera loses track of them. This also allows for the images targets to understand their position in relativity to other targets, meaning that the whole chess board can easily be viewed at once within having pop in and out. This does cause occasional glitches, however I feel on the whole, the whole pieces is a major success in generating a prototype for an augmented reality app like experience for chess which could be modified and added to at a future date:

If I were to further develop this piece, I would try to allow the app to play many different games generated from the same tile set along with twisting the rules of each of the games due to their digital environment. For example I could easily add randomly shifting pieces and dead spaces within chess as a game to make the whole thing more luck based and confusing. This would further confuse those who don’t have the augmented aspect via the app as the players would appear to do completely random things with an unknown set of tiles on a unfamiliar styled board.

Here is an image of the app icon, along with it displayed on my phone’s home screen along with different sized versions of the icon for the app itself. There is also a splash screen shown for the app which adds to the general immersion fort he whole program.

I feel overall the board within the unity application appears very alien which is the appearance I was aiming for. I do however want to make some form of case for it to make it a more complete experience.