@fischaelameergeildanke.com @SCRIPT18
Augmented Reality
with Web Technology
AR
VR AR MR
Virtual Reality
Virtual Reality
completely digital
Augmented Reality virtual overlay
Mixed Reality virtual and real world coincide
@fischaelameergeildanke.com @SCRIPT18
A Short History
of Augmented Reality
Ivan Sutherland
Ivan Sutherland’s
The Sword of Damocles
Ivan Sutherland’s
The Sword of Damocles
Sword of Damocles, 1812
Richard Westall
https://www.youtube.com/watch?v=ISJWZpFIAlQ
Tom Caudell
❝ Augmented Reality (AR) is a variation of
Virtual Environments (VE), or Virtual Reality as
it is more commonly called. ❞
Ronald Azura, 1997
❝ VE technologies completely immerse 

a user inside a synthetic environment. While
immersed, the user cannot see the 

real world around him. ❞
Ronald Azura, 1997
❝ In contrast, AR allows the user to see the
real world, with virtual objects superimposed
upon or composited with the real world.
Therefore, AR supplements reality, rather than
completely replacing it. ❞
Ronald Azura, 1997
The Touring Machine
@fischaelameergeildanke.com @SCRIPT18
Technology and Use Cases
of Augmented Reality Today
Innovation Trigger
Innovation Trigger
Peak of Inflated
Expectations
Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of

Enlightenment
Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of

Enlightenment
Plateau of
Productivity
Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of

Enlightenment
Plateau of
Productivity
Machine Learning
Blockchain
Augmented Reality
Virtual Reality
Camera
Tracking
GPS
Screen
PositionRotation
https://www.microsoft.com/de-de/hololens
http://www.metavision.com/
https://www3.lenovo.com/at/de/tango/
https://developer.apple.com/arkit/
https://developers.google.com/ar/
https://play.google.com/store/apps/details?id=com.evogames.stackitar
https://play.google.com/store/apps/details?id=com.roomle.android
https://play.google.com/store/apps/details?id=com.signalgarden.atomvisualizer
https://www.volkswagenag.com/de/group/research/virtual-technologies.html
https://www.linkedin.com/pulse/20140502184728-1282856-guide-to-augmented-reality-in-healthcare/
https://vrodo.de/augmented-reality-mit-hololens-microsoft-zeigt-holoportation/
https://techcrunch.com/2015/07/29/google-translates-app-now-instantly-translates-printed-text-in-27-languages/
https://www.urdesignmag.com/technology/2015/01/26/rideon-augmented-reality-ski-goggles/
@fischaelameergeildanke.com @SCRIPT18
Augmented Reality
and the Web?
Write Once, Runs Anywhere
Write Once, Runs Anywhere
No App Download Required
Camera
& Display
Camera
& Display
Tracking
Sensors
Camera
& Display
Tracking
Sensors
World
Recognition
WebGL API & WebVR API
WebGL API & WebVR API
Media Capture and Streams API
Marker-less AR
vs.
Marker-based AR
https://aframe.io/
@fischaelameergeildanke.com @SCRIPT18
Markerless Augmented Reality
with ARCore and ARKit
ARKit
by Apple
ARKit ARCore
by Apple by Google
ARKit ARCore
by Apple by Google
World/Motion Tracking Scene Analyzation Light Estimation
ARKit ARCore
by Apple by Google
WebAROnARKit WebAROnARCore
Two Experimental Browser Apps
https://github.com/google-ar/WebARonARKit https://github.com/google-ar/WebARonARCore
ARKit ARCore
by Apple by Google
three.ar.js
three.js for AR
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
https://furniture.3d.io/
<script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
let scene = new THREE.Scene();
let camera = new THREE.PerspectiveCamera(75,
window.innerWidth / window.innerHeight, 0.1, 10000);
let renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
function render() {
requestAnimationFrame(render);
renderer.render(scene, camera);
}
render();
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
<body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>
let scene = document.querySelector('a-scene');
let raycaster = document.querySelector('[ar-raycaster]');
let mark = document.querySelector('#intersection');
raycaster.addEventListener('raycaster-intersection', function(e) {
mark.setAttribute('position', e.detail.intersections[0].point);
mark.setAttribute('visible', true);
});
raycaster.addEventListener('raycaster-intersection-cleared', function(e) {
mark.setAttribute('visible', false);
});
window.addEventListener('click', function () {
let furniture = document.querySelector('[io3d-furniture]');
let position = mark.getAttribute('position');
furniture.setAttribute('position', position);
furniture.setAttribute('visible', true);
});
https://3d.io/
@fischaelameergeildanke.com @SCRIPT18
Marker-based AR
with AR.js
https://poly.google.com/view/77wHkzwlpOq
<script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>
<script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>
<script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>
<script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>
<a-scene arjs="trackingMethod:best;">
<a-assets>[…]</a-assets>
<a-anchor hit-testing-enabled="true">[…]</a-anchor>
<a-camera-static/>
</a-scene>
<a-assets>
<a-asset-item id="sun-obj" src="assets/sun.obj"></a-asset-item>
<a-asset-item id="sun-mtl" src="assets/sun.mtl"></a-asset-item>
</a-assets>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
<a-entity id="annyang" annyang-speech-recognition></a-entity>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
<a-entity id="annyang" annyang-speech-recognition></a-entity>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false"
speech-command__show="command: sun;
type: attribute; attribute: visible; value: true;"
speech-command__hide="command: hide;
type: attribute; attribute: visible; value: false;">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>
https://poly.google.com/view/77wHkzwlpOq
https://jeromeetienne.github.io/AR.js/three.js/examples/arcode.html
@fischaelameergeildanke.com @SCRIPT18
UX Design for
Augmented Reality
@fischaelameergeildanke.com @SCRIPT18
No Established Patterns Yet
Interpretability
Comfort
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc
Interpretability
Comfort
Safety
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc
User Input and Interaction
User Interface and Information
Interpretability
Context
Comfort
Safety
Illusion
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc
VoiceGestures Gaze Movement
Respond Immediately
User Input and
Interaction
Respond Immediately
Use Simple Interactions
User Input and
Interaction
Respond Immediately
Use Simple Interactions
Don’t Apply 2D patterns
User Input and
Interaction
User Interface
and Information
Provide Information in Context
DoDon’t
Provide Information in Context
Use Visual Indicators
User Interface
and Information
https://developer.apple.com/ios/human-interface-guidelines/technologies/augmented-reality/
Provide Information in Context
Use Visual Indicators
Onboarding is Important
User Interface
and Information
Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting
Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting
Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting
Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting
DoDon’t
https://medium.com/inborn-experience/onboarding-in-augmented-reality-mobile-application-6e08e1d338f8
Provide Information in Context
Use Visual Indicators
Onboarding is Important
Error Handling, too
User Interface
and Information
Creating an
Illusion
Light and Shadow are Important
https://www.youtube.com/watch?v=ttdPqly4OF8
Light and Shadow are Important
Avoid Interruptions
Creating an
Illusion
Light and Shadow are Important
Avoid Interruptions
Avoid Clutter
Creating an
Illusion
Consider
the Context
Consider a User’s Environment
Consider a User’s Environment
There’s No Optimal Setting
Consider
the Context
Comfort
and Safety
Avoid Fatiguing The User
Comfort
and Safety
Avoid Fatiguing The User
Support Short Usage
Comfort
and Safety
Avoid Fatiguing The User
Support Short Usage
Avoid Large Movements
Comfort
and Safety
Avoid Fatiguing The User
Support Short Usage
Avoid Large Movements
Respect Safe Space
Comfort
and Safety
@fischaelameergeildanke.com @SCRIPT18
The Future of AR
in the Web
WebXR as an Extension
The Future of
AR in the Web
WebXR as an Extension
Declarative WebGL
The Future of
AR in the Web
WebXR as an Extension
Declarative WebGL
3D Positions for Elements
The Future of
AR in the Web
https://www.youtube.com/watch?v=8GXUlijX60E
Get involved!
The Future of
AR in the Web
@fischaelameergeildanke.com @SCRIPT18
Thank you!
https://github.com/jeromeetienne/AR.js/
https://github.com/google-ar/three.ar.js/
https://research.mozilla.org/mixed-reality/
https://github.com/mozilla/webxr-api
https://webxr.io/webar-playground/app/
Libraries
https://github.com/mozilla-mobile/webxr-ios
https://github.com/google-ar/WebARonARKit
https://github.com/google-ar/WebARonARCore
Browsers
https://3d.io/docs/api/1/aframe-components.html
https://3d.io/docs/api/1/furniture.html
https://poly.google.com/
https://sketchfab.com/
https://jeromeetienne.github.io/AR.js/three.js/examples/arcode.html
Assets

Creating Augmented Reality Apps with Web Technology