AR/VR Development

Verge3D allows for creating web-based Augmented Reality (AR) and Virtual Reality (VR) experiences running on top of the browser technology called WebXR (eXtended Reality on the Web). In addition to that you can use USDZ format and Apple AR Quick Look technology to preview your models in AR mode on iPhones and iPads.

Contents

Virtual Reality (VR)

Setting Up

HTC and Oculus devices should work with WebXR out of the box. Use Google Chrome browser for desktop-connected devices such as HTC Vive. Use the browser which comes with your headset in case of standalone devices, e.g run your VR apps in Oculus Browser on Oculus Quest 1/2.

WebXR requires a secure context. Verge3D apps must be served over HTTPS/SSL, or from the localhost URL.

Creating VR Apps

The VR mode can be checked with check VR mode and set up using enter VR mode puzzle.

Interaction with 3D objects is performed by using the gaze-based reticle pointer automatically provided for VR devices without controllers (such as cardboards).

For VR devices with controllers, interaction is performed by the virtual ray casted from the controllers.

You can use the standard when hovered or when clicked puzzles to capture user events as well as VR-specific on session event.

For usage example, check out the following demo from the asset store: Industrial Robot.

Augmented Reality (AR)

Setting Up

You can run your Verge3D-based augmented reality applications on mobile devices with Android or iOS/iPadOS operating systems.

WebXR requires a secure context. Verge3D apps must be served over HTTPS/SSL, or from the localhost URL.

Android

To enable augmented reality, you need an Android device which supports ARCore technology and latest Google Chrome browser. You also need to install Google Play Services for AR. The installation of this package is prompted automatically upon entering AR mode for the first time, if not pre-installed.

iOS/iPadOS

For iPhones/iPads you have two choices:

  1. install WebXR Viewer browser
  2. preview assets in USDZ format using the conventional Safari browser

Mozilla's WebXR Viewer is a Firefox-based browser application which supports the AR technology on Apple devices (starting from iPhone 6s). This app is available from the Apple App Store. This solution has an advantage, that you can use the same WebXR technology (as well as Puzzles logic) that you use to run AR on Android devices. The disadvantage is obvious — you (as well as your users) should install WebXR Viewer browser manually.

With USDZ assets, you can use the Apple AR Quick Look technology to run simplified augmented reality experience in the default Safari browser. This approach requires you to use completely different rendering engine integrated in iOS, as such you can't make complex materials or use Puzzles logic for your apps running in AR mode.

Creating WebXR-based AR apps (Android, WebXR Viewer)

The AR mode can be set up for any Verge3D app using the enter AR mode puzzle.

Upon entering AR mode you will be able to position your 3D content in the "real" coordinate system, which is aligned with your mobile device. In addition to that, you can detect horizontal surfaces (tables, shelves, floor etc) by using the detect horizontal surface AR puzzle.

Also, to see the real environment through your 3D canvas, you should enable the transparent background option in the configure application puzzle.

For usage example, check out the following demo from the asset store: Augmented Reality.

Creating USDZ-based AR apps

To enter AR Quick Look mode you need to create USDZ asset first. To do this, you can export your scene (or a single model) in runtime (on the fly) or use your favorite modelling suite to make it ahead of time.

Preparing USDZ asset in runtime

This approach is based on using export to USDZ puzzle to generate USDZ (see Puzzles snippet below).

Preparing USDZ asset in modelling suites

At the moment, USDZ export supported natively only in 3ds Max. For Blender you can try unofficial BlenderUSDZ addon.

USDZ is different from USD, being a container which combines USD models with media data, such as PNG, JPEG, MP3. Thus, modelling suites which able to export to USD not necessary support USDZ.

If you use macOS/iOS/iPadOS, there are plenty of tools available on Apple website to convert your models stored in OBJ/FBX/glTF/USD formats to USDZ.

Loading USDZ

To preview USDZ asset, exported by the modelling suite or some other tool, you need to create an HTML link with the image inside it:

<a id="enter_AR_button" rel="ar" href="model.usdz"> <img src="media/enter_AR_button.png"> </a>

The link should reference a USDZ model asset, e.g model.usdz. The image should reference picture which shows some button with icon or text, e.g "Enter AR". In the example above, the image located at media/enter_AR_button.png inside the application directory.

When the user clicks on such link on Apple device, the system will detect USDZ asset referenced by the link and open an AR Quick Look mode so that you can position the model in the real world:

With Puzzles, the same link can be created on demand:

To generate and load USDZ assets in runtime use the following code:

Check out the differences — we used export to USDZ puzzle here and assigned additional download attribute to the link.

For usage example, check out the following demo from the asset store: Augmented Reality.

Chrome Debugging

For faster iterations during development of WebXR-based apps (but not USDZ-based), you can use Chrome's port forwarding feature to run your AR or VR app in mobile browser. This will save you from re-uploading it to a remote web server each time.

What's Next

To find out how to script your AR/VR app with Puzzles, read the following section. Also, check out the Physics Guide if you're going to simulate a realistic virtual environment with colliding objects and/or character.

For usage example, check out the following demo from the asset store: Snowballs VR.

Got Questions?

Feel free to ask on the forums!