Add text about extra views

[Manishearth]

I chose to write the text such that user-agents are allowed to expose additional views and content is expected to handle it, but we recommend that user-agents don't do this.

Depends on immersive-web/webxr-ar-module#57

Fixes #1045

cc @cabanier @thetuvix

---

Preview | Diff

[cabanier]

Since almost all content will only work on mono or stereo devices, maybe a feature that lets the site opt into multiple views would be better.

[Manishearth]

Yeah I was considering this! I'm totally open to that option as well, I wasn't sure which was better. I was a bit concerned because I wasn't sure if content could get this right for the scope of all future secondary views. E.g. quad views give you two views for each eye, which is an additional assumption broken.

@toji, what do you think? I could instead write spec text for a secondary-views feature in this spec, and first-person observer views just become a spec concept (and not a feature) in the AR module. Such a feature would basically be asking authors to certify:

• they expect the views array to change size
• they can handle any number of views
• they can handle multiple views with the same eye, with the stipulation that the first two will always be the primary L/R views if necessary

[asajeffrey]

I'm not sure about that, especially if we can get the frameworks like babylon.js or three.js to support multiple views.

[cabanier]

What if your experience uses multiview? Will that easily work if there are suddenly more displays?

[Manishearth]

@asajeffrey the feedback from @thetuvix was that:

• most applications rarely expect extra views
• specifically in the first-person observer case, if the application is not going to handle extra views then we should instead let the device APIs synthesize an FPO view over exposing an unused view

so there's a general feeling that content should signal support for extra views, and a stronger feeling that at least FPO views should be specifically requested

[thetuvix]

Our experience from HoloLens is that the easy part is getting an engine like Unity or Babylon to plop a scene camera at every view and produce some pixels. However, in practice, apps scripted in those engines tend to assume stereo unless tested from that extra perspective, and so their own shaders will make broken assumptions (e.g. in instancing, assumptions about the view origin, etc.).

It is also valuable to know whether a given app/page has opted-in because then you can do a better fallback for the pages that don't:

• On HoloLens 2, if an app doesn't opt into the first-person observer view, we do the least-bad thing and distort the left or right eye image. While this isn't ideal, it's better than rendering black or an otherwise broken image from an app that wasn't hardened correctly for that third view.
• On Starbreeze quad view headsets with wide-left and wide-right displays, if the app knows how to render 4 views, it's given one view that represents each display. If the app doesn't opt-in, however, it's given 2 stereo views that are each larger and overlap multiple displays. They don't cover the device's full FOV, but they cover much more than if the app rendered for the inner displays and then provided left the buffers black for the outer displays.

[Manishearth]

Yeah, it seems like this should be per-view-configuration-style opt in then.

[cabanier]

Doing so would make the spirit of this note normative.

[cabanier]

Maybe we can add a new XREye enum for the observer.

[Manishearth]

Maybe we can add a new XREye enum for the observer.

This was discussed a bit in the call and folks seemed largely against it

[cabanier]

This was discussed a bit in the call and folks seemed largely against it

There needs to be a way to tell if a view is an observer so the author can render things slightly differently to it. If you render to a quadview or cave system, how would you know which view is the observer view?

[Manishearth]

For quad views it will be the only one with a None XREye.

I'm open to adding this enum variant, I just think this should be discussed separately since folks didn't seem to like it during the call.

[thetuvix]

It seems valuable to keep the XREye attribute as is to tell you if a given view is left, right or mono, as this information is still useful for quad views and other secondary stereo views in the future when apps need to condition their behavior for left or right eye renders. Modules that provide for feature opt-in for additional kinds of views can then introduce an isFirstPersonObserver boolean attribute or such on XRView to let the app know which view is which.

[Manishearth]

That works for me!

[cabanier]

Alternatively XRViewerPose could expose the observer view as another attribute, ie:

[SecureContext, Exposed=Window] interface XRViewerPose : XRPose {
  [SameObject] readonly attribute FrozenArray<XRView> views;
  readonly attribute XRView? observerView;
};

Then, instead of the array adding/removing the extra view, the attribute would sometimes be null.

[Manishearth]

I think we should be conditioning content to expect anything in the views array. It's an array for a reason, and it seems better to get it to design code that just renders to all elements of the array it's given rather than getting it to handle each case separately.

My current thinking, at any rate, is that we should add a secondary-view feature, remove the FPO feature, and spec FPOs purely qualitatively in the AR module along with a single bool flag.

[cabanier]

I think we should be conditioning content to expect anything in the views array. It's an array for a reason, and it seems better to get it to design code that just renders to all elements of the array it's given rather than getting it to handle each case separately.

It's weird that the views array changes at runtime.
You will also need to update the views array definition. Will the views array change over time as well, or only the XRViews?

My current thinking, at any rate, is that we should add a secondary-view feature, remove the FPO feature, and spec FPOs purely qualitatively in the AR module along with a single bool flag.

Yes. That sounds like the right approach

[Manishearth]

Closed in favor of #1083