Skip to content

Camera injection: Onfido document capture fails (AVCaptureSession.connections and preview-layer metadata rects are not emulated) #989

Open

Description

@gelasdev

Description

With idb camera inject, an app that uses the Onfido iOS SDK (32.6.3) shows the injected video in its document capture preview, but the capture cannot finish:

  1. Pressing the shutter fails with Onfido's own error "Couldn't establish active camera connection on photo capture". capturePhoto(with:delegate:) is never called.
  2. With (1) fixed, the photo is taken, but cropping fails with "Something went wrong whilst cropping the captured image".

Both come from AVFoundation APIs that the virtual camera in Shims/Shimulator/VideoInjection does not answer yet:

  1. Before the capture, Onfido.CameraLayerView.checkPhotoCaptureConnectionIsActive() reads AVCaptureSession.connections. The shim does not swizzle it, so for a session that holds the virtual input it returns an empty array. The README already lists "a session's connections" as not covered.
  2. To compute the crop area, Onfido calls AVCaptureVideoPreviewLayer.metadataOutputRectConverted(fromLayerRect:) (metadataOutputRectOfInterestForRect:). With no native connection behind the layer, the system returns the layer rect with x/y and width/height swapped but not normalised, for example {244.7, 12.6, 237.7, 376.9} instead of values in 0...1. Onfido then multiplies it by the image size (1080x1440) and the crop rect falls far outside the image.

Reproduction

  1. Build idb from main (6079df9) and boot an iOS 26.5 simulator (Xcode 26.6, macOS 26.6.2).
  2. idb camera inject document.mp4 --loop --udid <UDID> (any video of a document), then launch the app through the companion (idb launch --udid <UDID> <bundle>).
  3. Open the Onfido document step and press the shutter.

Observed with LLDB attached to the app process:

-[AVCaptureSession connections] -> ()            # empty
  Onfido.CameraLayerView.checkPhotoCaptureConnectionIsActive() -> Bool
  Onfido.CameraLayerView.capturePhoto(onCompletion:)
  Onfido.PhotoCaptureView.didTapCaptureButton()

Onfido logs OnfidoFlowError.exception, message "Couldn't establish active camera connection on photo capture", withError: nil.

After adding AVCaptureSession.connections, breakpoints inside getFrameCapture(_:capturePhoto:) show:

areaOnScreen (layer)                     {12.56, 244.72, 376.88, 237.66}
getRelativeImageAreaToCrop -> (expected 0...1)  {244.72, 12.56, 237.66, 376.88}
convertRelativeRectToAbsoluteRect(size 1080x1440) -> {264297.6, 18090, 256673.0, 542700}

Solution

A pull request adds both to the shim:

  • AVCaptureSession.connections returns the connections of the virtual outputs that have input ports, after any native ones, the same way inputs and outputs are answered.
  • AVCaptureVideoPreviewLayer's metadataOutputRectOfInterestForRect:, rectForMetadataOutputRectOfInterest:, captureDevicePointOfInterestForPoint: and pointForCaptureDevicePointOfInterest: use one transform between the layer and the normalised native frame: the frame placed by videoGravity, turned by the preview connection's rotation, and mirrored when the connection is, as AVFoundation does on a device.

With these changes Onfido captures the front and the back of a document and reaches its selfie step.

Additional Information

  • idb revision: 6079df9 (main, 2026-10-24)
  • Xcode 26.6 (17F113), macOS 26.6.2, iPhone 17 Pro simulator, iOS 26.5
  • Onfido iOS SDK 32.6.3

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions