Skip to content

Conversation

@ngokevin
Copy link
Member

@ngokevin ngokevin commented Jul 14, 2017

http://aframe.io/blog/motion-capture
dmarcos/aframe-motion-capture-components#32

Changes Proposed

Motion Capture Features

  • Press the camera icon in the top left or press m shortcut to toggle motion capture tools above scene graph. Opening the tools will inject the motion capture components.
  • Specify recording name to save as (within Indexed DB) with a recording button to start recording. When we press the recording button with the controller toggle on, then we press any button on controller to start recording and then press any button five times quickly to stop recording. There will be a 5 second countdown before recording, shown both in the 2D UI and in-VR camera. There will also be a 2D button to stop recording too.
  • Select a recording from Indexed DB via dropdown. All recordings will be saved locally. Then we can press replay button to start recording. During replay, we can either use the Inspector camera (and even modify the scene live), the first-person player camera, or the spectator camera (hit q outside Inspector).
  • To easily transfer recordings between computers, we can export recording by saving as a file, upload button to gist.github.com and get a URL back. Or delete recordings.
  • We can add recordings into our recording DB from file system or from a URL.
  • The Inspector will populate Indexed DB with a sample recording with head and two controllers if the database is empty. Developers will have to set IDs rightHand/leftHand to see controllers, however, I added a note in the UI when this sample recording is selected. Or perhaps when we select a recording, I should always should which IDs it is controlling.

screen shot 2017-07-22 at 5 05 52 am

@ngokevin
Copy link
Member Author

ngokevin commented Jul 18, 2017

@ngokevin ngokevin changed the title [wip] motion capture tools Jul 18, 2017
@dmarcos
Copy link
Member

dmarcos commented Jul 18, 2017

I did a quick first run. Problems:

  1. When the recording starts the inspector closes, what makes everything a bit confusing. I know that the scene is paused and might be difficult to resume / pause with the inspector opened. We could maybe at least to add any recording feedback while recording is happening with the inspector closed. A blinking red dot, a timer? It would be interesting to have a button to stop recording and then the inspector opens back up again when pressed. I would probably hide the Inspect Scene button as well.
  2. When a recording is playing I can't start recording. Replaying should stop and recording start
@ngokevin
Copy link
Member Author

  1. The Inspector should close because the Inspector adds helpers and extras. We want to be sure we are recording with the pure scene. The Inspector closing adds a level of feedback that the recording has started (versus nothing on the screen changing). -- I will make the Recording button do a countdown for now, and we can add more feedback later. -- What I intend is to have the Inspect Scene button act as the Stop Recording button. I need to make the Inspector overlay a button on top.

  2. OK.

@ngokevin
Copy link
Member Author

ngokevin commented Jul 18, 2017

I also added a few more tasks for myself in the PR description via the checkboxes after doing a run because sometimes I'd rather not have to open the Inspector on each refresh. People can import the components, take recordings with the Inspector, but be able to auto-replay them on refresh via like query param or component configuration.

@ngokevin
Copy link
Member Author

Ready for another try-out. https://ngokevin.github.io/aframe-inspector/example/controllers.html

Camera switching may be weird sometimes because the Inspector examples immediately boot into the Inspector, and vrdisplayactivate will kick you into VR while in the Inspector. Sometimes recording will work fine, but best to exit VR and re-enter the Inspector and record. In normal use case, it is not a big problem because entering the Inspector will exit VR.

@ngokevin ngokevin requested a review from fernandojsg July 22, 2017 13:48
@ngokevin
Copy link
Member Author

mocadev2

@ngokevin
Copy link
Member Author

Also finding it useful being able to change the speed of the recording. I wrote a script to double the speed by dividing the timestamps in half, speeds up each test iteration for me (e.g., watching a 20 second recording). Maybe later will add a UI tool for that.

@ngokevin
Copy link
Member Author

ngokevin commented Aug 9, 2017

@fernandojsg
Copy link
Member

@ngokevin I've just fetched it, npm install and npm start but I get:

ERROR in ./~/react-file-reader-input/lib/index.js                                                                                
Module not found: Error: Cannot resolve module 'prop-types' in C:\code\aframeinspector\node_modules\react-file-reader-input\lib 
 @ ./~/react-file-reader-input/lib/index.js 29:17-38                                                                             
webpack: Failed to compile.                                                                                                      

Did you forget to add some dependency in package.json? Or am I doing something wrong? :\

@ngokevin
Copy link
Member Author

OK, can you try again? I think I needed to move a devDep to a dep in react-file-reader-input

@fernandojsg
Copy link
Member

@ngokevin I've tried again and that error is gone, but as long as I click on the motion capture button and the inspector injects the aframe-motion-capture component I get the following error on the console:

[inspector] Injecting <script src="https://rawgit.com/ngokevin/aframe-motion-capture/mocapinspector/dist/aframe-motion-capture-components.min.js"></script> for motion capture.
SceneGraph.js:107 Uncaught TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at SceneGraph.js:107
    at treeIterate (SceneGraph.js:105)
    at treeIterate (SceneGraph.js:130)
    at EventEmitter.SceneGraph._this.rebuildOptions (SceneGraph.js:134)
    at EventEmitter.emit (events.js:96)
    at Events.emit (Events.js:16)
    at MutationObserver.<anonymous> (Main.js:61)

/cc @dmarcos

@ngokevin
Copy link
Member Author

Is this A-Frame master? Will try again later.

@dmarcos
Copy link
Member

dmarcos commented Aug 29, 2017

When testing this:

  1. I click on the play button for the Sample Recording and I get the following errors on on console on Chrome.: errors

  2. In Firefox. I see an Indexed DB error and the Sample Recording does not show up in the list. erros

@dmarcos
Copy link
Member

dmarcos commented Aug 29, 2017

When trying to record:

  • In both Chrome and Firefox: I introduce test as a recording name and play record. It enters fullscreen (testing without headset) I move with the keyboard and press escape. When back to the inspector my recording does not show up in the list and I see the same errors as in my previous comment (constructor of null in Chrome and IndexedDB errors on Firefox)
@ngokevin
Copy link
Member Author

OK will patch it up soon. I think it was due to an A-Frame regression (constructor is null in buildData), was working flawless for me otherwise.

@ngokevin ngokevin closed this Sep 2, 2017
@ngokevin ngokevin reopened this Sep 2, 2017
@ngokevin ngokevin force-pushed the mocap branch 2 times, most recently from d840e6e to 7ec5699 Compare September 2, 2017 10:53
@ngokevin
Copy link
Member Author

ngokevin commented Sep 3, 2017

OK fixed the A-Frame error, try again.

@ngokevin
Copy link
Member Author

ngokevin commented Sep 3, 2017

nvm need to fix things...code has rotted since rebase

@ngokevin ngokevin force-pushed the mocap branch 3 times, most recently from dbd0654 to e5359e4 Compare September 3, 2017 16:09
@ngokevin
Copy link
Member Author

ngokevin commented Sep 3, 2017

OK, all fixed up again. Had to fix several bugs with the motion capture components (changes to A-Frame tracked-controls system which motion capture replayer wraps), and since A-Frame no longer cloned data on getAttribute, motion-capture-recorder had to clone so it wasn't save the same object for all of its poses.

@fernandojsg
Copy link
Member

IMHO too much verbose info on the component. I would get rid of the long explanation on the toggle recording checkbox and on the controllers sample info, maybe adding then to a tooltip next to the button if needed, because right now is taking more than half of my screen's height.
image

I would change the color of the Record and Replay instead of using underline as the first reaction would be to try to click it thinking it's a link.

I'll be out without access to a Vive so I can't test the functionality with it, @dmarcos could you try it?

In the meanwhile I've been trying with the laptop without any headset and it's confusing when you click the record button as it just go fullscreen without any other action. Should we just remove the record button if not controls or headset detected? And just let the user to replay recordings?

@ngokevin
Copy link
Member Author

ngokevin commented Sep 8, 2017

I wouldn't remove the text, they are critical instructions and it's a checkbox that changes behavior. It'd be confusing to remove the instructions and just have a bunch of shape icons. The tools are collapsible if you don't want to see.

The tools are primarily for headset but still works for 2D. Unticking the checkbox will start recording without controller trigger. I can change color of headings but I use this heavily so it works, put maybe a month into it, and it'd help everyone who is still putting the headset on and off every code change.

@fernandojsg
Copy link
Member

Ok to the comments text. I still don't undestand the behaviour in 2D as I don't see any feedback that I'm recording or so, it just go fullscreen and after exiting I can't find the recorded session.
Am I doing something wrong?

@fernandojsg
Copy link
Member

I mean, if it's not still supported the 2D mode for recording, we could just merge it and then we could fix/disable the 2d mode recording in another PR, and then improve the support for it just not to delay this PR as it's already being delayed a lot (Sorry for that :)

@fernandojsg
Copy link
Member

@ngokevin any thoughts on my previous comments?

@ngokevin
Copy link
Member Author

@fernandojsg Sorry, I'm back! I added some text display to the camera Press any controller or keyboard button to record, and support pressing keyboard button to start. This gives an indicator for both 2D and VR mode. Should be good now!

@fernandojsg
Copy link
Member

@ngokevin welcome back! 👍
I've tried it and it works great in 2D! I've just found a small bug that make it throws an error when using the keyboards to move around the scene.

image

document.addEventListener('keyup', function keyboardStart () {
textEntity.parentNode.removeChild(textEntity);
self.countdownRecording();
sceneEl.removeEventListener('keyup', buttonStart);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it must be document and keyboardStart:
document.removeEventListener('keyup', keyboardStart);

@ngokevin
Copy link
Member Author

ok fixed

textEntity.parentNode.removeChild(textEntity);
self.countdownRecording();
sceneEl.removeEventListener('keyup', buttonStart);
sceneEl.removeEventListener('keyup', keyboardStart);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sceneEl => document as you're adding the listener to document isn't it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, fixed again

@fernandojsg
Copy link
Member

Supercool!!! 🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

@fernandojsg fernandojsg merged commit 5b54b14 into aframevr:master Sep 30, 2017
@ngokevin
Copy link
Member Author

ngokevin commented Sep 30, 2017

horray, i'm gonna cut a release of inspector and later on do a blog post

@fernandojsg
Copy link
Member

@ngokevin cool! 👍

@Utopiah
Copy link

Utopiah commented Oct 3, 2017

Bit confusing to get a fixed point of view when the recording is finished, might we less disorienting to leave the camera moving with the HMD but display a message like "remove the HMD and go edit in the inspector" after.

Anyway that's great, can't wait to read the blog post!

@nikgraf
Copy link

nikgraf commented Oct 4, 2017

Really amazing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants