Wednesday, May 25, 2016

Dev update #12 - FreeTrack and other tracking options



This week we focused on improving our head tracking to reduce drift. We're adding FreeTrack support and alternative sensor fusion algorithms. 

Desktop client version is 0.5.4.4, mobile app version is 0.8.7. 

Why are we releasing update on Wednesday instead of Thursday this week? Tomorrow is public holiday in our country so everything (including our office building) is closed. Most of the team will be away on this long holiday weekend but we will catch up and respond to all e-mails and comments on Monday. 

Alternative tracking (mobile)

Until now we were using Cardboard API as the only tracking option. Today we are adding more options. Cardboard API is still the most tested with probably the lowest latency but many of you asked for something more stable. We decided to use alternative sensor fusion algorithms to reduce drift. Keep in mind that using magnetometer with magnet-button-cardboard can result in odd results.

We are also adding option to disable tracking completely because some people wanted 100% stable picture for cinema-like apps.

Our new options are:

TrackerRequired sensorsProsCons
Cardboard API
Gyroscope
Accelerometer
Lowest latencyHighest drift
Use magnetometer
(stability)
Gyroscope
Accelerometer
Magnetometer
Lowest driftHighest latency

May interfere with cardboard magnet-button
Use magnetometer
(responsiveness)
Gyroscope
Accelerometer
Magnetometer
Very stable
with low drift
Latency still higher than Cardboard API

May interfere with cardboard magnet-button
No gyroscope
Accelerometer
Magnetometer
Works without
gyroscope
Extremely shaky
No trackingnoneNo drift :)No tracking at all

You can access new options by tapping more in top-right part of the connection screen.

We will try to reduce delays on driftless options but if you are looking mostly ahead without much head movement you might want to use magnetometer optons for more stable experience. Remember that you can reset head position by pressing Ctrl+Alt+Shift+R.

External tracking support (FreeTrack 2.0 protocol)




Many people already have their own DIY tracking and we received many requestes to provide a way to use FreeTrack-formatted data. This is our first iteration of this protocol but we wanted to get it out ASAP so you can start experimenting with your devices. We tested with OpenTrack (FreeTrack 2.0 protocol).

Positional data is supported by SteamVR driver only currently. Oculus emulation layer will use rotational data only.

49 comments:

  1. Hi, Tracking is always disabled for me. Any way to revert the update?

    ReplyDelete
    Replies
    1. Did you update both mobile and desktop app?

      Delete
    2. Thanks for the reply, restart seems to have fixed the issues :)

      Delete
  2. Wow!!! I was already impressed by your software, but it keeps getting better! Positional tracking will be awesome, but with diy sets it's doubtfull how good it will be. Going to spend some hours on freetrack this week ;)
    Thanks for your hard work and making this free for everybody!

    ReplyDelete
  3. Freetrack:

    It works pretty well , I tried OpenTrack with EdTracker and smoothness is very good. It would be interesting to test with infrared LEDs and a webcam to see if you can get 6DOF .


    Problem with video:

    QuickSync / Media Foundation - 1080p / 720p or any resolution - 60 FPS:

    Start taking 35 / 45FPS
    If I screenshot I have 59.9FPS
    After 3 or 4 minutes 34 / 45FPS again.

    QuickSync - 1440p - 60 / 30FPS :

    The video begins to 59FPS but after 2 seconds down to 30FPS . There is no way to go up .

    ReplyDelete
    Replies
    1. I read your reports on previous post but I forgot to respond. We're gathering all Intel QS data because there are some cases when it fails to keep up to correct framerate and some cases where if fails completely.

      We don't have any new workarounds and hacks currently but we're collecting specs and settings where it fails.

      Delete

    2. No problem , I just want to help.

      With the Foundation Media Codec happens exactly the same.

      At 1080p is not able to run at 60FPS on mobile , although the image if it is properly encoded . If I make a screenshot is set to 59FPS , but to back down 3/4 min to 35 / 45FPS.

      At 1440p directly does not rise above 30FPS.

      What I find curious is that doing screenshot if starts running at 60FPS ( in 1080 )

      Delete

    3. I tried to lock the frequency of the CPU and the GPU , but still get the same result ...

      35 / 45FPS and when I make a screenshot 60FPS for 3/4 min .

      There is something that does not work well when decoding the image and display it on the screen, it's as if the cell came into " energy saving " mode but is not, and when I screenshot began to work well for a while.

      Delete
  4. Does the Freetrack mode also work with phone sensors?

    ReplyDelete
    Replies
    1. We wanted to have a mixed mode (FreeTrack position + phone orientation) but we decided to skip this for now. It was a little buggy.

      Delete
  5. Excelent update!! Thanks for your work!!

    ReplyDelete
  6. When are you going to add non-video encoder GPU compatibility, for example for 820M?

    ReplyDelete
    Replies
    1. That's at least one month away.

      Delete
  7. I really want to try the positional tracking but i cant find a good guide on how to make the kit and how to set it up. If you guys have any link or free time please make a video or a tutorial on how to make a kit and how to set it up and use it with Vridge.
    Thank you for all your hardwork.

    ReplyDelete
    Replies
    1. Hello, first of all you need this things :
      1. Camera , here is a list of recomended - http://www.free-track.net/english/hardware/camera.php. (i'm using ps3 eye)
      2. Point model. You have to build point model with led lights that your camera gonna capture for tracking. In free track is four different models, but best one for the vr pos tracking is 3 point model. You can read more here - http://www.free-track.net/english/hardware/point_model.php
      3. You need a software , chosee that you like, OpenTrack, FreeTrack, IrTrack and etc. All of them using same protocol FreeTrack 2.0
      Actually it seems a little bit hard but it's pretty easy , go on youtube and search "Freetrack" for better understanding.
      Good luck with it !

      Delete
    2. awesome, i went through the links you mentioned but somehow after reading your message its more clear and understandable.
      Thank you.
      i ll definitely look into it.

      Delete
    3. Also you can go deeper if you would like )
      You can use other software and ways of tracking . You will need to convert them to freetrack 2.0 protocol (vridge using this one)
      I'm planning to use Aruco marker tracking , i think is much better
      Here is some examples that i found on youtube
      https://www.youtube.com/watch?v=tfvsHUgMU5g
      https://www.youtube.com/watch?v=Xn5YW5Anjl4 (have no idea which language he speaks)
      Tell me if you want to know how to do that , it's even more easier i think )

      Delete
    4. How do you track when you turn away with stuck? I can't seem to get multiple images working with opentrack

      Delete
    5. How do you track when you turn away with stuck? I can't seem to get multiple images working with opentrack

      Delete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Hello, i got DIY HMD that uses DK2 firmware, and cuz it DIY i don't have any Positional Tracking , I know that it's possible to use dk2 with riftcat application , but is there any chance to use this freetrack function not only with streaming on mobile devices ? I mean can i use it with DK2 ? Do you use any drivers so steamVR can work with freetrack ? How to do that ?

    ReplyDelete
    Replies
    1. Yes, we coded FreeTrack support into our driver/server. There is no built-in option in SteamVR or Oculus runtime to accept FreeTrack input.

      Delete
  10. If you start_server to work on the drifting, why not working with gearvr hardware too ?

    ReplyDelete
    Replies
    1. Writing alternative tracking algorithm is a lot faster in terms od dev-time compared to switching to GearVR SDK & environment.

      Delete
  11. So frustrating.. can't get this software to work at all :/

    ReplyDelete
  12. Hi vridge/riftcat is just amazingm
    Thank you so much for creating it.

    So can freetrack now be used to emulate oculus positional camera/ vive lighthouse???....can I use it in steam vr as a tracker?

    Any progress on injectors? I.e vorpx, verio perception etc?

    ReplyDelete
    Replies
    1. You can use any FreeTrack-compatible tracker to feed positional+rotational data into VRidge so it does work kind of like Vive lighthouse (without controller tracking).

      No progress on 3d injectors so far.

      Delete
    2. But you can use it only when vridge server runing , yeah ? So i can't use it with my dk2 ? Only smartphones ?

      Delete

  13. I can confirm that there is the same problem with a different terminal :

    Meizu MX5 ( Helio X10) :

    NVENC - 1080 @ 60FPS - 9Mbps = 35 / 45FPS

    Meizu Pro 5 ( Exynos 7420 )

    NVENC - 1080 @ 60FPS - 9Mbps = 35 / 45FPS

    When I make a screenshot is set to 59FPS for 2/3 minutes.

    ReplyDelete
  14. Thank you for the article. It's really worth reading.

    ReplyDelete
  15. Sensores are bottlenecking my fps range, when i turf off phone sensor i get + 60 fps, when i turf it on o get 25~30 unplayable fps, how can i get remove this bottleneck using phone sensors? Can it be a USB cable bottleneck? I am using usb 3.0

    ReplyDelete
  16. Sensores are bottlenecking my fps range, when i turf off phone sensor i get + 60 fps, when i turf it on o get 25~30 unplayable fps, how can i get remove this bottleneck using phone sensors? Can it be a USB cable bottleneck? I am using usb 3.0

    ReplyDelete
    Replies
    1. Is it the same with all tracking modes?

      Delete
  17. This Riftcat/Vridge thing is great... The resolution upscaling option is great but like many others I've had problems with drift, I used OpenTrack with the Accela filter (and 1 other I can't remember) and the drift was completely gone, you had to tune it for latency vs stability, but when you did the drift was gone. The only thing is, they use Euler angles so there's gimbal lock which is sad (FreePie IMU and HeadTracker both). It might something different than gimbal lock, but my roll and pitch are tied.

    That last part has nothing to do with you, just saying you guys are great for working around that problem and also pointing out algorithms that work.

    ReplyDelete
  18. This Riftcat/Vridge thing is great... The resolution upscaling option is great but like many others I've had problems with drift, I used OpenTrack with the Accela filter (and 1 other I can't remember) and the drift was completely gone, you had to tune it for latency vs stability, but when you did the drift was gone. The only thing is, they use Euler angles so there's gimbal lock which is sad (FreePie IMU and HeadTracker both). It might something different than gimbal lock, but my roll and pitch are tied.

    That last part has nothing to do with you, just saying you guys are great for working around that problem and also pointing out algorithms that work.

    ReplyDelete
    Replies
    1. Thanks for the tip with OT settings. BTW did you use beta or stable Android app? The beta app uses updated Google VR algorithm for tracking with less drift.

      Delete
  19. hi there, I'm using an S6 with the GearVR in a computer with an AC wifi connection and an Nvidia 9700. So far so good, I can play Project Cars and LFS. But....

    The head tracking needs to be adjusted now and then... and.. the worst part is that the headtracking is painfully slow. I cannot turn my head with a smooth movement, it seems to do like "steps" and its very crude.

    Question:

    Would an external headtracking via de sony ps3 camera and opentrack give me smoother head tracking movement????

    ReplyDelete
    Replies
    1. It's probably the Samsung jitter bug where frames are sometimes displayed at incorrect timing/out of order causing it to be jerky/stuttering. We are working on it. I don't think external tracking would help in this case since it seems to be rendering (graphics) issue.

      Delete
  20. Hi Guys.
    Is it possible to use The leap motion LEDs in the unit to track your position?
    eg. Leap Motion attached to the HMD with a PS eye camera tracking the IR LED movements using Freetrack....
    Thanks guys, solid work.

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Update. I've purchased facetracknoir and found that in a dark room with a non reflective vr hmd, (I'll have to paint mine) you can track the 3 LEDs of the leap motion device with some success. You can't move your head fast or turn your head much at all but it works.... Only problem is use need some light I would of thought, to use leap motion as a controller to emulate the vive. This is untested as it's not yet night here to test properly.
    Any more input from others would be great. Thanks

    ReplyDelete
  24. Please! I want use Vridge only as translate Position Tracking Freetrack to SteamVR. It's need for PC HMD, thats working directly with SteamVr but not have Positional Tracking (Deepoon E2, Pimax 4k). It's possible? Or

    ReplyDelete
  25. Hello. Does your software support vr box headset?

    ReplyDelete
    Replies
    1. That depends on the phone your using....?

      Delete
    2. Hi. If it's all-in-one headset then we don't support those. It may or may not work.

      Delete
  26. These flexispy reviews would help you a lot if you are going to use this app.

    ReplyDelete