Monday, August 7, 2017

Dev update #32 - NOLO untethered



Today we are releasing version 1.5.0 to the beta channel. It comes with NOLO VR wireless support. It's not 100% finished but we wanted to start gathering some user reports to make the next version better based on your feedback.

Updates:
[1.5.2] - We're changing the way tracking status is interpreted. If you had tracking issues with 1.5.0, try again with the new version. It should be available in Google Play 7PM UTC at the latest.

Known issues:
  • Tracking may have decreased send rate, resulting in more frame skips (up to 20 dropped rotation frames per second).
  • Controller rotation and position may go out of sync in respect to head marker after recentering.


Nolo untethered

General info

In dev update 30 [http://blog.riftcat.com/2017/06/dev-update-30.html] we described reasoning behind implementing NOLO VR wireless mode. Today we are releasing first version of it to the beta channel.

There are few things that are not working correctly yet:
  • There are no vibrations yet.
  • Recentering may yield unstable results. You can turn controllers off and on again while holding them forward to make sure it resets properly.
  • Sometimes controller rotation jumps ~30 degrees to the left. We're not sure if it happens to our hardware unit only. We asked for another set from NOLO to test it further.
We plan to continue our work to fix above problems. We should have another update ready for you next week.

Connecting

You need to update both desktop and mobile app to the latest beta (1.5.0).

To use NOLO VR in wireless mode simply plug it into your Android micro USB port with USB data cable. VRidge popup will ask you for permission to use this USB device. Check "remember my answer" and then tap "allow". From now on, VRidge will automatically start the moment you plug your NOLO VR device. You can later revoke the permission by going to Android application settings if you want.

Tracking modes

When NOLO VR is plugged in, you will have another option in VRidge mobile app settings. You can toggle between NOLO VR sensors and phone sensors to handle rotational tracking. Your experience may vary from device to device. If you find the NOLO VR rotational tracking to be laggy in your case, try switching to phone sensors and see if it's smoother if you use your phone as rotational tracker.

Keep in mind that "Phone sensors" mode is still experimental as it fuses data together from two reference systems to create final 3D pose. It might yield incorrect results (controllers could be backwards, yaw might be skewed, etc.) This mode needs some extra tweaking and we might drop it completely in the future if no one uses it. Let us know if you find experience less laggy in this experimental mode.

Usage

If you started VRidge while NOLO VR was connected it will work out of the box. You don't need to install anything on your PC. We added two key combos for extra convenience. These hotkeys don't work in experimental mixed tracking mode (with phone sensors as rotational tracker).


Double tap menu buttons (upper buttons) while controllers are next to each other to recenter position and rotation

Double tap menu buttons (upper button) while controllers are apart to turn around

Rotation is still being tweaked so you might get odd results sometimes. The threshold between "next to each other" and "apart" is 10cm.

Testing sensors

If you want to see raw NOLO VR hardware data you can long press NOLO VR option in VRidge settings. This option is visible only when VRidge was started with NOLO VR connected to USB port with granted permission to read USB data. It can be useful to test if your trackers drift or have some other oddities. Rotation is displayed as quaternions which are very unreadable structure from human perspective but all you need to know about them is the fact, that those numbers shouldn't change a lot when your controllers or head marker are not being rotated.

Blurry right eye (certain Galaxy devices)

We've been tracking this bug for a while but we could never reproduce it, which made it really hard to fix. Recently few redditors suggested that it's a problem limited to Snapdragon only.
Galaxy S7 and S8 come in two variants. US edition comes with Qualcomm Snapdragon chipset and almost all other countries get Exynos edition. We checked and it seems that we have ~10 confirmations total that it's indeed a Snapdragon only problem.
Now that we know that we cannot reproduce it with our unit we took some other steps to attempt to fix it. We will soon have upgraded Gear VR renderer. We're going to send test APK to few users that are willing to test if the new version still has the problem. We are also in contact with some people at Samsung. We are trying to create minimum reproducible test case so we can ask them to help with this problem. We will keep updating you on this topic. With all the additional details and help, maybe we will be able to fix this once and for all.

Update: It seems to be fixed in updated rendering framework internally. Next Gear VR update will no longer have this problem. Many thanks to all the people who volunteered to test our debug builds!

API v3

This beta update also includes new version of API which uses Protocol Buffers as serialization layer. We needed to move from default .NET serialization because it had few things that were not very portable (byte layout, paddings, endianness, etc.). We also created Java library that is used to communicate between NOLO VR devices and VRidge. After we clean up some code and write some docs, we will add it to our VRidge API repo as another way to communicate with VRidge. Making software mods and plugins is pretty cool and can result in awesome new things being created so we want to share with you the same tools and layers that we use when connecting external tracking systems like NOLO VR.

Under Fire VR

We recently hired new programmers. They are still learning their way around VR so they created a small game as an introduction to VR world. You can check it out here:

https://play.google.com/store/apps/details?id=com.Riftcat.UnderFire

It's pretty cool so give it a try if you liked Scorched Earth back in the 90s. Our new developers now moved to other tasks that are much more related to VRidge.

More controllers option

Thanks to our new developers and expanded API we are planning to release few additional controller/tracking options. The first one to come will be based on Xbox controller. It will allow you to move in roomscale game and emulate controller rotation and position with controller analog sticks. It's not very immersive but it may find its use if you are developing VR game without motion control hardware. It's also quite viable option to play some atmospheric experiences where you want to explore areas but lack necessary hardware to trigger some gameplay things.
More advanced tracking options are coming too but it's too early to share details about it.

Other changes and fixes

  • We fixed a bug that prevented VRidge from working on Android O. It's working without any problems now on our Pixel XL with latest Android O (8.0, compilation id: OPP4.170623.014).
  • Connection screen will now turn upside down properly if phone is held upside down.
  • Settings screen will no longer freeze when SteamVR settings file is not writeable.
  • Mobile tracking will now properly switch off when VRidge receives full pose matrix or rotation quaternion through VRidge API (GitHub #12)

What's next?

  • We need to finish tweaking nolo wireless mode. This should be done in a week or two.
  • We're still reworking majority of the codebase for VRidge 2.0 revamp.
  • Our new developers are working on new tracking options.
  • Once 2.0 is out we will focus on optimizng latency and smoothness, jitter reduction and iOS version.

40 comments:

  1. Replies
    1. Report back: Works great for me! Even from a wireless laptop to wireless phone, because I'm pretty much in the same room as the 5GHz router it's working well! Thanks for the hard work RiftCat team!!!

      Delete
  2. I'm having trouble finding the newest version of vridge... It only gives me the version 1.4.1 from ConstructVR. Any Tipps?

    ReplyDelete
    Replies
    1. This update is available for Android only currently. We're waiting for new Gear VR to ship so we can test passthrough port to see if it works correctly with NOLO.

      Delete
    2. But where do i download the android version?
      Thanks

      Delete
  3. Holy.... Effin.... Crap...
    yall FINALLY did it...
    and its a thing a beauty.
    *clap clap clap*
    Took me a bit to make it work... but it was well worth it.
    A work of art. Played kingspray... wholly untethered, and i must say it was awesome.

    ReplyDelete
  4. Not sure if this is a NOLO + RiftCat issue or just a RiftCat + Revive issue, but I am having no luck getting Revive to work with NOLO Untethered + RiftCat.

    The only game that actually started up was Bullet Train and it wouldn't actually render / receive input from RiftCat.

    Anyone else had any luck with Revive + RiftCat + NOLO (WIRELESS, I saw videos of it working wired before) ?

    ReplyDelete
    Replies
    1. Try older version 1.1.6 of Revive.

      Delete
    2. @Unknown Hey just wanted to say thanks for the tip, it worked! I was able to get into Lucky's Tale and was about try Robo Recall with my NOLO before it stopped (maybe it's one game per Revive start?)

      I think for Oculus software I'll stick to using my Rift for now, but I'm glad it's at least *possible* to use the NOLO with it too :)

      Delete
  5. I never got the message asking if Vridge can use the usb device when I connected my headset, I just get a quick message at the bottom of the screen saying Nolo device connected, but I get no tracking and the Nolo VR option is not present in the tracking options of my mobile app. I also can't find where I can change app settings to try to manually allow permissions.

    ReplyDelete
    Replies
    1. I'm having similar issues. Can't get tracking working. Using my laptop as a hotspot (and running riftcat from the laptop). Not sure if this is related to the issue, since I'm not connecting both devices through a router.

      Delete
    2. I can nolo working wireless with vhui64 (Virtual here usb client) on my phone and the Nolo driver running on my PC, but I can't do anything about the phone permissions for vridge on my phone. Storage is the only option. I'm getting the 30 degree offset for the controllers bug also so I might just have to go back to non-beta anyway.

      Delete
    3. I misread the 30 degrees to the left in the blog, my controllers seems to rotate fine, its just that they are off to my left by quite a bit.

      Delete
  6. @Che Ortega look slightly right then recalibrate one of the controllers at a time (bottom button double tap). It's a NOLO shortcut

    ReplyDelete
    Replies
    1. Thanks for the tip, but unfortunately the controllers are still aligned off to my left. My right controller is about where my left hand would be and everything is spaced correctly (my left controller is way off to my left). The Beta has not been kind to me... I also can't find any permissions for usb access for Vridge in my settings. I've only seen permissions for storage which it is granted.

      Delete
    2. Have you tried swapping which hand the controller is in? also, try not to use the RiftCat double tap shortcuts (upper button) for now because they shifted my controllers off to my left as well.

      And hmm, try enabling USB Debug mode / developer options maybe?

      Delete
    3. Thanks again @Daven Bigelow, your advice you put on the Nolo Kickstarter page to change the desktop settings to phone sensors worked like a charm. I've got NOLO headtracking back. Now my issues are down to my controllers be way off to my left for some reason. That and I'm having an issue where my VRidge runtime will not work again after it's been closed. I have to do a full restart of my computer to get it to work again. I'm hoping that's due to it being in beta.

      Delete
    4. Try double tapping the bottom button on the controllers (one controller at a time) while facing forwards to fix the "to the left" issue. Also can try swapping hands!

      Delete
    5. I figured out how to get rid of the orientation issue. I'm not sure why its not in any of the instructions, but I was not aware that I should put the two nolo controllers next to the headset when I push the nolo setup button on the headtracker. I never had to do that before, but when I did that it worked. I read it in one of the nolo reddits.

      I tried double tapping, but the issue with double tapping the controllers is now apparently its the same as hitting the setup button on the headtracker, so when I double tap the buttons my POV gets instantly put onto the floor, so I have to take off the headset, put everything on the floor and hit the setup button on the headtracker.

      I'm also having a big problem with latency after a couple of minutes of starting up steam VR. Everything is fine at first, but then my riftcat display will be several second behind, while the mirrored display on my computer screen is exactly where it should be, but on my phone it is several seconds behind and gets even worse over time.

      The last issue I'm having is with the Riftcat runtime. It works fine the first time I open steam VR through riftcat, but if I close it, it won't come up again unless I restart my PC. I didn't have this issue when I was using the non-beta.

      Delete
  7. I am unable to get the wireless Nolo to work. As soon as I connect the head tracker to my phone the Nolo driver immediately disconnects everything else and does not recognize anything. I am using an lg g4.

    ReplyDelete
    Replies
    1. The NOLO PC Software should be closed when using wireless

      Delete
    2. This comment has been removed by the author.

      Delete
  8. ty for the nice updates.when we can expect the moonlight support any date?

    ReplyDelete
  9. I have been unable to get NOLO untethered to work... Perhaps it is because I am using my Laptops Killer 1435 Network card? would a USB dongle work better? I don't get how...

    ReplyDelete
    Replies
    1. What part of it isn't working? if your laptop finds the phone over WiFi (5GHz AC standard recommended) then it's not the ethernet port!

      Once your phone and laptop are on the same network (laptop can be plugged in by ethernet) and RiftCat finds it, that part of the setup should be okay! (assuming it streams well).

      The NOLO then should be plugged directly from headset marker into the phone using the OTG cable that comes with it.

      Are you using the latest beta software for RiftCat on PC and your Android phone?

      You'll want to sign up to beta test on the phone: https://play.google.com/apps/testing/com.riftcat.vridge

      Make sure any other software that controls USB devices (eg: VirtualHere) are uninstalled on your phone, or they'll block RiftCat.

      Then while RiftCat is open plug in the NOLO head marker and it should ask to open automatically when NOLO detected ,choose yes.

      Then on PC all you need to do is close the NOLO software and in RiftCat set it to Phone Sensors only.

      Turn on your base station and controllers if you haven't already, then you're good to go!

      Hope that helps!

      Delete
  10. My head tracker turns off right when I begin to stream and both controllers do not track. This is very frustrating. In the steamvr menu they are lit up green and seem to be detected though. I do not know what is happening.

    ReplyDelete
    Replies
    1. The head marker is directly connected to the phone, and you're not using software like VirtualHere on your phone? The RiftCat app opens when you plug NOLO into phone?

      Delete
    2. The head marker is connected directly to my LG G4. I am not using any other software. The vridge app does open up when I plug in my headtracker.

      Delete
    3. What are your PC settings? @Daven Bigelow pointed out to me that I needed to change my PC settings over to "phone sensors" under the tracking option and not freetrack anymore like it is for a NOLO usb connected setup. This make sense since the NOLO headtracker would be considered being part of the phones tracking in the wireless setup.

      Delete
    4. I got it to work finally. I reinstalled riftcat and it started. The audio streaming seemed to be broken though it was just high pitched static.

      Delete
    5. Glad to hear the streaming is almost working (shame about the audio, maybe there is a way to turn that off and plug wireless headphones into the PC?)

      Delete
    6. In the PC application, there is a checkbox for the audio streaming in the settings (Other Settings).

      Delete
  11. hi, I have problems with audio streaming. when i ebable it, my LG G4 goes into an infinite loop of disconnect/re-connect. if I disable audio everything works good. i also tried with my old moto G 1st gen and audio worked good, so is there any incompatibility with the LG G4? server version 1.5.0 and mobile version 1.5.2 on both phones. thank you.

    ReplyDelete
  12. get up moonlight fast, nobody can use your product for more than 10 mins, stop playing with extra features, first things first.

    ReplyDelete
    Replies
    1. Not sure what you mean "nobody can use your product for more than 10 mins "

      it works fine with HVEC and 5GHz wireless streaming over an AC network for me?

      Delete
    2. Ah can't edit, but for play times I get about 2 hours to 2.5 hours before my phone battery dies (a phone dedicated to VR so from 100% to 0%), which is pretty good since it's also powering the NOLO headmarker over OTG too and is a 2015 ZTE Axon.

      Delete
    3. VRidge works fine for hours at a time. I hope they don't waste time on extra features like moonlight when there's basic things that still ned to be worked on, like completing Nolo support and increasing the stability of the desktop software.

      Delete
    4. Agree on that one. Fix the laggy steaming issue first please! I can not advise anybody to use vridge/riftcat at this moment. You'll invest a lot of time in a system that just doesn't work properly. We NEED the moonlight update!!

      Delete
  13. So.. I'm going to give this one more try. Setup: S8,2017gearvr,i5,1080nvidia,4x cam. I had a Netgear Nighthawk X8(v1) but it was laggy as h*ll.
    Can anybody tell me what router I should use? It will be in the same room and not other devices will be connected.
    Thanks for the reply

    ReplyDelete