Dev update #33 - CPU optimizations and NOLO tweaks

We're deploying few tweaks and fixes. They are already available in Gear VR beta channel and should be available for all other Android devices later today.

Latest beta versions are:

• Gear VR: VRidge for Gear VR 1.5.0
• Android: VRidge 1.5.3 

This is mobile side update only. You don't need to update desktop clients.

---

Tracking update rate fixed

In version 1.4 we introduced a new networking protocol but the implementation wasn't optimized too well. It resulted in many tracking frame drops every second (up to 20). If you are used to 60+ FPS gaming it probably felt terrible because the tracking update rate was hovering around 40 updates per second.

CPU optimizations

We're also changing synchronization model to put lower load on CPU. It should greatly reduce heat generation and battery drain compared to v1.4. On average, your CPU load should be reduced by ~50% compared to v1.4. In case of Pixel XL or Galaxy S6, VRidge CPU load should now stay below 20-30% (measured with Android Studio + ADB).

NOLO wireless changes

• We're following an advice from NOLO and we're changing default tracking mode so the phone sensors are used for rotational tracking. You can still switch to NOLO IMU in mobile app settings but generally we found that phone sensors have lower tracking latency.
• We weren't aware that NOLO hardware has recenter built-in at the firmware level. See calibration paragraph below for details.
• We're removing nolo flip-around hotkey. It might come back in the future if many people request but currently it was causing too many problems with mixed tracking mode.

NOLO wireless calibration

1. Put your headset on, turn on controllers and spin them around for 10-15 seconds. It can take up to 10-15 seconds for the magnetometer to become stable. It will be probably in wrong rotation when it stops drifting.
2. Double tap system (bottom) button on both controllers. Re-center is per controller so you can recenter them individually. VRidge will also register this hotkey as "recenter" so it won't go out of sync.
3. [Optional] Press system button on any of your NOLO controllers and go to settings in SteamVR VR dashboard. Hit "reset position in there" to make sure you are at the center of playing field.

If you feel that your height is wrong or controllers are out of sync with your head rotation, run SteamVR room setup by right-clicking on SteamVR floating window and selecting "room setup" in there. You may need to run room setup again if your relative position to the base station changes.

NOLO wireless + VRidge for Gear VR 2017

Galaxy phones plugged into Gear VR 2017 can host both Gear VR and NOLO devices which enables fully wireless experience in this scenario.

1. Install VRidge for Gear VR 1.5 Beta (if you have installation problems, please see this article).
2. Plug in your NOLO into Gear VR with Micro USB - USB-C cable. You need to have it connected before you start VRidge.
3. Start VRidge for Gear VR app so it's on "Please insert your phone into Gear VR" screen.
4. Insert your phone into Gear VR.
5. [Just once if you select "always use this app"] You will be asked which app do you want to use with NOLO. Select VRidge for Gear VR by looking at it in VR and tapping your GVR touch pad. You can select "always use this app" to skip this step in the future. You can always reset this "always use this app" by long pressing VRidge icon to access Android app info and then clearing defaults in "set as default" subscreen. If your VRidge (Cardboard) is always launching first without an option to select GVR edition, you might need to clear its defaults in the same way.
6. Start SteamVR as usual in desktop app.

Things that are missing

We want to finish NOLO integration by implementing following things:

1. Vibrations.
2. Top-down tracking mode, or at least trying to do it with mobile SDK.
3. Throwing things. Currently controllers don't provide their acceleration / velocity in mobile SDK so throwing motion doesn't work properly in some games.

We need to work with NOLO devs to code 3&4 properly so we don't know which of these will come first. They are quite busy with their driver too so it sometimes takes a bit to get all the answers and changes we need. We can sort of do 3&4 on our own but we don't want to release it as a temporary hack without future-proofing it by working with NOLO devs directly. We want to be sure that our hacks are not broken by some future firmware update

In the meantime we're still working on a lot of things that are not related to NOLO, we should have more news in upcoming weeks.

Construct VR shutting down

Yesterday Construct VR announced that it's shutting down its platform. We are looking at other solutions to host VRidge for GearVR app but we don't have replacement ready yet.

We're very surprised to see this announcement. Construct VR team and platform was a pleasure to work with and we're really sad that their journey ends here. We wish them luck in their future endeavours. Your contribution to VR world will be remembered!

Dev update #32 - NOLO untethered

Today we are releasing version 1.5.0 to the beta channel. It comes with NOLO VR wireless support. It's not 100% finished but we wanted to start gathering some user reports to make the next version better based on your feedback.

Updates:
[1.5.2] - We're changing the way tracking status is interpreted. If you had tracking issues with 1.5.0, try again with the new version. It should be available in Google Play 7PM UTC at the latest.

Known issues:

• Tracking may have decreased send rate, resulting in more frame skips (up to 20 dropped rotation frames per second).
• Controller rotation and position may go out of sync in respect to head marker after recentering.

---

Nolo untethered

General info

In dev update 30 [http://blog.riftcat.com/2017/06/dev-update-30.html] we described reasoning behind implementing NOLO VR wireless mode. Today we are releasing first version of it to the beta channel.

There are few things that are not working correctly yet:

• There are no vibrations yet.
• Recentering may yield unstable results. You can turn controllers off and on again while holding them forward to make sure it resets properly.
• Sometimes controller rotation jumps ~30 degrees to the left. We're not sure if it happens to our hardware unit only. We asked for another set from NOLO to test it further.

We plan to continue our work to fix above problems. We should have another update ready for you next week.

Connecting

You need to update both desktop and mobile app to the latest beta (1.5.0).

To use NOLO VR in wireless mode simply plug it into your Android micro USB port with USB data cable. VRidge popup will ask you for permission to use this USB device. Check "remember my answer" and then tap "allow". From now on, VRidge will automatically start the moment you plug your NOLO VR device. You can later revoke the permission by going to Android application settings if you want.

Tracking modes

When NOLO VR is plugged in, you will have another option in VRidge mobile app settings. You can toggle between NOLO VR sensors and phone sensors to handle rotational tracking. Your experience may vary from device to device. If you find the NOLO VR rotational tracking to be laggy in your case, try switching to phone sensors and see if it's smoother if you use your phone as rotational tracker.

Keep in mind that "Phone sensors" mode is still experimental as it fuses data together from two reference systems to create final 3D pose. It might yield incorrect results (controllers could be backwards, yaw might be skewed, etc.) This mode needs some extra tweaking and we might drop it completely in the future if no one uses it. Let us know if you find experience less laggy in this experimental mode.

Usage

If you started VRidge while NOLO VR was connected it will work out of the box. You don't need to install anything on your PC. We added two key combos for extra convenience. These hotkeys don't work in experimental mixed tracking mode (with phone sensors as rotational tracker).

Double tap menu buttons (upper buttons) while controllers are next to each other to recenter position and rotation

Double tap menu buttons (upper button) while controllers are apart to turn around

Rotation is still being tweaked so you might get odd results sometimes. The threshold between "next to each other" and "apart" is 10cm.

Testing sensors

If you want to see raw NOLO VR hardware data you can long press NOLO VR option in VRidge settings. This option is visible only when VRidge was started with NOLO VR connected to USB port with granted permission to read USB data. It can be useful to test if your trackers drift or have some other oddities. Rotation is displayed as quaternions which are very unreadable structure from human perspective but all you need to know about them is the fact, that those numbers shouldn't change a lot when your controllers or head marker are not being rotated.

Blurry right eye (certain Galaxy devices)

We've been tracking this bug for a while but we could never reproduce it, which made it really hard to fix. Recently few redditors suggested that it's a problem limited to Snapdragon only.
Galaxy S7 and S8 come in two variants. US edition comes with Qualcomm Snapdragon chipset and almost all other countries get Exynos edition. We checked and it seems that we have ~10 confirmations total that it's indeed a Snapdragon only problem.
Now that we know that we cannot reproduce it with our unit we took some other steps to attempt to fix it. We will soon have upgraded Gear VR renderer. We're going to send test APK to few users that are willing to test if the new version still has the problem. We are also in contact with some people at Samsung. We are trying to create minimum reproducible test case so we can ask them to help with this problem. We will keep updating you on this topic. With all the additional details and help, maybe we will be able to fix this once and for all.

Update: It seems to be fixed in updated rendering framework internally. Next Gear VR update will no longer have this problem. Many thanks to all the people who volunteered to test our debug builds!

API v3

This beta update also includes new version of API which uses Protocol Buffers as serialization layer. We needed to move from default .NET serialization because it had few things that were not very portable (byte layout, paddings, endianness, etc.). We also created Java library that is used to communicate between NOLO VR devices and VRidge. After we clean up some code and write some docs, we will add it to our VRidge API repo as another way to communicate with VRidge. Making software mods and plugins is pretty cool and can result in awesome new things being created so we want to share with you the same tools and layers that we use when connecting external tracking systems like NOLO VR.

Under Fire VR

We recently hired new programmers. They are still learning their way around VR so they created a small game as an introduction to VR world. You can check it out here:

https://play.google.com/store/apps/details?id=com.Riftcat.UnderFire

It's pretty cool so give it a try if you liked Scorched Earth back in the 90s. Our new developers now moved to other tasks that are much more related to VRidge.

More controllers option

Thanks to our new developers and expanded API we are planning to release few additional controller/tracking options. The first one to come will be based on Xbox controller. It will allow you to move in roomscale game and emulate controller rotation and position with controller analog sticks. It's not very immersive but it may find its use if you are developing VR game without motion control hardware. It's also quite viable option to play some atmospheric experiences where you want to explore areas but lack necessary hardware to trigger some gameplay things.
More advanced tracking options are coming too but it's too early to share details about it.

Other changes and fixes

• We fixed a bug that prevented VRidge from working on Android O. It's working without any problems now on our Pixel XL with latest Android O (8.0, compilation id: OPP4.170623.014).
• Connection screen will now turn upside down properly if phone is held upside down.
• Settings screen will no longer freeze when SteamVR settings file is not writeable.
• Mobile tracking will now properly switch off when VRidge receives full pose matrix or rotation quaternion through VRidge API (GitHub #12)

What's next?

• We need to finish tweaking nolo wireless mode. This should be done in a week or two.
• We're still reworking majority of the codebase for VRidge 2.0 revamp.
• Our new developers are working on new tracking options.
• Once 2.0 is out we will focus on optimizng latency and smoothness, jitter reduction and iOS version.