Thursday, December 7, 2017

Dev Update #35 - VRidge 2.0 Beta is here!

This is the biggest update so far. VRidge got rewritten from scratch and it has to be released in a different way than our usual update method.

Remember that this is a Beta version of VRidge 2.0 - we need your feedback to fix all potential issues that might come up (there are over 10 000 different Android devices out there - it’s hard to test them all). Please send us feedback, suggestions and bug reports to - it is crucial if we want this update to become a success.

You will be able to track the progress of the bug fixes on this public Trello board: 

What you will find in the VRidge 2.0 Update?

Completely new user interface 

New design with user experience as primary objective. Registration will no longer be required to try VRidge!

Possibility to change quality while streaming 

Need to find the perfect setup? You will no longer need to restart VRidge endlessly to find most suitable settings. Just use LIVE streaming settings to optimize your VRidge experience (advanced live settings are also available).

Much better stability 

Completely new software architecture allowed us to make VRidge software much more stable. No more full software restarts when something fails.

Integrations support (easy add-ons to VRidge) 

We now have much better tools to integrate third party solutions. More on that soon!

Moonlight support 

If you have a modern NVidia card you can use Moonlight streaming instead of VRidge. You might receive better results in terms of latency and image clarity. To learn how to use Moonlight with VRidge - read about it in the Moonlight Basic Guide. 

Multi-language support

We have implemented multi language support. We support English and Polish languages for now. Tell us which languages you would like to see. More coming soon!

Some other stuff 

We won’t spoil everything. You can discover more changes by joining the Beta :)

Minor patches

  • Added diagnostic mode counters (
  • Added native error logging
  • Added custom FPS that can be set in config mid-stream (won't persist through restart, just for testing purposes now)
  • Fixed FPS setting not changing VR FPS which caused increasing latency
  • Fixed installer freezing at 80%
  • Fixed crash loop after switching encoders
  • Fixed Riftcat not being able to stop SteamVR if SteamVR was started as an admin
  • Fixed Oculus SDK games crashing when running without admin.
  • Fixed FreeTrack sometimes crashing app.
  • Fixed NOLO wired (desktop driver) positional tracking.
  • Fixed trial timer sometimes ticking down incorrectly (visual error only).
  • Fixed resolution sometimes not refreshing in UI in basic mode.
  • Fixed file lock problem causing both apps to lock up if error was encounter during settings change.
  • Fixed moonlight distort not being refreshed when .cfg changed.
  • Fixed background update check crashing RiftCat client when Internet went down mid runtime.

The instructions below will help you join the VRidge 2.0 Beta program. 

RiftCat Client (Windows side) 

The client got reworked completely. That’s why you need to install separate application to join the Beta. You can find a RiftCat 2.0 Beta installer HERE or by pressing the button below:
Install the client like any other Windows program. The installer is fairly easy to use and should get you up and running fast.

VRidge App (normal Android version) 

The mobile VRidge 2.0 application is also distributed through a separate channel. Please follow this LINK or by pressing the button below:
You will be prompted to install separate VRidge 2.0 application. Google Play will install the VRidge 2.0 App on your phone after your acceptance.

You can now connect this newly installed app with RiftCat desktop Beta client - enjoy and let us know if you encounter any issues.

VRidge App (GearVR version) 

The VRidge 2.0 application for GearVR can be found on Sideload. Just follow this LINK or by pressing the button below:
The installation process is the same as in normal installation of VRidge for GearVR. To learn how to install applications from Sideload, simply read this guide:

Friday, November 3, 2017

Dev update #34

In short

VRidge 2.0 will be released to beta channel in few weeks. NOLO update won't require hardware firmware update but will be delayed until 2.0.

Full version

NOLO update


When we released NOLO wireless update we estimated that we will be able to release another update in September that will contain rest of the features that were missing from the initial update. As you already know, we missed this deadline by two months. The following post is not an excuse because we definitely could have handled this better. Here's the story behind the delays.
We originally designed VRidge-API in early 2017 as a part of our shared vision to make NOLO work with VRidge seamlessly. Basically, it's an open interface that allows anyone (with some basic coding skills) to seamlessly intercept and manipulate VRidge tracking in any way. You can override rotations, positions, offsets, correct drifting, send controller data, etc. Everything will be automatically combined with whatever is sent from mobile phone in a non-conflicting way.
After it was ready, NOLO developers decided to take a new approach. They designed their own SteamVR driver that sends some of data directly to SteamVR (controllers) and some to VRidge (head data). This was released as NOLO windows driver delivered as their software installer package. This had some benefits and drawbacks. One benefit is making their hardware compatible with other SteamVR HMDs. The major drawback is that we had pretty much zero control over problems happening in this scenario. We were trying to support this mode both with support advice and some patches on our side but in the end - major part of this codebase was simply not accessible.
After some time, we decided to develop “NOLO wireless mode”. We changed VRidge API so it can be accessed over Wi-Fi. We then decided to create full NOLO windows driver equivalent on the Android side so we can control all data flow. This was supposed to allow us to fix bugs in a less hacky way. Open specification of VRidge API allows anyone (including NOLO or any developer) to connect ANY hardware connected to Android with VRidge. Our implementation was released with 1.5 for both Gear VR and Android.
Nevertheless, even with rewriting the driver on our side, we encountered quite a few challenges. The only way to communicate with NOLO hardware on the Android side is the NOLO Android SDK. We decided to use it but this has resulted in few minor and major challenges.


The initial release wasn't working at all for nearly all of the people
It turned out that we were developing using a version that was behaving in a different way than consumer sent units. After we reported this we were sent another hardware kit with CV specs.
Our fix: We corrected the comms shortly after it was reported thanks to few helpful users that sent us diagnostic data from their CV units before our new hardware kit was shipped.
Random controller button press events are triggered every now and then
This is caused by NOLO Android SDK returning data that is randomly simply wrong. Based on our experience - the longer the devices are turned on and the more of them are communicating at the same time, the more likely it is for SDK to send random button presses. It can manifest itself usually with random SteamVR screenshots being triggered. All button states are stored into single number encoded as a 32-bit bitmask. This bitmask turns into something like 10011011101101010111011001111001 (some random buttons pressed) instead of 00000000000000000000000000000000 (no buttons pressed). Screenshot combo requires only two bits to be set, so it's highly likely that this random number will land a screenshot combo.
Our fix: We added filtering that requires the same bitmask to be set for at least two consecutive frames. This added extra frame of delay but filtered nearly all of those random bit flips.
Our fix #2*: "Nearly all" was not enough because for some users it was still happening (silicon lottery – not all units work equally good). We decided to add a configurable filtering window size (3-10) frames to de-noise the button input.
Wireless mode wasn't working with ceiling mounts properly.
This was caused by lack of any configuration/setup with NOLO hardware. With Valve Lighthouse there's a setup stage where all Lighthouse devices communicate with each other to automagically prepare 3D representation of the room. With NOLO Windows driver you had to swap a DLL to trigger ceiling mode because there was no configuration step. You had to physically alter the code (the DLL) to switch it every time you made changes to your NOLO base station position.
Our fix*: We added a checkbox in mobile settings that triggers the celling mode by transforming the room coordinates from front-facing scenario to top-down scenario.
NOLO hardware is hard to use with multiple apps installed that can use NOLO hardware on the Android.
This is caused by Android SDK architecture. Most hardware products are split into driver and whatever uses the driver. For example, Vive has a runtime (SteamVR) and API (OpenVR). Multiple OpenVR programs can use hardware or act as games while SteamVR resolves all conflicts. The exact same scenario applies to Oculus Runtime/Home and Oculus SDK or even VRidge.exe and VRidge-API.
NOLO works in a different way. Driver/runtime and client/access layer is mixed into single Android binary (it's even more complicated on the PC side but that's another story). With Android it caused conflicts because you had to always ask Android OS to grant USB permission for NOLO hardware. It became quite annoying during development because clicking on the same box every single restart is not the best UX. We then decided to make use of "Remember this decision" checkbox so once VRidge was granted NOLO hardware access, it was always auto-started when it was connected with permissions.
This turned out to create another problem because Android doesn't allow two apps (for example VRidge and some native NOLO android app) to both have access to the device. You now have to go to app settings and tap "Clear Defaults when you wanted to switch NOLO-using apps (even VRidge and VRidge for Gear VR). This could have been resolved by some sort of authoritative NOLO runtime/service on the Android side and we hope that something like this will be created in the future to prevent app conflicts.
Throwing motion does not work in some games.
Throwing can be implemented in several ways. The recommended way for SteamVR games is to ask controller for its velocity and acceleration and transfer the values to thrown objects while detaching them from the controller's grasp. That's kind of how physics work and it is the most accurate way to create a throwing motion. Unfortunately, the NOLO controllers were not providing acceleration and velocity at all and therefore - throwing in some games resulted in grenades dropping right down your feet.
We won't go into details of the long story of multiple NOLO SDKs and firmware updates taking over a month of trial and error changes to make it work at the hardware level. Adapting to every set of changes took a lot of time and eventually - none of these changes were good enough to be released.
The technical root cause of most of the problems in this story is the fact that providing acceleration and velocity triggers SteamVR positional interpolation that makes the controllers movement smoother. Unfortunately, those values need to be perfect - otherwise it causes a lot of terrible jitter of SteamVR trying to adjust position and velocity that are not in sync with each other. As you may have noticed with all those delays - this did not work out.
The throwing motion was the most time-consuming thing that delayed the whole update. This has also delayed all other fixes marked with * because we were always switching SDKs and firmware and it always seemed that the final working version is just around the corner. Now that we still can't define how far “the corner” actually is, we decided to go another way. We are dropping the idea of relying on firmware update that adds velocity hardware data and do it our own way. It might be not 100% physics accurate but at least it will throw your grenades in the correct direction.
The challenges listed above are only several of the technical obstacles that we had to overcome.

NOLO changes release date

This is basically "long story short" version of the above paragraph in case you skipped over the explanation and only want the most important info from your perspective. The new approach to fix throwing problems will delay the nolo update until 2.0 update. We could have taken few days to reimplement it now but this will delay 2.0 version which is a priority since only few percent of VRidge users have NOLO devices.

VRidge 2.0


As explained before, the whole NOLO integration thing took way longer than we expected. It pushed back 2.0 ETA from Q2 to Q4 – that's a lot. Fortunately, we weren't focused 100% on NOLO. There were multiple periods where we waited for new 3rd party update and we took the time to continue working on 2.0. The current scheduled beta release date is late November, maybe early December. We still have few external events that might divert our attention for few days but we should be able to make it in late November.


2.0 is mostly about stability and creating a better groundwork for future improvements. RiftCat was initially developed as a VR downloader, "Steam for VR" of sorts. This was before SteamVR/OpenVR was even a thing so we had quite a few people using it and over 100+ games available for DK1/DK2. VRidge was created as an extra tool for people without real HMDs to experience those games. There was no positional tracking and no controllers back then, when we started working on it. It was pretty good plug and play solution that allowed you select Oculus game and just launch it.
Then things started changing, SteamVR became a major player in the VR industry, positional tracking became a standard, runtimes became much more complex, motion controllers were shipped. This caused a lot of refactoring on the VRidge side to adapt to the industry that was developing in a lightning speed. There are still hundreds of games that were great for DK1/DK2 but they are unusable now because they were not adapted to breaking changes of VR runtimes.
This caused a great deal of dirty hacks caused by quick refactoring. Those kinds of quick fixes stack up and slowly become a monster that is hard to maintain.


No more cascading crashes
We want to solve those problems by decoupling a lot of things that are dependent on each other now. We know that with current VRidge version one thing going wrong can cause entire chain of crashes resulting in a need to restart of VRidge, RiftCat, SteamVR, game that you have been playing.
Settings adjustable on the fly
Some settings will be changeable on the fly. Not all of the settings can be changed dynamically but some of them, like bitrate, can be adjusted easily without restating a thing. This will allow greater ability to find your best settings.
New Android app
Completely redone with new unified renderer designed for Daydream, Google Cardboard and Gear VR. One codebase that will automatically switch backends depending on your phone capabilities. We still need to test it on wider set of devices but generally the whole thing seems to run smoother than 1.x versions.
All settings on the PC
You won't need to switch between PC and mobile apps back and forth to adjust settings – all of them will be stored on your PC and sent to the mobile along with the stream.
Desktop mirror and new latency timings
This is a groundwork for time warp and “fake 3D” and latency optimizations. We're creating 2.0 as an expandable platform that we can continue improving with post 2.0 update. We created a desktop mirroring mode to test it with some high-framerate cameras. This mode will be disabled by default as it's currently serving debug and diagnostic purposes but we might leave an option to enable it if you want to test streaming only.
We also might share some of the 240 FPS recordings in the next posts that will come in few weeks because seeing “instant” screen refresh going slowly from the top to the bottom of the screen is very interesting.
Moonlight mode
We mentioned decoupling modules in 2.0 and this is one of the major features that will benefit from it. You will be able to run VRidge as tracking-only service and start NVIDIA GameStream that will stream properly formatted window to your phone running Moonlight.
HDMI support
The window mentioned in paragraph above will also allow you to send properly formatted stream to any display that you have attached to your GPU, including DIY headsets with HDMI passthrough. Tracking will have to rely on VRidge-API or VRidge running in tracking only mode the background.
New RifCat client
We're removing all the things that are very rarely used like game library/downloads. These games mostly serve as demos and a quick way to test. We still leave an option to run non-SteamVR old Oculus SDK games because there are some experiences that weren't ported to new runtimes and some people still ask for help with them through our support channel.
New client will automatically start VRidge in the background and begin pairing/connecting to be ready for the moment SteamVR is started. It will remember your phone, your settings and will take a lot less clicks to get VR running.
The client will also allow for an easier integration of “plugins”. For example, PS Move could be one of those plugins. We already have few of them in late stage of development, so you will hear more about those integrations post 2.0 release. Some of them might turn out as a great addition.

No registration required
Less registration forms and formalities. Just download it and start testing VRidge.
…and more
Those are the features that are scheduled to be released in 2.0 update. Most likely all of them will make it to beta release in few weeks unless we find some game-breaking bugs. Of course, as with every new release, new bugs will arise but we will continue directly updating beta app to make it stable ASAP (by the end of the year) and start working on the 2.1 in December/January.

Post 2.0

There will be two major focus points post 2.0 – time warp / latency optimizations and iOS. Both of them will greatly benefit from more adaptable codebase of 2.0 release so that’s why we wanted to finish it first.
We also want to look into ARCore. It’s available as preview SDK by Google and it seems that it has great inside-out positional tracking capabilities. We really want to know if camera feed decoding & processing will leave enough room for our VR stream decoding & processing. If it’s viable, it might greatly enhance mobile VR immersion. Moving in VR really adds a lot to the experience.
We still have fake 3D on our task board but optimization and iOS will be a higher priority.
We are really sorry that it takes us so long to bring you an update. Please, just give us a few weeks more and we will deliver. This will be the biggest update that we ever shipped for VRidge.

Wednesday, August 23, 2017

Dev update #33 - CPU optimizations and NOLO tweaks

We're deploying few tweaks and fixes. They are already available in Gear VR beta channel and should be available for all other Android devices later today.

Latest beta versions are:
  • Gear VR: VRidge for Gear VR 1.5.0
  • Android: VRidge 1.5.3 
This is mobile side update only. You don't need to update desktop clients.


Tracking update rate fixed

In version 1.4 we introduced a new networking protocol but the implementation wasn't optimized too well. It resulted in many tracking frame drops every second (up to 20). If you are used to 60+ FPS gaming it probably felt terrible because the tracking update rate was hovering around 40 updates per second.

CPU optimizations

We're also changing synchronization model to put lower load on CPU. It should greatly reduce heat generation and battery drain compared to v1.4. On average, your CPU load should be reduced by ~50% compared to v1.4. In case of Pixel XL or Galaxy S6, VRidge CPU load should now stay below 20-30% (measured with Android Studio + ADB).

NOLO wireless changes

  • We're following an advice from NOLO and we're changing default tracking mode so the phone sensors are used for rotational tracking. You can still switch to NOLO IMU in mobile app settings but generally we found that phone sensors have lower tracking latency.
  • We weren't aware that NOLO hardware has recenter built-in at the firmware level. See calibration paragraph below for details.
  • We're removing nolo flip-around hotkey. It might come back in the future if many people request but currently it was causing too many problems with mixed tracking mode.

NOLO wireless calibration

  1. Put your headset on, turn on controllers and spin them around for 10-15 seconds. It can take up to 10-15 seconds for the magnetometer to become stable. It will be probably in wrong rotation when it stops drifting.
  2. Double tap system (bottom) button on both controllers. Re-center is per controller so you can recenter them individually. VRidge will also register this hotkey as "recenter" so it won't go out of sync.
  3. [Optional] Press system button on any of your NOLO controllers and go to settings in SteamVR VR dashboard. Hit "reset position in there" to make sure you are at the center of playing field.

If you feel that your height is wrong or controllers are out of sync with your head rotation, run SteamVR room setup by right-clicking on SteamVR floating window and selecting "room setup" in there. You may need to run room setup again if your relative position to the base station changes.

NOLO wireless + VRidge for Gear VR 2017

Galaxy phones plugged into Gear VR 2017 can host both Gear VR and NOLO devices which enables fully wireless experience in this scenario.
  1. Install VRidge for Gear VR 1.5 Beta (if you have installation problems, please see this article).
  2. Plug in your NOLO into Gear VR with Micro USB - USB-C cable. You need to have it connected before you start VRidge.
  3. Start VRidge for Gear VR app so it's on "Please insert your phone into Gear VR" screen.
  4. Insert your phone into Gear VR.
  5. [Just once if you select "always use this app"] You will be asked which app do you want to use with NOLO. Select VRidge for Gear VR by looking at it in VR and tapping your GVR touch pad. You can select "always use this app" to skip this step in the future. You can always reset this "always use this app" by long pressing VRidge icon to access Android app info and then clearing defaults in "set as default" subscreen. If your VRidge (Cardboard) is always launching first without an option to select GVR edition, you might need to clear its defaults in the same way.
  6. Start SteamVR as usual in desktop app.

Things that are missing

We want to finish NOLO integration by implementing following things:
  1. Vibrations.
  2. Top-down tracking mode, or at least trying to do it with mobile SDK.
  3. Throwing things. Currently controllers don't provide their acceleration / velocity in mobile SDK so throwing motion doesn't work properly in some games.
We need to work with NOLO devs to code 3&4 properly so we don't know which of these will come first. They are quite busy with their driver too so it sometimes takes a bit to get all the answers and changes we need. We can sort of do 3&4 on our own but we don't want to release it as a temporary hack without future-proofing it by working with NOLO devs directly. We want to be sure that our hacks are not broken by some future firmware update

In the meantime we're still working on a lot of things that are not related to NOLO, we should have more news in upcoming weeks.

Construct VR shutting down

Yesterday Construct VR announced that it's shutting down its platform. We are looking at other solutions to host VRidge for GearVR app but we don't have replacement ready yet.

We're very surprised to see this announcement. Construct VR team and platform was a pleasure to work with and we're really sad that their journey ends here. We wish them luck in their future endeavours. Your contribution to VR world will be remembered!

Monday, August 7, 2017

Dev update #32 - NOLO untethered

Today we are releasing version 1.5.0 to the beta channel. It comes with NOLO VR wireless support. It's not 100% finished but we wanted to start gathering some user reports to make the next version better based on your feedback.

[1.5.2] - We're changing the way tracking status is interpreted. If you had tracking issues with 1.5.0, try again with the new version. It should be available in Google Play 7PM UTC at the latest.

Known issues:
  • Tracking may have decreased send rate, resulting in more frame skips (up to 20 dropped rotation frames per second).
  • Controller rotation and position may go out of sync in respect to head marker after recentering.

Nolo untethered

General info

In dev update 30 [] we described reasoning behind implementing NOLO VR wireless mode. Today we are releasing first version of it to the beta channel.

There are few things that are not working correctly yet:
  • There are no vibrations yet.
  • Recentering may yield unstable results. You can turn controllers off and on again while holding them forward to make sure it resets properly.
  • Sometimes controller rotation jumps ~30 degrees to the left. We're not sure if it happens to our hardware unit only. We asked for another set from NOLO to test it further.
We plan to continue our work to fix above problems. We should have another update ready for you next week.


You need to update both desktop and mobile app to the latest beta (1.5.0).

To use NOLO VR in wireless mode simply plug it into your Android micro USB port with USB data cable. VRidge popup will ask you for permission to use this USB device. Check "remember my answer" and then tap "allow". From now on, VRidge will automatically start the moment you plug your NOLO VR device. You can later revoke the permission by going to Android application settings if you want.

Tracking modes

When NOLO VR is plugged in, you will have another option in VRidge mobile app settings. You can toggle between NOLO VR sensors and phone sensors to handle rotational tracking. Your experience may vary from device to device. If you find the NOLO VR rotational tracking to be laggy in your case, try switching to phone sensors and see if it's smoother if you use your phone as rotational tracker.

Keep in mind that "Phone sensors" mode is still experimental as it fuses data together from two reference systems to create final 3D pose. It might yield incorrect results (controllers could be backwards, yaw might be skewed, etc.) This mode needs some extra tweaking and we might drop it completely in the future if no one uses it. Let us know if you find experience less laggy in this experimental mode.


If you started VRidge while NOLO VR was connected it will work out of the box. You don't need to install anything on your PC. We added two key combos for extra convenience. These hotkeys don't work in experimental mixed tracking mode (with phone sensors as rotational tracker).

Double tap menu buttons (upper buttons) while controllers are next to each other to recenter position and rotation

Double tap menu buttons (upper button) while controllers are apart to turn around

Rotation is still being tweaked so you might get odd results sometimes. The threshold between "next to each other" and "apart" is 10cm.

Testing sensors

If you want to see raw NOLO VR hardware data you can long press NOLO VR option in VRidge settings. This option is visible only when VRidge was started with NOLO VR connected to USB port with granted permission to read USB data. It can be useful to test if your trackers drift or have some other oddities. Rotation is displayed as quaternions which are very unreadable structure from human perspective but all you need to know about them is the fact, that those numbers shouldn't change a lot when your controllers or head marker are not being rotated.

Blurry right eye (certain Galaxy devices)

We've been tracking this bug for a while but we could never reproduce it, which made it really hard to fix. Recently few redditors suggested that it's a problem limited to Snapdragon only.
Galaxy S7 and S8 come in two variants. US edition comes with Qualcomm Snapdragon chipset and almost all other countries get Exynos edition. We checked and it seems that we have ~10 confirmations total that it's indeed a Snapdragon only problem.
Now that we know that we cannot reproduce it with our unit we took some other steps to attempt to fix it. We will soon have upgraded Gear VR renderer. We're going to send test APK to few users that are willing to test if the new version still has the problem. We are also in contact with some people at Samsung. We are trying to create minimum reproducible test case so we can ask them to help with this problem. We will keep updating you on this topic. With all the additional details and help, maybe we will be able to fix this once and for all.

Update: It seems to be fixed in updated rendering framework internally. Next Gear VR update will no longer have this problem. Many thanks to all the people who volunteered to test our debug builds!

API v3

This beta update also includes new version of API which uses Protocol Buffers as serialization layer. We needed to move from default .NET serialization because it had few things that were not very portable (byte layout, paddings, endianness, etc.). We also created Java library that is used to communicate between NOLO VR devices and VRidge. After we clean up some code and write some docs, we will add it to our VRidge API repo as another way to communicate with VRidge. Making software mods and plugins is pretty cool and can result in awesome new things being created so we want to share with you the same tools and layers that we use when connecting external tracking systems like NOLO VR.

Under Fire VR

We recently hired new programmers. They are still learning their way around VR so they created a small game as an introduction to VR world. You can check it out here:

It's pretty cool so give it a try if you liked Scorched Earth back in the 90s. Our new developers now moved to other tasks that are much more related to VRidge.

More controllers option

Thanks to our new developers and expanded API we are planning to release few additional controller/tracking options. The first one to come will be based on Xbox controller. It will allow you to move in roomscale game and emulate controller rotation and position with controller analog sticks. It's not very immersive but it may find its use if you are developing VR game without motion control hardware. It's also quite viable option to play some atmospheric experiences where you want to explore areas but lack necessary hardware to trigger some gameplay things.
More advanced tracking options are coming too but it's too early to share details about it.

Other changes and fixes

  • We fixed a bug that prevented VRidge from working on Android O. It's working without any problems now on our Pixel XL with latest Android O (8.0, compilation id: OPP4.170623.014).
  • Connection screen will now turn upside down properly if phone is held upside down.
  • Settings screen will no longer freeze when SteamVR settings file is not writeable.
  • Mobile tracking will now properly switch off when VRidge receives full pose matrix or rotation quaternion through VRidge API (GitHub #12)

What's next?

  • We need to finish tweaking nolo wireless mode. This should be done in a week or two.
  • We're still reworking majority of the codebase for VRidge 2.0 revamp.
  • Our new developers are working on new tracking options.
  • Once 2.0 is out we will focus on optimizng latency and smoothness, jitter reduction and iOS version.

Tuesday, July 4, 2017

Dev update #31 - sound streaming and networking protocol changes

Version 1.4 is an update of both mobile app and desktop app. It's not backwards compatible so you need to update both apps. It's currently available in beta channel.

This update includes two major features - sound streaming (off by default, see paragraph below) and networking protocol changes (designed to decrease visual artifacting).

Keep in mind that it’s a first iteration of those features so you might encounter game-breaking bugs and might need to rollback to stable. You might notice few more minor bugfixing updates on weekly basis in July while we fix reported problems.

There's no Gear VR version yet so keep using stable channel if you are Gear VR user.

Update (11 July): Gear VR beta version is now available here.

Known issues:
  • Higher bitrates may introduce higher than usual latency.
  • Streams may partially freeze (rotational tracking stops working) after few minutes. - Fixed in Android 1.4.1 / PC 1.4.3


Sound streaming

We've been delaying it for a while for multiple reasons. We wanted it work with the widest set of devices possible and it took some time to tune all parameters and codec settings to work on all of our test devices. It's first public release so it's highly likely that it won't work anyway on some phones that we didn't test so please send your feedback and bug reports to If you have problems with audio on your Android device, please follow the steps below:
  1. Go to Android app Settings -> Diagnostics and check "Enable diagnostics" in there.
  2. Reproduce the problem by streaming (or trying to stream) with sound for at least 20-30 seconds. Try few times, you can restart mobile app between tries.
  3. Go to Android app Settings -> Diagnostics again and tap "Send bug report". Enter your e-mail address so we can contact you and maybe ask some more questions that could help us fix it.
Currently sound streaming is disabled by default. In this state, it doesn't use any extra resources on mobile or PC side. If you want to enable sound streaming you need to go to PC app settings, expand "Other settings" and check "Sound streaming" box. It's currently buried this deep because it's first public release of this feature so we expect many problems and device specific issues. Once we polish it and fix reported problems, we're going to move it somewhere else in UI. We're changing this screen in 2.0 anyway so we might figure out a better layout for the settings.

Network protocol changes

This is major protocol change and a part of 2.0 changes that should make things better (maybe not initially, but once we tweak it properly). Generally, it should make streaming more stable (in terms of artifacting). This unfortunately breaks compatibility with older versions of VRidge so 1.4 versions of mobile app will require 1.4 versions of desktop app (and vice versa). We know that 1.3 mobile app doesn't work properly on some older devices so stick to stable version if you are using one of those: Galaxy Grand Prime, Galaxy S5 Mini, Blue Advance 5.0. We should have one of those devices at the office soon so we can check out what's wrong with 1.3 and GPU used by those devices.

If you don't want to be bored by tech details, you can skip rest of this section and read last paragraph only.

Until now we were using UDP datagrams to stream video data and tracking data. Adding sound streaming forced us to change our architecture so we can fit more channels (video and sound). Currently new networking layers is set to work in RUDP mode - which means that it will retransmit lost network data. This should mean that artifacts should be reduced by a great lot. Let us explain why.

Video stream is transmitted as a mix of keyframes (I-frames) and predicted frames (P-frames). Predicted frames mostly store differential information (e.g. "the bullet from previous frame moved 5 pixels to the right"). This uses much less data because only movement needs to be transmitted, not the full picture again. This has one major drawback though. If one frame is lost, all following frames that refer to it, will have some sort of artifacts because all next frames are based (at least partially) on information that wasn't received. It's even worse when an important I-frame is lost because it's followed by movement of things that do not exist yet in your stream. It may look like this:

It usually fixes itself in a second or two because new I-frame (keyframe) is received and it clears out all visual artifacts that accumulated since the missing frame. Digging down deeper, it's even more complicated because one picture frame usually consists of multiple (sometimes even a hundred) of network frames. If any of these frames gets lost, you get partially broken picture. This manifests itself with floating pixels and frames falling apart.

This is why we want to test how RUDP works out in this scenario. We're starting with RUDP enabled for all packets. Based on user feedback we're going to tweak it. There are few configurations we can use - we can use RUDP on per-packet basis so we can, for example, only mark keyframes as "critically important" or use RUDP for audio channel only.

Once we receive some feedback we're going to set some decent defaults and possibly make rest of it configurable. Some people prefer raw frames arriving ASAP even if one is broken every now and then. Some would prefer a 5 ms lag spike once every few seconds to get rid of all artifacting. We need your feedback to get answers to some questions: How does the new streaming protocol work for you? Does it make things better?

Let's find out - we're listening at

What's next?

  • We are working to update our API to v3 which uses Protocol Buffers instead of default .NET serialization which is kind of bad for cross-platform & language compatibility.
  • We continue our work on NOLO wireless mode. This should be out this July. It should exclude all potential lag and CPU spikes that could be caused by NOLO-SteamVR-VRidge interaction.
  • We recently got hold of Galaxy S8 so we want to test if the right-blurry-eye problem that some users report with S7 is reproducible with S8. Maybe we'll be able to fix it. 
  • 2.0 rework is still in progress but 1.4 protocol changes is one of the things that will play a major role in 2.0 so we want to test it properly with this beta update.
  • We need to figure out a way of deploying beta updates to Gear VR apps (we would like to have the same structure of stable/beta channels with Gear VR app too).

Friday, June 23, 2017

Dev update #30

In this blog post you can read about our recent developments in area of NOLO VR integration. We also talk about 1.3 which enabled everyone to integrate 3rd party trackers and included some SteamVR driver improvements.

We also try to explain problems that caused sound streaming (now scheduled to be released in beta channel on July 4th) and other updates to be delayed.


NOLO performance and wireless mode

There are two ways NOLO VR can work with VRidge. Currently only one is avaialble.

1. PC-based.

NOLO VR tracker is connected to the PC USB. It communicates with NOLO VR driver and runtime software. The NOLO VR software then communicates with RiftCat or SteamVR directly to provide absolute tracking data of headset and controllers.

2. Android-based.

NOLO VR tracker is connected to the Android USB OTG. In this scenario Android device is the host and it can communicate with tracker hardware to get tracking data of head marker and controllers.

Currently only #1 is implemented which is a bit limiting. The biggest downside is that you need to be cable-connected to your PC. We also don't control the software part of the process, we rely on NOLO VR software to tell us everything and we directly forward all the data into VR runtime. This is also limiting in terms of latency optimization because we don't communicate with the hardware directly. We also can't optimize CPU usage. We basically can't do anything other than listening to tracking data provided by NOLO VR runtime. Is has an ability to fully control rotational and positional tracking of HMD and controllers.

With #2 option we will be able to ask hardware directly for its sensor data and process it on our own. It should be lower latency and (depending on their hardware) very light on Android CPU usage. We haven't started working on it but we want to explore this scenario to find "the best" NOLO VR experience with RiftCat. This has a one downside though. The tracker will be connected to Android, making it fully wireless solution, but you lose ability to USB tether the stream connection.

We might try to re-implement #1 with a more direct approach (where our software queries their hardware directly instead of leaving all the comms to NOLO). It has a potential to be better in terms of latency and resource usage but wireless mode comes first.

Version 1.3 known issues

First days of 1.3 indicated that everything was going well. Shortly after we've started getting reports that some phones can't stream at all. It turns out that Galaxy S5 Mini, Samsung Galaxy Grand Prime, BLU Advance 5.0 (and some other phones using Mali 400 MP4 GPU) don't interpret video stream properly.

Those are not mainstream phones anymore but we had to start investigating the problem anyway because we don't want to leave people behind with each update. We still haven't pinpointed the problem and we'll probably defer the further debugging to 2.0. Version 2.0 will include many changes that include altered stream configs so we don't want to spend much time on this problem now because this will delay 2.0 which may fix it anyway. Version 1.0 of Android app is still compatible with latest desktop client (stable channel) so if your phone is affected - you may download and install it manually to keep using VRidge.

We also found that with 1.3 we broke our Android crash reporting & analytics. We use Google Play Services and its linked services to gather crash reports and related usage data. We rely on these reports to find out if changes we make work properly on all devices. With 1.3 we made crash reports mostly useless because of faulty configs. We tried to fix it and we now released 1.3.3 with Firebase reporting (also a Google service) that should be more reliable for crash tracking.

Virtual driver mode

As a minor change in 1.3 we changed our default SteamVR driver to use virtual display mode. It should have made the experience a bit more smoother. According to user feedback it indeed works better in this mode so we made it the new default. Your RiftCat config automatically switched to virtual display driver in 1.3 update. During the development we found another reason to add gamma slider so we'll most likely include gamma slider in 2.0 update. This will help users with screens that have less than ideal whiteness balance.

Beta & Stable

All of our updates are in stable channel now. This will give us more freedom in testing sound streaming, moonlight mode (and other 2.0 changes). If we break something in beta channel, users will still be able to revert to stable which contains everything we released so far.

VRidge API changes

We are making some changes to VRidge API for next API release.
Serialization will use Protocol Buffers. Reasons for this change are discussed here.
Remote API connections will work. Currently they are bugged (see here for a workaround).

Development status

We recently heard few opinions that we publish so many facebook posts without an actual updates on our side. Social media, support and development is handled by different people. If we stopped the social media activity it wouldn't change anything in terms of development. All the social media posts are our attempt to reach out to a wider audience. Maybe it results in an extra vridge license purchase a day, maybe it doesn't. We are not sure because measuring marketing is hard and we're still learning how to do those things without a huge budget. Coding (and this dev blog) is done by developers, social media is handled by other people. You can easily spot the difference in writing style.

Nonetheless, we still don't release as many updates as we would like to. We often get distracted from our main tasks by things that we cannot control. We rely on many components that often change outside of our control. Each Windows, SteamVr, GPU driver, Android OS (the list goes on) update we study the changes to make sure we will be ready once the update reaches user PCs. We are usually always ready to adapt to new changes but sometimes we need to drop everything and work on some emergency patches.

It's not always bad things that come from outside source. Sometimes a new game comes out that many user really want to try. We then try to make sure it's gonna work nicely not only with premium $600+ headsets but with our little tool too. We sometimes e-mail the developers to ask about their game, learn how its VR mode is implemented. Our goal is to not put any extra workload on the game developers (because they have their own goals - they gotta create the games we all love :). That's why it takes time to sometimes find creative ways to trick the game that "render please, we're totally Vive!".

Anyway, the problem with those things is that each time we drop current tasks to handle the "external event" (good or bad), it delays our plans a bit. Hours add up to days, days add up to weeks. It's also mentally distracting because we sometimes need to work on several totally different things in the same day. You don't need to be a developer to know the feeling that comes when you need to drop whatever you are currently doing to handle some outside interruption. It sometimes takes time time to get into creative zone again, to focus again after spending few hours on something totally unrelated to your original task.

We decided that we should try working with extra devs. We still don't know how well it will work out. We can't afford to transfer a senior developer with 20 years of experience @ Samsung or any other "big" company. They will be new to the system it's gonna take some time to teach them VRidge codebase and the ideas behind it. Nonetheless it's an investment that we want to try because VR is still rapidly changing and the more keyboards we can man, the more updates we'll be able to push. We already started getting our new friends familiar with VR, showing how VR experiences are made and what makes them work the best. We'll have more news in this area soon.

2.0 version will include Moonlight mode, HDMI output and will generally be more stable than currently released version. We'll keep updating you on its progress once we get back to work on it. As we promised earlier - "2.0" version name is only indicative of major codebase rework, you won't need to buy anything else - it will be a free update.

We're far from excited to write a blog post without any actual downloadable feature update but we hope that you can bear with us a little bit longer. Sound streaming is already merged into the main codebase and should be out it less than 2 weeks. After it's published, one developer comes back to 2.0 rework full-time and the other one focuses on wireless NOLO + some extra API changes to accommodate other controller types that are going to work with VRidge.

Friday, April 21, 2017

Dev update #29 - an update on our progress

This is text update only, we want to explain what we are doing and why we think it's a best way to do currently. 

VRidge core improvements 

Moonlight, stability & launch flow improvements, fake VR groundwork


VRidge was first designed as Oculus proxy interceptor that acted like Oculus DK2. Its work flow was quite simple because VR was simple back then. There were no controllers, no always-on services, no Oculus Home / SteamVR dashboard like environments. You started a game by running its executable and VRidge process lifetime was limited to game process lifetime.

We added SteamVR support exactly year ago, on 21th April 2016. It was first SteamVR-mobile HMD bridge back then and it helped us grew a lot. Then we continued adding more and more features and changes like UI slider for supersampling (which was later added by SteamVR itself), better support for controllers, FreeTrack, 1440p, Intel and Software encoders, Gear VR, partial Daydream support and lately - VRidge Tracking API.

The shape of VR has changed from headsets being simple 3D glasses to full room scale systems running multiple services in OS background. The problem is that the core of the VRidge remained mostly the same and if you ever worked on a big project, you know that when goals and requirements shift, the project maintainability suffers. The VRidge was designed with pre-release Oculus headsets in mind when there were no controllers or room scale lighthouse systems. And those legacy foundations slow us down a lot.

Core rework

That's why we are currently doing major rewrite of the VRidge core. It's actually going pretty good and we already made a lot of progress. In our development version VRidge was decoupled from SteamVR launch flow. We dropped that annoying chain of dependencies that required restarting all components (mobile, SteamVR, VRidge, Riftcat) when any of the mentioned components ran into some sort of problem. We can start and stop SteamVR/VRidge at any time and in any order - they will connect without problems. 

Another thing that rework enables is modular architecture. We can now substitute capture, encoding and output components during runtime. When we detect SteamVR connection being dropped, the stream automatically switches off to screen duplication stream. While it's not very useful at this moment (because VRidge doesn't have any Remote Desktop functions) it lays a solid foundation for future Fake VR. It can capture any window with very little overhead (<1% CPU usage, sub-frame latency, duplication tech stack very similar to Virtual Desktop, Bigscreen or OBS).

Another great thing that we already have working is a plug and play Moonlight support. It works as easy as possible. We have special VridgeView.exe that can be added to GameStream library in GeForce Experience panel. Then we simply start the VridgeView through Moonlight and it automatically connects to VRidge runtime and switches VRidge to fullscreen SBS output that is handled by Moonlight. And it works quite amazingly since as we said before - we think that NVIDIA GameStream + Moonlight pair is a "relationship goal" for any streaming software.

This also allows HDMI (or any video signal cable) output. You can use the VRidgeView.exe which will find its way to your chosen display and show VR on your device's display with no added latency. This is great thing for DIY fans that enjoy building their own HMDs.

We also want to unify Android rendering. Currently Gear VR and Android use completely different rendering engines. This makes it complicated because Daydream's immersive mode requires another VR renderer. We have a way to combine them all into one which would finally allow proper implementation of Daydream's immersive mode. We didn't want to write separate renderer for Daydream only because writing three things for three platforms that do essentially the same thing is a hell to maintain. With unified renderer, it should be a much less bumpy road.

We are also moving all settings to desktop app since adjusting some settings on the mobile makes absolutely no sense. Trying to adjust FOV with your headset off is an annoying guessing game.

Let's not forget iOS and possibly other mobile clients. This modular rework will remove a lot of spaghetti netcode which will then make it easier to port the mobile app over to iOS.

To sum things up, here's our current progress of the most important components:

☑ SteamVR driver connectivity rework (plug and play startup).
☑ Moonlight streaming support.
☑ HDMI output.
☑ High performance desktop capture engine for the future.
☑ Encoder quality settings adjustable on the fly.
☑ Encoder migration: NVENC.
☐ Encoder migration: MF (including AMD, Intel QS, Software).
☑ Network output
☐ Android network connectivity.
☑ Android decoder (not changed). 
☐ Android unified renderer. 
☐ Android background mode (for Moonlight mode tracking).

Scope of the work above is enough to call the reworked release VRidge 2.0 but those are only numbers without any other meaning than a label (no price changes or anything, everyone will get this update).

Since it's a major rework we cannot provide a testable version because Android connectivity wasn't migrated yet. Nonetheless we hope that this will allow us smoother development after most of legacy code is cleaned. During last year VR matured with a release of SteamVR and Oculus CVs. Controller interfaces are pretty well documented. VR environment APIs are pretty stable. With a more solid set of VR standards we are confident that we won't repeat the story pictured below:
by: Manu Comet

Sound progress

We planned to release it faster but we ran into some problems during testing on other machines. As it turns out - different sound chips use different audio formats so we are still testing and adapting to new motherboards/sound cards that we can find. After it's released to beta channel we will probably find some more bugs with exotic sound devices but that's what the beta channel is for.

SideloadVR & ConstructVR

Apparently SideloadVR is still online so we are going to update our Gear VR app on SideloadVR to match ConstructVR's latest version. It should be updated soon.

VRidge API changes

We are making some changes to VRidge API for next API release.
1. API call to re-center view.
2. Haptic feedback response for controllers.
3. A way to mark HMD trackers state as "out of tracking range".

Those changes were requested by 3rd party device manufacturers. The changes will be reflected in next week's API release on VRidge API GitHub.

Update release schedule

  • VRidge API will be updated by the end of next week.
  • Soon everything in current beta channel will move to stable channel. We want our stable version to be compatible with NOLO VR hardware when Kickstarter backers receive their units. 
  • Sound and protocol changes will be deployed to beta as soon as they are release-ready. Most likely it will be released in early May.
  • Core rework is estimated to be released to beta by the end of Q2, hopefully. It's our biggest change since release, but for a good reason.

Monday, March 27, 2017

Dev update #28 - VRidge tracking API

Vridge API is out in beta. Networking and audio changes are nearly done. More exciting features coming in next months.


VRidge API out in beta channel

VRidge API is a way to interact with VRidge tracking internals. This is the first step in making our platform more compatible with third party software and hardware solutions. We start with two endpoints that will allow you to take full control of head tracking system and controllers.

You can check out the example implementation and desktop app sample on our github here:

API is versioned so your plugins and drivers should work even after we alter the protocol. Perfect backwards compatibility is something we cannot promise for sure but we designed the API in a way that should allow easy iterations - we have several ways to keep both new and old versions of API channels running side by side. Backwards compatibility will be our priority when we make changes to the API.

We start with two endpoints that can be used separately:

Controller endpoint allows you to send VR motion controller state without writing a full OpenVR driver. This allows a more stable experience since only one driver is loaded in SteamVR. You can send multiple controller states, move them around, press and touch buttons and interact with touchpad. You can also use any language of your choice to send driver states as long as you send properly formatted packets.

Head tracking endpoint allows you to control head tracking in a variety of modes. You can use it to provide positional, rotational or combined data. You can also read mobile sensor data and provide an offset. If you find it necessary, you can modify phone tracking data in real time before it's used for rendering in VR.

We already have plans to expand API further to create a way to expose compressed or raw video output. Ultimate goal is to have everything pluggable/modifiable. VR hardware and software world is very fragmented now without a clear standard that has to be followed*. That's why we want to make parts of VRidge modifiable by controller and HMD manufacturers.

*OpenXR by Khronos is a good attempt to bring it all together. We hope that it succeeeds because with OpenVR, Oculus VR, OSVR and all the bridges in between them - the current state of VR SDK is somehow chaotic.

In the future we will also start moving more and more of our comm protocols into the open API because it allows creating 3rd party apps. We want to keep improving our Android app and eventually release an iOS app but if someone wants to create an alternative app specifically crafted for their purposes - we want to support it. There are multiple reasons to do so. Maybe you need a branded app for your company. Maybe you want to demo your architectural designs with some extra options. Or maybe you want to try to hack a way to run VR on TI-84 (after seeing Quake on oscilloscope or Symmetra being played with a microwave we believe that everything is possible).

While hacking and modding is fun this is our ultimate but not immediate goal. Our next plans are another round of performance/user experience improvements.

What's next?

Protocol, audio streaming, stable channel update (April)

There isn't much to say about it. The work on audio and protocol changes was suspended while we were trying to finish the API. It will be probably our next release since it's mostly complete and just needs some testing. Next blog post will be most likely focused on those changes with audio & protocol update being available in beta channel. We will probably push current beta to stable channel then too.

Smoothness, performance, latency - Moonlight & HDMI (Q2)

We tried to tackle it several times and there's still unresearched option of time warp providing significant improvements here. Trying to create the greatest VR streaming experience would be easier if we had something to compare to. That's why before we start working on time warp (which can improve the experience but we don't know it for sure) we want to create something that should have been added a long time ago.

NVIDIA Gamestream combined with open source Moonlight project is a great streaming solution. Probably the best one we've seen. We want to support it with VRidge app acting as a background tracker connected to VRidge runtime on the desktop which translates mobile tracking data into HMD tracking data and HMD frames into Moonlight & Gamestream compatible data. We want it to serve as a point of comparision of our own streaming.

Once we have Gamestream & Moonlight as a reference of perfect streaming, we want to improve our own streaming and tweak it for VR.

AMD users - don't worry, our streaming will still be available and improved in the future. We just want to take an advantage of pre-build solution because Team Green is large majority of our userbase. And the streaming is already there, we just need to hook into it.

While we are reworking rendering output we want to add rendering to HDMI connected displays* too. We already have render-to-screen working internally because we use it for development (finding bugs is easier when you can use your 2nd screen instead of phone). We just need to wrap it up in nice GUI switches and add some stability to it.

*You can substitute HDMI with DisplayPort/D-SUB/DVI. It will also have an option to simply render to window.

Less restarts, more stability

We all hate the "Your PC is going to restart in 5 minutes and there's nothing you can do." messages. Especially when you're in the middle of ranked match and you don't want your teammates to lose and your MMR to drop. This kind of experience ruins the fun and we want to address it in VRidge too. Current runtime flow requires restarting nearly everything (mobile app, SteamVR & game, VRidge, sometimes RiftCat dekstop app) every time even the smallest thing go wrong.
  • Connection interrupted? Sometimes you won't be able to reconnect. Gotta restart everything.
  • You want to change bitrate? Restart VRidge, SteamVR, game, mobile app.
  • USB cable disconnected? You need to restart everything. Oh, and Riftcat dekstop app crashes too.
Combine this with fragmented settings (some options needs to be configured on the mobile side, some are configured in the desktop app) and questionable design decisions (changing IPD & scale requires taking the phone outside the viewer).

We want to fix as many of these. Our goal is for VRidge to run always and connect with mobile app and SteamVR in a hot-pluggable way. Some settings would still require restarts but we want to limit those to situations when it absolutely needs to happen. We also want to move all settings to desktop app.

Minor tweaks and other features

We will have smaller tweaks and some new motion control options available in the meantime too. Moonlight, HDMI and anti-restart updates might take a while to be released (late Q2 is the most likely possibility). That's why we will try to squeeze in some extra side options that were prototyped as side projects but we're not ready to talk about them yet.

Fake VR is still on the table but performance has higher priority since over 50% of our feedback survey requests can be answered by having Moonlight as an option. iOS app will have to wait until we complete moving our protocol into open API.

Friday, February 24, 2017

Cloudflare security incident statement

Few hours ago Cloudflare informed the world that they discovered a security problem that could result in sensitive data being leaked. You can read it all over the news.

Cloudflare is used by over 4 million websites across the Internet, including Medium, YCombinator, Bitpay, Patreon, TransferWise, Coinbase, Authy, Okcupid, Zendesk, Curse network, Discord, among many other big websites. You can download compressed list of all potentially vulnerable websites here.

RiftCat also uses Cloudflare but we received an e-mail from Cloudflare informing us that they didn't find any of our data leaked:

Fortunately, your domain is not one of the domains where we have discovered exposed data in any third party caches. 

The leak isn't RiftCat specific. Any* of the users visiting any of the 4M websites could have seen part of someone's else data in website source. For example, while visiting there was an astronomically small chance of someone's Uber/Fitbit/Okcupid** password being hidden in page source (and the other way around).

Chances of your sensitive data being leaked from the affected sites are very, very small. According to Cloudflare one out of 3 million requests (0.00003%) contained hidden parts of someone's else request[1]. Those hidden parts were probably some sort of generic web traffic because most web traffic is just people browsing pages and not entering their passwords. It was also random and hidden by nature of the problem so it's unlikely that someone could use this as an attack on any of the services listed.

An analogy of this event in real world scenario would be a 0.00003% chance of finding totally random item from someone's else house under your door rug. If you haven't looked under the mat during this unlikely event, it was gone (unless it landed under bot's door rug which is also quite rare)

Most of publicly cached mixed pages are already removed. Cloudflare found 150 page requests publicly available in web caches.

RiftCat data was not found in any of the publicly leaked data. 

Nonetheless you might want to change your passwords around the Internet if you want to minimize your chances of being compromised. It's highly recommended that you don't ever use the same password twice. Stay secure.

*It might be limited to users using Cloudflare scrape shield only but it's not confirmed.
** According to TechCrunch they were confirmed as affected. See [3]

Further reading:

Thursday, February 9, 2017

Dev update #27

This post is text only without downloadable updates available in Riftcat client. We want to update you on our current work and talk about the future.


Sideload VR replacement

January ended with very unfortunate news. Sideload VR - a great source for Gear VR prototype apps - announced that it's closing down. You can see Sideload's author reasoning in the announcement post here.

We needed a replacement. First we thought that we will have to write our own repackaging service but fortunately such alternative to Sideload VR already exists. From now on you will be able to find Gear VR version on ConstructVR.

Desktop client links will be updated with next downloadable update.


Slightly over a week ago NOLO VR launched their Kickstarter campaign. We tried the hardware, we have units at our office and we can surely say that it's not a vaporware. It works just as advertised. There is no fake camera work or pre-rendered animations.

We are in touch with NOLO VR team and we are currently working to prepare a software side on our end so their hardware can work out of the box with VRidge. We tried Leap Motion, PSMoveService and other experimental drivers but NOLO VR was definitely the easiest and most reliable of them.

Their goal is to provide end user experience of nearly Vive-level tracking. It uses the same underlying technology (Lighthouse) so the tracking is very precise indeed.

Our goal is to provide a software support where you simply plug their hardware in and it just works. We don't want to make you install multiple layers of FreeTrack/FreePie like software relays. Simply plug it in, start VRidge and enjoy the next level of streamed VR.

VRidge API

While we are implementing NOLO VR support we decided to expand this into another component of VRidge. You can already do a lot with FreeTrack and OpenVR drivers so one could argue that we don't need another standard.

XKCD: Standards
But if you actually tried to use FreeTrack, custom OpenVR drivers and VRidge all together - you might have noticed several problems - black screens, different coordinate systems mixing up, etc. We are now working on providing a unified interface to keep everything loaded as one driver in one 3D space. Here are some key features that will be available with VRidge API.
  • Writing positional or positional and rotational data to override VRidge tracking. Similar to FreeTrack but with an ability to switch modes in real time without restarting VRidge. 
  • Reading and writing all VRidge tracking data before it's used to render a frame. You could use external sensor data and combine them with VRidge provided mobile sensor data to combine both data streams into a more precise measurement. It works in real time with ~0.1 ms response time.
  • Reading data and providing an offset. You could use external sensors to compare its data with phone measurements and provide an offset that corrects drift.
  • Writing OpenVR controller 3D position and button presses with an ability to choose the origin point of 3D tracking.
  • A way to do all of the above remotely. Wireless controllers won't need any desktop relays as long as they have some sort of wifi connectivity. It enables quite creative ideas of using mobile phones as motion controllers, similar to Daydream controller emulation.
The API is currently using ZeroMQ as a protocol. This is industry tested standard used by (according to their website) AT&T, Cisco, EA, Los Alamos Labs, NASA, Weta Digital, Zynga, Spotify, Samsung Electronics, Microsoft, CERN, among others. It allows developers to connect to VRidge API in any programming language easily. It doesn't matter if you use Python, Java, C#, C++, Objective-C or any other supported language (full list here).

We plan to provide samples with sources in Java (Android) and C# (WPF). We want to create a direct connector between VRidge and PSMoveService too because it's very popular and can serve as another example.

Audio and network protocol changes

Audio streaming is done but we decided to refactor networking a bit. Until now we used raw UDP datagrams with frame data but while we were adding audio stream we decided that it's time to upgrade to rUDP. It should reduce artifacting on 2.4 GHz networks. There's no reason to reinvent the wheel so we decided to use ENET. The same library that is used by Moonlight + Gamestream. We hope that retransmission will allow us to smooth out the experience further

Survey results and our next steps

According to our survey 80% of you noticed an improvement when it comes to smoothness in 1.2 update. This is a very good news and it motivates us to optimize further now that we can see that users actually notice not only features but the behind-the-scenes data flow optimizations too.
Here are some more facts:
  • One out of the three people use Gear VR.
  • Majority of people found that HEVC improves the exprience.
  • The most common video cards are GTX 970, 1070 and 1080. Only 20% of respondents used AMD cards.
  • Only 15% of users use WiFi 2.4 Ghz. WiFi 5 GHz and USB tethering are equally popular.
Last but definitely not least - the most important question of all of them. We are creating this software for you and your feedback is the most important factor in our decision making.

You can see raw data above. This is how we see it.
  1. You want a better controller support. We think that VRidge controller API described earlier will be a step in the right direction.
  2. Second most popular option is Fake VR. It was discussed earlier and we already did some work in this area. We plan to resume working on it after controller API, sound streaming and protocol changes are out.
  3. Third is jitter reduction. We hope that improved networking protocol will reduce it further but we still have timewarp on our task board.
  4. Reducing latency. This is a tricky one but time warp has a potential to reduce perceivable latency greatly.
Other answers (10%) contain many sound streaming requests which is already nearly complete.

We now want to finish controller API (#1), sound streaming and networking changes. After we're done with it, most likely one developer will focus on Fake VR (#2) and the other one will start prototyping a time warp (#3 and #4). As an additional step to improve on option #3 and #4 we want to add Moonlight compatible mode eventually.

We recently discussed Moonlight mode on reddit. We want to try time warp first because of reasons described in the discussion but Moonlight & Gamestream are always in our minds as a great example how to stream things perfectly.

We hope that this answers your feedback. Of course all of this can change because we can't predict the future but we hope that no major obstacles appear and we can actually deliver features you want the most.

Thursday, January 12, 2017

Dev update #26

This update took us much longer than expected but we finally believe that we made an improvement that is noticeable and worth giving a shot if you experienced terrible stuttering on your Galaxy S6-7 and some other phones.

We also bring some long term bug fixes like 30 and 60 FPS staying at the actually 30/60FPS, some Redmi Note and Zenfone fixes and several SteamVR improvements. There's an experimental version of HEVC too.

This update is released as version 1.2 (both mobile and desktop) through beta channels. Go to Riftcat desktop settings > Updates and get beta version there if you want to try it out.

Gear VR edition will be updated early next week if no major bugs are reported. We don't have beta channel on Sideload VR so we want to test it for few days on regular Android devices because it's much easier to rollback to stable version with Google Play.


Jitter / stutter improvements

TL;DR: It should be better now, more improvements will come in the future.

This took us a while. The problem first appeared when we released Gear VR edition in September. Some users were reporting various levels of stuttering and frame skips. We wanted to fix it quickly but after we investigated the problem it turned out that the underlying cause was really complex.

We first suspected frames being out of order but after high framerate recording it turned out that it wasn't true. We then investigated frames lost on the network but after adding some extra metrics we found that it wasn't packet loss. All frames were arriving on the phones yet the overall feel was very jittery.

We then started adding metrics and timings for every part of the virtual HMD pipeline. There are multiple places where VRidge needs to wait for something (game, steamvr, encoder, network transfer, decoder, Android vsync, etc.). If any of these components lags for even few miliseconds - it throws the whole thing out of sync.

We found that most of the microstutters came from network and Android decoder. Network micro lags are reasonable because that's how networks work. Android decoder optimization is a tougher nut to crack. It's performance is dependant on installed hardware and there are hundreds of chipset variants. The most annoying thing about is its inconsistence when it comes to decode times. For everything to be fluid, everything needs to take roughly the same amount of time every frame. Decoding time was varying greatly from few miliseconds to 12ms (with random spikes as high as 40). This kind of randomness is very bad for fluid playback.

Usually it's countered by buffering but it's not an option when we are trying to achieve VR-bearable levels of latency. We can't buffer even a second ahead to counter random spikes of interference.

Previously our FPS was stuck at almost 58 FPS. Some people suggested to turn it up to 60 and see if it fixes the stuttering. It didn't because the problem wasn't amount of frames per second. Generally to achieve fluid motion you need both high FPS and proper synchronization and intervals between them. An average of 60 FPS isn't good if sometimes the frames comes in burst of 2-3 at the same time with 30-40ms breaks after.

We decided to rework most of the timing and synchronization that was controlling capture, encoding and decoding. New frame rate is using better methods that combine multiple methods of timing and synchronization into one loop that is now much more accurate. Previously we had an average of 58.582 FPS in our capture and encoding module. Now it's running at 60.073 FPS with proper and stable intervals. This alone should lower deviation from 60 FPS from 2.5% to 0.1%.

Then we decided to rework Android decoding & rendering. Previously if no new decoded frame was ready at the beginning of the render cycle it would skip a frame. We now wait as much as possible for decoder to finish its work before giving up on the frame. You can control it by going to settings and changing frame timing:
  • Prefer low latency (new default) will try to keep the latency minimum but still wait as much as possible for decoder to finish its work.
  • Balanced will wait and buffer up to 1 frame if inconsistent decoding is detected.
  • Prefer smoothness will buffer up to 2 frames.
There are 2 legacy timing modes that are generally inferior to Prefer low latency but we are leaving them because there are 500+ different phone models running VRidge (according to Play Store data) and new modes might not work for someone.

We might make this even more configurable but please give us feedback if you feel the difference between those new timing modes and old ones.

With new timing we also improved 30 FPS mode for low end devices. Previously it would render the game at 60 (58.5 actually) FPS and skip every other frame during encoding. Now 30 FPS mode will slow the game down to 30 FPS. This saves up some CPU and GPU load and allows game to run in a smoother fashion instead of rendering the frames that were being dropped anyway.

This problem was happening across all devices but it was mostly noticeable on Gear VR edition and Google Pixel which use low persistence displays. It didn't look too bad on regular screens because it was hidden by micro blurs and smudges which are common among the regual 60 Hz displays.

This is not the end of our work to counter the stuttering. There are still frame skips and some of them are not avoidable. We cannot control a game that might drop a frame, we can't force network to give 100% of its resource to VRidge and we cannot make background Android apps drop all of its work and give 100% of device power to VRidge. Vive counters this with async reprojection. Oculus has great async timewarp and spacewarp. These technologies increase smoothness greatly and this is something we eventually want to do too. It's not an easy task and it's especially harder because we have to account for variable network latency but this is our goal. It won't come this month or the next one but our goal is perfect smoothness and we want to comes as close to it as possible.

Previous version
New version:

It might be hard to notice on the recording (you can click samples above and slow it down with gfycat speed controls to emphasize frame skips) but you should be able to feel the difference in VR.

VRidge for Gear VR needs an update for those improvements to take effect (sometime early next week).

Google VR SDK update to 1.1

We updated VR backend on Android to the most recent version. It enables some new features but it has some changes you might not like. Google decided to force its UI to be always on so we cannot dim the buttons and middle-screen divider anymore. On the bright side it's a bit dimmer now and it shouldn't be visible through lenses anyway.

New 1.1 SDK brings some new features too. Daydream-ready devices should take adventage of new async reprojection. It's not accounting for network latency but it should lower the perceivable latency by a small amount. Every little bit helps so we are leaving this enabled by default but you might turn it off in app settings by toggling display stabilization on/off. For non-daydream devices this option enables old "shaky" stabilization which works in a different way but it also may lower the perceivable latency too.

Daydream phones will also enter into low persistence VR mode with VRidge increasing smoothness greatly.

Daydream view and controller support is still not available because it requires much more work. We received our Daydream View only few days ago so we didn't have much time to integrate new features. Stay tuned for more Daydream news in the future.

SteamVR improvements

Sometimes we get e-mails about certain SteamVR games not working with VRidge. We always investigate those reports and try to bring compatibility to most titles available on the market. Some people reported Serious Sam VR having terrible framerate which was result of a interaction between the game itself and SteamVR which was happening on 3rd party HMD drivers. We reported those issues to SteamVR and Serious Sam VR devs and the problem was resolved shortly in SteamVR beta and now is also fixed in SteamVR live channel. This also solves Unity 5.5 FPS issues with VRidge. Again, we cannot be grateful enough to SteamVR devs taking all reports and suggestions from both game and HMD developers seriously and solving them very quickly.

Sometimes it's up to us to fix our bugs and add missing features to bring better compatibility. This update brings virtual presence sensor which should prevent some games from pausing thinking that HMD was taken off. We also add opt-in virtual positional sensor. Some games were simply refusing to work without some sort of positional tracking so you can now enter Riftcat desktop settings and turn "SteamVR Fake Sensor" on.

We also updated our HMD driver to the neweset OpenVR standards and fixed one minor memory leak which could end up increasing RAM usage over few hours to gigabytes.

If one of the games below wasn't working for you with VRidge, you can try again today with new beta version: Serious Sam VR, Edge of Nowhere, Herobound and Pinball FX 2 VR.

High Efficiency Video Coding

We're also adding HEVC as an opt-in feature. It's still highly unstable on certain devices so it's not enabled by default. It's currently avaialble only for Media Foundation encoder and GeForce 960 or newer GPUs. It also requires relatively new phone. It works alright on our Galaxy S6 (Exynos chipset).

It doesn't work correctly on our Pixel XL. When the phone receives HEVC frames it simply crashes the phone completely causing a full device reboot. It seems that there's a bug in the hardware codec because no software should be able to cause a hard crash like that. We wlll investigate this further and report it to as a bug if it's not fixable on our side and no workarounds can be developed.

VRidge for Gear VR needs an update for HEVC to work so wait until next week if you want to try it.

Redmi Note 2 and Asus Zenfone 2 fixes

We found a bug that was crashing decoder on Redmi Note 2. It should no longer crash with stream fixes enabled. We also applied workaround for Zenfone 2 suggested by Moonlight dev (thanks!). Intel chipsets should now work correctly on both Android 5.0 and 6.0 without 3s lag.

Minor changes

  • Disabled stream adjustments for Google Nexus 5 and Samsung Galaxy S5. It was sometimes crashing the app.
  • Display stabilization is now toggleable in mobile options. It's a matter of preference so you can now turn it on or off. It's on by default (it was always on before).
  • Changed IPD & Scale adjustment settings icon. New Google VR UI icons cannot be disabled so we had to use something else.
  • Increased maximum IPD by 50%.

What's next?

We are still working on sound streaming and we want to get back to "Fake VR" so you can stream regular games into your Android phone. We suspended work on both to bring improvements described above but we will now be able to come back to new features.

We also have a version that works on some Android 4.x  devices. We asked you to e-mail us if you want to test this version and we have a group of people willing to test it before making it avaialble for everyone. If you sent us an e-mail about it - sorry for not getting back to you earlier. Android 4.x was delayed by this update unfortunately. We will reply to you with a testable version link as soon as we are done making this update "stable enough".

We want to make above changes stable first so please give us your feedback and send bug reports. We have several of the most popular phones among our users and we test all changes on them but it's impossible to predict behaviors on all phones on the market. You can send e-mails to or send us a message or comment through any of the social media channels (Facebook/Twitter/reddit). We always read everything and while it may take some time to respond to everyone - we are not ignoring any of the comments.


Post edit history:
13 Jan 00:00 UTC - Added clarifications about VRidge for Gear VR requiring an update for HEVC and jitter improvements to work.