Dev update #26

This update took us much longer than expected but we finally believe that we made an improvement that is noticeable and worth giving a shot if you experienced terrible stuttering on your Galaxy S6-7 and some other phones.

We also bring some long term bug fixes like 30 and 60 FPS staying at the actually 30/60FPS, some Redmi Note and Zenfone fixes and several SteamVR improvements. There's an experimental version of HEVC too.

This update is released as version 1.2 (both mobile and desktop) through beta channels. Go to Riftcat desktop settings > Updates and get beta version there if you want to try it out.

Gear VR edition will be updated early next week if no major bugs are reported. We don't have beta channel on Sideload VR so we want to test it for few days on regular Android devices because it's much easier to rollback to stable version with Google Play.

---

Jitter / stutter improvements

TL;DR: It should be better now, more improvements will come in the future.

This took us a while. The problem first appeared when we released Gear VR edition in September. Some users were reporting various levels of stuttering and frame skips. We wanted to fix it quickly but after we investigated the problem it turned out that the underlying cause was really complex.

We first suspected frames being out of order but after high framerate recording it turned out that it wasn't true. We then investigated frames lost on the network but after adding some extra metrics we found that it wasn't packet loss. All frames were arriving on the phones yet the overall feel was very jittery.

We then started adding metrics and timings for every part of the virtual HMD pipeline. There are multiple places where VRidge needs to wait for something (game, steamvr, encoder, network transfer, decoder, Android vsync, etc.). If any of these components lags for even few miliseconds - it throws the whole thing out of sync.

We found that most of the microstutters came from network and Android decoder. Network micro lags are reasonable because that's how networks work. Android decoder optimization is a tougher nut to crack. It's performance is dependant on installed hardware and there are hundreds of chipset variants. The most annoying thing about is its inconsistence when it comes to decode times. For everything to be fluid, everything needs to take roughly the same amount of time every frame. Decoding time was varying greatly from few miliseconds to 12ms (with random spikes as high as 40). This kind of randomness is very bad for fluid playback.

Usually it's countered by buffering but it's not an option when we are trying to achieve VR-bearable levels of latency. We can't buffer even a second ahead to counter random spikes of interference.

Previously our FPS was stuck at almost 58 FPS. Some people suggested to turn it up to 60 and see if it fixes the stuttering. It didn't because the problem wasn't amount of frames per second. Generally to achieve fluid motion you need both high FPS and proper synchronization and intervals between them. An average of 60 FPS isn't good if sometimes the frames comes in burst of 2-3 at the same time with 30-40ms breaks after.

We decided to rework most of the timing and synchronization that was controlling capture, encoding and decoding. New frame rate is using better methods that combine multiple methods of timing and synchronization into one loop that is now much more accurate. Previously we had an average of 58.582 FPS in our capture and encoding module. Now it's running at 60.073 FPS with proper and stable intervals. This alone should lower deviation from 60 FPS from 2.5% to 0.1%.

Then we decided to rework Android decoding & rendering. Previously if no new decoded frame was ready at the beginning of the render cycle it would skip a frame. We now wait as much as possible for decoder to finish its work before giving up on the frame. You can control it by going to settings and changing frame timing:

• Prefer low latency (new default) will try to keep the latency minimum but still wait as much as possible for decoder to finish its work.
• Balanced will wait and buffer up to 1 frame if inconsistent decoding is detected.
• Prefer smoothness will buffer up to 2 frames.

There are 2 legacy timing modes that are generally inferior to Prefer low latency but we are leaving them because there are 500+ different phone models running VRidge (according to Play Store data) and new modes might not work for someone.

We might make this even more configurable but please give us feedback if you feel the difference between those new timing modes and old ones.

With new timing we also improved 30 FPS mode for low end devices. Previously it would render the game at 60 (58.5 actually) FPS and skip every other frame during encoding. Now 30 FPS mode will slow the game down to 30 FPS. This saves up some CPU and GPU load and allows game to run in a smoother fashion instead of rendering the frames that were being dropped anyway.

This problem was happening across all devices but it was mostly noticeable on Gear VR edition and Google Pixel which use low persistence displays. It didn't look too bad on regular screens because it was hidden by micro blurs and smudges which are common among the regual 60 Hz displays.

This is not the end of our work to counter the stuttering. There are still frame skips and some of them are not avoidable. We cannot control a game that might drop a frame, we can't force network to give 100% of its resource to VRidge and we cannot make background Android apps drop all of its work and give 100% of device power to VRidge. Vive counters this with async reprojection. Oculus has great async timewarp and spacewarp. These technologies increase smoothness greatly and this is something we eventually want to do too. It's not an easy task and it's especially harder because we have to account for variable network latency but this is our goal. It won't come this month or the next one but our goal is perfect smoothness and we want to comes as close to it as possible.

Previous version

New version:

It might be hard to notice on the recording (you can click samples above and slow it down with gfycat speed controls to emphasize frame skips) but you should be able to feel the difference in VR.

VRidge for Gear VR needs an update for those improvements to take effect (sometime early next week).

Google VR SDK update to 1.1

We updated VR backend on Android to the most recent version. It enables some new features but it has some changes you might not like. Google decided to force its UI to be always on so we cannot dim the buttons and middle-screen divider anymore. On the bright side it's a bit dimmer now and it shouldn't be visible through lenses anyway.

New 1.1 SDK brings some new features too. Daydream-ready devices should take adventage of new async reprojection. It's not accounting for network latency but it should lower the perceivable latency by a small amount. Every little bit helps so we are leaving this enabled by default but you might turn it off in app settings by toggling display stabilization on/off. For non-daydream devices this option enables old "shaky" stabilization which works in a different way but it also may lower the perceivable latency too.

Daydream phones will also enter into low persistence VR mode with VRidge increasing smoothness greatly.

Daydream view and controller support is still not available because it requires much more work. We received our Daydream View only few days ago so we didn't have much time to integrate new features. Stay tuned for more Daydream news in the future.

SteamVR improvements

Sometimes we get e-mails about certain SteamVR games not working with VRidge. We always investigate those reports and try to bring compatibility to most titles available on the market. Some people reported Serious Sam VR having terrible framerate which was result of a interaction between the game itself and SteamVR which was happening on 3rd party HMD drivers. We reported those issues to SteamVR and Serious Sam VR devs and the problem was resolved shortly in SteamVR beta and now is also fixed in SteamVR live channel. This also solves Unity 5.5 FPS issues with VRidge. Again, we cannot be grateful enough to SteamVR devs taking all reports and suggestions from both game and HMD developers seriously and solving them very quickly.

Sometimes it's up to us to fix our bugs and add missing features to bring better compatibility. This update brings virtual presence sensor which should prevent some games from pausing thinking that HMD was taken off. We also add opt-in virtual positional sensor. Some games were simply refusing to work without some sort of positional tracking so you can now enter Riftcat desktop settings and turn "SteamVR Fake Sensor" on.

We also updated our HMD driver to the neweset OpenVR standards and fixed one minor memory leak which could end up increasing RAM usage over few hours to gigabytes.

If one of the games below wasn't working for you with VRidge, you can try again today with new beta version: Serious Sam VR, Edge of Nowhere, Herobound and Pinball FX 2 VR.

High Efficiency Video Coding

We're also adding HEVC as an opt-in feature. It's still highly unstable on certain devices so it's not enabled by default. It's currently avaialble only for Media Foundation encoder and GeForce 960 or newer GPUs. It also requires relatively new phone. It works alright on our Galaxy S6 (Exynos chipset).

It doesn't work correctly on our Pixel XL. When the phone receives HEVC frames it simply crashes the phone completely causing a full device reboot. It seems that there's a bug in the hardware codec because no software should be able to cause a hard crash like that. We wlll investigate this further and report it to as a bug if it's not fixable on our side and no workarounds can be developed.

VRidge for Gear VR needs an update for HEVC to work so wait until next week if you want to try it.

Redmi Note 2 and Asus Zenfone 2 fixes

We found a bug that was crashing decoder on Redmi Note 2. It should no longer crash with stream fixes enabled. We also applied workaround for Zenfone 2 suggested by Moonlight dev (thanks!). Intel chipsets should now work correctly on both Android 5.0 and 6.0 without 3s lag.

Minor changes

• Disabled stream adjustments for Google Nexus 5 and Samsung Galaxy S5. It was sometimes crashing the app.
• Display stabilization is now toggleable in mobile options. It's a matter of preference so you can now turn it on or off. It's on by default (it was always on before).
• Changed IPD & Scale adjustment settings icon. New Google VR UI icons cannot be disabled so we had to use something else.
• Increased maximum IPD by 50%.

What's next?

We are still working on sound streaming and we want to get back to "Fake VR" so you can stream regular games into your Android phone. We suspended work on both to bring improvements described above but we will now be able to come back to new features.

We also have a version that works on some Android 4.x  devices. We asked you to e-mail us if you want to test this version and we have a group of people willing to test it before making it avaialble for everyone. If you sent us an e-mail about it - sorry for not getting back to you earlier. Android 4.x was delayed by this update unfortunately. We will reply to you with a testable version link as soon as we are done making this update "stable enough".

We want to make above changes stable first so please give us your feedback and send bug reports. We have several of the most popular phones among our users and we test all changes on them but it's impossible to predict behaviors on all phones on the market. You can send e-mails to support@riftcat.com or send us a message or comment through any of the social media channels (Facebook/Twitter/reddit). We always read everything and while it may take some time to respond to everyone - we are not ignoring any of the comments.

---

Post edit history:
13 Jan 00:00 UTC - Added clarifications about VRidge for Gear VR requiring an update for HEVC and jitter improvements to work.