Friday, April 21, 2017

Dev update #29 - an update on our progress



This is text update only, we want to explain what we are doing and why we think it's a best way to do currently. 
---

VRidge core improvements 

Moonlight, stability & launch flow improvements, fake VR groundwork

Reasons

VRidge was first designed as Oculus proxy interceptor that acted like Oculus DK2. Its work flow was quite simple because VR was simple back then. There were no controllers, no always-on services, no Oculus Home / SteamVR dashboard like environments. You started a game by running its executable and VRidge process lifetime was limited to game process lifetime.

We added SteamVR support exactly year ago, on 21th April 2016. It was first SteamVR-mobile HMD bridge back then and it helped us grew a lot. Then we continued adding more and more features and changes like UI slider for supersampling (which was later added by SteamVR itself), better support for controllers, FreeTrack, 1440p, Intel and Software encoders, Gear VR, partial Daydream support and lately - VRidge Tracking API.

The shape of VR has changed from headsets being simple 3D glasses to full room scale systems running multiple services in OS background. The problem is that the core of the VRidge remained mostly the same and if you ever worked on a big project, you know that when goals and requirements shift, the project maintainability suffers. The VRidge was designed with pre-release Oculus headsets in mind when there were no controllers or room scale lighthouse systems. And those legacy foundations slow us down a lot.

Core rework

That's why we are currently doing major rewrite of the VRidge core. It's actually going pretty good and we already made a lot of progress. In our development version VRidge was decoupled from SteamVR launch flow. We dropped that annoying chain of dependencies that required restarting all components (mobile, SteamVR, VRidge, Riftcat) when any of the mentioned components ran into some sort of problem. We can start and stop SteamVR/VRidge at any time and in any order - they will connect without problems. 

Another thing that rework enables is modular architecture. We can now substitute capture, encoding and output components during runtime. When we detect SteamVR connection being dropped, the stream automatically switches off to screen duplication stream. While it's not very useful at this moment (because VRidge doesn't have any Remote Desktop functions) it lays a solid foundation for future Fake VR. It can capture any window with very little overhead (<1% CPU usage, sub-frame latency, duplication tech stack very similar to Virtual Desktop, Bigscreen or OBS).

Another great thing that we already have working is a plug and play Moonlight support. It works as easy as possible. We have special VridgeView.exe that can be added to GameStream library in GeForce Experience panel. Then we simply start the VridgeView through Moonlight and it automatically connects to VRidge runtime and switches VRidge to fullscreen SBS output that is handled by Moonlight. And it works quite amazingly since as we said before - we think that NVIDIA GameStream + Moonlight pair is a "relationship goal" for any streaming software.

This also allows HDMI (or any video signal cable) output. You can use the VRidgeView.exe which will find its way to your chosen display and show VR on your device's display with no added latency. This is great thing for DIY fans that enjoy building their own HMDs.

We also want to unify Android rendering. Currently Gear VR and Android use completely different rendering engines. This makes it complicated because Daydream's immersive mode requires another VR renderer. We have a way to combine them all into one which would finally allow proper implementation of Daydream's immersive mode. We didn't want to write separate renderer for Daydream only because writing three things for three platforms that do essentially the same thing is a hell to maintain. With unified renderer, it should be a much less bumpy road.

We are also moving all settings to desktop app since adjusting some settings on the mobile makes absolutely no sense. Trying to adjust FOV with your headset off is an annoying guessing game.

Let's not forget iOS and possibly other mobile clients. This modular rework will remove a lot of spaghetti netcode which will then make it easier to port the mobile app over to iOS.

To sum things up, here's our current progress of the most important components:

☑ SteamVR driver connectivity rework (plug and play startup).
☑ Moonlight streaming support.
☑ HDMI output.
☑ High performance desktop capture engine for the future.
☑ Encoder quality settings adjustable on the fly.
☑ Encoder migration: NVENC.
☐ Encoder migration: MF (including AMD, Intel QS, Software).
☑ Network output
☐ Android network connectivity.
☑ Android decoder (not changed). 
☐ Android unified renderer. 
☐ Android background mode (for Moonlight mode tracking).

Scope of the work above is enough to call the reworked release VRidge 2.0 but those are only numbers without any other meaning than a label (no price changes or anything, everyone will get this update).

Since it's a major rework we cannot provide a testable version because Android connectivity wasn't migrated yet. Nonetheless we hope that this will allow us smoother development after most of legacy code is cleaned. During last year VR matured with a release of SteamVR and Oculus CVs. Controller interfaces are pretty well documented. VR environment APIs are pretty stable. With a more solid set of VR standards we are confident that we won't repeat the story pictured below:
by: Manu Comet

Sound progress

We planned to release it faster but we ran into some problems during testing on other machines. As it turns out - different sound chips use different audio formats so we are still testing and adapting to new motherboards/sound cards that we can find. After it's released to beta channel we will probably find some more bugs with exotic sound devices but that's what the beta channel is for.

SideloadVR & ConstructVR

Apparently SideloadVR is still online so we are going to update our Gear VR app on SideloadVR to match ConstructVR's latest version. It should be updated soon.

VRidge API changes

We are making some changes to VRidge API for next API release.
1. API call to re-center view.
2. Haptic feedback response for controllers.
3. A way to mark HMD trackers state as "out of tracking range".

Those changes were requested by 3rd party device manufacturers. The changes will be reflected in next week's API release on VRidge API GitHub.

Update release schedule

  • VRidge API will be updated by the end of next week.
  • Soon everything in current beta channel will move to stable channel. We want our stable version to be compatible with NOLO VR hardware when Kickstarter backers receive their units. 
  • Sound and protocol changes will be deployed to beta as soon as they are release-ready. Most likely it will be released in early May.
  • Core rework is estimated to be released to beta by the end of Q2, hopefully. It's our biggest change since release, but for a good reason.

Monday, March 27, 2017

Dev update #28 - VRidge tracking API


Vridge API is out in beta. Networking and audio changes are nearly done. More exciting features coming in next months.

---

VRidge API out in beta channel


VRidge API is a way to interact with VRidge tracking internals. This is the first step in making our platform more compatible with third party software and hardware solutions. We start with two endpoints that will allow you to take full control of head tracking system and controllers.

You can check out the example implementation and desktop app sample on our github here:


API is versioned so your plugins and drivers should work even after we alter the protocol. Perfect backwards compatibility is something we cannot promise for sure but we designed the API in a way that should allow easy iterations - we have several ways to keep both new and old versions of API channels running side by side. Backwards compatibility will be our priority when we make changes to the API.

We start with two endpoints that can be used separately:

Controller endpoint allows you to send VR motion controller state without writing a full OpenVR driver. This allows a more stable experience since only one driver is loaded in SteamVR. You can send multiple controller states, move them around, press and touch buttons and interact with touchpad. You can also use any language of your choice to send driver states as long as you send properly formatted packets.

Head tracking endpoint allows you to control head tracking in a variety of modes. You can use it to provide positional, rotational or combined data. You can also read mobile sensor data and provide an offset. If you find it necessary, you can modify phone tracking data in real time before it's used for rendering in VR.

We already have plans to expand API further to create a way to expose compressed or raw video output. Ultimate goal is to have everything pluggable/modifiable. VR hardware and software world is very fragmented now without a clear standard that has to be followed*. That's why we want to make parts of VRidge modifiable by controller and HMD manufacturers.

*OpenXR by Khronos is a good attempt to bring it all together. We hope that it succeeeds because with OpenVR, Oculus VR, OSVR and all the bridges in between them - the current state of VR SDK is somehow chaotic.

In the future we will also start moving more and more of our comm protocols into the open API because it allows creating 3rd party apps. We want to keep improving our Android app and eventually release an iOS app but if someone wants to create an alternative app specifically crafted for their purposes - we want to support it. There are multiple reasons to do so. Maybe you need a branded app for your company. Maybe you want to demo your architectural designs with some extra options. Or maybe you want to try to hack a way to run VR on TI-84 (after seeing Quake on oscilloscope or Symmetra being played with a microwave we believe that everything is possible).

While hacking and modding is fun this is our ultimate but not immediate goal. Our next plans are another round of performance/user experience improvements.



What's next?


Protocol, audio streaming, stable channel update (April)

There isn't much to say about it. The work on audio and protocol changes was suspended while we were trying to finish the API. It will be probably our next release since it's mostly complete and just needs some testing. Next blog post will be most likely focused on those changes with audio & protocol update being available in beta channel. We will probably push current beta to stable channel then too.

Smoothness, performance, latency - Moonlight & HDMI (Q2)

We tried to tackle it several times and there's still unresearched option of time warp providing significant improvements here. Trying to create the greatest VR streaming experience would be easier if we had something to compare to. That's why before we start working on time warp (which can improve the experience but we don't know it for sure) we want to create something that should have been added a long time ago.

NVIDIA Gamestream combined with open source Moonlight project is a great streaming solution. Probably the best one we've seen. We want to support it with VRidge app acting as a background tracker connected to VRidge runtime on the desktop which translates mobile tracking data into HMD tracking data and HMD frames into Moonlight & Gamestream compatible data. We want it to serve as a point of comparision of our own streaming.

Once we have Gamestream & Moonlight as a reference of perfect streaming, we want to improve our own streaming and tweak it for VR.

AMD users - don't worry, our streaming will still be available and improved in the future. We just want to take an advantage of pre-build solution because Team Green is large majority of our userbase. And the streaming is already there, we just need to hook into it.

While we are reworking rendering output we want to add rendering to HDMI connected displays* too. We already have render-to-screen working internally because we use it for development (finding bugs is easier when you can use your 2nd screen instead of phone). We just need to wrap it up in nice GUI switches and add some stability to it.

*You can substitute HDMI with DisplayPort/D-SUB/DVI. It will also have an option to simply render to window.

Less restarts, more stability

We all hate the "Your PC is going to restart in 5 minutes and there's nothing you can do." messages. Especially when you're in the middle of ranked match and you don't want your teammates to lose and your MMR to drop. This kind of experience ruins the fun and we want to address it in VRidge too. Current runtime flow requires restarting nearly everything (mobile app, SteamVR & game, VRidge, sometimes RiftCat dekstop app) every time even the smallest thing go wrong.

  • Connection interrupted? Sometimes you won't be able to reconnect. Gotta restart everything.
  • You want to change bitrate? Restart VRidge, SteamVR, game, mobile app.
  • USB cable disconnected? You need to restart everything. Oh, and Riftcat dekstop app crashes too.

Combine this with fragmented settings (some options needs to be configured on the mobile side, some are configured in the desktop app) and questionable design decisions (changing IPD & scale requires taking the phone outside the viewer).

We want to fix as many of these. Our goal is for VRidge to run always and connect with mobile app and SteamVR in a hot-pluggable way. Some settings would still require restarts but we want to limit those to situations when it absolutely needs to happen. We also want to move all settings to desktop app.

Minor tweaks and other features

We will have smaller tweaks and some new motion control options available in the meantime too. Moonlight, HDMI and anti-restart updates might take a while to be released (late Q2 is the most likely possibility). That's why we will try to squeeze in some extra side options that were prototyped as side projects but we're not ready to talk about them yet.

Fake VR is still on the table but performance has higher priority since over 50% of our feedback survey requests can be answered by having Moonlight as an option. iOS app will have to wait until we complete moving our protocol into open API.

Friday, February 24, 2017

Cloudflare security incident statement

Few hours ago Cloudflare informed the world that they discovered a security problem that could result in sensitive data being leaked. You can read it all over the news.

Cloudflare is used by over 4 million websites across the Internet, including Medium, YCombinator, Bitpay, Patreon, TransferWise, Coinbase, Authy, Okcupid, Zendesk, Curse network, Discord, among many other big websites. You can download compressed list of all potentially vulnerable websites here.

RiftCat also uses Cloudflare but we received an e-mail from Cloudflare informing us that they didn't find any of our data leaked:

Fortunately, your domain is not one of the domains where we have discovered exposed data in any third party caches. 

The leak isn't RiftCat specific. Any* of the users visiting any of the 4M websites could have seen part of someone's else data in website source. For example, while visiting riftcat.com there was an astronomically small chance of someone's Uber/Fitbit/Okcupid** password being hidden in page source (and the other way around).

Chances of your sensitive data being leaked from the affected sites are very, very small. According to Cloudflare one out of 3 million requests (0.00003%) contained hidden parts of someone's else request[1]. Those hidden parts were probably some sort of generic web traffic because most web traffic is just people browsing pages and not entering their passwords. It was also random and hidden by nature of the problem so it's unlikely that someone could use this as an attack on any of the services listed.

An analogy of this event in real world scenario would be a 0.00003% chance of finding totally random item from someone's else house under your door rug. If you haven't looked under the mat during this unlikely event, it was gone (unless it landed under bot's door rug which is also quite rare)

Most of publicly cached mixed pages are already removed. Cloudflare found 150 page requests publicly available in web caches.


RiftCat data was not found in any of the publicly leaked data. 

Nonetheless you might want to change your passwords around the Internet if you want to minimize your chances of being compromised. It's highly recommended that you don't ever use the same password twice. Stay secure.

*It might be limited to users using Cloudflare scrape shield only but it's not confirmed.
** According to TechCrunch they were confirmed as affected. See [3]

Further reading:

Thursday, February 9, 2017

Dev update #27


This post is text only without downloadable updates available in Riftcat client. We want to update you on our current work and talk about the future.

---

Sideload VR replacement

January ended with very unfortunate news. Sideload VR - a great source for Gear VR prototype apps - announced that it's closing down. You can see Sideload's author reasoning in the announcement post here.

We needed a replacement. First we thought that we will have to write our own repackaging service but fortunately such alternative to Sideload VR already exists. From now on you will be able to find Gear VR version on ConstructVR.

Desktop client links will be updated with next downloadable update.

NOLO VR

Slightly over a week ago NOLO VR launched their Kickstarter campaign. We tried the hardware, we have units at our office and we can surely say that it's not a vaporware. It works just as advertised. There is no fake camera work or pre-rendered animations.

We are in touch with NOLO VR team and we are currently working to prepare a software side on our end so their hardware can work out of the box with VRidge. We tried Leap Motion, PSMoveService and other experimental drivers but NOLO VR was definitely the easiest and most reliable of them.

Their goal is to provide end user experience of nearly Vive-level tracking. It uses the same underlying technology (Lighthouse) so the tracking is very precise indeed.

Our goal is to provide a software support where you simply plug their hardware in and it just works. We don't want to make you install multiple layers of FreeTrack/FreePie like software relays. Simply plug it in, start VRidge and enjoy the next level of streamed VR.

VRidge API

While we are implementing NOLO VR support we decided to expand this into another component of VRidge. You can already do a lot with FreeTrack and OpenVR drivers so one could argue that we don't need another standard.

XKCD: Standards
But if you actually tried to use FreeTrack, custom OpenVR drivers and VRidge all together - you might have noticed several problems - black screens, different coordinate systems mixing up, etc. We are now working on providing a unified interface to keep everything loaded as one driver in one 3D space. Here are some key features that will be available with VRidge API.
  • Writing positional or positional and rotational data to override VRidge tracking. Similar to FreeTrack but with an ability to switch modes in real time without restarting VRidge. 
  • Reading and writing all VRidge tracking data before it's used to render a frame. You could use external sensor data and combine them with VRidge provided mobile sensor data to combine both data streams into a more precise measurement. It works in real time with ~0.1 ms response time.
  • Reading data and providing an offset. You could use external sensors to compare its data with phone measurements and provide an offset that corrects drift.
  • Writing OpenVR controller 3D position and button presses with an ability to choose the origin point of 3D tracking.
  • A way to do all of the above remotely. Wireless controllers won't need any desktop relays as long as they have some sort of wifi connectivity. It enables quite creative ideas of using mobile phones as motion controllers, similar to Daydream controller emulation.

The API is currently using ZeroMQ as a protocol. This is industry tested standard used by (according to their website) AT&T, Cisco, EA, Los Alamos Labs, NASA, Weta Digital, Zynga, Spotify, Samsung Electronics, Microsoft, CERN, among others. It allows developers to connect to VRidge API in any programming language easily. It doesn't matter if you use Python, Java, C#, C++, Objective-C or any other supported language (full list here).

We plan to provide samples with sources in Java (Android) and C# (WPF). We want to create a direct connector between VRidge and PSMoveService too because it's very popular and can serve as another example.

Audio and network protocol changes

Audio streaming is done but we decided to refactor networking a bit. Until now we used raw UDP datagrams with frame data but while we were adding audio stream we decided that it's time to upgrade to rUDP. It should reduce artifacting on 2.4 GHz networks. There's no reason to reinvent the wheel so we decided to use ENET. The same library that is used by Moonlight + Gamestream. We hope that retransmission will allow us to smooth out the experience further

Survey results and our next steps

According to our survey 80% of you noticed an improvement when it comes to smoothness in 1.2 update. This is a very good news and it motivates us to optimize further now that we can see that users actually notice not only features but the behind-the-scenes data flow optimizations too.
Here are some more facts:

  • One out of the three people use Gear VR.
  • Majority of people found that HEVC improves the exprience.
  • The most common video cards are GTX 970, 1070 and 1080. Only 20% of respondents used AMD cards.
  • Only 15% of users use WiFi 2.4 Ghz. WiFi 5 GHz and USB tethering are equally popular.

Last but definitely not least - the most important question of all of them. We are creating this software for you and your feedback is the most important factor in our decision making.


You can see raw data above. This is how we see it.

  1. You want a better controller support. We think that VRidge controller API described earlier will be a step in the right direction.
  2. Second most popular option is Fake VR. It was discussed earlier and we already did some work in this area. We plan to resume working on it after controller API, sound streaming and protocol changes are out.
  3. Third is jitter reduction. We hope that improved networking protocol will reduce it further but we still have timewarp on our task board.
  4. Reducing latency. This is a tricky one but time warp has a potential to reduce perceivable latency greatly.


Other answers (10%) contain many sound streaming requests which is already nearly complete.

We now want to finish controller API (#1), sound streaming and networking changes. After we're done with it, most likely one developer will focus on Fake VR (#2) and the other one will start prototyping a time warp (#3 and #4). As an additional step to improve on option #3 and #4 we want to add Moonlight compatible mode eventually.

We recently discussed Moonlight mode on reddit. We want to try time warp first because of reasons described in the discussion but Moonlight & Gamestream are always in our minds as a great example how to stream things perfectly.

We hope that this answers your feedback. Of course all of this can change because we can't predict the future but we hope that no major obstacles appear and we can actually deliver features you want the most.

Thursday, January 12, 2017

Dev update #26

This update took us much longer than expected but we finally believe that we made an improvement that is noticeable and worth giving a shot if you experienced terrible stuttering on your Galaxy S6-7 and some other phones.

We also bring some long term bug fixes like 30 and 60 FPS staying at the actually 30/60FPS, some Redmi Note and Zenfone fixes and several SteamVR improvements. There's an experimental version of HEVC too.

This update is released as version 1.2 (both mobile and desktop) through beta channels. Go to Riftcat desktop settings > Updates and get beta version there if you want to try it out.

Gear VR edition will be updated early next week if no major bugs are reported. We don't have beta channel on Sideload VR so we want to test it for few days on regular Android devices because it's much easier to rollback to stable version with Google Play.


---

Jitter / stutter improvements

TL;DR: It should be better now, more improvements will come in the future.

This took us a while. The problem first appeared when we released Gear VR edition in September. Some users were reporting various levels of stuttering and frame skips. We wanted to fix it quickly but after we investigated the problem it turned out that the underlying cause was really complex.

We first suspected frames being out of order but after high framerate recording it turned out that it wasn't true. We then investigated frames lost on the network but after adding some extra metrics we found that it wasn't packet loss. All frames were arriving on the phones yet the overall feel was very jittery.

We then started adding metrics and timings for every part of the virtual HMD pipeline. There are multiple places where VRidge needs to wait for something (game, steamvr, encoder, network transfer, decoder, Android vsync, etc.). If any of these components lags for even few miliseconds - it throws the whole thing out of sync.

We found that most of the microstutters came from network and Android decoder. Network micro lags are reasonable because that's how networks work. Android decoder optimization is a tougher nut to crack. It's performance is dependant on installed hardware and there are hundreds of chipset variants. The most annoying thing about is its inconsistence when it comes to decode times. For everything to be fluid, everything needs to take roughly the same amount of time every frame. Decoding time was varying greatly from few miliseconds to 12ms (with random spikes as high as 40). This kind of randomness is very bad for fluid playback.

Usually it's countered by buffering but it's not an option when we are trying to achieve VR-bearable levels of latency. We can't buffer even a second ahead to counter random spikes of interference.

Previously our FPS was stuck at almost 58 FPS. Some people suggested to turn it up to 60 and see if it fixes the stuttering. It didn't because the problem wasn't amount of frames per second. Generally to achieve fluid motion you need both high FPS and proper synchronization and intervals between them. An average of 60 FPS isn't good if sometimes the frames comes in burst of 2-3 at the same time with 30-40ms breaks after.

We decided to rework most of the timing and synchronization that was controlling capture, encoding and decoding. New frame rate is using better methods that combine multiple methods of timing and synchronization into one loop that is now much more accurate. Previously we had an average of 58.582 FPS in our capture and encoding module. Now it's running at 60.073 FPS with proper and stable intervals. This alone should lower deviation from 60 FPS from 2.5% to 0.1%.

Then we decided to rework Android decoding & rendering. Previously if no new decoded frame was ready at the beginning of the render cycle it would skip a frame. We now wait as much as possible for decoder to finish its work before giving up on the frame. You can control it by going to settings and changing frame timing:

  • Prefer low latency (new default) will try to keep the latency minimum but still wait as much as possible for decoder to finish its work.
  • Balanced will wait and buffer up to 1 frame if inconsistent decoding is detected.
  • Prefer smoothness will buffer up to 2 frames.

There are 2 legacy timing modes that are generally inferior to Prefer low latency but we are leaving them because there are 500+ different phone models running VRidge (according to Play Store data) and new modes might not work for someone.

We might make this even more configurable but please give us feedback if you feel the difference between those new timing modes and old ones.

With new timing we also improved 30 FPS mode for low end devices. Previously it would render the game at 60 (58.5 actually) FPS and skip every other frame during encoding. Now 30 FPS mode will slow the game down to 30 FPS. This saves up some CPU and GPU load and allows game to run in a smoother fashion instead of rendering the frames that were being dropped anyway.

This problem was happening across all devices but it was mostly noticeable on Gear VR edition and Google Pixel which use low persistence displays. It didn't look too bad on regular screens because it was hidden by micro blurs and smudges which are common among the regual 60 Hz displays.

This is not the end of our work to counter the stuttering. There are still frame skips and some of them are not avoidable. We cannot control a game that might drop a frame, we can't force network to give 100% of its resource to VRidge and we cannot make background Android apps drop all of its work and give 100% of device power to VRidge. Vive counters this with async reprojection. Oculus has great async timewarp and spacewarp. These technologies increase smoothness greatly and this is something we eventually want to do too. It's not an easy task and it's especially harder because we have to account for variable network latency but this is our goal. It won't come this month or the next one but our goal is perfect smoothness and we want to comes as close to it as possible.

Previous version
New version:

It might be hard to notice on the recording (you can click samples above and slow it down with gfycat speed controls to emphasize frame skips) but you should be able to feel the difference in VR.

VRidge for Gear VR needs an update for those improvements to take effect (sometime early next week).

Google VR SDK update to 1.1

We updated VR backend on Android to the most recent version. It enables some new features but it has some changes you might not like. Google decided to force its UI to be always on so we cannot dim the buttons and middle-screen divider anymore. On the bright side it's a bit dimmer now and it shouldn't be visible through lenses anyway.

New 1.1 SDK brings some new features too. Daydream-ready devices should take adventage of new async reprojection. It's not accounting for network latency but it should lower the perceivable latency by a small amount. Every little bit helps so we are leaving this enabled by default but you might turn it off in app settings by toggling display stabilization on/off. For non-daydream devices this option enables old "shaky" stabilization which works in a different way but it also may lower the perceivable latency too.

Daydream phones will also enter into low persistence VR mode with VRidge increasing smoothness greatly.

Daydream view and controller support is still not available because it requires much more work. We received our Daydream View only few days ago so we didn't have much time to integrate new features. Stay tuned for more Daydream news in the future.

SteamVR improvements

Sometimes we get e-mails about certain SteamVR games not working with VRidge. We always investigate those reports and try to bring compatibility to most titles available on the market. Some people reported Serious Sam VR having terrible framerate which was result of a interaction between the game itself and SteamVR which was happening on 3rd party HMD drivers. We reported those issues to SteamVR and Serious Sam VR devs and the problem was resolved shortly in SteamVR beta and now is also fixed in SteamVR live channel. This also solves Unity 5.5 FPS issues with VRidge. Again, we cannot be grateful enough to SteamVR devs taking all reports and suggestions from both game and HMD developers seriously and solving them very quickly.

Sometimes it's up to us to fix our bugs and add missing features to bring better compatibility. This update brings virtual presence sensor which should prevent some games from pausing thinking that HMD was taken off. We also add opt-in virtual positional sensor. Some games were simply refusing to work without some sort of positional tracking so you can now enter Riftcat desktop settings and turn "SteamVR Fake Sensor" on.

We also updated our HMD driver to the neweset OpenVR standards and fixed one minor memory leak which could end up increasing RAM usage over few hours to gigabytes.

If one of the games below wasn't working for you with VRidge, you can try again today with new beta version: Serious Sam VR, Edge of Nowhere, Herobound and Pinball FX 2 VR.



High Efficiency Video Coding

We're also adding HEVC as an opt-in feature. It's still highly unstable on certain devices so it's not enabled by default. It's currently avaialble only for Media Foundation encoder and GeForce 960 or newer GPUs. It also requires relatively new phone. It works alright on our Galaxy S6 (Exynos chipset).

It doesn't work correctly on our Pixel XL. When the phone receives HEVC frames it simply crashes the phone completely causing a full device reboot. It seems that there's a bug in the hardware codec because no software should be able to cause a hard crash like that. We wlll investigate this further and report it to as a bug if it's not fixable on our side and no workarounds can be developed.

VRidge for Gear VR needs an update for HEVC to work so wait until next week if you want to try it.



Redmi Note 2 and Asus Zenfone 2 fixes

We found a bug that was crashing decoder on Redmi Note 2. It should no longer crash with stream fixes enabled. We also applied workaround for Zenfone 2 suggested by Moonlight dev (thanks!). Intel chipsets should now work correctly on both Android 5.0 and 6.0 without 3s lag.

Minor changes


  • Disabled stream adjustments for Google Nexus 5 and Samsung Galaxy S5. It was sometimes crashing the app.
  • Display stabilization is now toggleable in mobile options. It's a matter of preference so you can now turn it on or off. It's on by default (it was always on before).
  • Changed IPD & Scale adjustment settings icon. New Google VR UI icons cannot be disabled so we had to use something else.
  • Increased maximum IPD by 50%.

What's next?

We are still working on sound streaming and we want to get back to "Fake VR" so you can stream regular games into your Android phone. We suspended work on both to bring improvements described above but we will now be able to come back to new features.

We also have a version that works on some Android 4.x  devices. We asked you to e-mail us if you want to test this version and we have a group of people willing to test it before making it avaialble for everyone. If you sent us an e-mail about it - sorry for not getting back to you earlier. Android 4.x was delayed by this update unfortunately. We will reply to you with a testable version link as soon as we are done making this update "stable enough".

We want to make above changes stable first so please give us your feedback and send bug reports. We have several of the most popular phones among our users and we test all changes on them but it's impossible to predict behaviors on all phones on the market. You can send e-mails to support@riftcat.com or send us a message or comment through any of the social media channels (Facebook/Twitter/reddit). We always read everything and while it may take some time to respond to everyone - we are not ignoring any of the comments.

---

Post edit history:
13 Jan 00:00 UTC - Added clarifications about VRidge for Gear VR requiring an update for HEVC and jitter improvements to work.

Thursday, November 24, 2016

Dev update #25



This is quick update to let you know what we're working on. Desktop app will let you know that auto-update is ready soon (version 1.1.3).

Stable update for desktop app (v. 1.1.3)

We're pushing desktop beta updates to stable branch. Everyone will be able to take advantage of software encoder on low end PCs. This is last resort option because this will put heavy load on CPU. You should only use it if you don't have any compatible hardware (recent GPU or Intel Core CPU).

This update also includes some stability tweaks, UI changes, newsletter opt-in and Gear VR edition download button.

Motion controller black screen

Several people reported that black screen no longer happens in Steam VR beta when you use multiple drivers (e.g. VRidge and PSMoveService). Check it out and send log files* if it still blacks out for you.

*vrserver.txt located in Steam/logs (by default: C:\Program Files (x86)\Steam\logs).


Work in progress: sound streaming

The sound streaming work is done on the Windows and networking side. We just need to finish up Android decoding and playback. It's still on track to be released as an update sometime in December.


Work in progress: android improvements - jitter, Android 4.x, HEVC

During our jitter research we decided that it's worth a shot to try synchronous video decoder instead of asynchronous (which is recommended by Google). This will allow for VRidge to work on Android 4.x devices.

Most of the older phones that weren't upgraded to Android 5.0+ aren't great VR devices but there are some phones that have 720p+ display there were simply forgotten by carriers and were left with Android 4.x OS. If you have decent Android 4.x based phone with 720p (or better) display and you want to try VRidge on it - e-mail us at support@riftcat.com with "Android 4 testing" as subject line. We'll contact you when we have something worth testing.

Creating synchronous decoder as an option forced us to do some refactoring of our decoding & rendering flow. This will make it easier to implement HEVC/H265 coding which is ~40% more efficient than H264 that we use now. This means 40% better quality at the same bitrate. We hope that we don't run into major roadblocks on the PC side. We'll start with NVIDIA GPUs first for HEVC.


We are also in process of getting Daydream & Pixel to explore new VR possibilities on Google's new flagship device but it's too early to talk about the details.

Thursday, November 3, 2016

Dev update #24


Regrouping and reorganizing

You may have noticed that our development slowed down lately so we want to explain why it happened.

VRidge userbase grew a lot during this year. When we first shown our prototype proof of concept in March we expected some people to be interested in beta testing but we didn't expect thousands.
We always love chatting about development, receiving feedback, solving user problems. Our devs were always directly responding to e-mails. This includes feedback, suggestions, bug reports, pc upgrade advice, general VR chat. It was taking more and more of our devtime.

Now VRidge's userbase is approaching 100 000 and the old ways that used to work had to be changed. Even if we hear back only from a small subset, let's say a 3% of our userbase - that's 3 thousand people.

That's why our devs will no longer be the frontline of our support system because it simply takes too much time discussing every single case individually. Our support system will of course continue to work but we'll try to write a general FAQ items on each type of issue so people can fix most of the problems on their own. We'll have a dedicated person handling support tickets that will be communicating with our devs on a regular basis.

Dev note:  We love solving problems directly but this way we'll be able to fix underlying problems faster instead of developing duct-tape workarounds for each invididual case. :) We'll also continue to hang around reddit. We didn't post too much lately but we read every single discussion.

If you contacted us lately and haven't received a response for a few days, don't worry - we'll try to cach up with everyone soon.

We're also reassigning our dev time. Until now we focused 100% on fixing problems instead of adding new features but there's a good percentage of our users running VRidge without any major problems. That's why we'll have 1 developer assigned to fixing problems and optimizing experience (including "the jitter", read more in the last paragraph of this post) while the other developer will work on adding new features.

Sound streaming

If you have decent ac router and non-crowded 5 GHz band you can usually stream wirelessly with quality, lag and reliability comparable to USB cable. Many people asked us for sound streaming for fully untethered experience. Being able to sit down on your couch without a bunch of wires limiting your movement sounds very nice. You will be able to turn it on/off because there's no need to stream another data channel and eat up extra bandwidth if you want to use PC headphones anyway. We want to deliver this feature in December.

PS Move / controller black screen

We tried to fix it on our side. We exchanged our data and test results with PSMoveService contributors but it seems that the problem is not linked to VRidge itself but to the way SteamVR handles multiple drivers. It also happened with Oculus DK2 with PSMoveService (without VRidge) or with Leap Motion + VRidge (without PSMS). PSMoveService on its own works fine. VRidge does work on its own as well. But it seems that when two drivers are interacting with SteamVR together they may randomly "collide" when they try to send messages at the same time. It's a speculation because it's hard to know what exactly is happening because SteamVR runtime isn't open source. You can read full discussion here. We'll definitely let you know if we have more news on this topic.

Jitter

It's a bit long so there's a short version at the bottom. Scroll down if you want to skip this wall of text.

We tested, we measured, we analyzed rendering with high FPS cameras, we spent many, many hours trying to exclude all possible factors of this problem with various phones and display types. We know that in fact 60 (58 actually but this doesn't change anything) frames are reaching the phone (assuming good connection conditions) but something's not quite right - it feels jittery.

Nearly all (90%) of jitter reports come from S6 & S7 (Exynos chipsets) but there were few Snapdragons reports too. It's hard to tell if those Snapdragons experience the same underlying problem as Exynos devices. Samsung devices also use different type of screen than other mobile phones (AMOLED).

We knew that it's visible the most in Gear VR version but it also happens in Cardboard version on S6 & S7 too. After extensive testing we found that other phones also experience not exactly silky-smooth rendering during head movements but it's not quite as noticeable. It may manifest itself in bluriness instead of jitter.

Human perception is a tricky subject and some people don't see a difference between 30/60/144 FPS. People can play games with lowend GPUs at 20 FPS while some others consider the experience ruined if it drops below 60 FPS. The worst thing about it is that it's hard to objectively judge how smooth our small changes affect the experience. We do some changes, we test them again and compare, we show it to different people but when we try to describe how smooth it feels on a scale of 1-10 we get very random results. We literally have to stare at the distance through the window for a while to "reset" our eyes to see differences between our test versions. That's why it takes so long to achieve the perfect streaming.

We tried to use numeric metrics like FPS or frame latency but even we use high FPS camera to record the screen it's hard to catch everything because human eyes don't see the same way camera does. We don't have a brain refresh rate or eye capture rate. We see motion and we know when it's fluid and when it's not. We can get used to non-perfect rates but some some people may get a headache after a while and that's not something we think is acceptable. We know that even regular VR HMDs can trigger headaches and motion sickness in some people so the problem is not 100% solvable but VRidge has a lot of room for improvement.

Since all of our tools and metrics failed to give our definitive, measurable answer that can be used to compare different versions of VRidge we decided that we need more metrics, tools, numbers, graphs, timings, all kinds of data. We wanted to fix it quickly, to find the the wrong line of code, the mistimed part of the pipeline but it seems that there's no magical, easy way of "fixing" it.

With one of our developers assigned to new features and one free to dig deep into the VRidge internals, we think this approach will be the better way of handling the jitter problem. It takes more time but this we can do it the right way instead of the quick way, which doesn't always work. Will it work? We very much hope so.

You can think of VRidge as a street between two points - your phone and the PC. The problem is that there's a bunch of intersections - sensor reading & fusion, sending sensor data, SteamVR/Oculus driver interaction, game capture, encoding, sending, transmission error concealment, receiving, reconstructing, decoding, rendering. A small traffic jam at any of those intersection can stall the whole flow. Cities have cameras to monitor traffic flow and to optimize urban planning. We need similar tools monitoring flow at each part of the pipeline.

We found one thing that can possibly reduce jitter a little bit in non-GearVR Android version. This will be probably pushed to beta Android channel next week. It's the first step of fixing the problem but definitely not the last one.

TL;DR; We didn't find a quick solution for old problem so we took the long way to diagnose them more thoroughly. Some of you are quite happy with current version. so we're splitting devtime 50:50 with 1 dev assigned to new features and 1 to optimize and fix old problems.