Thursday, November 24, 2016

Dev update #25



This is quick update to let you know what we're working on. Desktop app will let you know that auto-update is ready soon (version 1.1.3).

Stable update for desktop app (v. 1.1.3)

We're pushing desktop beta updates to stable branch. Everyone will be able to take advantage of software encoder on low end PCs. This is last resort option because this will put heavy load on CPU. You should only use it if you don't have any compatible hardware (recent GPU or Intel Core CPU).

This update also includes some stability tweaks, UI changes, newsletter opt-in and Gear VR edition download button.

Motion controller black screen

Several people reported that black screen no longer happens in Steam VR beta when you use multiple drivers (e.g. VRidge and PSMoveService). Check it out and send log files* if it still blacks out for you.

*vrserver.txt located in Steam/logs (by default: C:\Program Files (x86)\Steam\logs).


Work in progress: sound streaming

The sound streaming work is done on the Windows and networking side. We just need to finish up Android decoding and playback. It's still on track to be released as an update sometime in December.


Work in progress: android improvements - jitter, Android 4.x, HEVC

During our jitter research we decided that it's worth a shot to try synchronous video decoder instead of asynchronous (which is recommended by Google). This will allow for VRidge to work on Android 4.x devices.

Most of the older phones that weren't upgraded to Android 5.0+ aren't great VR devices but there are some phones that have 720p+ display there were simply forgotten by carriers and were left with Android 4.x OS. If you have decent Android 4.x based phone with 720p (or better) display and you want to try VRidge on it - e-mail us at support@riftcat.com with "Android 4 testing" as subject line. We'll contact you when we have something worth testing.

Creating synchronous decoder as an option forced us to do some refactoring of our decoding & rendering flow. This will make it easier to implement HEVC/H265 coding which is ~40% more efficient than H264 that we use now. This means 40% better quality at the same bitrate. We hope that we don't run into major roadblocks on the PC side. We'll start with NVIDIA GPUs first for HEVC.


We are also in process of getting Daydream & Pixel to explore new VR possibilities on Google's new flagship device but it's too early to talk about the details.

Thursday, November 3, 2016

Dev update #24


Regrouping and reorganizing

You may have noticed that our development slowed down lately so we want to explain why it happened.

VRidge userbase grew a lot during this year. When we first shown our prototype proof of concept in March we expected some people to be interested in beta testing but we didn't expect thousands.
We always love chatting about development, receiving feedback, solving user problems. Our devs were always directly responding to e-mails. This includes feedback, suggestions, bug reports, pc upgrade advice, general VR chat. It was taking more and more of our devtime.

Now VRidge's userbase is approaching 100 000 and the old ways that used to work had to be changed. Even if we hear back only from a small subset, let's say a 3% of our userbase - that's 3 thousand people.

That's why our devs will no longer be the frontline of our support system because it simply takes too much time discussing every single case individually. Our support system will of course continue to work but we'll try to write a general FAQ items on each type of issue so people can fix most of the problems on their own. We'll have a dedicated person handling support tickets that will be communicating with our devs on a regular basis.

Dev note:  We love solving problems directly but this way we'll be able to fix underlying problems faster instead of developing duct-tape workarounds for each invididual case. :) We'll also continue to hang around reddit. We didn't post too much lately but we read every single discussion.

If you contacted us lately and haven't received a response for a few days, don't worry - we'll try to cach up with everyone soon.

We're also reassigning our dev time. Until now we focused 100% on fixing problems instead of adding new features but there's a good percentage of our users running VRidge without any major problems. That's why we'll have 1 developer assigned to fixing problems and optimizing experience (including "the jitter", read more in the last paragraph of this post) while the other developer will work on adding new features.

Sound streaming

If you have decent ac router and non-crowded 5 GHz band you can usually stream wirelessly with quality, lag and reliability comparable to USB cable. Many people asked us for sound streaming for fully untethered experience. Being able to sit down on your couch without a bunch of wires limiting your movement sounds very nice. You will be able to turn it on/off because there's no need to stream another data channel and eat up extra bandwidth if you want to use PC headphones anyway. We want to deliver this feature in December.

PS Move / controller black screen

We tried to fix it on our side. We exchanged our data and test results with PSMoveService contributors but it seems that the problem is not linked to VRidge itself but to the way SteamVR handles multiple drivers. It also happened with Oculus DK2 with PSMoveService (without VRidge) or with Leap Motion + VRidge (without PSMS). PSMoveService on its own works fine. VRidge does work on its own as well. But it seems that when two drivers are interacting with SteamVR together they may randomly "collide" when they try to send messages at the same time. It's a speculation because it's hard to know what exactly is happening because SteamVR runtime isn't open source. You can read full discussion here. We'll definitely let you know if we have more news on this topic.

Jitter

It's a bit long so there's a short version at the bottom. Scroll down if you want to skip this wall of text.

We tested, we measured, we analyzed rendering with high FPS cameras, we spent many, many hours trying to exclude all possible factors of this problem with various phones and display types. We know that in fact 60 (58 actually but this doesn't change anything) frames are reaching the phone (assuming good connection conditions) but something's not quite right - it feels jittery.

Nearly all (90%) of jitter reports come from S6 & S7 (Exynos chipsets) but there were few Snapdragons reports too. It's hard to tell if those Snapdragons experience the same underlying problem as Exynos devices. Samsung devices also use different type of screen than other mobile phones (AMOLED).

We knew that it's visible the most in Gear VR version but it also happens in Cardboard version on S6 & S7 too. After extensive testing we found that other phones also experience not exactly silky-smooth rendering during head movements but it's not quite as noticeable. It may manifest itself in bluriness instead of jitter.

Human perception is a tricky subject and some people don't see a difference between 30/60/144 FPS. People can play games with lowend GPUs at 20 FPS while some others consider the experience ruined if it drops below 60 FPS. The worst thing about it is that it's hard to objectively judge how smooth our small changes affect the experience. We do some changes, we test them again and compare, we show it to different people but when we try to describe how smooth it feels on a scale of 1-10 we get very random results. We literally have to stare at the distance through the window for a while to "reset" our eyes to see differences between our test versions. That's why it takes so long to achieve the perfect streaming.

We tried to use numeric metrics like FPS or frame latency but even we use high FPS camera to record the screen it's hard to catch everything because human eyes don't see the same way camera does. We don't have a brain refresh rate or eye capture rate. We see motion and we know when it's fluid and when it's not. We can get used to non-perfect rates but some some people may get a headache after a while and that's not something we think is acceptable. We know that even regular VR HMDs can trigger headaches and motion sickness in some people so the problem is not 100% solvable but VRidge has a lot of room for improvement.

Since all of our tools and metrics failed to give our definitive, measurable answer that can be used to compare different versions of VRidge we decided that we need more metrics, tools, numbers, graphs, timings, all kinds of data. We wanted to fix it quickly, to find the the wrong line of code, the mistimed part of the pipeline but it seems that there's no magical, easy way of "fixing" it.

With one of our developers assigned to new features and one free to dig deep into the VRidge internals, we think this approach will be the better way of handling the jitter problem. It takes more time but this we can do it the right way instead of the quick way, which doesn't always work. Will it work? We very much hope so.

You can think of VRidge as a street between two points - your phone and the PC. The problem is that there's a bunch of intersections - sensor reading & fusion, sending sensor data, SteamVR/Oculus driver interaction, game capture, encoding, sending, transmission error concealment, receiving, reconstructing, decoding, rendering. A small traffic jam at any of those intersection can stall the whole flow. Cities have cameras to monitor traffic flow and to optimize urban planning. We need similar tools monitoring flow at each part of the pipeline.

We found one thing that can possibly reduce jitter a little bit in non-GearVR Android version. This will be probably pushed to beta Android channel next week. It's the first step of fixing the problem but definitely not the last one.

TL;DR; We didn't find a quick solution for old problem so we took the long way to diagnose them more thoroughly. Some of you are quite happy with current version. so we're splitting devtime 50:50 with 1 dev assigned to new features and 1 to optimize and fix old problems.