Thursday, November 3, 2016

Dev update #24

Regrouping and reorganizing

You may have noticed that our development slowed down lately so we want to explain why it happened.

VRidge userbase grew a lot during this year. When we first shown our prototype proof of concept in March we expected some people to be interested in beta testing but we didn't expect thousands.
We always love chatting about development, receiving feedback, solving user problems. Our devs were always directly responding to e-mails. This includes feedback, suggestions, bug reports, pc upgrade advice, general VR chat. It was taking more and more of our devtime.

Now VRidge's userbase is approaching 100 000 and the old ways that used to work had to be changed. Even if we hear back only from a small subset, let's say a 3% of our userbase - that's 3 thousand people.

That's why our devs will no longer be the frontline of our support system because it simply takes too much time discussing every single case individually. Our support system will of course continue to work but we'll try to write a general FAQ items on each type of issue so people can fix most of the problems on their own. We'll have a dedicated person handling support tickets that will be communicating with our devs on a regular basis.

Dev note:  We love solving problems directly but this way we'll be able to fix underlying problems faster instead of developing duct-tape workarounds for each invididual case. :) We'll also continue to hang around reddit. We didn't post too much lately but we read every single discussion.

If you contacted us lately and haven't received a response for a few days, don't worry - we'll try to cach up with everyone soon.

We're also reassigning our dev time. Until now we focused 100% on fixing problems instead of adding new features but there's a good percentage of our users running VRidge without any major problems. That's why we'll have 1 developer assigned to fixing problems and optimizing experience (including "the jitter", read more in the last paragraph of this post) while the other developer will work on adding new features.

Sound streaming

If you have decent ac router and non-crowded 5 GHz band you can usually stream wirelessly with quality, lag and reliability comparable to USB cable. Many people asked us for sound streaming for fully untethered experience. Being able to sit down on your couch without a bunch of wires limiting your movement sounds very nice. You will be able to turn it on/off because there's no need to stream another data channel and eat up extra bandwidth if you want to use PC headphones anyway. We want to deliver this feature in December.

PS Move / controller black screen

We tried to fix it on our side. We exchanged our data and test results with PSMoveService contributors but it seems that the problem is not linked to VRidge itself but to the way SteamVR handles multiple drivers. It also happened with Oculus DK2 with PSMoveService (without VRidge) or with Leap Motion + VRidge (without PSMS). PSMoveService on its own works fine. VRidge does work on its own as well. But it seems that when two drivers are interacting with SteamVR together they may randomly "collide" when they try to send messages at the same time. It's a speculation because it's hard to know what exactly is happening because SteamVR runtime isn't open source. You can read full discussion here. We'll definitely let you know if we have more news on this topic.

Jitter

It's a bit long so there's a short version at the bottom. Scroll down if you want to skip this wall of text.

We tested, we measured, we analyzed rendering with high FPS cameras, we spent many, many hours trying to exclude all possible factors of this problem with various phones and display types. We know that in fact 60 (58 actually but this doesn't change anything) frames are reaching the phone (assuming good connection conditions) but something's not quite right - it feels jittery.

Nearly all (90%) of jitter reports come from S6 & S7 (Exynos chipsets) but there were few Snapdragons reports too. It's hard to tell if those Snapdragons experience the same underlying problem as Exynos devices. Samsung devices also use different type of screen than other mobile phones (AMOLED).

We knew that it's visible the most in Gear VR version but it also happens in Cardboard version on S6 & S7 too. After extensive testing we found that other phones also experience not exactly silky-smooth rendering during head movements but it's not quite as noticeable. It may manifest itself in bluriness instead of jitter.

Human perception is a tricky subject and some people don't see a difference between 30/60/144 FPS. People can play games with lowend GPUs at 20 FPS while some others consider the experience ruined if it drops below 60 FPS. The worst thing about it is that it's hard to objectively judge how smooth our small changes affect the experience. We do some changes, we test them again and compare, we show it to different people but when we try to describe how smooth it feels on a scale of 1-10 we get very random results. We literally have to stare at the distance through the window for a while to "reset" our eyes to see differences between our test versions. That's why it takes so long to achieve the perfect streaming.

We tried to use numeric metrics like FPS or frame latency but even we use high FPS camera to record the screen it's hard to catch everything because human eyes don't see the same way camera does. We don't have a brain refresh rate or eye capture rate. We see motion and we know when it's fluid and when it's not. We can get used to non-perfect rates but some some people may get a headache after a while and that's not something we think is acceptable. We know that even regular VR HMDs can trigger headaches and motion sickness in some people so the problem is not 100% solvable but VRidge has a lot of room for improvement.

Since all of our tools and metrics failed to give our definitive, measurable answer that can be used to compare different versions of VRidge we decided that we need more metrics, tools, numbers, graphs, timings, all kinds of data. We wanted to fix it quickly, to find the the wrong line of code, the mistimed part of the pipeline but it seems that there's no magical, easy way of "fixing" it.

With one of our developers assigned to new features and one free to dig deep into the VRidge internals, we think this approach will be the better way of handling the jitter problem. It takes more time but this we can do it the right way instead of the quick way, which doesn't always work. Will it work? We very much hope so.

You can think of VRidge as a street between two points - your phone and the PC. The problem is that there's a bunch of intersections - sensor reading & fusion, sending sensor data, SteamVR/Oculus driver interaction, game capture, encoding, sending, transmission error concealment, receiving, reconstructing, decoding, rendering. A small traffic jam at any of those intersection can stall the whole flow. Cities have cameras to monitor traffic flow and to optimize urban planning. We need similar tools monitoring flow at each part of the pipeline.

We found one thing that can possibly reduce jitter a little bit in non-GearVR Android version. This will be probably pushed to beta Android channel next week. It's the first step of fixing the problem but definitely not the last one.

TL;DR; We didn't find a quick solution for old problem so we took the long way to diagnose them more thoroughly. Some of you are quite happy with current version. so we're splitting devtime 50:50 with 1 dev assigned to new features and 1 to optimize and fix old problems.

32 comments:

  1. Okay, so what are the improvements that will bring the new version?

    ReplyDelete
  2. Great work guys, for me vridge is the best tool for phone vr experience.
    Is possible include HDMI option? I mean, there are many vr android hmd in the market, I have two with 1080p and 2k resolution vr hmd , these includes android and niburu, I can run vridge very good , better frame rate than phone, but these vr hmd includes hdmi port, I love the possibility to add hdmi option on vridge for these hmd.
    Thank you all vridge team

    ReplyDelete
    Replies
    1. We are considering it, but no promises.

      Delete
  3. A simple output to HDMI option would be great for those with a PSVR! :) I'd happily pay extra for it!!

    In this mode you would disable everything to do with the phone/Android - just outputting the video frame to a chosen HDMI port. The movement/tracking would be set to FreeTrack for everything.

    This would be a killer feature!

    ReplyDelete
    Replies
    1. We are considering it, but no promises.

      Delete
  4. Let's concentrate on the ONE FEATURE that Vridge needs desperately to compete: The ability to play PC games (ala Trinus). Do this and you will blow the other competitors out of the water. Ignore this and you will founder.

    ReplyDelete
    Replies
    1. An easy start would be to add in dolphin vr 😆. Most of the hard work is already done for you. All ya gota do is stream it with sound...

      Delete
    2. Sounds is something we are working on right now. Regarding the ability to play games like Trinus is something we want to do.

      Dolphin VR works if you are using version for 0.6-0.8 SDK. Newer ones for 1.3+ SDK MIGHT work with Revive, but I haven't tested it myself so I'm not sure.

      Delete
    3. Dolphin VR works well with SteamVR. Download the proper version. Don't fall into the trap of coding non VR games. Every game is different and their none VR UI is a mess. The riftcat crew would only spend their limited resources on broken experiences. By focusing the effort on VR games Vridge will only get better for future AAA VR games.

      Delete
    4. At the moment I've setup a launch path for dolphin vr in the nvidia driver app (whatever it's called).
      Then I just simply stream the program through moonlight and play from there. That simple and it works perfectly. Completely cable free with sound and controller support from anywhere in the house.

      Delete
  5. I hope you guys release the iOS version soon, that would be great!

    ReplyDelete
  6. when moonlight support??? this mode stream is amazing

    ReplyDelete
    Replies
    1. I would argue that riftcats streaming service is just as good. Maybe look at your router.

      Delete
  7. Please sort out Oculus 1.3 SDK compatibility AND h.265 codec..
    These are the two major things holding back vridge!

    ReplyDelete
    Replies
    1. You can use revive to add in 1.3 for now. Works great

      Delete
    2. From my understanding, there are legal issues with the 1.3 SDK, and since revive works great, there is no need to dance with that.

      Delete
    3. h.265 is on our radar. We won't be adding 1.3 SDK support since Revive works well enough.

      Delete
  8. Are there any plans to support the Gear VR gyroscope? It's much better than the built in one on my S7 Edge and I guess that's why it's juddery.

    ReplyDelete
    Replies
    1. GearVR version already utilizes all GearVR sensors. S7 Edge is having some issues with juddering right now on both GearVR and regular version. We are currently trying to fix it.

      Delete
    2. Good to hear you are working on it. I tested the app with sideload in S7 (gearvr) and I was not able to remove the lag/jiddering. I have really good 5GHz network and even the lowest resolution was not enough. Good to hear that you are working on this so I can retest it again later. For now, I'm just waiting. :)

      Delete
  9. When you talk about "jitter", what EXACTLY do you mean? Maybe a video would be a good idea to define it? That's a way to help to get rid of the subjectivity

    ReplyDelete
    Replies
    1. Ah sorry, it's actually fixed by setting tracking to Freetrack orientation and position.

      The effect (using phone orientation) is just a lack of smooth head tracking, same as you get with the Gear VR running Cardboard/non-Oculus apps that can't use its hardware gyro etc.

      Works amazingly well now. Awesome work. Thanks :)

      Delete
  10. Are you guys able to make use of the daydream apis / VR mode in android 7.0? On my pixel, the output is so laggy and jittery as to be uncomfortable but having just tried out daydream (on my cardboard headest) the tracking/smoothness was a huge step up from cardboard apps on the same device.

    ReplyDelete
    Replies
    1. We do want to fully support daydream. We recently ordered Pixel, but it hasn't arrived yet. Once we have it at our office we'll try to optimize VRidge for it.

      Delete
  11. Please add Direct Mode support for SteamVR. Without direct mode impossible to use such important feature as Asynchronous Reprojection which can significantly increase ingame fps on a weak system (laptops).

    P.S. Sorry for bad English

    ReplyDelete
    Replies
    1. It already works in Direct Mode, but we are proccessing VR games a bit differently than Vive/Oculus Headsets so it's not that simple to get Asynchronous Reprojection to work with VRidge.

      Delete
    2. So probably problem on my side, because my SteamVR start in Extended Mode and if I press "Enable DirectMode" it restart again in Extended or just tell than SteamVR don't work correctly and lost the stream.


      P.S. I use it on laptop with external GTX 960, connected via miniPCI-E x1 bandwidth (instead of normal x16), but Driver and Steam saw this as normal GTX 960 and it couldn't be a reason for this problem.

      As i guessing, async reprojection works on SteamVR side (directly with game, not with VR headset) and Riftcat just capture and encoding final video stream (already with additional frames, made by reprojection). Probably I'm wrong.

      Delete
  12. If Someone needs it. https://github.com/Shockfire/FakeVive Google Earth Vr with ps move and riftcat. It's cool!!!!

    ReplyDelete
  13. Question, what are the chances of purchasing "just" the vibe controllers or the (my preference) oculus touch controllers with sensors, and using them with vridge??? Will steam vr detect them with the mobile hmd?

    ReplyDelete
    Replies
    1. You need the basic station too. And The basic station not see your Hmd. I think it's not really work how you want to.

      Delete