Dev update #12 - FreeTrack and other tracking options

This week we focused on improving our head tracking to reduce drift. We're adding FreeTrack support and alternative sensor fusion algorithms. 

Desktop client version is 0.5.4.4, mobile app version is 0.8.7. 

Why are we releasing update on Wednesday instead of Thursday this week? Tomorrow is public holiday in our country so everything (including our office building) is closed. Most of the team will be away on this long holiday weekend but we will catch up and respond to all e-mails and comments on Monday. 

Alternative tracking (mobile)

Until now we were using Cardboard API as the only tracking option. Today we are adding more options. Cardboard API is still the most tested with probably the lowest latency but many of you asked for something more stable. We decided to use alternative sensor fusion algorithms to reduce drift. Keep in mind that using magnetometer with magnet-button-cardboard can result in odd results.

We are also adding option to disable tracking completely because some people wanted 100% stable picture for cinema-like apps.

Our new options are:

Tracker	Required sensors	Pros	Cons
Cardboard API	Gyroscope Accelerometer	Lowest latency	Highest drift
Use magnetometer (stability)	Gyroscope Accelerometer Magnetometer	Lowest drift	Highest latency
			May interfere with cardboard magnet-button
Use magnetometer (responsiveness)	Gyroscope Accelerometer Magnetometer	Very stable with low drift	Latency still higher than Cardboard API
			May interfere with cardboard magnet-button
No gyroscope	Accelerometer Magnetometer	Works without gyroscope	Extremely shaky
No tracking	none	No drift :)	No tracking at all

You can access new options by tapping more in top-right part of the connection screen.

We will try to reduce delays on driftless options but if you are looking mostly ahead without much head movement you might want to use magnetometer optons for more stable experience. Remember that you can reset head position by pressing Ctrl+Alt+Shift+R.

External tracking support (FreeTrack 2.0 protocol)

Many people already have their own DIY tracking and we received many requestes to provide a way to use FreeTrack-formatted data. This is our first iteration of this protocol but we wanted to get it out ASAP so you can start experimenting with your devices. We tested with OpenTrack (FreeTrack 2.0 protocol).

Positional data is supported by SteamVR driver only currently. Oculus emulation layer will use rotational data only.

Dev update #11

We don't have much to show this week but most of our work will carry over to future updates. We spent a lot of time trying to counter sensor drift but our algorithms are not stable and tested enough to release them. Expect more news on that next update.

Desktop client version is 0.5.4.3, mobile app version is 0.8.6.

Changes

• Log me in automatically should work more reliably. Let us know if RiftCat client forgets your password again.
• VRidge should no longer fall behind and crash when encoding 1440p on previous generation NVIDIAs with NVENC checked on. See 1440p compatibility paragraph below.
• RiftCat Client should no longer freeze while connecting to phone (MTU discovery timeout freeze).
• RiftCat Client should no longer crash during phone discovery (System.Net.Sockets.TcpListener.AcceptTcpClient crash)

1440p compatibility

After last week's 1440p update we received some reports about 1440p not working with some video cards. Here's a list of compiled results and reports. 

Generation	Example hardware	1440p @ 60 FPS	1440p @ 50 FPS	1080p @ 60 FPS
AMD GCN 1.2+	R9 285, R9 380, R9 Fury X, R9 390***	✓	✓	✓
NVIDIA Maxwell	GTX 750, GTX 830M-860M, GTX 930+	✓	✓	✓
Intel HD & Iris (Haswell+)	Intel Core i3-4xxx+	✓	✓	✓
NVIDIA Kepler	GTX 650+, GTX 740+	❌	✓*	✓
GCN 1.0 & 1.1 AMDs	R9 240, R9 290, HD 7700+	❌**	❌**	✓
Intel HD (Ivy Bridge)	Intel Core i3-3xxx	?	?	?
Intel HD (Sandy Bridge)	Intel Core i3-2xxx	?	?	?

* Use 60 FPS option. VRidge will try to encode at highest possible framerate.
** According to AMD developer it's not possible to encode video with resolutions beyond 1080p with GCN 1.0 & 1.1 AMD.
*** There are two 390 variants. One is GCN 1.1, other is GCN 1.2. Removed - 390 is GCN 1.1.

We did not test all of this ourselves. If it's wrong, please let us know so we can correct it. Please keep in mind that some laptop (-M suffix) variants use different chipsets than non-M variants.

Dev update #9

This week brings rewritten SteamVR driver (no more flickers and upside down textures) and mobile optimizations. We wanted to refactor some code and fix some bugs before moving on to bigger features. 

You might notice slightly lower latency (especially in SteamVR mode). If you are S7/S7 Edge user, please check it out too because this updated resolved artifacting for at least one person. :)

Desktop client version is 0.5.4.1, mobile app version is 0.8.4.

---

Please reinstall SteamVR driver after upading (close Steam first)

---

SteamVR driver update

Updated - known issues:

• SteamVR games flicker in some cases. Reported on both Nvidia and AMD. 
• The Lab may crash during the game on loading screens.
• VR Render resolution is now 1920x1080 previously it was 2364x1461. Small quality drop might be noticeable for people with high-res phones. It will be tied to streaming quality next update.

We've rewritten our driver to use automatic driver registration process so it won't get overwritten by SteamVR updates. It has lower latency than previous driver and enables use of many SteamVR features: seamless VR context switching between apps, multi-layer compositor and more. It also fixes some problems with textures being upside-down or aligned incorrectly in some games. It will also allow us more flexible development in the feature.

You will need one last reinstall (click the Install SteamVR button). Remember to close Steam first. Our old driver will be removed in the process.

If Riftcat does not locate vrpathreg.exe file correctly, please point it to the file manually. It should be located in SteamVR\bin\win32 directory.

Android memory optimizations

We've updated large part of stream handling and networking modules on the Android side. It should now deliver frames more reliably with lower latency. 

It also manages memory a lot more efficiently. This result in less micro-hangs caused by memory cleanups (GC) which could cause frame drops @ decoder. 

Android (optional) stream changes for certain phones

We're also adding two experimental options. They can reduce or improve performance so we're not enabling them by default. The first one (Intel) can improve performance for Asus Zenfone 2 (and other Intel-based phones). The second might help with most various phones (but is targeted at certain Snapdragon models). 

You can access this screen by tapping More in top-right corner of the connection screen.

If your stream is lagging behind and/or showing blocky-floaty artifacts, try one or both. Please let us know if it changes anything.

Hand Tracking for SteamVR + VRidge

Cameron Swisher wrote a guide on how to use Leap Motion as Vive Controllers. According to Cameron it works with VRidge so we thought we should share it since it allows to play games like The Lab (some tests are broken) which before were inaccessible.

Warning: This program is still very new and prone to be buggy and glitchy, so expect bugs and for it to not be 100% perfect. Will make a guide later on how to compile this manually for unreleased builds.

1. Make sure you have these before installing the SteamVR Leap Motion Driver:

• Install "Leap Motion Orion SDK". (Click on the Orion Beta (VR) button), Account is required to download the software
• Install SteamVR (Most of you should already have it)
• Install the Visual C++ 2015 Update 2 redistributables (32 and 64 bits) https://www.microsoft.com/en-us/download/details.aspx?id=51682

2. Download the file called "Steamvr_leap_driver_(version name_.zip" (created by cbuchner1 @ Github) (If you have installed Alpha One version before, make sure to uninstall Alpha One and install Alpha Two).

3. Extract zip file and run setup.exe.

4. Go through the installation process (just keep clicking "Next").

5. Once the installation process is done just close the window.

6. If everything worked, controller icons in SteamVR window should lit up green now (try restarting SteamVR if it doesn't for some reasons).

What about controls?

Currently following gestures are possible (subject to change a lot, so visit here to keep up to date):

• Bending of the index finger maps to the trigger button, like you would fire a gun
• Clenching the middle, ring, pinky finger to a fist maps to the grabbing buttons
• The thumbpress gesture (just point the thumb in the direction of your palm) touches and clicks the trackpad
• Flat hand held in front of you, palm towards your face is used for application menu button
• Flat hand held in front of you, palm away from you is used for system menu button

Leap Motion in action

Note: The Lab is working, but some tests are broken currently.

Note: Minecraft does indeed work with both, the program and VRidge, see the link for details. Very buggy since the hand tracking program is still new, but it will get better as time goes on. The Minecrift program (linked above) is still kinda new and some bugs need to be worked out. The main movement method is teleportation.


You can find the list of currently known issues/bugs here.


Cameron will be updating his guide and you can send him your feedback in the google docs here.

Dev update #8

This week we are not deploying desktop client patch - Android is getting small update with version 0.8.3. We optimized several things in mobile app but most of the week was spent on improving our SteamVR driver (new version is not ready yet) and searching for Android artifacting solution.

Small QOL changes

Several people suggested to hide the vertical white bar because it was visible in the viewport. It will now automatically hide after 10 seconds. Settings button will now automatically dim by 70% too.

If you want to align your cardboard to the screen, tap anywhere to show the vertical bar again (it will disappear after 30 seconds after manual toggle).

We also added "Never ask me again" option for bluetooth prompt so it won't annoy you anymore. We're leaving a small note at the bottom if it's online.

There were some other changes, optimizations mostly but nothing worth writing about. Some bytes were spared. Some bytes move a little faster.

SteamVR driver progress

As some of you noticed you had to reinstall our SteamVR driver after yesterday's Steam update. Next version of our driver will be properly registered with Steam API and won't require reinstallation anymore. Actually it won't even require manual installation, it will be done automatically.

We are not entirely sure but after some changes in our pipeline it seems that SteamVR latency is even lower now. It's hard to measure but it all seems more fluent.

New version will also allow better context switching between VR apps. More on this when it's out.

The ongoing battle with artifacts - frame loss / latency @ decoder

This is very annoying problem because whenever we finally think "Okay, that was it, fixed" something new comes up. This is the issue that can be identified by tapping top-left part of the streaming screen and seeing high (above 0.1 ms) decoder latency and constantly raising Frames lost @ decoder counter.

We had ~30 reports of this happening and we can now share some data - this is top 8 of reported problems.

Phone name	Reported cases	Install count	%	Chip
Asus Zenfone 2	5	89	5.62%	Intel® Atom™ Quad Core Z3580
Galaxy S4	5	84	5.95%	Exynos 5410 | Snapdragon 600
Galaxy S7	2	125	1.60%	Exynos 8890 | Snapdragon 820
Galaxy S7 Edge	2	339	0.59%	Exynos 8890 | Snapdragon 820
Redmi Note 2	2	60	3.33%	MediaTek Helio X10
Nexus 6	1	110	0.91%	Snapdragon 805
HUAWEI Honor 6	1	8	12.50%	Hisilicon Kirin 910 Octa Core
Xperia Z C6603	1	28	3.57%	Snapdragon S4

Galaxy S4 has the most problems. There were 2 users reporting hard device crashes. 

Zenfone 2 also has constant artifacting. No crashes were reported but it is still unusable if it's turning into blocky pixel mess after several seconds.

Galaxy S7/Edge seems to have something special about its decoder too.

These numbers are high because when we look at top 10 most popular devices of our users (S6 & OnePlus One leading the pack) there are 2000+ installs shared between them and 0.1% error report rate. Also, the reported errors are probably caused by something else because there were more issues than artifacting there.

One thing I would like to know. Is there anyone that got VRidge to work successfuly with bolded phones in the table? 

Please let us know if you got it to work. I know about one Zenfone 2 user that fixed it by changing CPU governor to interactive but the same didn't work for other phones. Send us an e-mail, PM @ reddit or use any other contact channel listed at the right column, at the top of the page.

Dev update #7 - The one with SteamVR

Before we start, we want to say one thing:

It's a beta! Please do not buy SteamVR games specifically for VRidge.

This technology in current state relies on many third party components and compatibility cannot be guaranteed. You can test SteamVR compatibility with many free SteamVR games available on Steam.

We've been working on SteamVR integration for quite some time and we finally can release a version that should work more often than not. I'm cross-posting these notes to reddit because many of you already have accounts there and it's an easy way to discuss and troubleshoot as a community.

https://www.reddit.com/r/vridge/comments/4fu4sn/dev_update_7_the_one_with_steamvr/

SteamVR

"It's happening!"

Please keep in mind that this tech is still in experimental / unstable stages. Save your games often because crashes can and will happen. Not all games work and compatibility may come and go with game updates. We tried some most-wanted titles and here are quick summaries what you can expect:

• Euro Truck Simulator 2
• You need to go to Steam properties of ETS and use beta branch named "Oculus". Then you need to add -openvr in launch arguments. Disable Steam Desktop Theater for this game so it won't launch in flat mode.
• Elite: Dangerous
• It works but has issues with blurry text. This problem is common even on real HTC Vive devices. You can read some research here.
• We recommend using 1920x1080 for stream resolution. Make sure your phone has 1080p display as well.
• If you are getting black screen, see this post by@adrensnyder.
• Virtual Desktop / Steam Desktop Theater
• Those apps have extra lag and capture problems because essentialy everything is captured and rendered twice as many times as native VR games. We are trying to figure out ways to reduce the latency in those cases.
• We recommend using 1920x1080 for stream resolution. Make sure your phone has 1080p display as well.

There are few steps you need to follow before SteamVR games can work.

1. Install SteamVR runtime in your Steam.

2. Install VRidge SteamVR driver. SteamVR path should be filled automatically. If it's empty, manually point it to vrpathreg.exe, which is located somewhere in your SteamVR location. This can be done after connecting your mobile app to the client. 

3. Launch SteamVR. You need to launch SteamVR while your mobile app is connected to RiftCat. If you launch SteamVR without mobile app paired, close it and start it again after connecting mobile app.

If you can't see this button, start SteamVR from your library (it's a regular game)

If you are asked to run "Room setup" please choose standing experience and enter your height when you are asked to. Otherwise your camera may be too close to the ground for comfortable use.

SteamVR setup/tutorial VR mode is upside down and aligned incorrectly. It won't affect other games.

4. Small SteamVR window will open. If your mobile device was connected when you started SteamVR, it should display Connected and SteamVR games will show Play in VR as an option.

That's it! You should be able to play a lot of amazing SteamVR games now. If your mobile device drops connection you can reconnect VRidge by reopening mobile app. It shouldn't require game/SteamVR restart.

If you are stuck with Headset not detected message, close SteamVR, make sure your mobile device is connected to Riftcat client and start SteamVR again.

If you have problems, please reach out to us through our help center (we will keep adding known issues with SteamVR there soon) or reddit (so we can share workaround and solutions as a community):

https://www.reddit.com/r/vridge/comments/4fu4sn/dev_update_7_the_one_with_steamvr/

Android CPU usage optimized == a lot less pixel artifacts

We solved one very common cause of pixel artifacts happening due to "Frame loss @ decoder". We optimized some of our algorithms and the overall CPU usage dropped by ~50%. This is important because Android devices will generate less heat and therefore Android OS won't downclock CPU to counter the heat.

Core frequency downclocking due to high heat (thermal throttling) fixed pixel aritfacts for nearly all of our devices. There is still room for more optimizations so if you are still experiencing high frame loss @ decoder, don't worry - we will keep trying to optimize and find all the problems.

We have a request - can anyone with Asus Zenfone 2 reply in comments if they got VRidge working correctly? It seems that everyone with this specific phone model is getting constant artifacts.

Memory leaks fixed in NVENC and Media Foundation encoder modules

Some of you reported that staying in-VR with VRidge for longer than 30-40 minutes resulted in VRidge crashing due to lack of memory. We fixed two memory leaks that caused RAM usage to grow 30-100 MB every minute. Sorry for the crashes, totally our fault.

Dev update #6 - Win 7, 8, 8.1 support for NVIDIA users

Update for NVIDIA users:

Windows 7 & 8 is still having some issues (infinite awaiting VR stream/crash) due to missing .DLL. Windows 8.1/10 should work. Possible workaround here.

---

The numbers in the title aligned quite nicely. This week we focused on NVIDIA support for Windows 7+. Next focus is solving artifacts @ Android's video decoder.

NVENC / Windows 7+ support for NVIDIAs

Note: If this update causes more artifacts for you, go to stream quality settings and disable NVENC. It's still highly experimental feature but it's enabled by default for all NVIDIA users.

We implemented NVENC. Until now we used Media Foundation to handle video encoding but unfortunately it requires Windows 8.1+ for AMD or Windows 10 for NVIDIA. It also has some undefined issues ("Not Enough Storage" errors and some other crashes) with AMD video cards, especially after WIndows 8.1->Windows 10 upgrade instead of clean installation.

NVENC is alternative encoding layer supported by NVIDIA driver. It has some extra features for low-latency streaming and anti-lag measures (error concealment). They are not all enabled yet because they amplify Android decoder problems (read about it later in this post) but we already tested some techniques and you can expect less artifacts and even lower latency in the future thanks to NVENC capabilities.

AMD users might wonder "Can we get something like this too?" The answer is yes (via direct VCE implementation) but we don't have any ETAs yet. This might solve some crashes for AMD users, especially those with mobile Radeons.

The ongoing battle with artifacts

We can finally reproduce one major problem reported by many users -  frame loss @ video decoder. 

For some people it starts very shortly after start of the stream (basically preventing VRidge usage because it artifacts all the time). Other people experience it after several mintues (the higher quality settings, the sooner it happens). 

Now that we can finally reproduce it - it shouldn't take too long to finally solve this problem.

When can I play new VR games (like Elite: Dangerous, ETS, ATS, Virtual Desktop, etc.) with Riftcat? 

This is the question asked the most. One of our devs has been working for several weeks to get them working with VRidge. Expect more news on this matter next week.