{"id":700,"date":"2016-04-28T16:13:57","date_gmt":"2016-04-28T15:13:57","guid":{"rendered":"http:\/\/www.brucewiggins.co.uk\/?p=700"},"modified":"2017-01-06T11:30:01","modified_gmt":"2017-01-06T11:30:01","slug":"youtube-360-vr-ambisonics-teardown","status":"publish","type":"post","link":"https:\/\/www.brucewiggins.co.uk\/?p=700","title":{"rendered":"YouTube 360 VR Ambisonics Teardown!"},"content":{"rendered":"

UPDATE : 4th May 2016 – I’ve added a video using the measured filters. This will be useful for auditioning the mixes before uploading them to YouTube.<\/p>\n

So, I’ve\u00a0been experimenting with YouTube’s Ambisonic to Binaural VR videos. \u00a0They work, sound spacious and head tracking also functions (although there seems to be some lag, compared to the video – at least on my Sony Z3), but I thought I’d have a dig around and test how they’re implementing it to see what compromises they’ve made for mobile devices (as the localisation could be sharper…)<\/p>\n

Cut to the chase<\/strong> – YouTube are using short, anechoic Head Related Transfer Functions that also assume that the head is symmetrical. \u00a0Doing this means you can boil down the Ambisonics to Binaural algorithm to just four short Finite Impulse Response\u00a0Filters that need convolving in real-time with the B-Format channels (W, X, Y & Z in Furse Malham\/SoundField notation – I know YouTube uses ambiX, but I’m sticking with this for now!). \u00a0These optimisations are likely needed to make the algorithm work on more mobile phones.<\/p>\n

So,\u00a0how do I know this? \u00a0 I put\u00a0a test signal (log sine wave sweep) on each of the B-Format channels and then I recorded back the stereo signals allowing me to measure a left and right response for each of the four channels individually. \u00a0I carried this out when the phone was both facing front and at +90 degree to check the rotation algorithm<\/a> was working. \u00a0Below are the Head Related Impulse Responses (HRIRs) I got back (click for higher res) – these will also contain any filtering etc. from my phone and computer, but have turned out pretty well considering I had to hold the phone still in the correct position!<\/p>\n

\"You<\/a> \"You<\/a><\/p>\n

The fact that the left and right HRIRs are identical (or polarity inverted) show that they’ve used the symmetrical head assumption and the X and Y channels swapping between facing 0 and 90 degrees shows the rotation being carried out on\u00a0the Ambisonic channel signals. \u00a0Once you’ve got these HRIRs, generating the Left and Right headphones signals in the phone is (where the multiply with a circle indicates convolution). \u00a0However, this is likely to be carried out in the frequency domain where it’s more efficient.<\/p>\n

\"L=W\\otimes\"R=W\\otimes<\/p>\n

Below is also the frequency response of the four Ambisonic HRTFs where you can see YouTube cutting off the response at around 16.4kHz (again, click for higher resolution).<\/p>\n

\"WXYZ<\/a><\/p>\n

Once the W, X, Y & Z filters were obtained, although I sent log sine sweeps every 12 seconds (for a 10 second sweep), due to the fact I was recording in the analogue domain (no clock sync between phone and computer) some clock differences have caused the filters to be slightly mis-aligned. \u00a0This can be most easily seen if a source is simulated at 90 degrees with respect to the head of the listener, as this should exhibit the greatest amplitude difference between the ears once the Ambisonics to Binaural algorithm is implemented. \u00a0This was achieved when the an extra 2 samples (2\/48000 of a second) delay was added to each 12 second ‘jump’. \u00a0The frequency plots for a front facing head and a source at +90 degrees, and a +90 degrees facing head and a source at 0 degrees are shown below (so in the first plot, the left ear should be loudest, and in the 2nd plot the right ear should be loudest). \u00a0I’ve also trimmed the HRIRs to 512 samples and windowed the responses using a hanning window. \u00a0It is likely that the actual clock difference isn’t a multiple of 1 sample, so this method isn’t quite ideal!<\/p>\n

\"ILD<\/a> \"ILD<\/a><\/p>\n

These plots should be identical, but measurement inconsistencies can be noted between the far ear responses to the source (as these are lower level, they’re more prone to error), and by inspection it seems like the head at +90 degrees is a slightly better capture at this point. \u00a0Also, remember that the response above 16.4kHz is not worth worring about as YouTube filters out frequencies above this value.<\/p>\n

More to follow…<\/p>\n

Ambisonics To Stereo<\/h3>\n

An aside on stereo. \u00a0I’ve not put any plots up yet, but it seems like the Ambisonics to Stereo algorithm used (on non-android YouTube) is simply:<\/p>\n

\"L=W+Y\"
\n\"R=W-Y\"<\/p>\n

Google should really look into using UHJ<\/a> for their Ambisonics to Stereo conversion…for an example of the difference, listen to the audio on these two videos. \u00a0The first one is Ambisonics to UHJ, the second will be YouTube’s Ambisonics->Stereo algorithm detailed above (to carry out this test DO NOT use the Android YouTube app!)<\/p>\n

UHJ : https:\/\/youtu.be\/d2rrsjt44rs<\/a>
\nW\/Y Stereo :\u00a0https:\/\/youtu.be\/i3TjIiaKDVU<\/a><\/p>\n

How do they Sound?<\/h3>\n

Here’s video using the extracted filters in Reaper in order to convert the 1st order Ambisonic audio to binaural.<\/p>\n