Thursday, September 10, 2015

Temporal Reprojection and SAO

We've recently re-visited our AO solution with the goal of improving its performance on consoles. We currently use the Scalable Ambient Obscurance algorithm presented by Morgan McGuire. Out target was to bring down the cost of the entire effect to something between 1-1.5ms on Xbox One. To achieve this it was clear that we needed to reduce the number of taps we were taking for each AO sample. An important part of this work went towards improving the efficiency of the temporal reprojection of our AO buffer. I thought I'd share a few observations we've made along the way.

Distributing AO Samples

When reprojecting data the key to success is to make sure your samples are well distributed through time. The Halton sequence was made popular for reprojection methods after Brian Karis presented his High Quality Temporal Supersampling. It is a sequence that gives well distributed samples in space as well as in time.

If we add an offset to a single AO sample we can see that after 2π it repeats (as expected)



So we use 8 samples ranging from [0, 2π] distributed by using the first 8 terms of a base 3 Halton sequence: {1/3, 2/3, 1/9, 4/9, 7/9, 2/9, 5/9, 8/9} x 2π



And this is the result using 6 AO samples.



Notice that there is some banding that can appear. By adding a dithered offset to the current sample's radius we can remove this quite nicely. We use a 4x4 pattern based on the Bayer matrix.



And here are our samples distributed through time. If anyone has a better way of distributing these I would be very interested to know.



Reprojection function

When doing any temporal reprojection it is crucial to have a good reprojection function. I like to refer to this as a similarity function. Its purpose is to identify how likely it is that the reprojected samples correspond to the samples of the current pixel. When writing this function it's quite important to have a convenient way to visualize it. If the function gets complicated, it's a good idea to isolate the different terms of the function so that you can reason and debug them individually. The reprojection function we use is a combination of three main terms.

Disocclusion Term


This is a simple term which identifies depth differences and classifies previous pixels as disoccluded. We use the relative depth difference described by Huw Bowles in Iterative Image Warping.
depth_similarity = saturate(pow(prev_depth/current_depth, 4) + DEPTH_MIN_SIMILARITY);

Velocity Term

The second term is also very straight forward and consists simply of reducing the similarity for fast moving pixels. A moving pixel has less chance of reprojecting successfully than a still one.
velocity_similarity = saturate(velocity * VELOCITY_SCALAR);

Dangerous Samples Term

The idea is to identify if the AO samples we are gathering are touching moving objects. To do this efficiently, we encode a moving bit as part of the depth buffer info passed in to the SAO algorithm. If you use the mip chain described by the SAO paper, make sure that you forward that 'moving bit' to the lower levels of the mip chain. This idea was presented by Anton Michels during the Labs R&D: Rendering Techniques in Rise of the Tomb Raider presentation earlier this year. Since each AO sample will need to read the depth information we get the 'moving bit' read for free.
samples_similarity = saturate(num_moving_samples * MOVING_SAMPLES_SCALAR);
samples_similarity *= (LOW_VELOCITY_SIMILARITY - MOVING_SAMPLES_MIN_SIMILARITY);
samples_similarity = lerp(samples_similarity, prev_samples_similarity, 0.9);
samples_similarity = min(samples_similarity, current_samples_similarity);


To try and invalidate samples associated with fast moving object, the Dangerous Samples term is accumulated through time. An idea described quite well by Oliver Mattausch in his TSSAO Gpu-Pro2 article which he called 'smooth invalidation'.



Here's the kind of ghosting we get without identifying 'dangerous' samples.



With this term as part of the reprojection function we can eliminate most of the ghosting that arises from the temporal reprojection:


Putting it all together

The final similarity term is calculated by combining all terms together.
similarity = depth_similarity * LOW_VELOCITY_SIMILARITY - velocity_similarity;
similarity *= (LOW_VELOCITY_SIMILARITY - HIGH_VELOCITY_SIMILARITY);
similarity = saturate(similarity - samples_similarity);

Well, that's it! Nothing too ground breaking but I thought I'd share. If anyone has ideas or suggestions on how to improve any of this please let us know!

29 comments:

  1. None of these images are working for me :/

    ReplyDelete
    Replies
    1. They're hotlinked to dropbox which returns HTTP 429 (Too Many Requests) for excessive use.

      Delete
    2. Should work now (I uploaded the images to blogger directly). Thanks for letting me know!

      Delete
  2. Hi,

    is your code available somewhere? I'm struggling with exponential smoothing of GTAO and I could use some reference implementations.

    ReplyDelete
  3. Could you please elaborate on the "moving bit" from the Dangerous Samples part?
    The TSSAO paper mentions that we should compare all depth samples with their equivalents from the previous frame so this pretty much doubles our texture tap count! Is this right? You've mentioned that we'll have this "moving bit" for free, how exactly did you accomplish that?

    The Tomb Raider paper is nowhere to be found (even the paid ones are incorrect!) so I can't even check what Anton had to say about it.

    Thanks!

    ReplyDelete
  4. We are available 24×7 for you. You can also E-mail us for special requirements. We have Russian and Indian tastes for our clients. Just share your requirements with us and fulfill your lust with her. Aerocity Escorts | Delhi Escorts

    ReplyDelete
  5. Webroot.com/secure : Protect your system by downloading, installing, and activating Webroot SecureAnywhere on webroot.com/secure.Webroot.com/secure antivirus gives internet security and offline security to your devices for business and individual use. Webroot Install With Key Code . webroot.com/secure - 123.hp.com/laserjet | Webroot install with key code | webroot.com/secure123.hp.com/laserjet |

    ReplyDelete

  6. ufafc



    Webroot SecureAnywhere on webroot.com/secure.Webroot.com/secure antivirus gives internet security and offline security to your devices for

    ReplyDelete
  7. Hello Sir I saw your blog, It was very nice blog, and your blog content is awesome, i read it and i am impressed of your blog, i read your more blogs, thanks for share this summary.
    Facebook Not Working Properly

    ReplyDelete
  8. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    check here

    ReplyDelete
  9. Iwas more than happy to find this site. I want
    to to thank you ffor your time for this particularly
    wonderful read!! I definitely savored every little bit of it and i also have yyou saved
    aas a favorite to check out new stuff in your website.


    카지노사이트
    온라인카지노
    카지노사이트추천

    ReplyDelete
  10. Have you ever considered writing an e-book or guest authoring on other sites? I have a blog centered on the same information you discuss and would love to have you share some stories/information. I know my audience would enjoy your work. If you are even remotely interested, feel free to send me an e-mail


    카지노사이트
    바카라사이트
    카지노사이트홈

    ReplyDelete
  11. Just wish to say your article is as amazing. The clarity to your publish is simply spectacular and i could think you are a professional in this subject. Fine with your permission let me to grab your feed to stay up to date with impending post. Thanks one million and please keep up the enjoyable work.



    토토사이트
    토토사이트추천
    토토 대표 사이트

    ReplyDelete
  12. I am very satisfied with visiting here. Thank you for good content. And please visit my website as well.


    토토
    스포츠토토
    메이저사이트 목록

    ReplyDelete
  13. Hey friend, it is very well written article, thank you for the valuable and useful information you provide in this post. Keep up the good work! FYI, Pet Care adda
    Credit card processing, wimpy kid books free
    ,science a blessing or curse essay

    ReplyDelete
  14. Hello, I read this nice article. I think You put a best effort to write this perfect article. I appreciate your work. thank you so much.
    Killing Eve Jodie Comer Jacket

    ReplyDelete
  15. Extremely decent blog and articles. I am realy extremely glad to visit your blog. Presently I am discovered which I really need. I check your blog regular and attempt to take in something from your blog. Much obliged to you and sitting tight for your new post.메이저사이트모음

    ReplyDelete
  16. Digital Groups is the best mobile tower installation service provider in India. We provide the best mobile tower installation in every corner of India
    Mobile Tower Installation

    ReplyDelete
  17. It was nice to read your blog. Are you in need of professional assignment helpers? Well! You have reached the right place at MyCaseStudyHelp.Com where you receive a one-stop solution.We offer one of the finest Case Study Assignment Help for students' benefits. We have more than 5000 professional Case Study Writers experts to support students in doing their university projects. We provide assured quality and credibility. That's why a majority of students trust our Assignment Help.

    ReplyDelete