Rendering Synthetic Objects into Legacy Photographs



Kevin Karsch, Varsha Hedau, David Forsyth, Derek Hoiem
Presented at SIGGRAPH Asia 2011



[Prototype software now available]




Abstract

We propose a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. We demonstrate in a user study that synthetic images produced by our method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. We also collected new illumination and reflectance datasets; renderings produced by our system compare well to ground truth. Our system has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.


Materials and media


[Download pdf (low resolution, 3MB)]
 [Download pdf (high resolution, 54MB)]

[Download video (.mp4, H.264 compression)]


Slides


[Download pptx (rough transcript included in notes)]
 [Download pdf without videos]


BibTeX

@inproceedings{Karsch:SA2011,
 author = {Karsch, Kevin and Hedau, Varsha and Forsyth, David and Hoiem, Derek},
 title = {Rendering synthetic objects into legacy photographs},
 booktitle = {Proceedings of the 2011 SIGGRAPH Asia Conference},
 series = {SA '11},
 year = {2011},
 isbn = {978-1-4503-0807-6},
 location = {Hong Kong, China},
 pages = {157:1--157:12},
 articleno = {157},
 numpages = {12},
 url = {http://doi.acm.org/10.1145/2024156.2024191},
 doi = {http://doi.acm.org/10.1145/2024156.2024191},
 acmid = {2024191},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {computational photography, image-based rendering, light estimation, photo editing},
}