Point a traditional 3D sensor at a chrome-plated ball bearing and observe its failure. The laser light intended to bounce back to the detector and reveal the object\u2019s geometry instead reflects at angles determined by the surface\u2019s curvature, most of it traveling in unhelpful directions. The sensor, effectively, perceives almost nothing. The same issue arises with surgical tools coated in saline, the glass rear window of a self-driving car, and the polished aluminum fixtures on a factory inspection line. Specular surfaces (anything reflective, glassy, glossy) have been a persistent challenge for machine vision for many years.<\/p>\n
A team at the University of Arizona believes they\u2019ve discovered a workaround, and the solution is strikingly simple. Instead of creating a superior sensor or redesigning the projection patterns that standard 3D imagers depend on, they\u2019ve managed to utilize the walls and furniture surrounding a reflective object as an informal screen. The environment becomes the apparatus. \u201cEssentially everything around you becomes a virtual screen,\u201d explains Florian Willomitzer, who heads the Computational 3D Imaging and Measurement Lab at the university\u2019s Wyant College of Optical Sciences.<\/p>\n
The research, released this week in Nature Communications, integrates two measurement techniques that have traditionally been viewed as distinct fields. One technique, known as structured-light triangulation, is the backbone of 3D sensing: project a known pattern onto an object, observe how it deforms from the camera\u2019s perspective, and then apply geometry. It excels on matte surfaces but completely fails on mirrors. The other technique, called deflectometry, measures how a reflective surface alters the image of a large patterned screen placed nearby. Effective on chrome, ineffective on fabric.<\/p>\n
Historically, the issue has been that real-world scenes consist of multiple materials. \u201cThink about the interior of a car or a living room,\u201d says Willomitzer. \u201cSuch environments feature specular materials, like mirrors, glass, or polished metal finishes, alongside diffuse surfaces, such as walls, fabrics, and furniture.\u201d A surgical site similarly combines elements: glistening wet tissue and dull skin exist within the same field of view. Most 3D sensors, tailored to one type, simply abandon the other.<\/p>\n
The Walls Have Eyes<\/p>\n
The standard solution for deflectometry\u2019s screen problem has been to create larger screens. For car-body inspection, manufacturers have constructed tunnel-length setups, entire rooms outfitted with calibrated displays, spacious enough for an entire vehicle. Costly, unmovable, restricted to a single purpose. The Arizona method bypasses these issues by recognizing that diffuse surfaces (a white wall, a fabric couch, a matte-painted ceiling) function just like a calibrated screen if their spatial locations can be determined.<\/p>\n
Here\u2019s the clever part. A scanning laser traverses the room two times in perpendicular directions. An event camera (more on that shortly) captures wherever the laser makes contact. The team\u2019s algorithms process those signals and apply epipolar geometry (generally, the geometric rule that light from a known source must appear along a predictable line in the camera image) to distinguish direct reflections from indirect ones. Direct signals originate from matte surfaces; indirect signals, which bounce once off a matte wall before impacting a shiny object and reaching the camera, contain information about that shiny surface\u2019s shape. The matte surfaces, whose positions are now determined from the triangulation step, serve the function of the deflectometry screen. No calibration required. No fixed hardware necessary. The screen reconstructs itself based on whatever is present in the room.<\/p>\n
\u201cWe can utilize a laser scanner to capture everything in the room, including anything inside,\u201d says Aniket Dashpute, the paper\u2019s primary author, now a doctoral student at Rice University. \u201cWe subsequently apply our algorithms to segregate the diffuse from the specular surfaces and can ultimately use all measured diffuse scene components as a virtual screen for the deflectometry measurement of the specular components.\u201d A piece of uncalibrated cardboard, as it turns out, serves perfectly well as the screen, and since the system triangulates its position anew for each frame, it can even be moved around a stationary object to enhance coverage without needing to recalibrate anything.<\/p>\n
A Camera That Doesn\u2019t Take Pictures<\/p>\n
The event camera is perhaps the more unusual element of the duo. Unlike a traditional camera, which captures complete image frames at a constant rate, an event camera only records the moments when individual pixels change in brightness. Each pixel activates asynchronously, generating a timestamped signal the moment it detects a change. The outcome is a stream of sparse events rather than a series of dense frames, reminiscent of how retinal ganglion cells in the eye react to changes rather than to static lighting.<\/p>\n
The practical benefit is speed and dynamic range. \u201cThe event camera can manage vastly different light levels, from very dim to extremely bright,\u201d states Jiazhang Wang, a postdoctoral researcher on the team.<\/p>\n","protected":false},"excerpt":{"rendered":"
Point a traditional 3D sensor at a chrome-plated ball bearing and observe its failure. The laser light intended to bounce back to the detector and reveal the object\u2019s geometry instead reflects at angles determined by the surface\u2019s curvature, most of it traveling in unhelpful directions. The sensor, effectively, perceives almost nothing. The same issue arises […]<\/p>\n","protected":false},"author":1,"featured_media":372645,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"Default","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[179],"class_list":["post-372644","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-source-scienceblog-com"],"_links":{"self":[{"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/posts\/372644","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=372644"}],"version-history":[{"count":0,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/posts\/372644\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=\/wp\/v2\/media\/372645"}],"wp:attachment":[{"href":"https:\/\/wolfscientific.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=372644"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=372644"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wolfscientific.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=372644"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}