SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation

CVPR 2025

Duc-Hai Pham*1, Tung Do*1, Phong Nguyen1, Binh-Son Hua1,2, Khoi Nguyen1, Rang Nguyen1
1 Anonymous 2 Trinity College Dublin, Ireland
*Indicates Equal Contribution
Teaser image demonstrating Marigold depth estimation.

SharpDepth bridges metric accuracy with detailed boundary preservation, resulting in depth predictions that are both metrically precise and visually sharp.

Audio Presentation (by NotebookLM)

Video Demonstration

SharpDepth can enhance the high-frequency details of a metric depth estimator, allowing for better point-cloud reconstruction. We use gradio in this demonstration.

Overview

We present SharpDepth, a diffusion-based depth model for refining metric depth estimators, e.g., UniDepth, without relying on ground-truth depth data. Our method can recover sharp details in thin structures and improve overall point cloud quality.

The gallery below presents several images from the internet and a comparison of SharpDepth with the previous state-of-the-art metric depth like UniDepth. Use the slider and gestures to reveal details on both sides.

Gallery

How it works

Fine-tuning protocol

Given an input image I, we first use both pre-trained metric depth model fD and diffusion-based depth model fG to produce metric and affine-invariant depth output d and d~, respectively. Our goal is to generate a sharpened metric depth map, d^, using our proposed sharpening model, Gθ. This model architecture is based on state-of-the-art pre-trained depth diffusion models. Instead of naively relying on the forward process of diffusion model, we introduce a Noise-aware Gating mechanism, which provides explicit guidance to the sharpener Gθ on uncertain regions. To enable ground-truth free fine-tuning, we use SDS loss to distill fine-grained details from the pretrained diffusion depth model fG and Noise-Aware Reconstruction Loss to ensure accurate metric prediction.

SharpDepth training scheme

Comparison with other methods

Quantitative comparison of SharpDepth with SOTA metric depth estimators on several zero-shot benchmarks. Our method achieves accuracy comparable to metric depth models. Further evaluation on synthetic datasets (Sintel, UnrealStereo, and Spring) and a real dataset (iBims) shows that our method significantly outperforms UniDepth in both edge accuracy and completeness. By leveraging priors from the pre-trained diffusion model, our approach produces sharper depth discontinuities and achieves high accuracy across datasets. In contrast, discriminative-based methods often produce overly smooth edges, leading to higher completeness errors.

Comparison with other methods Comparison with other methods