当前位置:首页 >> >>

Bilateral Filtering and Anisotropic Diffusion Towards a Unified Viewpoint


Bilateral Filtering and Anisotropic Di usion: Towards a Uni ed Viewpoint
Danny Barash HP Laboratories { Israel
Abstract
Bilateral ltering has recently been proposed as a noniterative alternative to anisotropic di usion. In both these approaches, images are smoothed while edges are preserved. Unlike anisotropic di usion, bilateral ltering does not involve the solution of partial di erential equations and can be implemented in a single iteration. Despite the di erence in implementation, both methods are designed to prevent averaging across edges while smoothing an image. Their similarity suggests they can somehow be linked. Using a generalized representation for the intensity, we show that both can be related to adaptive smoothing. As a consequence, we also show that bilateral ltering can be applied to denoise and coherence-enhance degraded images with approaches similar to anisotropic di usion.
Keywords:

Bilateral Filtering, Anisotropic Di usion, Adaptive Smoothing, Denoising.

Address: HP Labs { Israel, Technion City, Haifa 32000, Israel. E-mail: barash@hpli.hpl.hp.com

1

1 Introduction
In a wide variety of applications, it is necessary to smooth an image while preserving its edges. Simple smoothing operations such as low-pass ltering, which does not take into account intensity variations within an image, tend to blur edges. Anisotropic di usion 3] was proposed as a general approach to accomplish edge-preserving smoothing. This approach has grown to become a well-established tool in early vision. This paper examines the relation between bilateral ltering, a recent approach proposed in 6], and anisotropic di usion. The paper is divided as follows. Section II presents the connection between anisotropic di usion and adaptive smoothing. The goal is to suggest a viewpoint in which adaptive smoothing serves as the link between bilateral ltering and anisotropic di usion. In Section III, adaptive smoothing is extended, which results in bilateral ltering. The possible uni cation of bilateral ltering and anisotropic di usion is then discussed. Sections IV and V take advantage of the resultant link, borrowing the use of the geometric interpretation to anisotropic di usion and applying it in bilateral ltering. Section IV examines the convolution kernel of a bilateral lter, from the standpoint that color images are 2D surfaces embedded in 5D (x,y,R,G,B) space. Consequently, Section V describes another idea from anisotropic di usion, that of coherence-enhancement of color images using a bilateral lter. In Section VI, conclusions are drawn and suggestions are given for future examination of the proposed uni ed viewpoint.

2 Anisotropic Di usion and Adaptive Smoothing
We rst examine the connection between anisotropic di usion and adaptive smoothing, which was outlined in 4]. Given an image I ( ) (~ ), where ~ = (x1 ; x2) denotes space coordinates, x x an iteration of adaptive smoothing yields: P+1 P+1 ( ) ( () = ( +1) (~ ) = =?1 P?1 I Px1 + i; x2 + j )w I x (1) +1 +1 ()
t t t t i j i

where the convolution mask w( ) is de ned as:
t

=?1

j

=?1 w

t

d( ) (x1 ; x2 ) ) (2) ; x2 ) = exp (? 1 2k2 where k is the variance of the Gaussian mask. In 4], d( ) (x1 ; x2 ) is chosen to depend on the w( ) (x
t t t

2

magnitude of the gradient computed in a 3 3 window: q d( ) (x1 ; x2 ) = G2 1 + G2 2 where, ! @I ( ) (x1 ; x2 ) @I ( ) (x1 ; x2 ) (G ; G ) = ;
t x x t t x1 x2

(3) (4)

@x1

@x2

2

noting the similarity of the convolution mask with the di usion coe cient in anisotropic di usion 3], 7]. It was shown 4] that equation (1) is an implementation of anisotropic di usion. Brie y sketched, lets consider the case of a one-dimensional signal I (x) and reformulate the averaging process as follows:
t

I +1 (x) = c1 I (x ? 1) + c2 I (x) + c3 I (x + 1)
t t t t

(5) (6)

with
c1 + c2 + c3 = 1

Therefore, it is possible to write the above iteration scheme as follows:
I +1 (x)
t t

? I (x) = c1(I (x ? 1) ? I (x)) + c3 (I (x + 1) ? I (x))
t t t t t

(7) (8) (9)

Taking c1 = c3, this reduces to:
I +1 (x)

? I (x) = c1(I (x ? 1) ? 2I (x) + I (x + 1))
t t t t

which is a discrete approximation of the linear di usion equation: However, when the weights are space-dependent, one should write the weighted averaging scheme as follows:
I +1 (x) = c (x ? 1)I (x ? 1) + c (x)I (x) + c (x + 1)I (x + 1)
t t t t t t t

@I = cr2I @t

(10) (11)

with
c (x ? 1) + c (x) + c (x + 1) = 1
t t t

This can be rearranged as:
I +1 (x)
t

? I (x) = c (x ? 1)(I (x ? 1) ? I (x)) + c (x + 1)(I (x + 1) ? I (x)) (12)
t t t t t t t

or
I +1 (x)
t

? I (x) = c (x + 1)(I (x + 1) ? I (x)) ? c (x ? 1)(I (x) ? I (x ? 1)) (13)
t t t t t t t

which is an implementation of anisotropic di usion, proposed by Perona and Malik 3]:
@I = r(c(x1 ; x2 )rI ) @t

(14)

3

where c(x1 ; x2) is the nonlinear di usion coe cient, typically taken as: c(x1 ; x2 ) = g (krI (x1; x2 )k)

(15)

where krI k is the gradient magnitude, and g(krI k) is an \edge-stopping" function. This function is chosen to satisfy g(x) ! 0 when x ! 1 so that the di usion is stopped across edges. Thus, a link between anisotropic di usion (14) and adaptive smoothing (1) is established. In the next section, we show the link between adaptive smoothing and bilateral ltering.

3 Bilateral Filtering and Adaptive Smoothing
Bilateral ltering was introduced 6] as a nonlinear lter which combines domain and range ~x ltering. Given an input image f (~ ), using a continuous representation notation as in 6], the output image ~ (~ ) is obtained by: hx R 1 R 1 f (~)c(~; ~ )s(f (~); f (~ ))d~ ~ x ~ ~x ~ (~ ) = ?1 ?1 (16) hx R 1 R 1 c(~; ~ )s(f (~); f (~ ))d~ x ~ ~x ?1 ?1 ~ where ~ = (x1 ; x2),~ = ( 1; 2) are space variables and f = (f ; f ; f ) is the intensity. The x full vector notation is used in order to avoid confusion in what follows. The convolution mask is the product of the functions c and s, which represent `closeness' (in the domain) and `similarity' (in the range), respectively.
R G B

E ectively, we claim that a discrete version of bilateral ltering can be written as follows (using the same notation as in the previous section, only I is now a 3-element vector which describes color images): P+ P+ I ( ) (x + i; x + j )w( ) = ~ ( +1) (~ ) = =? ~ (17) I x P? P+1 ( )2 +
S S t t t i S j S i

with the weights given by:
w( ) (~ ; ~) = exp( x
t

=?
S

S

j

=?
S

S

w

t

(18) 2 2 2 2 where S is the window size of the lter, which is a generalization of (1). In order to prove our claim and demonstrate the relation to (1), we use a generalized representation for the ~ intensity I . In principle, the rst element corresponds to the range and the second element corresponds to the domain of the bilateral lter. De ning the generalized intensity as: 9 8 ~x x b < I (~ ) ; ~ = ~ (19) I
D R

?(~ ? ~ )2 ) exp( ?(I (~) ? I (~ ))2 ) x x

:

R

D

;

4

we now take d( ) (~ ) to be the di erence between generalized intensities at two points in a x b b~ ~ ~ given S S window, I (~) ? I (x) , the latter being a global extension to (3). In (3), the gradient, being the local di erence between two neighboring points in a 3 3 window, was taken as a distance measure. Starting from (2), and setting k = 1 since the variances and are already included in the generalized intensity, we obtain: 1 b bx 2 ~ ~ w( ) (~ ) = exp(? I (~) ? I (~ ) ) = x 2 8 9 8 9 ~ ~x x 2 1 < I (~) ; ~ = ? < I (~ ) ; ~ = ) = exp(? 2 : ; : ;
t D R t

8 92 ~ ~x x 1 < I (~) ? I (~ ) ; ~ ? ~ = ) = exp(? 2 : ; 1 0 ~ ~x x 1 @ (I (~) ? I (~ ))2 + (~ ? ~ )2 A) = exp(? 2 2
R D R D R D

2 ~ x2 ~ x 2 = exp( ?( ? ~ ) ) exp( ?(I ( ) ?2 I (~ )) ) 2 2 2
R D D R

(20)

Because these are the weights used in the bilateral lter, as can be veri ed in (18), equation (20) provides a direct link between adaptive smoothing and bilateral ltering. In a general framework of adaptive smoothing, one can take spatial and spectral distance measures along with increasing the window size, abandoning the need to perform several iterations. Taken as such, we get the bilateral ltering implementation of 6] which can be viewed as a generalization of adaptive smoothing.

4 Geometric Interpretation
In the previous two sections, it was shown that anisotropic di usion and bilateral ltering can be linked through adaptive smoothing. Speci cally, the di usion coe cient in (14) relates to the convolution mask and in particular to the distance measure which is used in the bilateral lter. Similarly, the relation between anisotropic di usion and robust statistics was described in 1]. For illustration, Figure 2 demonstrates two di erent ways of performing edge-preserving smoothing on the original image in Figure 1. The result of using nonlinear di usion ltering and the result of bilateral ltering is similar but not identical, since the parameters are di erent and it was intentionally chosen to use a large window size with the bilateral lter and several iterations with anisotropic di usion. That is the most natural setup for the two to be used. 5

Figure 1: Original image: Laplace.

Figure 2: Edge-preserving smoothing: anisotropic di usion with 20 time-steps of = 1:0 (left) and Gaussian bilateral ltering with a 30 30 window size, = 5:0 and = 30:0 (right). and are bilateral ltering parameters, see 6] for details.
D R D R

6

In color images, it was demonstrated in 5] that the image can be represented as 2D surface embedded in the 5D spatial-color space and denoising can be achieved by using the Beltrami ow. It is possible to borrow this notion outlined in 5] and choose the following spectral distance measure for the bilateral lter: q ~) = ( R)2 + ( G)2 + ( B )2 I (~ ) ? I ( x (21) Note that only the spectral distance measure of the range part is given in (21) and can directly be installed in the similarity function s of the bilateral lter as implemented in 6]. The spatial distance measure of the domain part remains the same as with grey-level images. Written that way, one can distinguish between closeness in the domain and similarity in the range, with the advantage of treating the two separately. However, it is also possible to write (21) equivalently by combining the spatial and spectral distance terms. Using the generalized intensity de ned in (19), the full distance measure can be written as:
d( ) (x1 ; x2 ) =
t D R

2

D

bx b ~ ~ (I (~ ) ? I (~)) = ( x1 )2 + ( x2 )2 + 2(( R)2 + ( G)2 + ( B )2 ) (22)
2
D

where = = . Note that this distance measure can be plugged into the convolution mask of adaptive smoothing (2) as one term with k = . It is now possible to take advantage of a geometric interpretation in which color images are 2D surfaces embedded in the 5D (x; y; R; G; B ) space. Equation (22) is then analogous to the local measure: ds2 = dx2 + dy 2 + 2 (dR2 + dG2 + dB 2 ) (23) which is the geometric arclength in the hybrid spatial-color space discussed in 2], 5].

5 Coherence-Enhancement by Bilateral Filtering
In this section, a coherence-enhancement procedure based on the geometric interpretation given in (22) and outlined in 2], 7] is applied using a bilateral lter. The approach, originally proposed by Weickert for anisotropic di usion, is based on the idea that the amount and direction of di usion can be controlled by altering the image metric. Our goal is to take a familiar procedure in anisotropic di usion and apply it in the framework of bilateral ltering in order to achieve control on edge-preservation. Let us de ne a di erence operator, R(x1 ; x2) jR(x1 ; 2) ? R(x1 ; x2)j+jR( 1; x2) ? R(x1 ; x2)j, where the same de nition applies to G and B . The coordinates ~ = (x1 ; x2 ) represent x ~ = ( 1 ; 2 ) is a point within the mask. Furthermore, let the center of the lter mask, and R jR(x1 ; 2 ) ? R(x1 ; x2 )j = x + jR( 1 ; x2 ) ? R(x1 ; x2 )j = x, where x stands for either x1 = jx1 ? 1 j or x2 = jx2 ? 2 j. Using these de nitions, one can write the following identity which resembles the chain-rule:
x

7

R(x1 ; x2 ) = ( R

1 2

x1

x1 + R

x2

x2 )

(24)

where the same identity applies to G and B . We note that the above de nition of the di erence operator instead of the straight-forward R = jR( 1; 2) ? R(x1 ; x2 )j is meant to give a two-point support to (x1 ; x2) when calculating R , which is a better approximation then a single-point support in our attempt to mimic the derivative operator in the context of a geometric framework. It is now possible to write the distance measure of (22) as:
x

d( ) (x
t

1 ; x2 )

2

= ( x1

x2 )

a b b c

!

x1 x2

!

(25)

where,
a = 1+ b =
2( 2(

R 2 1 + G 2 1 + B 21 )
x

4 2 ( R 2 + G2 + B 2 ) 2 2 2 c = 1+ 4 which can be veri ed by starting from (25), using the de nitions for R 1 ; R 2 and identity in (24) to reach back (22). We obtain in (25) a symmetric positive de nite matrix M analogous to a `structure tensor' 7]. Diagonalizing the matrix M , we get:
x x x x x

R

x1

R2+ G
x

4

x

x

(26)
x1

x1

G2+ B
x

B 2)
x

M =U U

T

(27)

where U = (u1ju2) is a Hermitian matrix with the eigenvectors u1; u2 in its columns. The diagonal matrix consists of the eigenvalues 1; 2 ( 1 > 2) in its diagonal. For a 2x2 system which is symmetric (such as the matrix M , see (25)) the corresponding eigenvalues are given by: q 1 (28) = 2 (a + c (a ? c)2 + 4b2 ) 12 and the orthonormal eigenvectors, excluding the trivial case where b = 0, can be obtained from: ! q 2b 2 2 u1 k (29) c ? a + (a ? c) + 4b
;

8

60

80
50

100

100

120

150

140

160
200

180

250 50 100 150 200 250

20

40

60

80

100

120

140

Figure 3: Original Image (this is a color image)

60

80
50

100

100

120

150

140

160
200

180
250 50 100 150 200 250

20

40

60

80

100

120

140

Figure 4: Bilateral Filtering (this is a color image)

60

80
50

100

100

120

150

140

160
200

180
250 50 100 150 200 250

20

40

60

80

100

120

140

Figure 5: Bilateral Coherence Enhancement (color image)

9

In our example, we change the eigenvalues of M without altering its eigenvectors, in the same manner used in 2] for the purpose of coherence-enhancement: =
1

0

?1

0
2

!

(30)

where 1 is a positive scalar. Unlike the procedure for coherence- enhancement in anisotropic di usion, di using before changing its eigenvalues is not necessary in bilateral ltering. In the example, we start from an original image of a sh plotted in Figure 3. The image in Figure 4 is then obtained after applying bilateral ltering with = 10:0 and = 100:0, using the distance measure as in (21). Note that the scales of the sh got smoothed as a consequence of bilateral ltering. In Figure 5, the same bilateral ltering is applied and in addition is set to 0:1. One can verify by examining the color images that smoothing can be controlled by the variable . We note that in addition to coherence-enhancement demonstrated here, image enhancement using bilateral ltering can be thought of. Although the framework of bilateral ltering does not allow color image enhancement by inverse diffusion across an edge as mentioned in 2], since the bilateral lter coe cients will always remain positive even when the eigenvalues of the `structure tensor' are made negative, other approaches can be suggested. One possible approach is to design special closeness and similarity functions, tailored to obtain image enhancement. Another approach which can be considered is to multiply the coherence-enhancing bilateral lter by an additional enhancement factor which attempts to capture the geometry of an edge. Image enhancement by bilateral ltering remains an open and challenging area of application.
D R

6 Conclusions
The nature of bilateral ltering resembles that of anisotropic di usion. It is therefore suggested the two are related and a uni ed viewpoint can reveal the similarities and di erences between the two approaches. Once such an understanding is reached, it is possible to choose the desired ingredients which are common to the two frameworks along with the implementation method. The method can be either applying a nonlinear lter or solving a partial-di erential equation. Adaptive smoothing serves as a link between the two approaches, each of which can be viewed as a generalization of the former. In anisotropic di usion, the di usion coe cient can be generalized to become a `structure tensor' 7] which then leads to phenomena such as edge-enhancing and coherence-enhancing di usions. In bilateral ltering, the kernel (which plays the same role as the di usion coe cient) is extended to become globally dependent on intensity, whereas a gradient can only yield local dependency among neighboring pixels. 10

Thus, the window of the lter becomes much bigger in size than the one used in adaptive smoothing and there is no need to perform several iterations. We note that this extension is general on its own right, meaning that a variety of yet unexplored possibilities exist for constructing a kernel with an optimal window size, as well as designing the best closeness and similarity functions for a given application. The general hybrid spatial-color formulation 2], 5] provide a geometric interpretation with which the bilateral convolution kernel can be viewed as an approximation to the geometric arclength in the 5D hybrid spatial-color space. Ideas that are based on the geometric interpretation, such as coherence-enhancement, can be borrowed from anisotropic di usion and applied to some degree of approximation in bilateral ltering. Two practical goals seem to come up from comparing between anisotropic di usion and bilateral ltering. The rst is a further trial to reduce the number of iterations needed in anisotropic di usion (which can be achieved by e cient numerical schemes such as 8], less proned to stability problems) while retaining the same accuracy as in bilateral ltering. The second is to reduce the window size and investigate other means which aim at minimizing computations associated with bilateral ltering. Both approaches are related to each other, and an exchange of new ideas between one another can be rewarding.

7 Acknowledgment
The author would like to thank Renato Keshet, Michael Elad, Doron Shaked, Ron Maurer and Ron Kimmel for extensively reviewing and commenting on this report.

8 References

1] Black M.J., Sapiro G., Marimont D., Heeger D., \Robust Anisotropic Di usion," IEEE Transactions on Image Processing, Vol. 7, No. 3, p.421, 1998. 2] R. Kimmel, R. Malladi, N. Sochen, \Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images," International Journal of Computer Vision, in press. 3] P. Perona and J. Malik, \Scale-Space and Edge Detection Using Anisotropic Di usion," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 7, p.629, 1990. 4] P. Saint-Marc, J.S. Chen, G. Medioni, \Adaptive Smoothing: A General Tool for Early Vision," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 6, p.514, 1991. 11

5] N. Sochen, R. Kimmel, R. Malladi, \A Geometrical Framework for Low Level Vision," IEEE Transactions on Image Processing, Vol. 7, No. 3, p.310, 1998. 6] C. Tomasi and R. Manduchi, \Bilateral Filtering for Gray and Color Images," Proceedings of the 1998 IEEE International Conference on Computer Vision , Bombay, India, 1998. 7] J. Weickert, Anisotropic Di usion in Image Processing, Tuebner Stuttgart, 1998. ISBN 3-519-02606-6. 8] J. Weickert, B.M. ter Haar Romeny, M. Viergever, \E cient and Reliable Schemes for Nonlinear Di usion Filtering," IEEE Transactions on Image Processing, Vol. 7, No. 3, p.398, 1998.

12



相关文章:
更多相关标签: