Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-28T19:31:20.384Z Has data issue: false hasContentIssue false
  • English
  • Français

Review of Methods Inspired by Algebraic-Multigrid for Data and Image Analysis Applications

Published online by Cambridge University Press:  28 May 2015

Meirav Galun ,
Ronen Basri  and
Irad Yavneh
Meirav Galun*
Affiliation:
Weizmann Institute of Science, Rehovot, Israel
Ronen Basri
Affiliation:
Weizmann Institute of Science, Rehovot, Israel
Irad Yavneh
Affiliation:
Technion, Israel Institute of Technology, Haifa, Israel
*
*Email addresses: [email protected] (Meirav Galun), [email protected] (Ronen Basri), [email protected] (Irad Yavneh)
Get access

Abstract

Algebraic Multigrid (AMG) methods were developed originally for numerically solving Partial Differential Equations (PDE), not necessarily on structured grids. In the last two decades solvers inspired by the AMG approach, were developed for non PDE problems, including data and image analysis problems, such as clustering, segmentation, quantization and others. These solvers share a common principle in that there is a crosstalk between fine and coarse representations of the problems, with flow of information in both directions, fine-to-coarse and coarse-to-fine. This paper surveys some of these problems and the AMG-inspired algorithms for their solution.

Keywords

65F1065F5065H1068U10Algebraic MultigridImage AnalysisData AnalysisMultiscale AlgorithmsMultilevel Optimization
Type
Research Article
Information
Numerical Mathematics: Theory, Methods and Applications , Volume 8 , Issue 2 , May 2015 , pp. 283 - 312
Copyright
Copyright © Global-Science Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1]Alpert, S. , Galun, M. , Brandt, A. and Basri, R.Image segmentation by probabilistic bottom-up aggregation and cue integration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(2), pp. 315–327, 2012.CrossRefGoogle ScholarPubMed
[2]Arbelaez, P.Maire, M.Fowlkes, C. and Malik, J.Contour detection and hierarchical image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), pp. 898–916, 2011.CrossRefGoogle ScholarPubMed
[3]Bagon, S. and Galun, M.A multiscale framework for challenging discrete optimization, NIPS Workshop on Optimization for Machine Learning, 2012.Google Scholar
[4]Bagon, S. and Galun, M.Aunified multiscale framework for discrete energy minimization, Report, arXiv:1204.4867, 2012.Google Scholar
[5]Bagon, S.Discrete Energy Minimization, beyond Submodularity: Applications and Approximations, PhD Thesis, the Weizmann Insitute of Science, arXiv:1210.7362, 2012.Google Scholar
[6]Berkhin, P.A survey of clustering data mining techniques, Grouping multidimensional data, Springer, pp. 25–71, 2006.CrossRefGoogle Scholar
[7]Besag, J.On the statistical analysis of dirty pictures, Journal of the Royal Statistical Society, Series B (Methodological), pp. 259–302, 1986.Google Scholar
[8]Borg, I. and Groenen, P.J.Modern Multidimensional Scaling: Theory and Applications, Springer, 2005.Google Scholar
[9]Borzi, A. and Borzi, G.Algebraic multigrid methods for solving generalized eigenvalue problems, International journal for numerical methods in engineering, 65(8), pp. 1186–1196, 2006.CrossRefGoogle Scholar
[10]Boykov, Y.Veksler, O. and Zabih, R.Fast approximate energy minimization via graph cuts, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), pp. 1222–1239, 2001.CrossRefGoogle Scholar
[11]Brandt, A.Multi-level adaptive solutions to boundary-value problems, Mathematics of computation, 31(138), pp. 333–390, 1977.CrossRefGoogle Scholar
[12]Brandt, A. and Livne, O.E.Multigrid techniques: 1984 guide with applications to fluid dynamics, 67, SIAM, 2011.Google Scholar
[13]Brandt, A.Mccormick, S.F. and Ruge, J.W.Algebraic multigrid (AMG) for automatic multigrid solutions with application to geodetic computations, Report, Inst. for Computational Studies, Fort Collins, CO, October 1982.Google Scholar
[14]Briggs, W.L.Henson, V.E. and Mccormick, S.F.A Multigrid Tutorial, 2nd Ed., SIAM, 2000.Google Scholar
[15]Bronstein, M.M.Bronstein, A.M.Kimmel, R. and Yavneh, I.Multigrid multidimensional scaling, Numerical linear algebra with applications, 13(2–3), pp. 149–171, 2006.CrossRefGoogle Scholar
[16]Corpet, F.Multiple sequence alignment with hierarchical clustering, Nucleic acids research, 16(22), pp. 10881–10890, 1988.CrossRefGoogle ScholarPubMed
[17]Di, Z.Emelianenko, M. and Nash, S.G.Truncated newton-based multigrid algorithm for centroidal Voronoi diagram calculation, Numerical Mathematics: Theory, Methods Applications 5, pp. 242–259, 2012.Google Scholar
[18]Du, Q.Faber, V. and Gunzburger, M.Centroidal Voronoi tessellations: applications and algorithms, J. SIAM Review, 41(4), pp. 637–676, 1999,CrossRefGoogle Scholar
[19]Du, Q. and Emelianenko, M.Acceleration schemes for computing centroidal Voronoi tessellations, Numerical Linear Algebra with Applications 13(2–3), pp. 173–192, 2006.CrossRefGoogle Scholar
[20]Du, Q. and Emelianenko, M.Uniform convergence of a nonlinear energy-based multilevel quantization scheme, SIAM Journal on Numerical Analysis 46(3), pp. 1483–1502, 2008.CrossRefGoogle Scholar
[21]Eldar, Y.Lindenbaum, M.Porat, M. and Zeevi, Y.Y.The farthest point strategy for progressive image sampling, IEEE Transactions on Image Processing, 6(9), pp. 1305–1315, 1997.CrossRefGoogle ScholarPubMed
[22]Falgout, R.D.An introduction to algebraic multigrid, Computing in Science and Engineering, 8(6), pp. 24–33, 2006.CrossRefGoogle Scholar
[23]Felzenszwalb, P.D. and Huttenlocher, D.P.Efficient graph-based image segmentation, International Journal of Computer vision, 59(2), pp. 167–181, 2004.CrossRefGoogle Scholar
[24]Felzenszwalb, P.F. and Huttenlocher, D.P.Efficient belief propagation for early vision, International journal ofcomputer vision, 70(1), pp. 41–54, 2006.CrossRefGoogle Scholar
[25]Galun, M.Sharon, E.Basri, R. and Brandt, A.Texture segmentation by multiscale aggregation of filter responses and shape elements, ICCV proceedings, pp. 716–723, 2003.Google Scholar
[26]Gersho, A. and Gray, R.M.Vector quantization and signal compression, Springer, 1992.CrossRefGoogle Scholar
[27]Gray, R.M.Vector quantization, ASSP Magazine, IEEE, 1(2), pp. 4–29, 1984.CrossRefGoogle Scholar
[28]Hackbusch, W.On the multi-grid method applied to difference equations, Computing, 20(4), pp. 291–306, 1978.CrossRefGoogle Scholar
[29]Hackbusch, W.Multi-grid methods and applications, 4, Berlin: Springer-Verlag, 1985.Google Scholar
[30]Inglis, T.Sterck, H.D.Sanders, G.Djambazian, H.Sladek, R.Sundarara-Jan, S. and Hudson, T.J.Multilevel space-time aggregation for bright field cell microscopy segmentation and tracking, Journal of Biomedical Imaging, 8, 2010.Google Scholar
[31]Kappes, J.H.Andres, B.Hamprecht, F.A.Schnörr, C.Nowozin, S.Batra, D.Kim, S.Kausler, B.X.Lellmann, J.Komodakis, N. and Rother, C.A comparative study of modern inference techniques for discrete energy minimization problems, CVPR, 2013.Google Scholar
[32]Kim, T.Nowozin, S.Kohli, P. and Yoo, C.D.Variable grouping for energy minimization, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1913–1920, 2011.Google Scholar
[33]Kolmogorov, V. and Zabih, R.What energy functions can be minimized via graph cuts, IEEE Trans. Pattern Analysis and Machine Intelligence, 26(2), pp. 147–159, 2004.CrossRefGoogle ScholarPubMed
[34]Kolmogorov, V.Convergent tree-reweighted message passing for energy minimization, IEEE Trans. Pattern Analysis and Machine Intelligence, 28(10), pp. 1568–1583, 2006.CrossRefGoogle ScholarPubMed
[35]Komodakis, N.Towards more efficient and effective LP-based algorithms for MRF optimization, ECCV, pp. 520–534, 2010.Google Scholar
[36]Koren, Y.Yavneh, I. and Spira, A.A multigrid approach to the scalar quantization problem, IEEE Transactions on Information Theory, 51(8), pp. 2993–2998, 2005.CrossRefGoogle Scholar
[37]Koren, Y. and Yavneh, I.Adaptive multiscale redistribution for vector quantization, SIAM Journal on Scientific Computing, 27(5), pp. 1573–1593, 2006,CrossRefGoogle Scholar
[38]Kushnir, D.Galun, M. and Brandt, A.Fast multiscale clustering and manifold identification, Pattern Recognition, 39(10), pp. 1876–1891, 2006.CrossRefGoogle Scholar
[39]Kushnir, D.Galun, M. and Brandt, A.Efficient multilevel eigensolvers with applications to data analysis tasks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), pp. 1377–1391, 2010.CrossRefGoogle ScholarPubMed
[40]Linial, N.London, E. and Rabinovich, Y.The geometry of graphs and some of its algorithmic applications, Combinatorica, 15(2), pp. 215–245, 1995.CrossRefGoogle Scholar
[41]Liu, Y.Wang, W.Lvy, B.Sun, F.Yan, D.M.Lu, L. and Yang, C.On centroidal voronoi tessellationenergysmoothness and fast computation, ACM Transactions on Graphics (ToG), 28(4), 2009.CrossRefGoogle Scholar
[42]Lloyd, S.Least squares quantization in PCM, IEEE Transactions on Information Theory, 28(2), pp. 129–137, 1982.CrossRefGoogle Scholar
[43]Nadler, B. and Galun, M.Fundamental limitations of spectral clustering, Advances in Neural Information Processing Systems, 2006.Google Scholar
[44]Nash, S.G.A multigrid approach to discretized optimization problems. Optimization Methods and Software, 14(1–2), pp. 99–116, 2000.CrossRefGoogle Scholar
[45]Nasrabadi, N.M. and King, R.A.Image coding using vector quantization: A review, IEEE Transactions on Communications, 36(8), pp. 957–971, 1988.CrossRefGoogle Scholar
[46]Ng, A.Y.Jordan, M.I. and Weiss, Y.On spectral clustering: Analysis and an algorithm, Advances in neural information processing systems 2, pp. 849–856, 2002.Google Scholar
[47]Nowozin, S.Rother, C.Bagon, S.Sharp, T.Yao, B. and Kohli, P.Decision tree fields, IEEE International Conference on Computer Vision (ICCV), pp. 1668–1675, 2011Google Scholar
[48]Pearl, J.Probabilistic reasoning in intelligent systems: networks of plausible inference, 2nd Ed., Morgan Kaufmann, 1988.Google Scholar
[49]Rother, C.Kolmogorov, V.Lempitsky, V. and Szummer, M.Optimizing binary MRFs via extended roof duality, IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2007.Google Scholar
[50]Ruge, J.W. and Stuben, K.Algebraic multigrid, Multigrid methods 3, pp. 73–130, 1987.CrossRefGoogle Scholar
[51]Schlesinger, D. and Flach, B.Transforming an arbitrary minsum problem into a binary one, Tu, Fak. Informatik, 2006.Google Scholar
[52]Sharon, E.Brandt, A. and Basri, R.Fast multiscale image segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2000.Google Scholar
[53]Sharon, E.Brandt, A. and Basri, R.Segmentation and boundary detection using multiscale intensity measurements, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2001.Google Scholar
[54]Sharon, E.Galun, M.Sharon, D.Basri, R. and Brandt, A.Hierarchy and adaptivity in segmenting visual scenes, Nature 442(7104), pp. 810–813, 2006.CrossRefGoogle ScholarPubMed
[55]Shi, J. and Malik, J.Normalized cuts and image segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence, 22(8), pp. 888–905, 2000.Google Scholar
[56]Stuben, K.Algebraic multigrid (AMG): experiences and comparisons, Applied mathematics and computation, 13(3), pp. 419–451, 1983.CrossRefGoogle Scholar
[57]Szeliski, R.Zabin, R.Scharstein, D.Veksler, O.Kolmogorov, V.Agarwala, A.Tappen, M. and Rother, C.A comparative study of energy minimization methods for Markov Random Fields with smoothness-based priors, IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(6), 2008.CrossRefGoogle ScholarPubMed
[58]Trottenberg, U.Oosterlee, C.W. and Schuller, A.Multigrid, Academic press, 2000.Google Scholar
[59]Von Luxburg, U.A tutorial on spectral clustering.– Statistics and computing, 17(4), pp. 395–416, 2007.Google Scholar
[60]Watson, D.F.Computing the n-dimensional Delaunay tessellation with application to Voronoi polytopes, The computer journal, 24(2), pp. 167–172, 1981.CrossRefGoogle Scholar
[61]Xu, R. and Wunsch, D.Survey of clustering algorithms, IEEE Transactions on Neural Networks, 16(3), pp. 645–678, 2005.CrossRefGoogle ScholarPubMed