Submodular functions and optimization second edition annals of discrete mathematicsseries editor. Algorithms for optimizing the ratio of submodular functions graph gthat maps from any subset of the objects x uto the set of words x wcontained in the objects. We present sfo, a toolbox for use in matlab or octave that implements algorithms for mini mization and maximization of submodular functions. Finally, anyone looking for an additional level of abstraction from submodular functions should look at murotas monograph on discrete convex analysis mur03. To motivate the representation of combinatorial optimization. Multidocument summarization via budgeted maximization of. Rn r can be maximized efficiently, if it is concave.
However, vondrak showed that a graphic matroid rank function over. Many interesting problems involving submodular functions can be solved using only polynomially many queries to the oracle, e. A unifying view of the theory is shown by means of base polyhedra and duality for submodular and supermodular systems. The focus of this work is optimization of monotone submodular functions. Continuous relaxations for submodular optimization problems. Submodular functions and optimization pdf free download epdf. Curvature and optimal algorithms for learning and minimizing. Maximizationofsubmodular functions, however, is an npcomplete op. Adding modular functions to submodular functions preserves submodularity.
The quality of any subset x umay be measured as the. Submodular functions and optimization, volume 58 2nd edition. Submodular functions and optimization issn series by s. Maximizing nonmonotone submodular functions under matroid or. In this survey paper, we give overview on the fundamental properties of submodular functions and recent algorithmic devolopments of their minimization. You may be thinking you dont have to be a genius to make that inference so ill expand a little.
Machine learning and convex optimization with submodular. Optimization of submodular functions tutorial lecture i jan vondrak1 1ibm almaden research center san jose, ca jan vondrak ibm almaden submodular optimization tutorial 1 1. Conditional gradient method for stochastic submodular maximization. Minimizing nonsubmodular functions with graph cuts a. Since the publication of the 1st edition of this book fifteen years ago, submodular functions have been showing further increasing importance in optimization, combinatorics, discrete mathematics, algorithmic computer science, and algorithmic economics, and there have been made remarkable developments of theory and algorithms in submodular functions. Many combinatorial optimization problems can be solved optimally or nearoptimally in polynomial time only when the underlying function is submodular. Submodular functions and optimization by fujishige, s. Submodularity is a property of set functions with deep theoretical consequences and far reaching applications. The importance of submodular functions has been widely recognized in recent years in combinatorial optimization. Submodular functions and optimization 2nd edition isbn. Submodular functions and optimization sciencedirect.
Submodular optimization and approximation algorithms satoru iwata rims, kyoto university submodular functions cut capacity functions matroid rank functions entropy functions finite set. Submodular functions are a key concept in combinatorial optimization. Submodular optimization two main types of submodular optimization. Intersecting and crossing submodular functions 86 a tree representations of crossfree families 87 b crossing submodular functions 91 c intersecting submodular functions 101 3. Submodular functions and optimization satoru fujishige. Optimization and approximation submodular functions are discrete analogue of convex functions, arising in various fields of applied mathematics including game. Among these setfunctions, submodular functions play an important role, similar to convex functions on vector spaces. In 21 nemhauser and wosley presented a greedy based 1.
Before we explore this relation, we will need more notation. Since the publication of the 1st edition of this book fifteen years ago, submodular functions have been showing further increasing importance in optimization, combinatorics, discrete mathematics, algorithmic computer science, and algorithmic economics, and. In many combinatorial theorems and problems, submodularity is involved, in one form or another, and submodularity often plays an essential role in a proof or an algorithm. Think about functions applied to a set of sets of elements, you want to find t. Jan 25, 2007 submodular functions often arise in various fields of operations research including discrete optimization, game theory, queueing theory and information theory. Invited tutorial intellgent optimization with submodular functions at lion 2012 by andreas krause. Submodular functions, optimization, and applications to machine learning n spring quarter, lecture 5 n.
Its a field dedicated to the optimization of submodular functions. Functions which satisfy the second condition are commonly encountered in economics and game theory where utilities are involved. Using this reduction, one can translate the results for unconstrained and constrained submodular maximization to the drsubmodular setting for many types of constraints in a unified manner. A toolbox for submodular function optimization core. Submodular functions and optimization volume 58 annals. An analysis of approximations for maximizing submodular set. Submodular programs constrained optimization 228 a lagrangian functions and optimality conditions 229 b related problems 234 b. Highperformance implementation of the minimum norm point algorithm for submodular function minimization with several applications matlab toolbox for submodular function optimization maintained by andreas krause. Conjugate functions and a fencheltype minmax theorem for submodular and supermodular functions. Conditional gradient method for stochastic submodular. Nonlinear optimization with submodular constraints pages 223250 download pdf. At first glance it appears very similar to concavity, in other ways it resembles convexity.
Extended tutorial slides, updated july 6 2008 pdf, ppt software, materials and references. Indeed, limited prior work has shown improved results for constrained minimization. We discuss some of these examples further in the following. A few examples of such functions include cuts functions of graphs and hypergraphs, rank functions of matroids and covering functions. Submodular functions and optimization, volume 47 1st edition.
A good knowledge of convex analysis is assumed see, e. Algorithms for optimizing the ratio of submodular functions. One class, which contains the location problem, arises from matroids, another from the assignment problem and a third from boolean polynomials. Bilmes ee563spring 2018submodularity lecture 15 may 16th, 2018 f870 pg. Algorithms that involve submodular functions usually assume that they are given by a value oracle. In chapter 11, we present other combinatorial optimization problems which can be partially solved using submodular analysis, such as submodularfunction maximization and the optimization of di. Submodular minimization nonsmooth convex optimization parallel algorithm for special case 3. The class of sub modular functions is indeed special since submodu lar function minimization is known to be polynomial time, while submodular maximization. Online submodular minimization the journal of machine. Submodular systems and base polyhedra neoflows submodular analysis nonlinear optimization with submodular constraints submodular function minimization discrete convex analysis. Tutorials on submodularity in machine learning and computer vision at dagm 2012 and ecai 2012 by stefanie jegelka and andreas krause preliminary pdf. At rst glance it appears very similar to concavity, in other ways it resembles convexity. We will see certain properties of submodular functions which will help us solve the matroid optimization problem.
We are interested in the information retrieval problem of. Submodular programs unconstrained optimization 216 a minimizing submodular functions 217 b minimizing modular functions 223 7. Optimization of submodular functions tutorial stanford cs theory. Several hard combinatorial optimization problems can be posed in this framework. Journal of machine learning research open source software paper. Jan vondrak ibm almaden submodular functions and applications 7 28. Submodular functions are relevant to machine learning for at least two reasons. Sums of concave over modular functions are submodular fa xk i1 imia 5.
For this reason, an optimization problem which concerns optimizing a convex or concave function can also be described as the problem of maximizing or minimizing a submodular function subject to some constraints. Purchase submodular functions and optimization, volume 47 1st edition. Submodular functions and optimization fujishige, 2005. Structured sparsityinducing norms relaxation of the penalization of supports by submodular functions. In section 3 we present three classes of these problems. Pdf submodular function maximization semantic scholar. An introduction to submodular functions and optimization.
Convexity is a very natural property of various functions and domains occuring in such models. A fagma submodular when g concave and m nonnegative modular. Submodular functions and their applications ibm research. Chapter v nonlinear optimization with submodular constraints. Submodular functions and optimization volume 58 annals of. Introduction in a recent paper cornuejols, fisher and nemhauser 2 give bounds on approximations heuristics and relaxations for the uncapacitated location prob lem. Finally, anyone looking for an additional level of abstraction from submodular functions should look at murotas monograph on. At rst glance it appears very similar to concavity, in other ways it. Submodular function maximization is a central problem in combinatorial optimization, generalizing many important problems including max cut in directedundirected graphs and in hypergraphs, certain constraint satisfaction problems, maximum entropy sampling, and maximum facility location problems. Submodular functions have properties which are very similar to convex and concave functions. The concept of a submodular function in discrete optimization appears to be in several respects analogous to that of a convex function in continuous optimization. Submodular optimization and approximation algorithms satoru iwata rims, kyoto university submodular functions cut capacity functions.
It has been shown that any submodular function can be minimized in polynomial time schrijver,2000iwataetal. Pdf learning and optimization with submodular functions. It has widely been recognized that submodular functions play essential roles in efficiently solvable combinatorial optimization problems. Overview 1 the reason 2 online convex optimization oco 3 online submodular set function minimization 4 online continuous submodular maximization 5 online submodular minimization in real space quadratic submodular functions general continuous submodular functions menglong li uiuc short title april 26, 2018 2 24.
Maximizing nonmonotone submodular functions under matroid. Here we extend these results and generalize them to a larger class of problems. For example, the problem of finding a maximum weight independent. Related polyhedra 102 a generalized polymatroids 102 b polypseudomatroids 106 c ternary semimodular polyhedra 112 3. The authors also showed that current versions of the conditional gradient method a. Optimization of submodular functions tutorial lecture i. Submodular functions, optimization, and applications to. Convex analysis and optimization with submodular functions.