Aug 26

# A unified markov random field/marked point process image model and its application to computational materials

Both Markov Random Field (MRF) and Marked Point Process (MPP) models have their limitations in image analysis. While MRF is a pixel-based representation, it is useful for imposing local constraints, but global constraints are not easily modeled. On the contrary, it is convenient to model global constraints, such as geometric shape and object interactions, within the MPP  framework. However, such object-based MPP model has limited capability for imposing local constraints such as pixel-wise interactions. We have proposed a combined MRF/MPP model to incorporate local and global constraints within one unified framework. Unlike MRF or MPP, which only utilizes either local or global constraint for image analysis, our hybrid model addresses both constraints at pixel and object level simultaneously.

Let $X$ denote the segmented image, $W$ the object field and $Y$ the observed image. $X$ and $Y$ are defined on a lattice $S$; $s \in S$ represents a pixel site. $\Omega _{x}$ and $\Omega_{w}$ denote the space of all possible realizations of $X$ and $W$. $(w_1,...,w_n)\in \Omega _{w}$ is an object field realization, with each $w_i$ being an object. Let $D_{w_{i}}\subset S$ be the pixel site region projected by $w_{i}$ onto the image lattice. Then we can define the segmentation of object $i$ as $x_{w_{i}}=x\bigcap D_{w_{i}}$, i.e., the segmentation $x$ restricted to the region $D_{w_i}$.

A non-homogeneous Gibbs process is introduced to represent the hybrid model. For this process, the neighborhood system is defined as: $w_{i}$ and $w_{j}$ are neighbors, denoted $w_{i} \sim w_{j}$, if $D_{w_{i}}$ intersects $D_{w_{j}}$. The energy function for the Gibbs distribution is

where $V_o(y|{w_i})$ is an energy potential describing how well object $w_i$ fits the observed image, $V_s({x_{{w_i}}^{MAP}})$ is an energy potential based on the likelihood of the MAP segmentation for object $w_i$, and $V_p({w_i},{w_j})$ is a geometric prior, describing the interactions between objects.

We use the Candy model [1] as the object model, which describes each line segment as an object. Usually three types of line segments are considered, which consist of free segment, single segment and double segment. Free segment has no ending point connected, single segment has one ending point connected while double segment has both ending points connected (illustrated as red segment in Fig. 1). Then a prior is proposed to describe the interactions between line segments. A detailed explanation of such prior can be found in [1].

Fig. 1. three types of line segments

We integrated such Candy model into our hybrid model and applied it to a silicate image. The target is to extract the silicate structure and obtain the corresponding segmentation. We perform optimization with our model using reversible jump Markov chain Monte Carlo (RJMCMC). We compared our hybrid model with original MPP model for object identification part and MRF based Graph Cuts method for segmentation. The results (Fig.2 ) show our proposed algorithm performs better than basic MPP at object level as well as MRF based Graph Cuts method at segmentation level.

Fig. 2. Results comparison

[1] Point Processes for Unsupervised Line Network Extraction in Remote Sensing (Caroline Lacoste, Xavier Descombes, and JosianeZerubia, Transaction on PAMI, 2005)