[mlpack] Fwd: Degenerate cases and other GMM problems

[Ryan Curtin]

On Tue, Apr 09, 2013 at 02:14:09PM -0400, John Demme wrote:
> (missed reply-all a second ago)
> 
> That patch is somewhat similar to a mod I tried. Mine helped sometimes, but
> not in all situations. Both problems still occur with r14883:
> 
> $ ./gmm -i gmm_obs0.csv -g 5
> [DEBUG] Compiled with debugging symbols.
> [DEBUG] Iterations: 42
> [DEBUG] EMFit::Estimate(): initial clustering log-likelihood: 1.4249e+06
> 
> error: inv(): matrix appears to be singular
> 
> terminate called after throwing an instance of 'std::runtime_error'
>   what():
> Aborted (core dumped)

I have solved one of these, but I may not get to the second problem by
the end of the day.  r14884 commits the fix.

The issue was not that the entire covariance matrix was zero but instead
that only one element on the diagonal was zero -- which makes it
non-invertible.  This could happen when the points for a particular
cluster have no variance in one dimension (in your case, it was the
fifth dimension).  So instead of checking if the sum of the covariance
diagonal is zero, now it checks each element and adds a perturbation
accordingly.

The perturbation is small (1e-50) so it does not affect the results of
the phi() function to get the probability of a point.  At the same time,
it's large enough to make the matrix invertible.

When I have a solution for the second one, I'll let you know.

> On Tue, Apr 9, 2013 at 2:06 PM, Ryan Curtin <gth671b at mail.gatech.edu> wrote:
> > (r148883)

Oops, one too many 8s there.  I was thinking that number looked a little
large...

Thanks for pointing these issues out.  :)

Ryan

-- 
Ryan Curtin       | "Do I sound like I'm ordering a pizza?"
ryan at igglybob.com |   - John McClane