`Kernel methods' make up a large class of machine learning techniques. Each of these methods is characterized by its dependence on a kernel function. In rough terms, a kernel function is a general notion of similarity between two points, with its value large when objects are similar and its value small when objects are dissimilar (note that this is not the only interpretation of what a kernel is).
A kernel (or `Mercer kernel') takes two objects as input and returns some sort of similarity value. The specific details and properties of kernels are outside the scope of this documentation; for a better introduction to kernels and kernel methods, there are numerous better resources available, including http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html "Eric Kim's tutorial".
mlpack implements a number of kernel methods and, accordingly, each of these methods allows arbitrary kernels to be used via the
KernelType template parameter. Like the MetricType policy, the requirements are quite simple: a class implementing the
KernelType policy must have
The signature of the
Evaluate() function is straightforward:
The function takes two vector arguments,
b, and returns a
double that is the evaluation of the kernel between the two arguments. So, for a particular kernel , the
Evaluate() function should return .
b, of types
VecTypeB, respectively, will be an Armadillo-like vector type (usually
arma::sp_vec, or similar). In general it should be valid to assume that
VecTypeA is a class with the same API as
Note that for kernels that do not hold any state, the
Evaluate() method can be marked as
KernelType template policy is quite simple (much like the MetricType policy). Below is an example kernel class, which outputs
1 if the vectors are close and
Then, this kernel may be easily used inside of mlpack algorithms. For instance, the code below runs kernel PCA (
mlpack::kpca::KernelPCA) on a random dataset using the
ExampleKernel. The results are saved to a file called
results.csv. (Note that this is simply an example to demonstrate usage, and this example kernel isn't actually likely to be useful in practice.)
Some algorithms that use kernels can specialize if the kernel fulfills some certain conditions. An example of a condition might be that the kernel is shift-invariant or that the kernel is normalized. In the case of fast max-kernel search (mlpack::fastmks::FastMKS), the computation can be accelerated if the kernel is normalized. For this reason, the
KernelTraits trait class exists. This allows a kernel to specify via a
bool when these types of conditions are satisfied. Note that a KernelTraits class is not required, but may be helpful.
KernelTraits trait class is a template class that takes a
KernelType as a parameter, and exposes
bool values that depend on the kernel. Setting these values is achieved by specialization. The code below provides an example, specializing
KernelTraits for the
ExampleKernel from earlier:
At this time, there is only one kernel trait that is used in mlpack code:
false): if , then the kernel is normalized and this should be set to true.
mlpack comes with a number of pre-written kernels that satisfy the
These kernels (or a custom kernel) may be used in a variety of mlpack methods: