Overview / Usage

Support vector machines (SVMs) is a popular machine learning technique, which works effectively with balanced datasets. However, when it comes to imbalanced datasets, SVMs produce suboptimal classification models. On the other hand, the SVM algorithm is sensitive to outliers and noise present in the datasets. Therefore, although the existing class imbalance learning (CIL) methods can make SVMs less sensitive to class imbalance, they can still suffer from the problem of outliers and noise.

Fuzzy SVMs (FSVMs) is a variant of the SVM algorithm, which has been proposed to handle the problem of outliers and noise. In FSVMs, training examples are assigned different fuzzy-membership values based on their importance, and these membership values are incorporated into the SVM learning algorithm to make it less sensitive to outliers and noise. However, like the normal SVM algorithm, FSVMs can also suffer from the problem of class imbalance. In this paper, a method is presented to improve FSVMs for CIL (called FSVM-CIL), which can be used to handle the class imbalance problem in the presence of outliers and noise.

Methodology / Approach

However, in most real-world problems, the datasets are not completely linearly separable, although they are mapped into a higher dimensional feature space. Therefore, the constrains in the aforementioned optimization problem in (2) are relaxed by introducing a slack variable _εi ≥ _0,

By applying a kernel function, such that K(xi, xj) = Φ(xi) _· _Φ(_xj _), we would be able to transform the dual-optimization problem in above equation.

Let m+ represent the membership value of a positive-class example x+, while m− represents the membership of a negative class example x− in their own classes. In the proposed FSVMCIL method, we define these membership functions as in the main pictures of membership fucntions

​

Where f(xi) generates a value between 0 and 1, which reflects the importance of xi in its own class. Moreover, we assign the values for r+ and r− in order to reflect the class imbalance, such that r+ > r−. Therefore, a positive-class example can take a membership value in the [0, r+] interval, while a negative-class example can take a membership value in the [0, r−] interval. This way of assigning membership values serves both of the aforementioned goals. Therefore, the proposed FSVM-CIL method can be used to handle both the problems of class imbalance and outliers/noise together.

Technologies Used

The project is build in anaconda environment using Jupyter Norebooks in python language.

Libraries used are:

CVXOPT, sklearn, numpy, pandas etc.

Repository

https://github.com/adizz2407/Fuzzy-SVM