Frequency and Bias

An important consideration in any classification task is class frequency. Class imbalances are problematic because the classifier becomes less sensitive to the minority classes. Consider a training set with a majority class A and minority B. An algorithm trained on this imbalanced data will develop a bias toward predicting A just because it appears more often. Another dataset could in theory contain more of B, for example. This classifier would function poorly in this case because it learned a preference for A.

To avoid this bias, perform class balancing. There are different ways to accomplish balancing. One method is to oversample the minority classes by duplicating observations. However, oversampling can cause the algorithm to overfit, and the data becomes skewed toward the replicated observations. Alternatively, one could undersample the majority class, leading to a loss of valuable information for a classifier [1]. I prefer to maintain as much information as possible.

In this article, I explain variations of oversampling, including using existing observations from the minority to synthesise new data. Balancing via artificial means is known as Synthetic Minority Oversampling Technique, or SMOTE [3].

SMOTE and Tomek Links

A data point can be represented as a vector, where each entry of the vector is an attribute. SMOTE works by first selecting a feature vector from the minority class at random. Then, the algorithm chooses a random neighbouring feature vector from k-nearest (usually five) neighbours. The new, synthesised, feature vector lies at an arbitrary point along the line connecting the two [3]. Fig.1 depicts synthetic data generation in SMOTE with an example dataset.

The problem with SMOTE is that it can generate noise by interpolating points between outliers [2], and alone does not necessarily improve on the more straightforward random oversampling method. It is unlikely that SMOTE adds any additional information by using existing data, but SMOTE still shifts the bias toward the minority class [2]. Applying SMOTE may increase the sensitivity of the minority class but could decrease accuracy and precision. The authors of SMOTE found that combining SMOTE with under-sampling methods can improve classification performance [3].

To address some issues with SMOTE, particularly the case of synthetic noise, there is a modified version, SMOTE+Tomek, which tries to clean the feature space of synthetic noise. SMOTE+Tomek removes a majority class point with the nearest neighbour of another class. The pair of two closeby opposing types is called a Tomek link and is illustrated in Fig.2 [2].

Undersampling the majority class with Tomek links thus removes boundary cases between classes and class label noise.

Another variation of SMOTE is SMOTE+ENN. Considered an improvement to removing Tomek links, SMOTE+ENN deletes the k-nearest neighbours as well as the points of the Tomek link [4]. There are more variations of SMOTE, including KMeans+SMOTE [5] which applies KMeans clustering before SMOTE.

It is important to note that class balancing is performed after the train-test split otherwise SMOTE will interpolate points between test data in training data. In this scenario, the new feature vectors in the training set could leak information about the location of points in the test data.

An alternative method to deal with imbalanced data and avoid SMOTE entirely is using balanced ensemble classifiers. In balanced ensemble methods, bootstrapping can be used to sample the data so that the constituent classifiers (e.g., trees in a forest) train on subsets with the classes present in equal amounts [6].

Example in Python: imblearn library

from imblearn.pipeline import make_pipeline
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SMOTE

pipeline = [SMOTE(random_state=0),
           RandomForestClassifier(random_state=0, min_samples_split=4, n_estimators=500)
          ]
model = make_pipeline(*pipeline)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

See more on the imblearn website: https://imbalanced-learn.org/dev/references/generated/imblearn.over_sampling.SMOTE.html#

References:

[1] Ma Y, He H. Imbalanced learning: foundations, algorithms, and applications. John
Wiley & Sons; 2013

[2] imblearn library documentation; By the Imbalanced-learn developers. Available from:
https://imbalanced-learn.org/dev/index.html.
[3] Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-
sampling technique. Journal of artificial intelligence research. 2002;16:321-57.
[4] Batista GE, Bazzan AL, Monard MC, et al. Balancing Training Data for Automated
Annotation of Keywords: a Case Study. In: WOB; 2003. p. 10-8.
[5] Last F, Douzas G, Bacao F. Oversampling for imbalanced learning based on k-means
and smote. arXiv preprint arXiv:171100837. 2017.
[6] Chen C, Liaw A, Breiman L, et al. Using random forest to learn imbalanced data. University of California, Berkeley. 2004;110(1-12):24.
d case studies. MIT Press; 2020.

You built an ML with TensorFlow or Sci-kit learn and now want it deployed on a website. This article is a quick guide on loading an ML model in Python and using it to make predictions on a web app with Flask. This is based on my image classifier app found on my GitHub Fabio-RibeiroB/image_classifier_app: App to classify happy or sad images (github.com). In this app, the user uploads a file and presses predict. Specifically, the user uploads images for binary classification. It all depends on your model. You can change this code to upload CSV data instead, for example.

I appreciate this article is high-level and lacking detail. It is more than an outline to make it as short-form as possible. To see more, check out my aforementioned repo on GitHub. Anyway, let’s begin.

Save and Load

Let’s say you have a Sequential model that you compiled.

model = Sequential()
....some model
....
model.compile(....)

Now save the model, for example, as a .h5 file. I saved mine in a “models” folder. You can also use pickle to dump and load models as .pkl files.

from tensorflow.keras.models import load_model
import os
model.save(os.path.join('models','model.h5'))

Now load it in your Flask app.

from flask import Flask, render_template, request, redirect, flash, session # useful flask modules
from werkzeug.utils import secure_filename # security
import logging

logging.basicConfig(level=logging.DEBUG)
logging.info('program starting')

from tensorflow.keras.models import load_model
model = load_model('models/model.h5') # loaded model

Static Uploads Folder

We also need a folder where we can upload data for the model. Make a directory called static, and within that, a directory called uploads.

UPLOAD_FOLDER = './static/uploads'
ALLOWED_EXTENSIONS = {'png', 'jgp', 'jpeg'} # change depending on model
app = Flask(__name__)
app.secret_key = b'somesecretkey'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER

def allowed_file(filename):
    """
    Check the uploaded data is correct format
    """"
    return '.' in filename and \
          filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

Routes and Prediction

Now I create the necessary routes in your app that makes a prediction. We will later make a home.html will allow us to start a prediction. The predict.html page shows the results.

@app.route('/')
def home():
    return render_template('home.html')

@app.route('/predict', methods=['POST'])
def predict():
    file = request.files('file')
    # check uploaded file is okay in upload folder
    if file and allowed_file(file.filename):
        filename = secure_filename(file.filename)
        data_path = os.path.join(app.config['UPLOAD_FOLDER'], filename)
   
        file. save(data_path) # save data
        # read data in for example with pandas
        prediction = model.predict(data) # in my example prediction is one number, a probability.

        # remember to delete file after use

        os.remove(data_path)
        return render_template('predict.html', data=prediction)
   
if __name__ == "__main__":
    app.run(debug=True)

The above code saves the valid data in the uploads folder and loads the data. For example, you could load your data into a pandas data frame. The /predict route passes the prediction variable containing a single prediction. The prediction variable is passed to predict.html to render the result on the web page. If your model outputs a lot of predictions, like a CSV of predictions, these lines will need to be modified. You probably want the user to download the predictions as a CSV instead of displaying the results on the screen. In this case, you need a download button in your HTML.

However, continuing my example, we have a home page called home.html with a form for the user to upload data. I simply removed the rest of my HTML tags to declutter the code snippet below. See my repo for the entire HTML file.

In home.html

<form method="POST" action="{{url_for('home')}}"   enctype=multipart/form-data>
    <input type=file name=file>
    <input type=submit value=predict>
</form>

And now for prediction.html. This page will simply output the results with a back button.

{{data}}
<form>
<input type="button" value="Try again" on    click="history.back()">
</form>

Flask run, and the app should be running in local host.