From Python to Julia: Feature Engineering and ML | by Wang Shenghao | June, 2023

Julia-based approach to building fraud detection models

This is part 2 of my two part series on Getting Started with Julia for Applied Data Science. In the first article, we looked at some examples of performing simple data manipulation and exploratory data analysis with Julia. In this blog, we will work on building a fraud detection model to identify fraudulent transactions.

Briefly speaking, we used a credit card fraud detection dataset obtained from Kaggle. The dataset contains 30 features including transaction time, amount, and 28 principal component features obtained with PCA. Below is the screenshot of first 5 examples of dataset loaded as dataframe in Julia. Note that the Transaction Time feature records the elapsed time (in seconds) between the current transaction and the first transaction in the dataset.

Before training the fraud detection model, let’s get the data ready for the model to consume. Since the main purpose of this blog is to introduce Julia, we are not going to do any feature selection or feature synthesis here.

data segmentation

When training a classification model, the data is usually divided in a stratified manner for training and testing. The main objective is to maintain the distribution of the data with respect to the target class variables in both training and test data. This is especially necessary when we are dealing with highly imbalanced datasets. The MLDataUtils package in Julia provides a range of preprocessing functions, including data segmentation, label encoding, and feature normalization. The following code shows how to do stratified sampling using stratifiedobs Function from MLDataUtils. A random seed can be set so that identical data partitions are reproduced.

The usage of the stratifiedobbs function is similar to the train_test_split function from the sklearn library in Python. Note that the input features X must undergo the transpose twice to restore the original dimensions of the dataset. This can be confusing for a Julia newbie like me. I’m not sure why the author of MLDataUtils developed the function the way it is.

The equivalent Python scalar implementation is as follows.

feature scaling

As a recommended practice in machine learning, feature scaling brings features into the same or similar ranges of values ​​or distributions. Feature scaling helps improve the convergence speed when training a neural network, and also avoids the dominance of any individual feature during training.

Although we are not training a neural network model in this work, I would like to know how feature scaling can be done in Julia. Unfortunately, I could not find a Julia library that provides both the functions of scalar fitting and transforming features. The feature normalization functions provided in the MLDataUtils package allow users to obtain the mean and standard deviation of features, but they cannot be easily applied to a training/test dataset to transform features. Since the mean and standard deviation of the features can be easily calculated in Julia, we can manually implement the process of standard scaling.

The following code creates a copy of X_train and X_test, and calculates the mean and standard deviation of each feature in a loop.

The transformed and original features are shown as follows.

In Python, sklearn provides various options for scaling features, including normalization and parameterization. By declaring a feature scalar, scaling can be done with as few as two lines of code. The following code gives an example of using RobustScaler.

oversampling (by pycol)

Fraud detection datasets are generally severely imbalanced. For example, the ratio of negative to positive examples in our dataset is above 500:1. Since it is not possible to obtain more data points, undersampling will result in heavy loss of data points from the majority class, in which case oversampling becomes the best option. Here I apply the popular SMOTE method to construct a synthetic example for the positive class.

Currently, there is no working Julia library that provides an implementation of SMOTE. The classbalance package has not been maintained for over two years, and cannot be used with recent versions of Julia. Fortunately, Julia allows us to call ready-to-use Python packages using a wrapper library called PyCall.

To import Python libraries in Julia, we need to install PyCall and specify PYTHONPATH as an environment variable. I tried to create a python virtual environment here but it didn’t work. For some reason, Julia cannot recognize the virtual environment’s Python path. That’s why I have to specify system default python path. Next, we can import the Python implementation of SMOTE, which is provided in the unbalanced-learn library. pyimport The functions provided by PyCall can be used to import Python libraries in Julia. The following code shows how to activate PyCall and how to ask Python for help in the Julia kernel.

The equivalent Python implementation is as follows. We can see that the fit_resample function is used in the same way as in Julia.

Now we have reached the stage of model training. We’ll be training a binary classifier, which can be done with a variety of ML algorithms, including logistic regression, decision trees, and neural networks. Currently, the resources for ML in Julia are distributed across several Julia libraries. I want to list some of the most popular options with specific sets of models.

Here I am going to choose XGBoost, given its simplicity and superior performance on traditional regression and classification problems. The process for training an XGBoost model in Julia is similar to that in Python, although there are some minor differences in syntax.

The equivalent Python implementation is as follows.

Finally, let us see how our model performs by looking at the accuracy, recall as well as the time taken to train the model on the test data. In Julia, exact, recall metrics can be calculated using the EvalMetrics library. There is an alternative package MLJBase for the same purpose.

In Python, we can use scalar to calculate metrics.

So who is the winner between Julia and Python? To make a fair comparison, both models were trained with default hyperparameters, and learning rate = 0.1, no. Number of estimators = 1000. The performance metrics are summarized in the following table.

It can be seen that the Julia model achieves better accuracy and recall with a slightly longer training time. Since the XGBoost library used for training the Python model is written in C++ under the hood, while the Julia XGBoost library is completely written in Julia, Julia runs as fast as C++, as it claimed!

Hardware used for the above test: 11th Gen Intel® Core™ i7–1165G7 @ 2.80GHz – 4 cores.

Jupyter notebooks can be found on Github.

I would like to end this series with a summary of Julia libraries mentioned for various data science tasks.

Due to the lack of community support, the usefulness of Julia cannot be compared to Python at the moment. However, considering its improved performance, Julia still has great potential in the future.

Reference