Machine Learning

Install Stable Difussion Automatic111, Torch 2.0 and Fix RTX 4090 Performance

I use a clean installation of Ubuntu 23.04 Lunar Lobster and Nvidia driver 525. If you already have the driver installed, here are the steps to improve Automatic111 Stable Diffusion performance to 40-44 it/s

  1. Install required Anaconda

Ubuntu 23.04 default Python version is 3.11 version. In this case, I will using Anaconda to provide Python 3.10. Download Anaconda and install

chmod a+x /
Machine Learning

Solve Pandas Drop Duplicates still not unique in Value Counts

When using pandas drop duplicates, we may encountered rows that still have duplicating by checking via


Not sure why Pandas drop duplicates performance showing inconsistent result. However, to remove duplicate row, produce 100% unique based on index or key column, you can use this

df_unique = df_unique.drop(df_unique[df_unique["key_column_name"].duplicated()].index)
Machine Learning

Install LightGBM use GPU in Linux Ubuntu

LightGBM can work faster in GPU. In PyCaret, I’m passing parameter use_gpu=True in TSForecastingExperiment() and got errors:

[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1

To enable this, we need to uninstall the current LightGBM and re-install the LightGBM with GPU. For Linux Ubuntu, its better to install pre-requisite packages

sudo apt install cmake build-essential libboost-all-dev

Make sure you already have Nvidia Toolkit installed

sudo apt install nvidia-cuda-toolkit

The first option, is installation inside conda environment

pip uninstall lightgbm -y

conda install -c conda-forge gcc=12.1.0

pip install lightgbm --install-option=--gpu --install-option="--opencl-include-dir=/usr/local/cuda/include/" --install-option="--opencl-library=/usr/local/cuda/lib64/"

Second options, installation without environment

# Get LightGBM source.
git clone --recursive
cd LightGBM/python-package/
# cmake specifying locations of OpenCL files.
sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/ -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ ..
# Compile.
sudo make
# Install for Python, using what we just compiled.
python install --precompile

Install Stable Diffusion Windows and Fix RTX performance 2023

Many feedback about performance NVIDIA RTX after installing Stable Diffusion Automatic1111. I will explained a simple way to install and fix the RTX 4090 performance within 5 minutes

First, make sure you have Python 3.10 in your Windows. You can use Anaconda or native Python installation.

  1. Clone stable diffusion git repository to your local directory

2. Install Stable Diffusion with xformers

This part is tricky. By default, it will install Torch 2.1.0, however the latest xformers will required to use torch 2.0. Which later you will encountered the problems like :

AssertionError: Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check

The solution for installation both xformers and torch inside stable difussion is to pass the arguments in installation

./webui.bat --xformers

Fix Install Tensorflow 2 with GPU Simple Version 2023

Tensorflow running use GPU in Windows sometimes is difficult to do, where many articles not pointing exactly which Tensorflow version, NVIDIA drivers and other requirement needed to achieve it in Windows 11. Especially when you have NVIDIA RTX Graphic card like 4090 or 3090 or similar version.

I will help to explain on how to install it properly and make it run. Here are the steps to do:

1.Installing the Latest Anaconda.

Go to Anaconda website and download its community distribution. You can try to activate in your terminal windows with “conda activate” and you will enter your base.

If you are using powershelll, you can try “conda init powershell” to load the environment by default

2.Installing Microsoft Visual Studio 2022 (not to confuse with VSCode editor).

Choose the community version at and install Desktop Environment with C++