Imports We import modules from Hugging Face’s transforms, peft, and datasets libraries. from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline from peft import prepare_model_for_kbit_training from peft import LoraConfig, get_peft_model from datasets import load_dataset import transformers Additionally, we need the following dependencies installed for some of the previous modules to work. !pip install auto-gptq !pip install optimum !pip…

Optimising multi-model collaboration with graph-based orchestration Orchestra — photographer Arindam Mahanta by unsplashIntegrating the capabilities of various AI models unlocks a symphony of potential, from automating complex tasks that require multiple abilities like vision, speech, writing, and synthesis to enhancing decision-making processes. Yet, orchestrating these collaborations presents a significant challenge in managing the inner relations…

This article will show you different approaches you can take to create embeddings for your data Creating quality embeddings from your data is crucial for your AI system's efficacy. This article will show you different approaches you can use to convert your data from formats like images, texts, and audio, into powerful embeddings that can…

Visualizing satellite images captured over volcanos and wildfires in various spectral bands Sentinel-2 images captured over a volcano and a wildfire visualized with different spectral bands by the author🌟 Introduction 🔍 Sentinel-2 (Spectral Bands) 🌐 Downloading Sentinel-2 Images ⚙️ Processing Sentinel-2 Images (Clipping and Resampling) 🌋 Visualization of Sentinel-2 Images (Volcano) 🔥 Visualization of Sentinel-2…

Using OpenAI’s Clip model to support natural language search on a collection of 70k book covers In a previous post I did a little PoC to see if I could use OpenAI’s Clip model to build a semantic book search. It worked surprisingly well, in my opinion, but I couldn’t help wondering if it would…

An accessible walkthrough of fundamental properties of this popular, yet often misunderstood metric from a predictive modeling perspective Photo by Josh Rakower on UnsplashR² (R-squared), also known as the coefficient of determination, is widely used as a metric to evaluate the performance of regression models. It is commonly used to quantify goodness of fit in…

For additional ideas on how to improve the performance of your RAG pipeline to make it production-ready, continue reading here: This section discusses the required packages and API keys to follow along in this article. Required Packages This article will guide you through implementing a naive and an advanced RAG pipeline using LlamaIndex in Python.…

LDA Convergence Explained with a Dog Pedigree Model “What if my a priori understanding of dog breed group distribution is inaccurate? Is my LDA model doomed?” My wife asked. Welcome back to part 2 of the series, where I share my journey of explaining LDA to my wife. In the previous blog post, we discussed…

How to Stream and Apply Real-Time Prediction Models on High-Throughput Time-Series Data Photo by JJ Ying on UnsplashMost of the stream processing libraries are not python friendly while the majority of machine learning and data mining libraries are python based. Although the Faust library aims to bring Kafka Streaming ideas into the Python ecosystem, it…

Simplified utilizing the HuggingFace trainer object Image from Unsplash by Markus SpiskeHuggingFace serves as a home to many popular open-source NLP models. Many of these models are effective as is, but often require some sort of training or fine-tuning to improve performance for your specific use-case. As the LLM implosion continues, we will take a…