Optimize the Embedding Space for Improving RAG Image by author. AI generated.Embeddings are vector representations that capture the semantic meaning of words or sentences. Besides having quality data, choosing a good embedding model is the most important and underrated step for optimizing your RAG application. Multilingual models are especially challenging as most are pre-trained on…

Diving deeply into the working structure of the first version of gigantic GPT-models 2017 was a historical year in machine learning. Researchers from the Google Brain team introduced Transformer which rapidly outperformed most of the existing approaches in deep learning. The famous attention mechanism became the key component in the future models derived from…

The way you retrieve variables from Airflow can impact the performance of your DAGs Photo by Daniele Franchi on UnsplashWhat happens if multiple data pipelines need to interact with the same API endpoint? Would you really have to declare this endpoint in every pipeline? In case this endpoint changes in the near future, you will…

A few personal lessons learned from developing LLM applications Source DALL·E 3 prompted with “Operationalizing LLMs, watercolor”It’s been fun posting articles exploring new Large Language Model (LLM) techniques and libraries as they emerge, but most of the time has been spent behind the scenes working on the operationalization of LLM solutions. Many organizations are working…

When you think of AI, you might think of ChatGPT, AI-generated art, or maybe something like the Terminator. But let’s take a step back and ask the basic question, “What is AI?” AI is short for artificial intelligence — which may not tell us much because one of these words is problematic. The first word,…

What are they, where are they, and are they right for you? Photo by Hongwei FAN on UnsplashInput and output (I/O) operations refer to the transfer of data between a computer’s main memory and various peripherals. Storage peripherals such as HDDs and SSDs have particular performance characteristics in terms of latency, throughput, and rate which…

Run and evaluate monocular depth estimation models with Hugging Face and FiftyOne Monocular Depth heat maps generated with Marigold on NYU depth v2 images. Image courtesy of the author.Humans view the world through two eyes. One of the primary benefits of this binocular vision is the ability to perceive depth — how near or far…

To ensure you can follow along, we are using pandas 2.2.0, which is the latest version available at the time of writing this article. You are probably already familiar with performing aggregations in pandas using methods such as sum or min. You have also probably used these methods in combination with groupby. Therefore, it will…

1.1: What is Gradient Descent Image by DALL-E-2In machine learning , Gradient Descent is a star player. It’s an optimization algorithm used to minimize a function by iteratively moving towards the steepest descent as defined by the negative of the gradient. Like in the picture, imagine you’re at the top of a mountain, and your…

Discover the concepts and basic methods of causal machine learning applied in Python Photo by David Clode on UnsplashCausal inference has many tangible applications in a wide variety of scenarios, but in my experience, it is a subject that is rarely talked about among data scientists. In this article, we define causal inference and motivate…