The dataset used in Part 1 is simple and can be easily modeled with just a mixture of Gaussians. However, most real-world datasets are far more complex. In this part of the story, we will apply several synthetic data generators to some popular real-world datasets. Our primary focus is on comparing the distributions of maximum…

In this new post I present the outcome of my quest for the most advanced and powerful libraries for web-based data visualization and analysis as judged by me after a careful analysis of performance, flexibility, and richness of features. Some of the libraries I selected are not popular at all, but they offer surprising capabilities…

As human beings, we can read and understand texts (at least some of them). Computers in opposite “think in numbers”, so they can’t automatically grasp the meaning of words and sentences. If we want computers to understand the natural language, we need to convert this information into the format that computers can work with —…

How to know the unknowable in observational studies Introduction Problem Setup 2.1. Causal Graph 2.2. Model With and Without Z 2.3. Strength of Z as a Confounder Sensitivity Analysis 3.1. Goal 3.2. Robustness Value PySensemakr Conclusion Acknowledgements References The specter of unobserved confounding (aka omitted variable bias) is a notorious problem in observational studies. In…

Automation, machine learning and LLMs in the chip industry (source: chatGPT)I felt like one of those guys from Monsters Inc. You know, the ones in the big yellow hazmat suits. A necessary precaution! I was entering the most complex manufacturing environment in the world. One that requires so much precision that even microscopic particulates from…

How to Create a Speech-to-Text-to-Speech Program Image by Mariia Shalabaieva from unsplashIt’s been exactly a decade since I started attending GeekCon (yes, a geeks’ conference 🙂) — a weekend-long hackathon-makeathon in which all projects must be useless and just-for-fun, and this year there was an exciting twist: all projects were required to incorporate some form…

Understand Semantic Structures with Transformers and Topic Modeling We live in the age of big data. At this point it’s become a cliche to say that data is the oil of the 21st century but it really is so. Data collection practices have resulted in huge piles of data in just about everyone’s hands. Interpreting…

A handy reference on migrating bookmarks, terminal enhancements, and AWS Cli settings Image generated by author using midjourneyI recently received a new 16-inch MacBook Pro with the latest Apple M3 chip for my work computer. I had heard rave reviews about the blazing-fast Apple M1 and M2 chips, so I was incredibly excited to get…

Advanced techniques to process and load data efficiently AI-generated image using KandinskyIn this story, I would like to talk about things I like about Pandas and use often in ETL applications I write to process data. We will touch on exploratory data analysis, data cleansing and data frame transformations. I will demonstrate some of my…

Build your own book recommender with CatBoost Ranker 14 min read · 16 hours ago In today’s digital world, where information overload and wide product offer is the norm, being able to help customers find what they need and like can be an important factor to make our company stand out and…