The work is done in a Google Colab Pro with a V100 GPU and High RAM setting for the steps involving LLM. The notebook is divided into self-contained sections, most of which can be executed independently, minimizing dependency on previous steps. Data is saved after each section, allowing continuation in a new session if needed.…

Today, the world is abuzz with LLMs, short for Large Language models. Not a day passes without the announcement of a new language model, fueling the fear of missing out in the AI space. Yet, many still struggle with the basic concepts of LLMs, making it challenging to keep pace with the advancements. This article…

Gain a deeper understanding of Gaussian processes by implementing them with only NumPy. Gaussian Processes (GPs) are an incredible class of models. There are very few Machine Learning algorithms that give you an accurate measure of uncertainty for free while still being super flexible. The problem is, GPs are conceptually really difficult to understand. Most…

How to do poorly on Kaggle, and learn about RAG+LLM from it 23 min read · Dec 25, 2023 Image generated with ChatGPT+/DALL-E3, asking for an illustrative image for an article about RAG.Retrieval Augmented Generation (RAG) seems to be quite popular these days. Along the wave of Large Language Models (LLM’s),…

Ten of my LinkedIn posts on LLMs 1. Non-determinism in LLMs The best LLM use cases are where you use LLM as a tool rather than expose it directly. As Richard Seroter says, how many chatbots do you need? However, this use case of replacing static product pages by personalized product summaries is like many…

Experimenting with Large Language Models for free (Part 2) Photo by Glib Albovsky, UnsplashIn the first part of the story, we used a free Google Colab instance to run a Mistral-7B model and extract information using the FAISS (Facebook AI Similarity Search) database. In this part, we will go further, and I will show how…

Repositories with the most stars! Happy New Year 2024! As the first post in the new year, just like what I did before, I’m very curious about what were the most popular Python projects so far. GitHub is definitely the most suitable place to have these statistics. Although not all the open-sourced projects will be…

Use various data source types to quickly generate text data for artificial datasets. Image generated with DALL-E 3In a previous article, we explored creating many-to-one relationships between columns in a synthetic PySpark DataFrame. This DataFrame only consisted of Foreign Key information and we didn’t produce any textual information that might be useful in a demo…

Part 3: Causality Image by Cottonbro Studios from Pexels.comMy hope is that by the end of this article you will have a good understanding of how philosophical thinking around causation applies to your work as a data scientist. Ideally you will have a deeper philosophical perspective to give context to your work! This is the…

Discussion backed up by some concrete examples, sketching broad guidelines on how to develop better AI systems Photo by National Cancer Institute on UnsplashArtificial Intelligence has become an integral tool in scientific research, but concerns are growing that the misuse of these powerful tools is leading to a reproducibility crisis in science and its technological…