The Real Python Podcast

How do you process and classify text documents in Python? What are the fundamental techniques and building blocks for Natural Language Processing (NLP)? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, talks about how machine learning (ML) models understand text.

Jodie explains how ML models require data in a structured format, which involves transforming text documents into columns and rows. She covers the most straightforward approach, called binary vectorization. We discuss the bag-of-words method and the tools of stemming, lemmatization, and count vectorization.

We jump into word embedding models next. Jodie talks about WordNet, Natural Language Toolkit (NLTK), word2vec, and Gensim. Our conversation lays a foundation for starting with text classification, implementing sentiment analysis, and building projects using these tools. Jodie also shares multiple resources to help you continue exploring NLP and modeling.

Course Spotlight: Learn Text Classification With Python and Keras

In this course, you’ll learn about Python text classification with Keras, working your way from a bag-of-words model with logistic regression to more advanced methods, such as convolutional neural networks. You’ll see how you can use pretrained word embeddings, and you’ll squeeze more performance out of your model through hyperparameter optimization.

Topics:

• 00:00:00 – Introduction
• 00:02:47 – Exploring the topic
• 00:06:00 – Perceived sentience of LaMDA
• 00:10:24 – How do we get started?
• 00:11:16 – What are classification and sentiment analysis?
• 00:13:03 – Transforming text in rows and columns
• 00:14:47 – Sponsor: Snyk
• 00:15:27 – Bag-of-words approach
• 00:19:12 – Stemming and lemmatization
• 00:22:05 – Capturing N-grams
• 00:25:34 – Count vectorization
• 00:27:14 – Stop words
• 00:28:46 – Text Frequency / Inverse Document Frequency (TFIDF) vectorization
• 00:32:28 – Potential projects for bag-of-words techniques
• 00:34:07 – Video Course Spotlight
• 00:35:20 – WordNet and NLTK package
• 00:37:27 – Word embeddings and word2vec
• 00:45:30 – Previous training and too many dimensions
• 00:50:07 – How to use word2vec and Gensim?
• 00:51:26 – What types of projects for word2vec and Gensim?
• 00:54:41 – Getting into GPT and BERT in another episode
• 00:56:11 – How to follow Jodie’s work?
• 00:57:36 – Thanks and goodbye

Show Links:

• Why Google’s “sentient” AI LaMDA is nothing like a person.
• On NYT Magazine on AI: Resist the Urge to be Impressed | Emily M. Bender | Medium
• ELIZA - Wikipedia
• eliza.py - Python 2 version by Daniel Connelly
• dabraude/Pyliza: Python3 Implementation of Eliza
• magneticpoetry.com
• Natural Language Processing With Python’s NLTK Package – Real Python
• Practical Text Classification With Python and Keras – Real Python
• Sentiment Analysis: First Steps With Python’s NLTK Library – Real Python
• NLTK: Natural Language Toolkit
• spaCy · Industrial-strength Natural Language Processing in Python
• Natural Language Processing With spaCy in Python - Real Python
• Stemming - Wikipedia
• Lemmatization - Wikipedia
• Binary/Count Vectorization: sklearn.feature_extraction.text.CountVectorizer— scikit-learn
• TFIDF: sklearn.feature_extraction.text.TfidfVectorizer — scikit-learn
• Porter Stemmer: nltk.stem.porter module — NLTK
• Snowbal...

What’s the big breakthrough for Natural Language Processing (NLP) that has dramatically advanced machine learning into deep learning? What makes these transformer models unique, and what defines “attention?” This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, continues our talk about how machine learning (ML) models understand and generate text.

This episode is a continuation of the conversation in episode #119. Jodie builds on the concepts of bag-of-words, word2vec, and simple embedding models. We talk about the breakthrough mechanism called “attention,” which allows for parallelization in building models.

We also discuss the two major transformer models, BERT and GPT3. Jodie continues to share multiple resources to help you continue exploring modeling and NLP with Python.

Course Spotlight: Building a Neural Network & Making Predictions With Python AI

In this step-by-step course, you’ll build a neural network from scratch as an introduction to the world of artificial intelligence (AI) in Python. You’ll learn how to train your neural network and make predictions based on a given dataset.

Topics:

• 00:00:00 – Introduction
• 00:02:20 – Where we left off with word2vec...
• 00:03:35 – Example of losing context
• 00:06:50 – Working at scale and adding attention
• 00:12:34 – Multiple levels of training for the model
• 00:14:10 – Attention is the basis for transformer models
• 00:15:07 – BERT (Bidirectional Encoder Representations from Transformers)
• 00:16:29 – GPT (Generative Pre-trained Transformer)
• 00:19:08 – Video Course Spotlight
• 00:20:08 – How far have we moved forward?
• 00:20:41 – Access to GPT-2 via Hugging Face
• 00:23:56 – How to access and use these models?
• 00:30:42 – Cost of training GPT-3
• 00:35:01 – Resources to practice and learn with BERT
• 00:38:19 – GPT-3 and GitHub Copilot
• 00:44:35 – DALL-E is a transformer
• 00:46:13 – Help yourself to the show notes!
• 00:49:19 – How can people follow your work?
• 00:50:03 – Thanks and goodbye

Show Links:

• Recurrent neural network - Wikipedia
• Long short-term memory - Wikipedia
• Vanishing gradient problem - Wikipedia
• Vanishing Gradient Problem | What is Vanishing Gradient Problem?
• Attention Is All You Need | Cornell University
• Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar
• Standing on the Shoulders of Giant Frozen Language Models | Cornell University
• #datalift22 Embeddings paradigm shift: Model training to vector similarity search by Nava Levy - YouTube
• Transformer Neural Networks - EXPLAINED! (Attention is all you need) - YouTube
• BERT 101 - State Of The Art NLP Model Explained
• How GPT3 Works - Easily Explained with Animations - YouTube
• Write With Transformer (GPT2 Live Playground Tool) - Hugging Face
• Language Model with Alpa (GPT3 Live Playground Tool) OPT-175B
• Big Data | Music
• OpenAI API
• 🤗 (Hugging Face)Transformers Notebooks
• GitHub Copilot: Fly With Python at the Speed of Thought
• GitHub Copilot learned about the daily struggle of JavaScript developers after being trained on billions of lines of code. | Marek Sotak on Twitter
• Jodie Burchell’s Blog - Standard error
• Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) / Twitter
• JetBrains: Essential tools for software ...

Has the current growth of artificial intelligence (AI) systems made you wonder what the future holds for Python developers? What are the hidden benefits of learning to program in Python and practicing computational thinking? This week on the show, we speak with author Lawrence Gray about his upcoming book “Mastering Python: A Problem Solving Approach.”

Lawrence shares how learning Python helped him through a dark and trying time. He developed lifelong skills that he wants to pass along through teaching and authoring a book.

We discuss what you can do to prepare for a future where coding jobs are automated through AI. He shares ways that Python can help build the higher-order thinking skills required by future careers. We also talk about how Python can help with computational thinking and promote cognitive development.

This week’s episode is brought to you by Site24x7.

Course Spotlight: Looping With Python enumerate()

Once you learn about for loops in Python, you know that using an index to access items in a sequence isn’t very Pythonic. So what do you do when you need that index value? In this course, you’ll learn all about Python’s built-in enumerate(), where it’s used, and how you can emulate its behavior.

Topics:

• 00:00:00 – Introduction
• 00:02:29 – Learning Python and changing how you think
• 00:05:07 – What is the goal of the book?
• 00:05:59 – Discovering Python during a dark period
• 00:10:32 – What is unique to Python that helped you?
• 00:12:43 – How did you start teaching Python?
• 00:14:40 – Teaching Python to art students
• 00:18:52 – Sponsor: Site24x7.com
• 00:19:48 – Why write about computational thinking?
• 00:21:55 – Why learn Python now?
• 00:25:18 – Multiple modalities for Python
• 00:29:32 – Other optimistic thoughts on the future
• 00:34:15 – Reskilling a workforce
• 00:37:16 – What skills can be developed?
• 00:41:13 – Video Course Spotlight
• 00:42:40 – Bloom’s taxonomy
• 00:48:00 – Sharing the positive impact of Python
• 00:50:26 – What are you excited about in the world of Python?
• 00:51:32 – What do you want to learn next?
• 00:52:45 – How can people follow your work online?
• 00:52:47 – Thanks and goodbye

Show Links:

• The Future of Jobs Report 2023 - World Economic Forum
• Mastering Python: Get Access to Chapter 1
• Computational thinking - Wikipedia
• Bloom’s taxonomy - Wikipedia
• Probably Overthinking It – Data science, Bayesian Statistics, and other ideas
• scikit-learn: machine learning in Python - documentation
• Yellowbrick: Machine Learning Visualization - documentation
• PyCon US 2024 - PyCon US 2024
• Education Summit - PyCon US 2023
• PyTorch - Get Started
• Lawrence Gray Personal Site

Level up your Python skills with our expert-led courses:

• Using k-Nearest Neighbors (kNN) in Python
• How to Write Pythonic Loops
• Looping With Python enumerate()

Support the podcast & join our community of Pythonistas

What are real-life examples of using Python decorators? How can you harness their power in your code? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.

We discuss a recent article series that digs into Python decorators. The first two articles discuss the basics of constructing decorators. The third part describes how popular Python libraries use decorators with call interception, function registration, and enriching the behavior of a function.

Christopher shares a piece about the common pitfalls of working with the Pythondatetime library. The article considers how current third-party libraries don’t address most of these quirks and offers a potential solution with a new library.

We also share several other articles and projects from the Python community, including a couple of news items, a discussion about the popularity of the Rust language, handling unset values in FastAPI with Pydantic, working with Python’s mini-language for formatting strings, mocking Django queryset functions, and a modern replacement for the Requests library.

This week’s episode is brought to you by Sentry.

Course Spotlight: Python Decorators 101

In this course on Python decorators, you’ll learn what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions in Python. By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

Topics:

• 00:00:00 – Introduction
• 00:02:53 – Django security releases issued: 5.0.2, 4.2.10, and 3.2.24
• 00:03:10 – Python 3.12.2 and 3.11.8 are now available
• 00:03:21 – Introducing PSF Grants Program Office Hours
• 00:04:19 – Python’s Format Mini-Language for Tidy Strings
• 00:12:22 – Ten Python datetime Pitfalls
• 00:18:34 – Sponsor: Sentry
• 00:19:37 – Real Life Use of Decorators
• 00:29:18 – Handling Unset Values in FastAPI With Pydantic
• 00:35:43 – Video Course Spotlight
• 00:37:06 – The Python Rust-Aissance
• 00:50:19 – django-mock-queries: Mock Django Queryset Functions
• 00:53:09 – niquests: Requests but Multiplexed
• 00:55:55 – Thanks and goodbye

News:

• Django security releases issued: 5.0.2, 4.2.10, and 3.2.24
• Python 3.12.2 and 3.11.8 are now available
• Introducing PSF Grants Program Office Hours

Show Links:

• Python’s Format Mini-Language for Tidy Strings – In this tutorial, you’ll learn about Python’s format mini-language. See how to use it for creating working format specifiers and build nicely formatted strings and messages in your code.
• Ten Python datetime Pitfalls – It’s no secret that the Python datetime library has its quirks. Not only are there probably more than you think, but third-party libraries don’t address most of them! Arie created a new library to explore what a better datetime library could look like.
• Real Life Use of Decorators – Part 3 in a series on how Python decorators are used. This part covers real-life use cases including call interception, function registration, and behavioral enrichment.
• Handling Unset Values in FastAPI With Pydantic – When using the HTTP PATCH method only those fields that got changed are updated. Pydantic sets fields not given as arguments as None so there is no way to distinguish between an explicit None value and an unset field. This post explains how you process this scenario.

Discussion:

• The Python Rust-Aissance – Companies like Polars are showing how with Rust, Python developers now have a better, smoother path towards building high-performance libraries.
• Rye: A Python Developer Experience Vision Continued
• PyO3: Rust bindings for the Python interpreter
• Rust in Linux: Where we are...

Do you have gaps in your Python learning path? If you’re like me, you may have followed a completely random route to learn Python. This week on the show, David Amos is here to talk about the release of the Real Python book, “Python Basics: A Practical Introduction to Python 3”. The book is designed not only to get beginners up to speed but also to help fill in the gaps many intermediate learners may still have.

David has been working on the book for the last two years, and we dive into all the resources that come with it. These include code challenges, quizzes, and multiple projects that are designed to help you cement your learning. We also discuss the people and processes involved in creating, reviewing, and updating the book.

Spotlight: Python Basics: A Practical Introduction to Python 3

Go from beginner to intermediate in Python with this complete curriculum, up-to-date for Python 3.9. Python Basics includes exercises, interactive quizzes, and sample projects, so you’ll always know what to focus on next in order to build a strong Python foundation.

Topics:

• 00:00:00 – Introduction
• 00:01:36 – Python Basics: A Practical Introduction to Python 3
• 00:02:39 – How has feedback helped the process?
• 00:04:17 – Who is the intended audience?
• 00:06:02 – Covering how to get Python installed on different platforms?
• 00:11:05 – What topics does the book cover?
• 00:14:12 – What can be previewed on Real Python?
• 00:15:03 – What format can you get the book in?
• 00:18:30 – Code challenges included!
• 00:21:33 – What other resources are provided to help with cementing your learning?
• 00:22:41 – What versions of Python are covered?
• 00:23:08 – How does the book fit into the Real Python learning eco-system?
• 00:29:35 – Spotlight: How to get a preview of the book!
• 00:30:38 – What has the writing process been like?
• 00:33:21 – What does didactic mean, in terms of reviewing materials?
• 00:39:23 – What were areas you were excited about updating?
• 00:41:29 – Were there important things you felt needed to be added?
• 00:45:55 – Who worked on the book?
• 00:47:13 – What are you excited about in the world of Python?
• 00:48:13 – What do you want to learn next?
• 00:49:40 – Thanks and goodbye

Show Links:

• Python Basics: A Practical Introduction to Python 3
• Python 3 Installation & Setup Guide
• Create and Modify PDF Files in Python
• Python GUI Programming With Tkinter
• Object-Oriented Programming (OOP) in Python 3
• A Practical Introduction to Web Scraping in Python

Level up your Python skills with our expert-led courses:

• Grow Your Python Portfolio With 13 Intermediate Project Ideas
• Cool New Features in Python 3.9
• Unicode in Python: Working With Character Encodings

Support the podcast & join our community of Pythonistas