{"podcast":{"title":"CyberCode Academy","slug":"cybercode-academy-7578615","podcast_index_feed_id":7578615,"rss_url":"https://www.spreaker.com/show/6790974/episodes/feed","website_url":"https://www.spreaker.com/podcast/cybercode-academy--6790974","image_url":"https://d3wo5wojvuv7l.cloudfront.net/t_rss_itunes_square_1400/images.spreaker.com/original/5f51ccc7b22fdba95149ffa6346f1533.jpg","author":"CyberCode Academy","episode_count":311,"summary":"Welcome to CyberCode Academy — your audio classroom for Programming and Cybersecurity. 🎧 Each course is divided into a series of short, focused episodes that take you from beginner to advanced level — one lesson at a time. From Python and web development to ethical hacking and digital defense, our content transforms complex concepts into simple, engaging audio learning. Study anywhere, anytime — and level up your skills with CyberCode Academy. 🚀 Learn. Code. Secure. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy","last_synced_at":"2026-07-27T12:18:25.188608+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615"},"episode":{"title":"Course 40 - Web Scraping with Python | Episode 17: Mastering Requests, Regex, and Beautiful Soup","slug":"course-40-web-scraping-with-python-episode-17-mastering-requests-regex-and-beautiful-soup","published_at":"2026-07-27T06:00:02+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-17-mastering-requests-regex-and-beautiful-soup","show_page_url":"https://stenobird.com/podcast/cybercode-academy-7578615","url":"https://www.spreaker.com/episode/course-40-web-scraping-with-python-episode-17-mastering-requests-regex-and-beautiful-soup--72756892","audio_url":"https://dts.podtrac.com/redirect.mp3/api.spreaker.com/download/episode/72756892/python_web_scraping_with_beautifulsoup_and_regex.mp3","summary":"In this lesson, you’ll learn about: how Python retrieves web pages, how regex is used for pattern-based extraction, and how BeautifulSoup improves scraping by understanding HTML structure instead of treating it as plain text1. Fetching Web Content in Python🔹 HTTP Request FlowWeb scraping always starts with getting the page content.🔹 Libraries Used urllib → built-in, basic control httplib2 → low-level control requests → easiest and most popular 🔹 Requests Exampleimport requests response = requests.get(\"https://example.com\") html = response.text 🔹 User-Agent HandlingSome sites block bots, so you can:headers = {\"User-Agent\": \"Mozilla/5.0\"} requests.get(url, headers=headers) 👉 Key Insight Without proper headers, many sites will reject your scraper2. Regular Expressions (Regex Basics)🔹 Pattern Matching ConceptRegex treats web data as raw text patterns.3. Core Regex FunctionsFunctionBehaviormatch()checks start onlysearch()finds first match anywherefindall()returns all matches🔹 Special SymbolsSymbolMeaning\\ddigits\\wletters + numbers\\swhitespace🔹 Example Patternimport re re.findall(r\"\\d+\", \"Price is 123 dollars\") 👉 Key Insight Regex is powerful but fragile for HTML4. Advanced Regex Techniques🔹 Ranges & Groups [A-Z] → uppercase letters {3} → exact repetition ( ) → capture groups 🔹 Example: Extract Namesre.search(r\"(\\w+) (\\w+)\", \"John Smith\") 5. Real Web Scraping Use Cases🔹 Inspecting HTMLUsing browser tools, you can locate: items , headers contact details location data 🔹 Example Targets Phone numbers Zip codes City/state data 6. BeautifulSoup (Structured Parsing)🔹 DOM-Based ApproachBeautifulSoup understands HTML as a tree structure, not text.🔹 Basic Usagefrom bs4 import BeautifulSoup soup = BeautifulSoup(html, \"lxml\") print(soup.title.string) 🔹 Key Advantage Navigates tags…","meta_description":"In this lesson, you’ll learn about: how Python retrieves web pages, how regex is used for pattern-based extraction, and how BeautifulSoup improves scrapin…","key_points":[],"chapters":[],"topics":[],"duration_seconds":1279,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/cybercode-academy-7578615/episodes/course-40-web-scraping-with-python-episode-17-mastering-requests-regex-and-beautiful-soup/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-17-mastering-requests-regex-and-beautiful-soup.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}