{"podcast":{"title":"CyberCode Academy","slug":"cybercode-academy-7578615","podcast_index_feed_id":7578615,"rss_url":"https://www.spreaker.com/show/6790974/episodes/feed","website_url":"https://www.spreaker.com/podcast/cybercode-academy--6790974","image_url":"https://d3wo5wojvuv7l.cloudfront.net/t_rss_itunes_square_1400/images.spreaker.com/original/5f51ccc7b22fdba95149ffa6346f1533.jpg","author":"CyberCode Academy","episode_count":311,"summary":"Welcome to CyberCode Academy — your audio classroom for Programming and Cybersecurity. 🎧 Each course is divided into a series of short, focused episodes that take you from beginner to advanced level — one lesson at a time. From Python and web development to ethical hacking and digital defense, our content transforms complex concepts into simple, engaging audio learning. Study anywhere, anytime — and level up your skills with CyberCode Academy. 🚀 Learn. Code. Secure. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy","last_synced_at":"2026-07-27T12:18:25.188608+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615"},"episode":{"title":"Course 40 - Web Scraping with Python | Episode 7: Overcoming the JavaScript Challenge","slug":"course-40-web-scraping-with-python-episode-7-overcoming-the-javascript-challenge","published_at":"2026-07-17T06:00:02+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-7-overcoming-the-javascript-challenge","show_page_url":"https://stenobird.com/podcast/cybercode-academy-7578615","url":"https://www.spreaker.com/episode/course-40-web-scraping-with-python-episode-7-overcoming-the-javascript-challenge--72756776","audio_url":"https://dts.podtrac.com/redirect.mp3/api.spreaker.com/download/episode/72756776/scraping_javascript_websites_with_selenium_and_python.mp3","summary":"In this lesson, you’ll learn about: why JavaScript breaks traditional scrapers, how to detect dynamic content issues, and the tools used to scrape modern interactive websites1. Why Traditional Scraping Fails on Modern Websites🔹 The Core ProblemLibraries like Requests and Scrapy: Only download initial HTML Do NOT execute JavaScript 👉 Result: Missing data Empty elements Incomplete pages 🔹 What Actually Happens in Modern Sites Browser loads basic HTML JavaScript runs Data is fetched via APIs (AJAX/XHR) DOM updates dynamically 👉 Key Insight The real data often exists only after JavaScript execution2. How to Detect a “JavaScript Problem”🔹 Using Chrome DevToolsSteps: Open DevTools → Elements tab Disable JavaScript OR simulate slow network Reload page 🔹 What You’re Looking For Missing tables/content Empty elements Data appearing only after delay 👉 If content disappears → scraper will fail🔹 Pro TrickCheck Network → XHR/Fetch You might find the real API endpoint Sometimes you can skip browser automation entirely 3. Solution #1: Requests-HTML (Simple & Powerful)🔹 OverviewUse Requests-HTML Built on: Puppeteer via Pyppeteer 🔹 How It Works Loads page in headless browser Executes JavaScript Returns fully rendered HTML 🔹 Examplefrom requests_html import HTMLSession session = HTMLSession() r = session.get(\"https://example.com\") r.html.render() data = r.html.find(\"div.item\") 🔹 When to Use It Medium complexity sites Quick projects When you want minimal setup 4. Solution #2: Selenium (Full Control)🔹 OverviewUse Selenium Controls real browsers: Chrome Firefox 🔹 Key Feature: “Wait Until”from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC element = WebDriverWait(driver, 10).u…","meta_description":"In this lesson, you’ll learn about: why JavaScript breaks traditional scrapers, how to detect dynamic content issues, and the tools used to scrape modern…","key_points":[],"chapters":[],"topics":[],"duration_seconds":1032,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/cybercode-academy-7578615/episodes/course-40-web-scraping-with-python-episode-7-overcoming-the-javascript-challenge/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-7-overcoming-the-javascript-challenge.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}