{"podcast":{"title":"CyberCode Academy","slug":"cybercode-academy-7578615","podcast_index_feed_id":7578615,"rss_url":"https://www.spreaker.com/show/6790974/episodes/feed","website_url":"https://www.spreaker.com/podcast/cybercode-academy--6790974","image_url":"https://d3wo5wojvuv7l.cloudfront.net/t_rss_itunes_square_1400/images.spreaker.com/original/5f51ccc7b22fdba95149ffa6346f1533.jpg","author":"CyberCode Academy","episode_count":311,"summary":"Welcome to CyberCode Academy — your audio classroom for Programming and Cybersecurity. 🎧 Each course is divided into a series of short, focused episodes that take you from beginner to advanced level — one lesson at a time. From Python and web development to ethical hacking and digital defense, our content transforms complex concepts into simple, engaging audio learning. Study anywhere, anytime — and level up your skills with CyberCode Academy. 🚀 Learn. Code. Secure. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy","last_synced_at":"2026-07-27T12:18:25.188608+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615"},"episode":{"title":"Course 40 - Web Scraping with Python | Episode 13: Mastering Scrapy Shell, CSS, and XPath Selectors","slug":"course-40-web-scraping-with-python-episode-13-mastering-scrapy-shell-css-and-xpath-selectors","published_at":"2026-07-23T06:00:03+00:00","page_url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-13-mastering-scrapy-shell-css-and-xpath-selectors","show_page_url":"https://stenobird.com/podcast/cybercode-academy-7578615","url":"https://www.spreaker.com/episode/course-40-web-scraping-with-python-episode-13-mastering-scrapy-shell-css-and-xpath-selectors--72756861","audio_url":"https://dts.podtrac.com/redirect.mp3/api.spreaker.com/download/episode/72756861/prototyping_data_extraction_in_scrapy_shell.mp3","summary":"In this lesson, you’ll learn about: how to use Scrapy Shell for interactive crawling, how CSS selectors work for fast extraction, and how XPath enables advanced and flexible data targeting1. What is Scrapy Shell?🔹 Interactive Prototyping ToolScrapy Shell is a live testing environment where you can: Test selectors before writing spiders Inspect HTML responses instantly Experiment with scraping logic 🔹 Key Objects Inside Shell response → HTML content of the page request → HTTP request details spider → scraper context 👉 Key Insight You can test everything before writing real crawling logic2. Working with Live URLs and FilesScrapy Shell supports: 🌐 Live websites 📄 Local HTML files 👉 This makes it ideal for debugging broken or complex pages3. CSS Selectors (Fast & Simple)🔹 Basic Extraction🔹 Common SyntaxSelectorMeaning#idSelect by ID.classSelect by classtagSelect by tag🔹 Scrapy Shell Methodsresponse.css(\"title\").get() response.css(\"p\").get_all() 🔹 Extract Attributesresponse.css(\"img::attr(src)\").get() 👉 Key Insight CSS is perfect for quick, readable extraction4. Important Behavior: Cached Responses🔹 One Hidden DetailScrapy Shell: Works on cached HTML Won’t reflect live changes unless restarted 👉 Key Insight Always restart shell when debugging updated pages5. XPath Selectors (Advanced Power)🔹 Full DOM NavigationXPath lets you navigate HTML like a tree structure6. Absolute vs Relative XPath🔹 Absolute Path/html/body/div/p Starts from root Very strict 🔹 Relative Path//div/p Searches anywhere More flexible 7. Attribute Matching in XPath🔹 Using @response.xpath(\"//img[@id='logo']\").get() 🔹 Using Wildcardsresponse.xpath(\"//*[@id='main']\").get() 8. Using contains()🔹 Pattern Matchingresponse.xpath(\"//p[contains(text(), 'news')]\").get() 👉 Key Insight XPath is powerful for uncertai…","meta_description":"In this lesson, you’ll learn about: how to use Scrapy Shell for interactive crawling, how CSS selectors work for fast extraction, and how XPath enables ad…","key_points":[],"chapters":[],"topics":[],"duration_seconds":1239,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/cybercode-academy-7578615/episodes/course-40-web-scraping-with-python-episode-13-mastering-scrapy-shell-css-and-xpath-selectors/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/cybercode-academy-7578615/course-40-web-scraping-with-python-episode-13-mastering-scrapy-shell-css-and-xpath-selectors.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}