"What web scraping is: it goes out and pulls down all of the information that you want from a specific website."
"You don't have to be extremely familiar with HTML in order to scrape websites."
"Searching for a tag will get the first tag on the page, but the first tag might not always be what we want."
"With the findAll method, instead of just returning the first tag that matches, it will return a list of all the tags that match those arguments."
"If you run into something like this and you just want to skip by any missing information then what we can do is put that part of the code into a try-except block."
"Now that we've scraped the information that we want from our web page, now we can save this in any way that we'd like."
"Web scraping is a very powerful and useful tool whenever you're building not just web applications but programs in general with Python."
"Before web scraping, ensure compliance with privacy laws and platform terms of service to avoid legal issues."
"Scrapy is an open-source framework for extracting data from websites in a fast, simple, yet extensible way."
"If you're looking to do anything from small to large-scale scraping, it's much better to use something like Scrapy because it comes with a load of features built-in that you don't have to worry about."
"So once you've got your API key we can move back to our book Spider file and we will start adding in a new function which will help us send the traffic first to our new proxy provider."
"Settings in Scrapy contain everything related to how your spider and crawling operate, from obeying robots.txt to setting the number of concurrent requests."
"In part four of our Scrapy beginners course, we're going to look at how to create a Scrapy spider, use Scrapy shell to find CSS selectors, and extract data from multiple pages."
"Just a simple web search for 'Python web scraping' has tons of different how-to's of the way different people have done that for their various projects."
"Web scraping and GUI automation chapters. That's what I decided to make this talk about."
"...we have successfully scraped a website alright so that's it guys hopefully you enjoyed this and you know experiment with it if you want to use this site that's fine..."
"Hands down the best way to go for web scraping."
"This will be a great project to kind of hone in your skills on web scraping, data analysis, manipulation, and last but not least, visualization of the data."
"Thank you so much, I really appreciate it. See you tomorrow for the next installment of web scraping with Selenium."
"We hope that by the end of the series, you'll have a good idea of how to scrape websites using Python Scrapy."
"So I hope you now have enough to start scraping with the Python Scrapy framework."
"Selenium is a framework that allows us to scrape dynamic websites."
"You learn how to export the data we scraped into a CSV file."
"If you like to master web scraping in Python by solving code projects like this one, consider taking my course."
"This is how we web scrape the data; this is how we build a simple web crawler."
"I'm here to teach you how to web scrape with Python."
"In the absence of an API, if you learn web scraping, it is actually a very important tool for a data scientist and a data engineer to know because the entire Internet becomes your database."
"Beautiful Soup is another library commonly used for web scraping."
"Web scraping is basically the idea that we want to collect data from websites and parse it into a meaningful format."
"Web scraping is super empowering and really can help you leverage all of your cool data analyses."
"We're going to be pulling the HTML code and then making use of the structure of HTML to be able to find specific pieces of data that we want."
"Find all is going to give you back a result set with all of the elements on the entire page that match whatever you're asking soup to find."
"We can actually loop over this result set and just print out for every item I find in the result set, print out the item dot text."
"It's getting really powerful to just drill down deeper into only the information you want."
"These are a lot of the core building blocks of what happens in web scraping."
"The way that you're really going to learn web scraping is by trying a project like this, trying to develop strategies of how can I collect the information I want and then save it so that I can use it in a downstream analysis."
"Please do be responsible when you are web scraping."
"Shein's algorithms continuously comb the web to discover viral trends before they blow up."
"This is going to be one of the simplest and easiest ways to scrape any website in a matter of minutes."
"In this video, I'm going to show you how you can quickly and easily take your web scraper and then turn it into a fully functional API that we can deploy onto the web."
"We're going to be taking the quotes website here... and we're going to scrape it and we're going to create an API."
"What we want to do is we want to return a list of dictionary objects or JSON data of all of the quotes on that page."
"This is what we're doing with web scraping."
"We're going to fill out the form to search for programs that have the word 'Mobile' in them."
"That is how you scrape and pull content out of a web page so you can save that in a text file for access later on."
"Scrapy is the most complete web scraping framework in Python."
"Fetch helps us get the HTML markup of a website."
"Web scraping is more about extracting information from websites."
"Now you're ready to start scraping websites with Scrapy."
"Scrape web data at scale with zero infrastructure."
"This is probably one of the more simple methods for web scraping."
"We're going to walk through an end-to-end web scraping project where we scrape multiple types of data and we load it into pandas for easy analysis."
"Please suggest the best Python libraries for web scraping data."
"There are several libraries that are commonly used for web scraping in Python including Beautiful Soup, Scrapy, and Selenium."
"Beautiful Soup is widely used in web scraping applications and is considered one of the most popular and powerful web scraping tools available in Python."
"I was able to use a private helper method to encapsulate the functionality of actually getting the web page itself."
"Hello to everyone, this is our next tutorial in our series on web scraping using Python."
"It can be used to create screenshots and PDFs of pages, useful if you need to do some quick scraping of websites."
"Web scraping is a method of extracting huge amounts of data from a website and saving it to a local file or a database."
"Hope you guys have been enjoying my recent series on these web scraping tutorials."
"We're going to build a flow that's going to go out to a website, screen scrape some information off the website."
"When you talk Python and you talk web scraping, we're talking Beautiful Soup."
"We can use Python, we can use requests, Beautiful Soup to scrape this information."
"So now we've managed to scrape the first page and we know our functions work."
"We've now got a list of dictionaries with the title, the company, a salary, and a summary."
"This is pretty cool, you could do a lot more with this."
"Nice and easy web scraping with requests and Beautiful Soup again."
"This is another package in Python that you can use to help scrape dynamic content from websites."
"In this video, we're going to learn how to scrape content from a web page where some of the data is dynamically loaded by JavaScript."
"Now let's take a look at how we can use Selenium to do some web scraping, and of course, we'll do it in Jupyter for the interactiveness of it all."
"Beautiful Soup turns HTML into searchable text with Python."
"There's a lot more to it than you can do than just this but I just want to extract the main data that I wanted."
"Amazon to a degree wants people to scrape on their site so that you can actually buy things from their site."
"Now we're going to store our scraped data from Amazon into our Django model."
"We now have a way to scrape this data from Amazon, do it over time, and do it on our own schedule thanks to all of the tools involved."
"It's really useful to know how to do web scraping."
"Regular expressions are something that you may or may not use a whole lot, but when it comes to doing web scraping, you end up having to use regular expressions from time to time."
"So that should give us a timestamp related to this scraping event."
"Web scraping is a very, very valuable skill."
"Web scraping is an essential tool for your web developer toolkit."
"You understand the principles and you've really picked up the proper habits of how to do scraping properly."
"Turn any website in the whole world into an API with Google Chrome, Puppeteer, and Google Cloud."