A Guide to Web Scraping with Node.js

Here before fireship gets jailed for "stealing" data

Why do you put the Freemasonry symbol in your videos?

o my the voice is so high haha

Whats the name of intro music?

Seems like almost every project I’ve done in the past few years has required at least one web scraper to function. It does get really tiring having to keep them working, but I usually can’t find an API appropriate for my use case (or if I can, they’re from ridiculously overpriced SaaS providers).

bro sounds soooo young my god

AI voice

why is your voice high pitched in this one? AI voice huh........

An outstanding video that has been a valuable reference for our newcomers. We sincerely thank you for sharing!

bring back the MySQL pillow....

I only start understanding things better after watching your videos. It's such a great explanation and never boring.

Love how much more squeaky Jeff's voice was!

First time I heard smth like: in case you don't know jquery! :))

Can I still get a t-shirt?

Free shirt!?!?

Turn on autogenerated subs please

Hi Frd, How to by pass cloudflare in node js

I don't get the first one to work. Even when I copy your code. In Line 8 the arrow function inside the map fires a parsing error... (by eslint I guess). No idea what goes wrong.

This video is definitely not for a beginner.

Playwright is a great tool for doing this instead of using Puppeteer

Thanks for the insights!

Dude...this is exactly what I needed... why are you so helpful

For those getting ERR_REQUIRE_ESM error downgrade to node-fetch version 2.66
npm uninstall node-fetch
npm install [email protected]
if using TypeScript
npm install --save-dev @types/[email protected]

where subtitles

🔥🔥🔥

this is awesome man. Thank a lot!

there is onw usecase,
when i am hitting the endpoint, its returning the very minimum data, then its loading the subsequent data (similar to client side rendering). i need to scrape that data, which is loaded after the initial load.
how to scrape that data.
any help is appreciated
Thanks

I think it would be great if you make a video on how to make a telegram bot

Cant seem to get this to run. keep getting "Your function was killed because it raised an unhandled error." after npm run serve

Hmmm....keep getting res.text() is not a function

Is there a link to the github of the code you used in this video ?

Great video! If I want to create a robot that will login to a website and stay there till X product is available for purchase, than add it to a cart & buy it, what language/framework will u use?

Hello! Thanks for making this video!

Just wondering how you got puppeteer to work. Are you using WSL?

Very helpful sir. But I have a question: how we can get the all sections from a webpage and how to put them in clipboard?


Thanks un Advance 🙏

Is node js scraping is like python scrapy framework? which one better performe?

you fergot to mention the part where you say: "this video is educational only"
i'm suprised this has not gotten a DMCA strike already

Do Tampermonkey please, it's next level Pupeteer

"Awesome tutorial thank you. but for a non-tech user it is quiet hard to do a workable scraper for my WooCommerce store. As a side solution i am using eCommerce "ESCRAPER" maybe it helps somebody too.
But I am not giving up))) Thank you for your input!!!"

Thanks for this upload. I got into a little bit of web scraping a couple years ago but ran into a lot more complexity than I was expecting for a tiny little side project. Now that I know a lot more, and JS, hoping to add this to the toolbelt.

Any suggestions on the best tools for scraping content from locally running applications (not those running in a browser environment).

Is fireship setting the speed on videos 2x since i have to set it on 0.5 in order to understand?

Does cloud functions available for free version of firebase ?

Is the robots.txt file related to web scraping? I’ve got a hunch that it is

I didnt even know you can scrape with javascript... Im only a python focused dev and i only use JS for web development (MERN stack)... Thanks

I love your video, but can you make subtitle for non-English speakers like me because you talk pretty fast :D ( it is not complaining)

Can I have a shirt?
Great vid btw