The Intelligent Worker
Posts
🔀 Extract website data 10x faster

🔀 Extract website data 10x faster

+ 📊 Visualize datasets + 💻 CAPTCHAs are now broken

Henry Habib
August 21, 2023

ENTERPRISE DNA PRESENTS

Hello folks,

My first ever “real” job was working for a real estate company…

Every morning, I would spend the first hour going to 3 websites and pull data for new listings, and then copy them to an Excel file.

After a month, I wrote a program in Python and automated it - it took me 2 weeks and several Stack Overflow discussions.

Nowadays, AI and No-Code can do that for you in less than 2 minutes, and that is no exaggeration.

Here are the best ways to scrape data from any website and automate the boring parts of your job…

Subscribe | Sponsor

_{This post may contain referral, sponsored, and partnership links. If it does, it will be disclosed. If you use these links, we may earn a commission, with no cost to you.}

In this issue:

📕 Learn Something: Extract and monitor data from websites
💥 Trending Tool: Understand and visualize data in 1 click
🙃 Something Fun: Is there even a point to CAPTCHAs anymore?
❓ Big Question: What AI trend are you most worried about?

Let’s get right into it…

📕 LEARN SOMETHING
Extract and monitor data from websites

1/ Browse AI

This tool is likely the easiest way to scrape data from any website, with absolutely no code required.

Here is a quick bot I made that takes the top products from ProductHunt every day and populates an Excel file. I made this in 5 minutes.

Step 1: Select extract structured data

Step 2: Put in the URL you want to scrape

Step 3: Select the Robot and click Capture List

Step 4: Select the list and elements you want to extract

Step 5: Browse AI captures the data which can then be exported into Excel

In just 5 steps, I created a bot that I can run every day that will produce the following Excel file, from top products on ProductHunt.com

Here are some other features of Browse AI:

You can extract data on a schedule
You can get notified of any changes on the website’s data (great for monitoring ticket sales)
If your data is behind a log-in page, no sweat! Browse AI can automate log-ins for you
Note: You do have install a Chrome extension, but that’s it

Want to try it? We spoke with Browse AI got all our readers 10% off! (referral)

2/ Bardeen

This tool is very similar to Browse AI, except that it adds a whole element of Automation to scraping data from websites.

For example, you can build flows like:

Scrape data —> Populate Excel file —> Send me an email
Log on —> See if Data has Changed —> Send Slack Message

In fact, Bardeen has already created thousands of pre-automated flows for very common use cases

Another cool thing? You can get AI to create the automation for you! Check it out below

💥 TRENDING TOOL
Understand and visualize data in 30 seconds

ChartPixel takes any data you have and creates both insights and visualizations on it, instantly.

It’s basically a quick way to digest any dataset. It can’t go too detailed, but it’s useful if you want a quick overview.

Best part? The Beta version does not cost a thing!

Here are some charts that were made for me with an AI Salaries dataset:

🙃 SOMETHING FUN
Is there even a point to CAPTCHAs anymore?

A bombshell just dropped over the weekend. 💣

You know those annoying CAPTCHA’s that make you identify crosswalks in a collage of 9 images? Well, their purpose is to prevent bots.

Guess who can solve these now? That’s right, bots!

The kicker? AI bots can do it better than humans! A study by the University of California found that AI bots had an accuracy range of 85-100% whereas the human accuracy rate is only 50-85%.

This is a big deal! These CAPTCHAs also exist to prevent hackers from executing bad automations. We’ll need to solve this problem soon.

Also, I hate CAPTCHAs. Gene Tsudik, one of the study's researchers, commented on the general dislike for CAPTCHAs, questioning the global effort invested in solving them.

If bots can solve them now, why even have them?