> For the complete documentation index, see [llms.txt](https://docs.codewords.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.codewords.ai/automations/firecrawl-for-web-scraping.md).

# Firecrawl for web scraping

Learn how to use Firecrawl on CodeWords to scrape websites, extract clean text, capture screenshots, and monitor changes automatically — no coding or setup required.

### What is Firecrawl?

Firecrawl extracts text, data, and visuals from public web pages. It returns clean, structured content from a URL so you can use that data in your workflows.

On CodeWords, Firecrawl is built in — you can start scraping without extra setup, API keys, or configuration.

### When to use Firecrawl

**Best for:**

* Collecting article or blog content
* Tracking website or pricing updates
* Extracting product information from multiple pages
* Researching competitors or market trends
* Gathering content for summaries or analysis

**Avoid Firecrawl for:**

* Sites that require login
* Pages with hidden or interactive content — use the [Web Agent](https://docs.codewords.ai/features/web-agent) or [Chrome extension](https://docs.codewords.ai/features/chrome-extension) instead
* Tasks that need clicking, filling forms, or multi-step browsing

### How Firecrawl works

Firecrawl visits the URL you specify, processes the page, and returns the content in a clean format. You can:

* Extract the **text content** of the page
* Capture a **screenshot** for visual reference
* Combine both for a fuller overview

Then use that output in CodeWords — for example, summarize it with Cody, analyze trends, or pass it into other automations.

### Common use cases

**Research and analysis**\
Gather text from news articles, blogs, or reports to create summaries or insights.

**Competitor monitoring**\
Check and compare pricing, features, or content updates on competitor sites.

**Product data collection**\
Pull information from multiple product pages into one structured view.

**Visual monitoring**\
Capture screenshots to track layout or design changes on important pages.

### FAQs

<details>

<summary>Do I need to install or set up Firecrawl separately?</summary>

No. Firecrawl is built into CodeWords and works inside your workflows — no extra configuration or API key needed.

</details>

<details>

<summary>Can Firecrawl scrape pages that require a login?</summary>

No. Firecrawl only works with publicly accessible pages. For sites that need login, use the [Chrome extension](https://docs.codewords.ai/features/chrome-extension), which can work with your authenticated browser session.

</details>

<details>

<summary>What kind of data can Firecrawl extract?</summary>

Readable text, key page content, and full-page screenshots — useful for research, monitoring, and analysis.

</details>

<details>

<summary>How fast is Firecrawl?</summary>

Most pages process in about 2–5 seconds. Larger or more complex sites may take a bit longer.

</details>

<details>

<summary>What if a website blocks scraping?</summary>

Some sites restrict automated access. If you hit errors or missing data, ask Cody to retry with a short delay, or switch to the [Web Agent](https://docs.codewords.ai/features/web-agent) for interactive sites.

</details>

<details>

<summary>Is it legal to scrape any website?</summary>

Only scrape public pages and follow the site’s terms of service. Use Firecrawl responsibly for legitimate business or research purposes.

</details>

<details>

<summary>Can I schedule Firecrawl to run automatically?</summary>

Yes. Set up a [scheduled workflow](https://docs.codewords.ai/automations/schedules-and-triggers) to run Firecrawl on a cadence — useful for daily monitoring or reports.

</details>

<details>

<summary>Does Firecrawl save the data it collects?</summary>

Firecrawl itself doesn’t store your data permanently. You can save scraped information in your workflow with tools like Redis, Google Sheets, or other connected services.

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.codewords.ai/automations/firecrawl-for-web-scraping.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.