BestPromptFinder

Below is an instruction that describes a task. Write a respo...

The prompt

Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Create a web crawler in Python that starts from a website and looks for other links in the page then scrapes the page content. ### Input: Not applicable ### Output: import requests from bs4 import BeautifulSoup import re def crawl(url): visited = set() queue = [url] while queue: curr = queue.pop(0) visited.add(curr) response = requests.get(curr) if response.ok: soup = BeautifulSoup(response.text, 'html.parser') # scrape the page here # find all the links links = soup.find_all('a', href=re.compile('^http')) for link in links: if link['href'] not in visited: queue.append(link['href'])

Why this prompt

• AI-graded 55/100 for quality and structure
• Reusable template — swap in {LOCATION}
• 87% found it useful across Gemini, GPT-5.6, Claude

Source

Hugging Face

Related Coding prompts