
Build a Real Estate AI Agent with CrewAI & Bright Data
The real estate sector is undergoing significant shifts with intelligent robotics and AI systems entirely replacing manual work. Businesses in real estate often deal with disparate datasets, labor-intensive analysis, and enormous limits to scale. But what if such a system existed that could reason, adapt, and autonomously provide complete property services?
You will learn in this guide:
How modern AI agent frameworks integrated with web scraping infrastructure solve these issues.
How to create a modern real estate Agent using CrewAI and Bright Data’s MCP server .
Let’s get started!
What is CrewAI?
CrewAI is an open-source framework for orchestrating collaborative AI agents. With CrewAI, you can explicitly specify what an agent can do, their objectives, and the tools they are allowed to use. This enables the execution of complex, multi-step workflows in real estate by grouping agents into teams or crews.
CrewAI is composed of these essential components:
Agent. An LLM-driven worker with a defined role, a specific goal, and an optional backstory. The real estate domain context is relevant to the model.
Task. A single, well-bounded output and scoped job for one agent that has a clearly defined output serving as a quality control benchmark.
Tool. Private functions that an agent can call for domain-specific functionality, such as fetching property data or market analysis, or even using Bright Data’s MCP endpoint for scraping.
Crew. A real estate objective involves a collection of agents working collaboratively, with each performing their applicable assignments.
Process. The execution plan, which can be done in a sequential, parallel, or hierarchical manner, governs the order of tasks, their assignment, delegation, and repetition.
This mirrors a real estate team, property researchers handle data extraction, market analysts provide insights, client managers handle communication, and listing specialists oversee marketing.
To learn more about how CrawAI integrates with tools like Bright Data, check out this guide .
What is MCP?
MCP is an open JSON-RPC 2.0 standard that lets AI agents call external tools and data sources through a single, structured interface. Think of it as a universal connector for real estate data.
Bright Data’s MCP server turns that standard into practice by wiring an agent directly into Bright Data’s scraping stack, making real estate data extraction far more straightforward than traditional approaches:
Anti-bot bypass. Requests flow through Web Unlocker and a pool of 150M+ rotating residential IPs spanning 195 countries
Dynamic-site support. A purpose-built Scraping Browser renders JavaScript, so agents see fully loaded property listings
Structured results. Many tools return clean JSON, eliminating the need for custom parsers.
The server publishes over 50 ready-made tools, ranging from generic URL fetches to real estate-specific scrapers, so your CrewAI agent can retrieve property details, market data, or listing information with a single call.
What We Are Building: Real Estate Agents
We’ll be building a CrewAI real estate agent that will research real estate properties from Zillow page and return the details as a structured JSON output.
You can use it for other properties by changing the link and some parts of the code.
Prerequisites:
Before diving into the code, ensure you have the following setup:
Python 3.11 – Recommended for stability.
Node.js + npm – Required to run the Bright Data MCP server; download from the official site .
Python virtual environment
– Keeps dependencies isolated; see the
venv
docs
.
Bright Data account – Sign up and create an API token (free trial credits are available).
Nebius API key – Create a key in Nebius AI Studio (click + Get API Key ). You can use it for free. No billing profile is required.
Step 1. Environment Setup:
Run the following commands on your terminal to set up the project environment and install dependencies:
mkdir real-estate-ai-system && cd real-estate-ai-system
python -m venv venv
# macOS/Linux: source venv/bin/activate
# Windows: venv\\Scripts\\activate
pip install "crewai-tools[mcp]" crewai mcp python-dotenv pandas
Create a new file called
real_estate_agents.py
and add the following imports:
from crewai import Agent, Task, Crew, Process
from crewai_tools import MCPServerAdapter
from mcp import StdioServerParameters
from crewai.llm import LLM
import os
import json
import pandas as pd
from datetime import datetime
from dotenv import load_dotenv
load_dotenv()
Step 2. Brightdata MCP Server Configuration
Create a
.env
file in your project root with your credentials:
BRIGHT_DATA_API_TOKEN="your_api_token_here"
WEB_UNLOCKER_ZONE="your_web_unlocker_zone"
BROWSER_ZONE="your_browser_zone"
NEBIUS_API_KEY="your_nebius_api_key"
You need:
API token : Generate a new API token from your Bright Data dashboard
Web Unlocker zone : Create a new Web Unlocker zone for real estate sites
Browser API zone : Create a new Browser API zone for JavaScript-heavy property sites
Nebius API key : Already created in Prerequisites
Add this configuration to your
real_estate_agents.py
file:
llm = LLM(
model="nebius/Qwen/Qwen3-235B-A22B",
api_key=os.getenv("NEBIUS_API_KEY")
)
server_params = StdioServerParameters(
command="npx",
args=["@brightdata/mcp"],
env={
"API_TOKEN": os.getenv("BRIGHT_DATA_API_TOKEN"),
"WEB_UNLOCKER_ZONE": os.getenv("WEB_UNLOCKER_ZONE"),
"BROWSER_ZONE": os.getenv("BROWSER_ZONE"),
},
)
This launches
*npx @brightdata/mcp*
as a subprocess and exposes 50+ tools via the MCP standard for real estate data extraction.
Step 3. Agent and Task Definition
Here, we define the agent’s persona and the specific job it needs to do. When implementing CrewAI, prioritize task design, dedicating approximately 80% of your effort to it, and allocate only 20% to defining your agents. Update the
real_estate_agents.py
file to add agent and task definitions:
def build_scraper_agent(mcp_tools):
return Agent(
role="Senior Real Estate Data Extractor",
goal=(
"Return a JSON object with snake_case keys containing: address, price, "
"bedrooms, bathrooms, square_feet, lot_size, year_built, property_type, "
"listing_agent, days_on_market, mls_number, description, image_urls, "
"and neighborhood for the target property listing page. Ensure strict schema validation."
),
backstory=(
"Veteran real estate data engineer with years of experience extracting "
"property information from Zillow, Realtor.com, and Redfin. Skilled in "
"Bright Data MCP, proxy rotation, CAPTCHA avoidance, and strict "
"JSON-schema validation for real estate data."
),
tools=mcp_tools,
llm=llm,
max_iter=3,
verbose=True,
)
def build_scraping_task(agent):
return Task(
description=(
"Extract property data from <https://www.zillow.com/homedetails/123-Main-St-City-State-12345/123456_zpid/> "
"and return it as structured JSON."
),
expected_output="""{
"address": "123 Main Street, City, State 12345",
"price": "$450,000",
"bedrooms": 3,
"bathrooms": 2,
"square_feet": 1850,
"lot_size": "0.25 acres",
"year_built": 1995,
"property_type": "Single Family Home",
"listing_agent": "John Doe, ABC Realty",
"days_on_market": 45,
"mls_number": "MLS123456",
"description": "Beautiful home with updated kitchen...",
"image_urls": ["<https://example.com/image1.jpg>", "<https://example.com/image2.jpg>"],
"neighborhood": "Downtown Historic District"
}""",
agent=agent,
)
Here is what each parameter does:
role – Short job title, CrewAI injects the @role parameter into every system prompt.
goal – North-star goal; CrewAI compares it after each looping step to decide whether to stop.
backstory – Domain knowledge that helps guide the tone of the agent and reduces hallucination.
tools – Inject list of BaseTool objects (e.g., MCP search_engine, scrape_as_markdown).
llm – definition of the model CrewAI will use for each think → plan → act → answer routine.
max_iter – Hard cap on the number of internal loops the agent can do – v0.30 + uses 20 by default.
verbose – Streams every prompt, every thought, every tool call to stdout (for debugging).
description – An action-oriented instruction is injected on every turn.
expected_output – A formal contract for a valid answer (strict JSON, no following comma).
agent – Binds this task to a specific Agent instance for Crew.kickoff().
Step 4. Crew Assembly and Execution
This part assembles the agent and task into a Crew and runs the workflow. Add the Crew assembling and execution script to the
real_estate_agents.py
file:
def scrape_property_data():
"""Assembles and runs the scraping crew."""
with MCPServerAdapter(server_params) as mcp_tools:
scraper_agent = build_scraper_agent(mcp_tools)
scraping_task = build_scraping_task(scraper_agent)
crew = Crew(
agents=[scraper_agent],
tasks=[scraping_task],
process=Process.sequential,
verbose=True
)
return crew.kickoff()
if __name__ == "__main__":
try:
result = scrape_property_data()
print("\\n[SUCCESS] Scraping completed!")
print("Extracted property data:")
print(result)
except Exception as e:
print(f"\\n[ERROR] Scraping failed: {str(e)}")
Step 5. Running the Scraper
Run the command to execute the script from your terminal:
python real_estate_agents.py
You will see the agent’s thought process in the console as each agent plans and executes their tasks.

The agent thought process
The final output will be a clean JSON object:
{
"address": "123 Main Street, City, State 12345",
"price": "$450,000",
"bedrooms": 3,
"bathrooms": 2,
"square_feet": 1850,
"lot_size": "0.25 acres",
"year_built": 1995,
"property_type": "Single Family Home",
"listing_agent": "John Doe, ABC Realty",
"days_on_market": 45,
"mls_number": "MLS123456",
"description": "Beautiful home with updated kitchen...",
"image_urls": ["<https://example.com/image1.jpg>", "<https://example.com/image2.jpg>"],
"neighborhood": "Downtown Historic District"
}
Advanced Implementation Pattern
While our basic example shows the core ideas, real-world applications need more thought:
Market Analysis and Lead Generation
Building market intelligence requires agents who can analyze trends, identify opportunities, and generate qualified leads. These agents collaborate to deliver comprehensive market insights and identify prospects.
Add these market analysis agents to your
real_estate_agents.py
file:
def build_market_analysis_agent(mcp_tools):
return Agent(
role="Real Estate Market Analyst",
goal=(
"Analyze market trends, price movements, and investment opportunities. "
"Provide actionable insights for buyers, sellers, and investors based "
"on comprehensive market data and comparable property analysis."
),
backstory=(
"Senior market analyst with expertise in real estate economics, "
"property valuation, and investment analysis. Specializes in identifying "
"market trends, pricing anomalies, and investment opportunities using "
"statistical analysis and machine learning techniques."
),
tools=mcp_tools,
llm=llm,
max_iter=4,
verbose=True,
)
def build_lead_generation_agent(mcp_tools):
return Agent(
role="Real Estate Lead Generation Specialist",
goal=(
"Identify potential buyers and sellers based on market activity, "
"property searches, and behavioral patterns. Generate qualified "
"leads with contact information and engagement strategies."
),
backstory=(
"Lead generation expert with deep knowledge of real estate marketing, "
"customer behavior analysis, and digital prospecting. Experienced in "
"identifying high-value prospects and developing targeted outreach "
"campaigns for real estate professionals."
),
tools=mcp_tools,
llm=llm,
max_iter=3,
verbose=True,
)
def analyze_market_and_generate_leads(area_zip_code, price_range):
"""Perform market analysis and generate leads for a specific area."""
with MCPServerAdapter(server_params) as mcp_tools:
market_analyst = build_market_analysis_agent(mcp_tools)
lead_generator = build_lead_generation_agent(mcp_tools)
market_task = Task(
description=(
f"Analyze the real estate market for ZIP code {area_zip_code} "
f"within price range {price_range}. Research recent sales, "
"current listings, price trends, and market conditions. "
"Identify opportunities and provide investment recommendations."
),
expected_output="""{
"market_overview": {
"avg_price": "$000,000",
"median_price": "$000,000",
"price_trend": "increasing/decreasing/stable",
"days_on_market_avg": 00,
"inventory_levels": "high/medium/low"
},
"recent_sales": [],
"active_listings": 000,
"price_per_sqft_trend": "$000",
"investment_opportunities": [],
"market_forecast": "market_prediction",
"recommendations": []
}""",
agent=market_analyst,
)
lead_task = Task(
description=(
f"Generate qualified leads for {area_zip_code} area. "
"Identify potential sellers with properties likely to be listed, "
"buyers actively searching in the area, and investors looking "
"for opportunities. Include contact strategies and timing recommendations."
),
expected_output="""{
"potential_sellers": [],
"active_buyers": [],
"investor_prospects": [],
"lead_scoring": {
"high_priority": [],
"medium_priority": [],
"low_priority": []
},
"contact_strategies": [],
"follow_up_timeline": []
}""",
agent=lead_generator,
)
crew = Crew(
agents=[market_analyst, lead_generator],
tasks=[market_task, lead_task],
process=Process.sequential,
verbose=True
)
return crew.kickoff()
Client Interaction and Communication
Effective client management requires specialized agents who can handle communications, schedule appointments, and maintain relationships throughout the real estate process. These agents ensure consistent, professional client service.
Add these client management agents to your
real_estate_agents.py
file:
def build_client_communication_agent(mcp_tools):
return Agent(
role="Real Estate Client Relations Manager",
goal=(
"Manage client communications, schedule appointments, send follow-ups, "
"and maintain client relationships throughout the buying/selling process. "
"Provide personalized service and timely responses to client inquiries."
),
backstory=(
"Experienced client relations specialist with expertise in real estate "
"customer service, appointment scheduling, and relationship management. "
"Skilled in understanding client needs, managing expectations, and "
"maintaining long-term relationships for referrals and repeat business."
),
tools=mcp_tools,
llm=llm,
max_iter=3,
verbose=True,
)
def build_appointment_scheduler_agent(mcp_tools):
return Agent(
role="Real Estate Appointment Coordinator",
goal=(
"Schedule property viewings, client meetings, and follow-up appointments. "
"Coordinate between buyers, sellers, and agents to optimize scheduling "
"and maximize showing efficiency."
),
backstory=(
"Professional appointment coordinator with deep understanding of real "
"estate workflows, client preferences, and scheduling optimization. "
"Expert in managing complex calendars and coordinating multiple stakeholders."
),
tools=mcp_tools,
llm=llm,
max_iter=2,
verbose=True,
)
def handle_client_communication(client_inquiry, client_profile):
"""Process client inquiries and manage communications."""
with MCPServerAdapter(server_params) as mcp_tools:
communication_agent = build_client_communication_agent(mcp_tools)
scheduler_agent = build_appointment_scheduler_agent(mcp_tools)
communication_task = Task(
description=(
f"Process client inquiry: '{client_inquiry}' from client with "
f"profile: {client_profile}. Provide personalized response, "
"address their specific needs, and recommend next steps."
),
expected_output="""{
"response_message": "personalized_client_response",
"client_needs_assessment": {
"budget_range": "$000,000 - $000,000",
"preferred_locations": [],
"property_requirements": [],
"timeline": "timeframe"
},
"recommended_properties": [],
"next_steps": [],
"follow_up_schedule": "timing_recommendations"
}""",
agent=communication_agent,
)
scheduling_task = Task(
description=(
"Based on the client communication, schedule appropriate "
"follow-up appointments, property viewings, or consultation "
"meetings. Optimize scheduling for client convenience and "
"agent efficiency."
),
expected_output="""{
"scheduled_appointments": [],
"property_viewing_schedule": [],
"follow_up_reminders": [],
"calendar_integration": "scheduling_details"
}""",
agent=scheduler_agent,
)
crew = Crew(
agents=[communication_agent, scheduler_agent],
tasks=[communication_task, scheduling_task],
process=Process.sequential,
verbose=True
)
return crew.kickoff()
Property Listing and Marketing Automation
Marketing automation requires specialized agents who can create compelling listings, optimize them for search engines, and distribute them across multiple platforms.
Add this marketing agent to your
real_estate_agents.py
file:
def build_listing_manager_agent(mcp_tools):
return Agent(
role="Property Listing Marketing Manager",
goal=(
"Create compelling property listings, optimize for search engines, "
"and distribute across multiple platforms. Generate marketing materials "
"and track listing performance to maximize exposure and inquiries."
),
backstory=(
"Digital marketing specialist with expertise in real estate marketing, "
"SEO optimization, and multi-platform listing management. Experienced "
"in creating high-converting property descriptions and managing "
"marketing campaigns across MLS, Zillow, Realtor.com, and social media."
),
tools=mcp_tools,
llm=llm,
max_iter=4,
verbose=True,
)
Search and Discovery Features
Intelligent property search and recommendation systems enable clients to find properties that precisely match their specific needs and preferences. These agents provide personalized property discovery and recommendation services.
Add these search and discovery agents to your
real_estate_agents.py
file:
def build_search_agent(mcp_tools):
return Agent(
role="Intelligent Property Search Specialist",
goal=(
"Provide intelligent property search and recommendation services. "
"Understand client preferences, search multiple databases, and "
"deliver personalized property recommendations with detailed analysis."
),
backstory=(
"Search technology expert with deep understanding of real estate "
"databases, property matching algorithms, and client preference analysis. "
"Specializes in advanced search techniques and personalized recommendation "
"systems for optimal property discovery."
),
tools=mcp_tools,
llm=llm,
max_iter=4,
verbose=True,
)
def build_recommendation_agent(mcp_tools):
return Agent(
role="Property Recommendation Engine",
goal=(
"Analyze client behavior, preferences, and market data to generate "
"personalized property recommendations. Learn from client feedback "
"and continuously improve recommendation accuracy."
),
backstory=(
"Machine learning specialist with expertise in recommendation systems, "
"behavioral analysis, and predictive modeling for real estate. "
"Experienced in developing personalized recommendation engines that "
"learn from user interactions and market trends."
),
tools=mcp_tools,
llm=llm,
max_iter=3,
verbose=True,
)
def intelligent_property_search(search_criteria, client_preferences):
"""Perform intelligent property search with personalized recommendations."""
with MCPServerAdapter(server_params) as mcp_tools:
search_agent = build_search_agent(mcp_tools)
recommendation_agent = build_recommendation_agent(mcp_tools)
search_task = Task(
description=(
f"Search for properties matching criteria: {search_criteria}. "
f"Client preferences: {client_preferences}. Use advanced search "
"techniques across multiple platforms and databases. Prioritize "
"results based on client preferences and market conditions."
),
expected_output="""{
"search_results": [],
"total_matches": 0,
"search_filters_applied": [],
"alternative_suggestions": [],
"market_insights": {
"avg_price_in_area": "$000,000",
"market_trends": "trend_analysis",
"inventory_levels": "availability_status"
}
}""",
agent=search_agent,
)
recommendation_task = Task(
description=(
"Analyze search results and client preferences to generate "
"personalized recommendations. Rank properties by relevance, "
"identify hidden gems, and suggest alternative options that "
"might meet client needs."
),
expected_output="""{
"top_recommendations": [],
"personalization_score": "0-100",
"recommendation_reasoning": [],
"alternative_options": [],
"learning_insights": {
"preference_patterns": [],
"behavior_analysis": "client_behavior_summary"
}
}""",
agent=recommendation_agent,
)
crew = Crew(
agents=[search_agent, recommendation_agent],
tasks=[search_task, recommendation_task],
process=Process.sequential,
verbose=True
)
return crew.kickoff()
Deployment and Production Setup
Finally, let’s create a comprehensive orchestration system that coordinates all our specialized agents. This primary system class will serve as the central hub for all real estate operations.
Add this main system orchestration to your
real_estate_agents.py
file:
class RealEstateAgentSystem:
def full_property_analysis(self, property_url, client_profile=None):
with MCPServerAdapter(server_params) as mcp_tools:
research_agent = build_property_research_agent(mcp_tools)
market_analyst = build_market_analysis_agent(mcp_tools)
listing_manager = build_listing_manager_agent(mcp_tools)
research_task = build_property_research_task(research_agent, property_url)
market_task = Task(
description="Analyze market conditions for the researched property",
expected_output="Market analysis with trends and recommendations",
agent=market_analyst,
)
marketing_task = Task(
description="Create marketing strategy based on property and market analysis",
expected_output="Complete marketing campaign plan",
agent=listing_manager,
)
crew = Crew(
agents=[research_agent, market_analyst, listing_manager],
tasks=[research_task, market_task, marketing_task],
process=Process.sequential,
verbose=True
)
return crew.kickoff()
def client_service_workflow(self, client_inquiry, client_profile):
communication_result = handle_client_communication(client_inquiry, client_profile)
if "search" in client_inquiry.lower():
search_criteria = self.extract_search_criteria(client_inquiry)
search_result = intelligent_property_search(search_criteria, client_profile)
return {
"communication": communication_result,
"search_results": search_result
}
return communication_result
def extract_search_criteria(self, inquiry):
criteria = {
"price_range": "extracted_from_inquiry",
"location": "extracted_from_inquiry",
"property_type": "extracted_from_inquiry",
"bedrooms": "extracted_from_inquiry",
"bathrooms": "extracted_from_inquiry"
}
return criteria
def main():
system = RealEstateAgentSystem()
property_url = "<https://www.zillow.com/homedetails/661-Cranbrook-Rd-London-ON-N6K-1W8/2071250954_zpid/>"
try:
print("=== Starting Comprehensive Property Analysis ===")
analysis_result = system.full_property_analysis(property_url)
print("\\n=== Analysis Complete ===")
print("Extracted property data:")
print(json.dumps(analysis_result, indent=2) if isinstance(analysis_result, dict) else str(analysis_result))
client_inquiry = "I'm looking for a 3-bedroom house under $500,000 in downtown area"
client_profile = {
"name": "John Smith",
"budget": "$450,000",
"preferred_locations": ["downtown", "midtown"],
"timeline": "3 months"
}
print("\\n=== Processing Client Inquiry ===")
service_result = system.client_service_workflow(client_inquiry, client_profile)
print("\\n=== Client Service Complete ===")
print("Client service results:")
print(json.dumps(service_result, indent=2) if isinstance(service_result, dict) else str(service_result))
print("\\n=== Analyzing Market and Generating Leads ===")
market_leads = analyze_market_and_generate_leads("90210", "$500,000-$1,000,000")
print("\\n=== Market Analysis Complete ===")
print("Market analysis and leads:")
print(json.dumps(market_leads, indent=2) if isinstance(market_leads, dict) else str(market_leads))
except Exception as e:
print(f"\\n[ERROR] System execution failed: {str(e)}")
if __name__ == "__main__":
main()
Cost Optimization
Bright Data’s MCP is usage-based, so you are billed for every additional request you make. Here are a few tips to help you keep costs in check:
Request only the property fields you need instead of crawling entire listing sites or datasets.
Enable CrewAI’s tool-level cache to skip calls when property data hasn’t changed, saving both time and credits.
Default to the Web Unlocker zone and switch to the Browser API zone only when JavaScript rendering is essential for complex property sites.
Give every agent a sensible maximum iteration bound so they can’t loop forever on problematic listings.
Follow these practices, and your CrewAI agents will stay cost-efficient and reliable, ready for production real estate workloads.
Conclusion
Throughout this tutorial, you’ve learned how to build CrawAI real estate agents using the Bright Data’s MCP server .
We started by understanding what these technologies are and how they are used. Then we went further to build the real estate agent. Setting up the Bright Data’s MCP server, configuring the LLM, creating agents, defining tasks, and assembling the task Crew.
You can easily adapt these agents to other real estate targets. For example, to scrape from Realtor.com instead of Zillow, all you need to do is tweak your agent’s role, goal, and backstory, along with the task’s description and what you expect as output.