Monitor Crypto News for Trading Signals
Build an automated crypto news monitoring system that tracks breaking stories from major cryptocurrency news outlets. This use case demonstrates how to capture market-moving events in real-time and react faster than manual monitoring allows.
Prerequisites
- Supacrawler API key
- Basic understanding of CSS selectors
- Email address or webhook endpoint for notifications
Quick example
Monitor CoinDesk for new articles and get notified when headlines change:
curl -X POST https://api.supacrawler.com/api/v1/watch \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.coindesk.com/",
"frequency": "hourly",
"selector": "a.card-title",
"notify_email": "[email protected]",
"notification_preference": "changes_only"
}'
Configuration options
Frequency settings
Choose monitoring frequency based on market volatility and news importance:
- Every 5 minutes: Critical breaking news (custom cron:
*/5 * * * *
) - Hourly: Standard crypto news monitoring
- Every 30 minutes: High-activity periods (custom cron:
*/30 * * * *
)
CSS selectors for major crypto sites
CSS selectors can change when sites update their layouts. Always inspect the target page to verify current selectors before deploying.
- Name
CoinDesk
- Type
- a.card-title
- Description
Main headline links on the homepage
- Name
CoinTelegraph
- Type
- .post-card__title a
- Description
Article title links in the post cards
- Name
Decrypt
- Type
- h3 > a
- Description
Headline links within H3 elements
- Name
CryptoSlate
- Type
- .article-title a
- Description
Article title links in news listings
Multiple source monitoring
Monitor multiple crypto news sources to ensure comprehensive coverage:
import requests
import os
API_KEY = os.environ['SUPACRAWLER_API_KEY']
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json"
}
# Define crypto news sources
sources = [
{
"name": "CoinDesk",
"url": "https://www.coindesk.com/",
"selector": "a.card-title"
},
{
"name": "CoinTelegraph",
"url": "https://cointelegraph.com/",
"selector": ".post-card__title a"
},
{
"name": "Decrypt",
"url": "https://decrypt.co/",
"selector": "h3 > a"
}
]
# Create watch jobs for each source
for source in sources:
response = requests.post(
"https://api.supacrawler.com/api/v1/watch",
headers=headers,
json={
"url": source["url"],
"frequency": "hourly",
"selector": source["selector"],
"notify_email": "[email protected]",
"notification_preference": "changes_only"
}
)
result = response.json()
if result.get("success"):
print(f"{source['name']} monitor created: {result['watch_id']}")
else:
print(f"Failed to create {source['name']} monitor: {result.get('error')}")
Building a trading intelligence system
Architecture overview
Detection Layer
- Watch API monitors multiple news sources
- CSS selectors capture headline changes
- Configurable frequency based on market conditions
Processing Layer
- Webhook receives real-time notifications
- Keyword filtering for relevant terms
- Sentiment analysis for market impact
Action Layer
- Alert routing (email, SMS, Discord, Slack)
- Integration with trading platforms
- Portfolio management triggers
Storage Layer
- Historical news tracking
- Pattern analysis for future optimization
- Performance metrics and backtesting
Keyword filtering implementation
Process Incoming Alerts
import re
from typing import List, Dict
# Define market-moving keywords
BULLISH_KEYWORDS = [
'adoption', 'partnership', 'etf approval', 'institutional',
'bullish', 'surge', 'rally', 'breakout'
]
BEARISH_KEYWORDS = [
'regulation', 'ban', 'hack', 'bearish', 'crash',
'sell-off', 'sec action', 'investigation'
]
COIN_TICKERS = ['BTC', 'ETH', 'SOL', 'ADA', 'MATIC']
def analyze_headline(headline: str) -> Dict:
"""Analyze headline for trading relevance and sentiment."""
headline_lower = headline.lower()
# Check for coin mentions
mentioned_coins = [ticker for ticker in COIN_TICKERS
if ticker.lower() in headline_lower]
# Sentiment analysis
bullish_score = sum(1 for keyword in BULLISH_KEYWORDS
if keyword in headline_lower)
bearish_score = sum(1 for keyword in BEARISH_KEYWORDS
if keyword in headline_lower)
return {
'headline': headline,
'coins_mentioned': mentioned_coins,
'sentiment': 'bullish' if bullish_score > bearish_score
else 'bearish' if bearish_score > 0 else 'neutral',
'relevance_score': bullish_score + bearish_score + len(mentioned_coins),
'priority': 'high' if (bullish_score + bearish_score) >= 2 else 'normal'
}
# Example webhook handler
def process_watch_notification(payload):
"""Process incoming watch notification."""
changes = payload.get('changes', [])
for change in changes:
if change['type'] == 'added':
analysis = analyze_headline(change['content'])
if analysis['relevance_score'] >= 2:
send_priority_alert(analysis)
Best practices
Selector reliability
- Use stable CSS classes that are unlikely to change frequently
- Avoid selectors that depend on dynamic content or IDs
- Test selectors across different page loads to ensure consistency
- Have fallback selectors ready for when sites update their layouts
Rate limiting and costs
- Start with hourly monitoring and adjust based on market conditions
- Use custom cron expressions for high-frequency monitoring during volatile periods
- Monitor your API usage to optimize costs
- Consider pausing monitors during low-activity periods (weekends, holidays)
Alert fatigue prevention
- Implement keyword filtering to reduce noise
- Use different notification channels for different priority levels
- Set up digest emails for lower-priority news
- Create rules to suppress duplicate or similar headlines
Next steps
- Integrate with trading platforms: Connect alerts to your trading bots or portfolio management tools
- Add sentiment analysis: Use AI models to automatically assess market sentiment
- Historical analysis: Store alerts to identify patterns and optimize your strategy
- Multi-asset monitoring: Extend monitoring to DeFi protocols, NFT markets, and traditional finance news
Related resources
- Watch API Reference - Complete API documentation
- Webhook Integration - Set up real-time notifications
- CSS Selector Guide - Advanced selector techniques