155 lines
6.3 KiB
Markdown
155 lines
6.3 KiB
Markdown
# News & Weather API Integration Project
|
|
|
|
## Demo
|
|
[View Demo Video](https://drive.google.com/file/d/14sWoiC6sy2LxmZAAxY66vKKbOVURba_i/view?usp=sharing)
|
|
|
|
## Overview
|
|
This project integrates news scraping functionality with weather API services to provide a comprehensive news and weather dashboard application. It consists of both frontend and backend components, with automated processing capabilities driven by LLM agents.
|
|
|
|
## Project Structure
|
|
```
|
|
News-App/
|
|
├── Fields and responses.txt # Field definitions and API responses mapping
|
|
├── Interactive Weather API Explorer - WeatherAPI.com.html # Weather API documentation
|
|
├── Interactive Weather API Explorer - WeatherAPI.com1.html
|
|
├── logs-2025-11-19T10-29-00.txt # System logs
|
|
├── logs-2025-11-19T10-36-08.txt
|
|
├── ppt.txt # PowerPoint presentation notes
|
|
├── premise.txt # Project premise documentation
|
|
├── script.md # Automation scripts
|
|
├── tone.txt # Communication tone guidelines
|
|
├── weather_service.log # Weather service logs
|
|
├── .venv/ # Python virtual environment
|
|
├── news-frontend/ # Frontend application (React)
|
|
├── news-weather-backend/ # Backend application (Python Flask/FastAPI)
|
|
└── new_venv/ # Alternative Python virtual environment
|
|
```
|
|
|
|
## Technologies Used
|
|
|
|
### Backend (news-weather-backend)
|
|
- **Python**: Programming language
|
|
- **Flask/FastAPI**: Web framework (based on the project structure)
|
|
- **gnews**: Google News API wrapper for news scraping
|
|
- **Requests**: HTTP library for API calls
|
|
- **BeautifulSoup**: HTML parsing for web scraping
|
|
- **Pandas**: Data manipulation
|
|
- **Datetime**: Date/time handling
|
|
- **JSON**: Data serialization
|
|
- **Google News URL Decoder**: Specialized library for decoding Google News links
|
|
|
|
### Frontend (news-frontend)
|
|
- **HTML/CSS/JavaScript**: Base web technologies
|
|
- **Bootstrap**: Responsive design framework
|
|
- **AJAX**: Asynchronous data loading
|
|
- **Chart.js**: Data visualization (implied from structure)
|
|
|
|
### APIs & Services
|
|
- **News API**: News article retrieval
|
|
- **WeatherAPI**: Current and forecast weather data
|
|
- **RSS Feeds**: Alternative news sources
|
|
- **Google News**: News aggregation
|
|
- **OpenWeatherMap**: Backup weather data provider (likely)
|
|
|
|
## How to Run
|
|
|
|
### Backend Setup
|
|
1. Navigate to the `news-weather-backend` directory
|
|
2. Activate the Python virtual environment:
|
|
```bash
|
|
# Windows
|
|
.\.venv\Scripts\activate
|
|
|
|
# Linux/Mac
|
|
source .venv/bin/activate
|
|
```
|
|
3. Install required dependencies:
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
4. Set up environment variables:
|
|
```bash
|
|
# Create .env file with necessary API keys
|
|
echo "NEWS_API_KEY=your_api_key_here" >> .env
|
|
echo "WEATHER_API_KEY=YOUR_WEATHER_API_KEY_HERE" >> .env
|
|
```
|
|
5. Run the backend server:
|
|
```bash
|
|
python app.py
|
|
```
|
|
|
|
### Frontend Setup
|
|
1. Navigate to the `news-frontend` directory
|
|
2. Open the application:
|
|
```bash
|
|
# If using a local server (e.g., Live Server extension in VS Code)
|
|
# or simply double-click index.html to open in browser
|
|
```
|
|
3. Configure API endpoints in the frontend (usually in config.js or similar):
|
|
```javascript
|
|
const BACKEND_URL = 'http://localhost:5000'; // Default Flask port
|
|
```
|
|
|
|
## Features
|
|
- Real-time news and weather updates
|
|
- Automated news scraping from multiple sources
|
|
- Weather data integration from WeatherAPI.com
|
|
- Responsive dashboard interface
|
|
- RSS feed processing
|
|
- Scheduled data updates
|
|
- LLM-powered content summarization (future feature)
|
|
- Multi-source news aggregation
|
|
|
|
## Configuration Files
|
|
- `Fields and responses.txt`: Maps API response fields to application requirements
|
|
- `script.md`: Contains automation and deployment scripts
|
|
- `premise.txt`: Project premise and objectives documentation
|
|
- `.env`: Environment variables (not included in repo for security)
|
|
- Deployment configurations for both frontend and backend
|
|
|
|
## Future Enhancements
|
|
Instead of users manually running the services, the system will be enhanced to automatically trigger processing through LLM agents:
|
|
- Integration with CRM systems for automated insights
|
|
- Enhanced decision-making capabilities based on weather patterns and news trends
|
|
- Automated alert system for significant weather events
|
|
- Predictive analytics based on news sentiment and weather forecasts
|
|
- Smart recommendations engine
|
|
- Automated report generation
|
|
- Integration with scheduling systems for automated updates
|
|
|
|
The system will detect weather anomalies and breaking news events, automatically triggering notifications and reports without manual intervention. Through the CRM integration, sales teams can get timely alerts about weather impacts on customer operations or news events affecting industries they serve.
|
|
|
|
## Usage Scenarios
|
|
1. **News Monitoring**: Track relevant news stories with weather contexts
|
|
2. **Weather Impact Analysis**: Understand how weather affects news events
|
|
3. **Automated Reporting**: Generate combined news and weather reports
|
|
4. **CRM Integration**: Link weather events to customer impact assessments
|
|
5. **Decision Support**: Assist with decisions based on combined news/weather data
|
|
|
|
## Development Notes
|
|
- The backend handles all API calls and data processing
|
|
- The frontend provides a user-friendly interface
|
|
- News scraping is handled by gnews library
|
|
- Weather data comes from WeatherAPI.com
|
|
- All data is stored temporarily in-memory or in simple files (production would require persistent storage)
|
|
- The system is designed to work with both APIs simultaneously
|
|
|
|
## Troubleshooting
|
|
Common issues:
|
|
- API key limits exceeded (check logs in weather_service.log)
|
|
- Network connectivity issues with external APIs
|
|
- RSS feeds may occasionally return malformed XML
|
|
- Rate limiting from news providers
|
|
|
|
If experiencing issues:
|
|
1. Check logs in the backend
|
|
2. Verify API keys are valid and not rate-limited
|
|
3. Ensure internet connectivity
|
|
4. Confirm backend is running and accessible
|
|
|
|
## Security Considerations
|
|
- Never commit API keys directly to the repository
|
|
- Use environment variables for sensitive credentials
|
|
- Validate all inputs from external APIs
|
|
- Implement rate limiting to prevent abuse
|
|
- Sanitize all data before displaying in the frontend |