tejaswini/Data-Analytics
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Day 1 — Setup and Data Ingestion_updated

Browse Source 
Day 1 — Setup and Data Ingestion_updated
This commit is contained in:
tejaswini
2025-10-09 06:46:19 +00:00
parent 16afdbb00b
commit 90435d8ab7
1 changed files with 37 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
Day_001_Setup_and_Data_Ingestion.md
View File

@@ -1,57 +1,56 @@
# 📅 Day 1 — Setup and Data Ingestion
## 🎯 Goal
Establish the foundation for the project.
The primary objective is to create an **n8n workflow** that successfully fetches raw **sales** and **product review data** from their respective sources.
Establish the foundation of the project.
Create an **n8n workflow** that successfully fetches raw sales and product review data from their respective sources.
----
---
## 🧩 Tasks
### 1. Environment Setup
- Set up your **n8n instance**:
- Options: n8n Cloud, Docker, or local installation.
- Reference: [https://docs.n8n.io/hosting/](https://docs.n8n.io/hosting/)
- Create a **Python environment**.
- Install necessary libraries:
```bash
pip install pandas nltk
```
- Get your **n8n instance** running (n8n Cloud, Docker, or local install).
- Set up a **Python environment**:
```bash
pip install pandas nltk
2. Create a New n8n Workflow
Start with a Manual Trigger node (Start).
---
This will be replaced with an automated schedule on Day 5.
### 2. Create a New n8n Workflow
- Start with a **Manual Trigger** node (`Start`).
- This will later be replaced with an automated schedule on **Day 5**.
3. Fetch Sales Data
Add an HTTP Request node (or Database node like PostgreSQL).
---
Connect to your e-commerce platform’s API (e.g., Shopify /orders.json).
### 3. Fetch Sales Data
- Add an **HTTP Request** or **Database Node (PostgreSQL/MySQL)**.
- Connect to your e-commerce API endpoint (e.g., Shopify → `/orders.json`).
- Set up credentials (API Key, OAuth2, etc.).
- Test and verify that recent order data (e.g., order_id, product_id, price, quantity) is fetched successfully.
Set up authentication (API Key, OAuth2, etc.).
---
Test the node to ensure it pulls recent orders.
### 4. Fetch Product Reviews
- Add another **HTTP Request** node.
- Configure it to pull recent **product reviews** (e.g., review_text, rating).
- Test independently to ensure successful data retrieval.
4. Fetch Product Reviews
Add another HTTP Request node.
---
Connect to your website API or third-party service to fetch reviews.
### 5. Combine Data Streams
- Add a **Merge Node**.
- Connect both sales and review nodes.
- Set **Mode** → `Combine`.
- This ensures both data sets are merged and available for the next phase.
Test independently to ensure data retrieval.
---
5. Combine Data Streams
Add a Merge node.
## ✅ Deliverable
A manually triggered **n8n workflow** that:
- Pulls raw data from **two sources** (sales + reviews).
- Combines them using a **Merge node**.
- Outputs unified JSON data ready for analysis.
Connect Sales Data and Review Data nodes.
Set Mode → Combine.
This ensures both data sets are available for the next step.
✅ Deliverable
A manually triggered n8n workflow that:
Pulls raw data from two sources (sales + reviews).
Merges the data into a single workflow run.
💡 Solution
Workflow successfully merges sales and review data.
Outputs a combined JSON object, ready for Day 2 Python processing.