Graduate coursework · M.S. Big Data Analytics

Pricing, prediction, and pattern mining in Python.

Two end-to-end data mining projects from BGDA 510: predicting used-car prices from 15,232 marketplace listings, and modeling Titanic passenger survival. Each one follows the same workflow: clean and validate the data, compare models against a baseline, tune on training data only, and test once.

Used-car pricing Titanic survival

Price by brandLuxury brands sit far above the rest, one reason a log-price model works better.

Project 1 · Regression and classification

Used-car price prediction

Can a dealer estimate a fair price for a car from its age, mileage, engine, power, brand, and fuel type?

View code on GitHub (opens in a new tab)

15,232listings after cleaning

0.857test R² (variance explained)

$1,740typical error (test MAE)

0.89F1 for premium-car classifier

Data preparation

Removed 167 duplicate listings and 12 impossible mileage values, treated zero-seat cars as missing, and built leak-free scikit-learn pipelines for imputation, scaling, and encoding.

Modeling

Compared a mean baseline, linear, log-price, Ridge, Lasso, and decision-tree models with 5-fold cross-validation. Modeling log price cut RMSE from $6,255 to $4,214 and removed 591 negative price predictions.

Result

The tuned log-price Ridge model scored R² 0.857 on the untouched test set, close to its cross-validation score, so it generalizes well. A follow-up classifier flagged premium cars (over $10,000) with a cross-validated F1 of 0.89 (tuned gradient boosting).

Residual plot for the log-price model, with errors spread evenly around zero — **Fixing the model's shape**After switching to log price, errors are centered on zero and no prices are negative.

Line chart of training and cross-validation error by tree depth, showing overfitting past depth 15 — **Choosing model complexity**Training error keeps falling, but validation error rises past depth 15: a clear overfitting signal.

Also in this project: a market basket analysis of 2,000 simulated accessory purchases (support, confidence, and lift), where roof racks and bike racks showed a lift of 6.5.

Project 2 · Classification and association rules

Titanic survival: end-to-end data mining

Which passenger traits are linked to survival, and how well can a model predict it?

View code on GitHub (opens in a new tab)

891passenger records

83.9%test accuracy (random forest)

0.83recall at a 0.3 threshold

2.54lift of top association rule

Data quality

Built a data quality report (77% of cabin and 20% of age values missing), filled age by passenger title, and engineered Title, FamilySize, and IsAlone features.

Finding the real driver

Cherbourg passengers survived more often, but a cross-tab showed 51% of them were first class. Passenger class, not port, explains most of the gap.

Modeling

Compared five classifiers with 5-fold cross-validation, tuned a random forest with GridSearchCV (test F1 0.77), and mined association rules such as female and first class leading to survival (97% confidence).

Box plot of 5-fold cross-validation F1 scores for five classifiers, with random forest highest — **Comparing five models**Random forest had the highest and most consistent F1 across folds.

Line chart of precision rising and recall falling as the decision threshold increases from 0.3 to 0.7 — **Matching the threshold to the goal**For a rescue scenario, a 0.3 threshold misses the fewest survivors (recall 0.83) at the cost of more false alarms.

Bar charts of survival rate by age group and by boarding port, with children and Cherbourg highest — **Survival patterns**Children survived most often, and port differences trace back to passenger class.

What these projects show

Careful process, not just high scores.

Both projects use a held-out test set touched only once, compare every model to a simple baseline, and explain results in plain language. The full notebooks, with code and outputs, are on GitHub.

All projects Contact me