Loading Project Media
70K+
Cardiovascular Records
0.83
Leakage-free ROC-AUC
2
Clustering Strategies
(Overview)
PulseisacardiovascularMLinvestigationwhereIexploredwhetherclusteringcouldimprovediseaseprediction,anddiscoveredthatamajorperformancegainwasactuallycausedbytarget leakage.Ithenvalidatedthefindingthroughcross-validationandexperimentedwithleakage-freeclusteringapproaches.
(The Challenge)
ThemainchallengewasunderstandingwhyaddingclusteringinformationcausedtheXGBoostmodelperformancetoincreasedramatically.Hyperparametertuningproducedonlyasmallimprovement,whileintroducingthetargetvariableintoclusteringresultedinamuchlargerjump.Thisraisedthequestionofwhethertheimprovementrepresentedgenuinepredictiveinformationorhiddentargetleakage.
(The Solution)
Icomparedmultiplebaselinemodels,selectedXGBoost,experimentedwithK-ModesandK-Prototypesclustering,andtracedtheunexpectedperformanceincreasebacktotargetleakage.Ithenseparatedtarget-inclusiveandtarget-excludedclustering,evaluatedbothusingcross-validation,andconfirmedthattheapparentimprovementwaslargelycausedbyleakage.Finally,Iexperimentedwithleakage-freeclusteringtoinvestigatewhetherclusteringcouldprovidegenuinepredictivevaluewithoutusingthetarget.
- 01
Performedexploratorydataanalysis,datacleaning,preprocessing,andfeatureanalysisonalargecardiovasculardataset.
- 02
ComparedmultiplebaselinemachinelearningmodelsandselectedXGBoostbasedonitsoverallclassificationperformance.
- 03
ExperimentedwithhyperparametertuningandclusteringusingK-Modestoinvestigatewhetherunsupervisedgroupingcouldimprovecardiovascularprediction.
- 04
InvestigatedanunexpectedperformanceincreaseafterincorporatingclusterinformationintotheXGBoostmodel,leadingtothediscoveryoftargetleakage.
- 05
Validatedtheleakagethroughcross-validationandcontrolledexperimentscomparingtarget-inclusiveandtarget-excludedclusteringapproaches.
- 06
ExperimentedwithK-Prototypesclusteringtodevelopaleakage-freeclusteringapproachandevaluatewhethertheperformanceimprovementcouldbeachievedwithoutusingthetargetvariable.
Built with
PythonPandasNumPyScikit-learnXGBoostK-ModesK-PrototypesJupyter Notebook
Haveaprojectinmind?
dattaiman56@gmail.com