2018 - IDSS PW3

Group 3

Reference: O.W. Samuel, G.M. Asogbon, A.K. Sangaiah, P. Fang, G. Li (2017) - An integrated decision support system based on ANN and Fuzzy_AHP for heart failure risK prediction. Expert Syst. Appl. 68, 163-172.

Intro

• requirement.txt : list which packages used in this project.

• setup.py : for packing up all.

• main.py : integrate all function and add user interface.

• test.py : integrate all function for testing each function.

• preprocessing.py : load data and deal with missing data.

• faphy.py : Fuzzy_AHP using pairwise_matrix to get the attribute's weights.

• ann.py : train ANN to trained ANN for prediction.

• eval.py : evaluate the model, using sensitivity/specificity, evaluation metrics, ROC and performance plot.

data/ :

• processed_data.csv : original dataset from UCI data repository.
• weights : attribute's weights computed from Fuzzy_AHP.

results/ : store all images using in README.md

Infrastructure

Result

(BEST ONE: without missing data treatment, min_max scale X, with attribute weights)

• Programming Output
  
  
• Plot Curve
  

Evaluation

(left: hybrid method, right: conventional ANN method)

• evaluation metrics
  

• ROC
  

• performance plot
  
  

  	hybrid method	conventional ANN method
  Train Accuracy	84.13%	85.32%
  Test Accuracy	95.56%	91.11%
  Sensitivity	100.00%	91.30%
  Specificity	90.91%	90.91%
  FP rate	9.09%	9.09%
  FN rate	0.00%	8.70%
  Recall	100.00%	91.30%
  Precision	92.00%	91.30%
  F1	95.83%	91.30%

Comparison

seed = 1, batch_size = 50, iteration=2000,

X(attribute) scale	missing data	attribute weight	fix attribute w	ANN	test acc(train acc)
min_max	replace_mean	√	√	13-10-2	80.43(86.38)
normalise	replace_mean	√	√	13-10-2	82.61(86.38)
min_max	replace_med	√	√	13-10-2	78.26(85.21)
normalise	replace_med	√	√	13-10-2	80.43(85.6)
MICE	replace_med	√	√	13-10-2	82.61(86.38)
MICE	replace_med	√	√	13-10-2	82.61(86.38)
min_max	knn_1	√	√	13-10-2	78.26(85.6)
normalise	knn_1	√	√	13-10-2	80.43(86.38)
min_max	knn_3	√	√	13-10-2	78.26(85.99)
normalise	knn_3	√	√	13-10-2	78.26(85.99)
min_max	x	√	√	13-10-2	95.56(83.73)
normalise	x	√	√	13-10-2	93.33(84.13)
min_max	x	x	√	13-10-2	91.11(85.32)
min_max	x	√	√	13-10-1	93.33(83.73)

TO-DO List

• No-fixed attribute weight
• missing data treatment
• Evaluation
• Integrate 'missing data treatment'
• User Interface
• Pack project to .exe
• Word
• Slides