OHDSI Home | Forums | Wiki | Github

Why is MLP always the worst performer in ATLAS PLP?

In all the patient-level-prediction cases I went through, MLP is always the worst performer among all the 8 models. The AUC is around 0.5 even for training. Is there something wrong with the MLP implementation?

It uses python’s sklearn.neural_network and I’m not sure why it does poorly most of the time. I suspect it may be the hyper-parameters that are used by default:

activation=‘logistic’
alpha is c(0.3,0.01,0.001,0.000001)
learning_rate_init = 0.001
hidden_layer_sizes=(4, 2)
max_iter= 2000
early_stopping=True
n_iter_no_change = 10
solver = ‘Adam’
tol= 0.0001
beta_1 = 0.9
beta_2 = 0.999
epsilon = c(1,0.1,0.00000001)

It would be useful to test out alternative hyper-parameters and if we find better ones we can update the defaults.

t