[FeatureRequest] Option to reset already fitted model to its initial state

[rychuelektryk]

Hi,

I would like to have some kind of a clear method on a already learned model to restore it to the unlearned state. I guess it boils down to resetting model weights. Think you could add such an option in future?

My case:
In each iteration of my kfold procedure I would like to start fitting model from the beginning. I could copy the model in each iteration without copying weights but that leads to loosing already set callback and I would be forced to recompile the model for which I would need information about optimizer, metrics and loss which are not accessible from original model.

[zaleslaw]

Yeah, I see your need. If it's not private, please share your K-Fold

I think it could be solved as a resetting method or option, but could I ask you: do you need to reset the model only because of the problem with callbacks in #270 ?

[zaleslaw]

@rychuelektryk could you please comment on this thread?

[rychuelektryk]

Hi @zaleslaw, and sorry for late response

This issue is related to the callback issue I've described in other thread. Being unable to reset model weights forces me to make additional model compilation in my kfold method which also lets me set callback.

Below code snippets are simplified for the sake of readability

This is how my current kfold looks like

class KFoldEvaluator : IKFoldEvaluator {
    override fun evaluate(model: Functional, fitDataset: FitDataset, foldCount: Int, trainBatchSize: Int, validationBatchSize: Int, optimizer: Optimizer, loss: Losses, metric: Metrics): KFoldResult {

        fitDataset.toKFoldDatasets(foldCount).forEach { (trainFitDataset, validationFitDataset) ->
            val dumbModel = model.copy(copyWeights = false)

            dumbModel.compile(optimizer, loss, metric)

            dumbModel.fit(trainFitDataset.toOnHeapDataset(), validationFitDataset.toOnHeapDataset(), epochs = Int.MAX_VALUE, trainBatchSize, validationBatchSize)
        }
    }
}

And here is how I would like it to look like

class KFoldEvaluator : IKFoldEvaluator {
    override fun evaluate(model: Functional, fitDataset: FitDataset, foldCount: Int, trainBatchSize: Int, validationBatchSize: Int): KFoldResult {

		val dumbModel = model.copy(copyWeights = false)
		
        fitDataset.toKFoldDatasets(foldCount).forEach { (trainFitDataset, validationFitDataset) ->
            dumbModel.resetWeights()

            dumbModel.fit(trainFitDataset.toOnHeapDataset(), validationFitDataset.toOnHeapDataset(), epochs = Int.MAX_VALUE, trainBatchSize, validationBatchSize)
        }
    }
}

What do you think?

[zaleslaw]

Regarding reset weights, do you mean that KotlinDL should keep somewhere the initial weights (loaded from h5 model or generated by initializers) or reset the weights leads only to new random initialization without relation to previously initialized weights?

I guess that in the context of usage KFold you don't need a stable copy of initial weights and resetting means just re-run of all initializers to generate new weights. Correct me if I'm wrong