Add Dot layer

[mkaze]

Currently, the support for the Dot layer is missing and it would be great if we could add support for this layer as well. The desired PR addressing this issue should include:

• Implementation of layer class named as Dot inherited from the AbstractMerge class
• Documentation of layer.
• Test cases
• Support for export of layer to JSON (see ModelSaver.kt)
• Support for import of layer from JSON (see ModelLoader.kt)

Also, if needed, you can take a look at Keras documentation for this layer.

[zaleslaw]

Good issue, we could put it on the roadmap

[jyue86]

My friend, @RandyHuynh5815, and I would like to take a shot at this task.

[zaleslaw]

Hi! @justinyue1643, I assigned it to you, welcome to the KotlinDL!

[zaleslaw]

Hi, @justinyue1643. Are you going to continue to work on this issue? It will be moved to the 0.4 milestone.

[zaleslaw]

Hi @justinyue1643 I'm going to unassign this ticket from you during the next few days, write here if you want to keep it

[therealansh]

Hey @zaleslaw , I could try this one! Does the Ops right now have batch_dot? Also i could not find l2_normalize for normalizing.

[zaleslaw]

NOTE: Both these operations are implemented in Python using basic TF Ops

l2_normalize is missed because it's implemented in Python fully, we need to implement it in a KotlinDL like here

For example, I've created an implementation of l2_normalize below

fun  l2Normalize(scope: Scope?, x: Operand<Float>, axis: IntArray?): Operand<Float>? {
    val squareSum: Operand<Float> = ReduceSum.create(
        scope,
        Square.create(scope, x),
        Constant.create(scope, axis),
        ReduceSum.keepDims(Boolean.TRUE)
    )
    val invNorm: Operand<Float> = Rsqrt.create(
        scope,
        org.tensorflow.op.math.Maximum.create(
            scope, squareSum, Constant.create(scope, 1e-12f)
        )
    )
    return Mul.create(scope, x, invNorm)
}

Hope the same could be implemented for the batch_dot op, because it's a part of Keras, implemented in Python over the basic TF ops (looks like most of them are available)