Add–1d–layers #57
Add–1d–layers #57
Conversation
|
The following PR must be merged before this one will compile: |
Sources/DeepLearning/Layer.swift
Outdated
| /// - Parameters: | ||
| /// - filter: The filter (width, inputChannels, outputChannels). | ||
| /// - bias: The bias (dimensions: output channels). | ||
| /// - activation: The activation activation. |
There was a problem hiding this comment.
| /// - activation: The activation activation. | |
| /// - activation: The element-wise activation function. |
Please also make this change elsewhere, including for Conv2D.
Sources/DeepLearning/Layer.swift
Outdated
| /// padding. | ||
| /// | ||
| /// - Parameters: | ||
| /// - filter: The filter (width, inputChannels, outputChannels). |
There was a problem hiding this comment.
For the - Parameters: doc comment of memberwise initializers, please copy the doc comments from each individual stored property.
For example:
/// - Parameters:
/// - filter: The 3-D convolution kernel.
/// - bias: The bias vector.
/// ...Please also make this change elsewhere, including for Conv2D.
Sources/DeepLearning/Layer.swift
Outdated
| /// initialization with the specified seed. The bias vector is initialized with zeros. | ||
| /// | ||
| /// - Parameters: | ||
| /// - filterShape: The shape of the filter (width, inputChannels, outputChannels). |
There was a problem hiding this comment.
Please use `[dim0, dim1, ...]` syntax for specifying all shapes in doc comments.
| /// - filterShape: The shape of the filter (width, inputChannels, outputChannels). | |
| /// - filterShape: The 3-D shape of the filter, representing | |
| /// `[width, inputChannels, outputChannels]`. |
Co-Authored-By: dave-fernandes <[email protected]>
| public init( | ||
| filter: Tensor<Scalar>, | ||
| bias: Tensor<Scalar>, | ||
| activation: @escaping Activation, |
There was a problem hiding this comment.
We are using identity as the default activation in Dense and Conv2D.
| activation: @escaping Activation, | |
| activation: @escaping Activation = identity, |
There was a problem hiding this comment.
The init method in the struct declaration doesn't have a default. Only the ones in the extensions have default values. So this is consistent, but maybe not what is desired?
Sources/DeepLearning/Layer.swift
Outdated
| @differentiable | ||
| public func applied(to input: Tensor<Scalar>, in _: Context) -> Tensor<Scalar> { | ||
| return input.expandingShape(at: 1).maxPooled( | ||
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding |
There was a problem hiding this comment.
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding | |
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding |
Sources/DeepLearning/Layer.swift
Outdated
| @differentiable | ||
| public func applied(to input: Tensor<Scalar>, in _: Context) -> Tensor<Scalar> { | ||
| return input.expandingShape(at: 1).averagePooled( | ||
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding |
There was a problem hiding this comment.
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding | |
| kernelSize: (1, 1, poolSize, 1), strides: (1, 1, stride, 1), padding: padding |
| func testConv1D() { | ||
| let filter = Tensor<Float>(ones: [3, 1, 2]) * Tensor<Float>([[[0.33333333, 1]]]) | ||
| let bias = Tensor<Float>([0, 1]) | ||
| let layer = Conv1D<Float>(filter: filter, bias: bias, activation: identity, stride: 1, padding: .valid) |
Sources/DeepLearning/Layer.swift
Outdated
| let filterTensorShape = TensorShape([ | ||
| Int32(filterShape.0), Int32(filterShape.1), Int32(filterShape.2)]) | ||
| self.init( | ||
| filter: Tensor(glorotUniform: filterTensorShape, seed: seed), |
|
Regarding the APIs, I think MaxPool2D and AvgPool2D should use 2-tuples instead of 4-tuples for strides and poolSize. The 4-tuples are inconsistent with Conv2D. |
Co-Authored-By: dave-fernandes <[email protected]>
3 participants
Added Conv1D, MaxPool1D and AvgPool1D layers