def mutate(self): mut_prob = 0.3 mut_strength = 10 for param in self.parameters(): param.data += mut_strength * torch.randn_like(param) * (torch.rand(size=param.data.size()) < mut_prob).int()
The loop bypasses all tensors, for each tensor it performs the following actions:
Increments the tensor element by the product of three variables:
- mutation force
- tensor of the same dimension filled with random values in the interval [0..1]
- tensor of the same dimension filled randomly with zeros and ones, the ones are shed there with controllable probability because we want to control the mutation probability of the weights.
Here is the non-trivial magic in this last value.
(torch.rand(size=param.data.size()) < mut_prob).int()
What happens here: First a tensor filled with random variables is created.
Then each value is checked for the condition that it is less than the mutation probability.
The tenson is made of True/False Boolean values, and the closer the mutation probability is to zero, the less True it will be.
Then this tensor, as I understand it, is converted to a tensor containing integer values using TORCH.TENSOR.INT(). That is:
tensor([[False, False], [False, True], [False, True]])
tensor([[0, 0], [0, 1], [0, 1]], dtype=torch.int32)
In effect, this multiplication results in a zerosmasking of the tensor with random variables.
There is a strong suspicion that this should be done somehow easier, by some in-built method, but I haven't found it. Or maybe mutating the scales in this barbaric way is fundamentally wrong, I dunno.
Anyway, maybe someone will find it useful.