module owlite.nn.functions.clq
class CLQFunction
An implementation of QAT function using CLQ (Constrained Learned Quantization).
In CLQ(Constrained Learned Quantization) method, instead of using a fixed set of quantization levels, this method adapts the scales during training to minimize the impact on model performance. Learnable step_size allows the model to be better adapted to the distribution of fed data. ### Gradient of step_size.
When $$x$$ is input of $$FakeQuantize$$ and $$s$$ is step_size of $$FakeQuantize$$
$$ \dfrac{\partial \hat{x}}{\partial s}= \begin{cases} \left( -\dfrac{x}{|s|}+\left\lceil{\dfrac{x}{|s|}} \right\rfloor \right) \cdot \text{sign}(s) & \text{if, } \text{quant_min} < \dfrac{x}{|s|} < \text{qant_max} \ \ \text{quant_min} \cdot \text{sign}(s) &\text{if, }\dfrac{x}{|s|}\leq \text{quant_min} \ \ \text{quant_max}\cdot \text{sign}(s) &\text{if, } \dfrac{x}{|s|}\geq \text{quant_max} \end{cases} $$
method backward
python
backward(ctx: Any, *grad_outputs: Any) → Any
method forward
python
forward(
ctx: Any,
inputs: Tensor,
step_size: Tensor,
zero_point: Tensor,
grad_scale: float,
quant_min: int,
quant_max: int,
axis: int | None
) → Tensor
Updated: 2024-06-13T23:42:41