module owlite.nn.modules.fake_quantizer
class FakeQuantizer
An implementation of fake quantization (a.k.a. quantization simulation).
Attributes:
step_size
: The step size for the fake quantization. The larger the step size, the shorter the length of quantization intervals, and vice versa.zero_point
: The zero point for the fake quantization. When simulating symmetric quantization (i.e.self.symmetric
isTrue
), this parameter will be fixed to 0 and remain unchanged.
method __init__
python
__init__(
options: FakeQuantizerOptions,
enable: bool = True,
identification: str | None = None
)
Initialize a FakeQuantizer
instance.
Args:
options
(FakeQuantizerOptions): Options for the fake quantizer.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Raises:
ValueError
: ifoptions.ptq_calibration
is "percentile" butoptions.percentile
isNone
.
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
option
The options that current FakeQuantizer instance represents.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
The maximum value this fake quantizer can handle.
property
quant_min
The minimum value this fake quantizer can handle.
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
Apply fake quantization to the input tensor.
Args:
inputs
(torch.Tensor): A tensor to fake-quantize.
Returns:
torch.Tensor
: the fake-quantized tensor if this fake quantizer is enabled, the unchanged input tensor otherwise.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakeINTQuantizer
Fake integer quantizer.
method __init__
python
__init__(
options: FakeQuantizerOptions,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
)
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer underlying integer type.
e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] when self.target_dtype == "int8"
and self.narrow_range
is True
.
property
option
The options that current FakeQuantizer instance represents.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
property
quant_min
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
Apply fake quantization to the input tensor.
Args:
inputs
(torch.Tensor): A tensor to fake-quantize.
Raises:
ValueError
: If the fake quantizer has a negative step size or its channel size mismatches with theinputs
.
Returns:
torch.Tensor
: the fake-quantized tensor if this fake quantizer is enabled, the unchanged input tensor otherwise.
method invert_signedness
python
invert_signedness() → Self
Invert signedness of this fake quantizer's underlying integer type.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakePerChannelINTQuantizer
Fake quantizer that simulates per-channel quantization.
method __init__
python
__init__(
options: FakeQuantizerOptions,
channel: Channel,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
)
property
channel
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer underlying integer type.
e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] when self.target_dtype == "int8"
and self.narrow_range
is True
.
property
option
The options that current FakeQuantizer instance represents.
property
per_channel
Check if per-channel quantization is used.
Equivalent to self.channel is not None
.
Returns:
bool
:True
if per-channel quantization is used,False
otherwise.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
property
quant_min
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
method as_per_tensor
python
as_per_tensor() → FakePerTensorINTQuantizer
Create a new fake per-tensor quantizer with the same option (except for the per_channel
value).
The step_size
of the new fake per-tensor quantizer is initialized as the maximum element of self.step_size
.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
method invert_signedness
python
invert_signedness() → Self
Invert signedness of this fake quantizer's underlying integer type.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakePerTensorINTQuantizer
Fake integer quantizer that simulates per-tensor quantization.
method __init__
python
__init__(
options: FakeQuantizerOptions,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
)
property
channel
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer underlying integer type.
e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] when self.target_dtype == "int8"
and self.narrow_range
is True
.
property
option
The options that current FakeQuantizer instance represents.
property
per_channel
Check if per-channel quantization is used.
Equivalent to self.channel is not None
.
Returns:
bool
:True
if per-channel quantization is used,False
otherwise.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
property
quant_min
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
method as_per_channel
python
as_per_channel(channel: Channel) → FakePerChannelINTQuantizer
Create a new fake per-channel quantizer with the same option (except for the per_channel
value).
The step_size
and zero_point
of the new fake per-channel quantizer is initialized with shape (channel.size,)
filled with values in self.step_size
and self.zero_point
, respectively.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
Apply fake quantization to the input tensor.
Args:
inputs
(torch.Tensor): A tensor to fake-quantize.
Raises:
ValueError
: If the fake quantizer has a negative step size or its channel size mismatches with theinputs
.
Returns:
torch.Tensor
: the fake-quantized tensor if this fake quantizer is enabled, the unchanged input tensor otherwise.
method invert_signedness
python
invert_signedness() → Self
Invert signedness of this fake quantizer's underlying integer type.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakeFPQuantizer
Fake FP quantizer.
method __init__
python
__init__(
options: FakeQuantizerOptions,
enable: bool = True,
identification: str | None = None
)
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer.
Do not use narrow range for the FP fake quantizer.
property
option
The options that current FakeQuantizer instance represents.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
The maximum value this fake quantizer can handle.
property
quant_min
The minimum value this fake quantizer can handle.
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
Apply fake quantization to the input tensor.
Args:
inputs
(torch.Tensor): A tensor to fake-quantize.
Raises:
ValueError
: If the fake quantizer has a negative step size.
Returns:
torch.Tensor
: the fake-quantized tensor if this fake quantizer is enabled, the unchanged input tensor otherwise.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakePerChannelFPQuantizer
Fake quantizer that simulates per-channel quantization.
method __init__
python
__init__(
options: FakeQuantizerOptions,
channel: Channel,
enable: bool = True,
identification: str | None = None
)
property
channel
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer.
Do not use narrow range for the FP fake quantizer.
property
option
The options that current FakeQuantizer instance represents.
property
per_channel
Check if per-channel quantization is used.
Equivalent to self.channel is not None
.
Returns:
bool
:True
if per-channel quantization is used,False
otherwise.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
The maximum value this fake quantizer can handle.
property
quant_min
The minimum value this fake quantizer can handle.
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
method as_per_tensor
python
as_per_tensor() → FakePerTensorFPQuantizer
Create a new fake per-tensor fp quantizer with the same option (except for the per_channel
value).
The step_size
of the new fake per-tensor quantizer is initialized as the maximum element of self.step_size
.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
class FakePerTensorFPQuantizer
Fake integer quantizer that simulates per-tensor quantization.
method __init__
python
__init__(
options: FakeQuantizerOptions,
enable: bool = True,
identification: str | None = None
)
property
channel
property
learnzeropoint
Whether to update self.zero_point
during QAT.
Can be set to True
only if self.symmetric
is False
property
maxabs_bound
The maximum absolute value that this fake quantizer can handle.
property
narrow_range
Whether to apply narrow range clipping for this fake quantizer.
Do not use narrow range for the FP fake quantizer.
property
option
The options that current FakeQuantizer instance represents.
property
per_channel
Check if per-channel quantization is used.
Equivalent to self.channel is not None
.
Returns:
bool
:True
if per-channel quantization is used,False
otherwise.
property
qat_function
The autograd function of this fake quantizer.
property
quant_max
The maximum value this fake quantizer can handle.
property
quant_min
The minimum value this fake quantizer can handle.
property
unsigned
Whether this fake quantizer's underlying integer type is unsigned.
method as_per_channel
python
as_per_channel(channel: Channel) → FakePerChannelFPQuantizer
Create a new fake per-channel quantizer with the same option (except for the per_channel
value).
The step_size
and zero_point
of the new fake per-channel quantizer is initialized with shape (channel.size,)
filled with values in self.step_size
and self.zero_point
, respectively.
classmethod check_if_enabled
python
check_if_enabled(fake_quantizer: Optional[Self]) → TypeGuard[Self]
Check if the given fake quantizer (possibly None
) is enabled.
Args:
fake_quantizer
(FakeQuantizer | None): either a fake quantizer to verify orNone
.
Returns:
TypeGuard[FakeQuantizer]
:True
iffake_quantizer
is notNone
and it is enabled,False
otherwise.
classmethod create
python
create(
options: FakeQuantizerOptions | None,
channel: Channel | None = None,
enable: bool = True,
narrow_range: bool = False,
identification: str | None = None
) → FakePerTensorINTQuantizer|FakePerChannelINTQuantizer|FakePerChannelFPQuantizer|FakePerTensorFPQuantizer|None
Create a FakeQuantizer
instance based on the provided options.
Args:
options
(FakeQuantizerOptions | None): Options for the fake quantizer.channel
(Channel | None, optional): ifchannel
is provided andoptions.per_channel
isTrue
, a fake per-channel quantizer will be created. Otherwise, a fake per-tensor quantizer will be created.enable
(bool, optional): Whether to enable the fake quantizer. Defaults toTrue
.narrow_range
(bool, optional): Whether the fake quantizer should use narrow range. Defaults toFalse
. This is only required by FakeINTQuantizer. e.g. quantized integer values will be clipped into range [-127,127] instead of [-128,127] whenoptions.dtype == int8
andnarrow_range
isTrue
.identification
(str | None, optional): unique ID for the fake quantizer. Defaults toNone
.
Returns:
FakePerTensorINTQuantizer | FakePerChannelINTQuantizer | None
: If theoptions
is valid for quantization returns created fake quantizer,None
otherwise.
method disable
python
disable() → Self
Disable this fake quantizer.
method enable
python
enable() → Self
Enable this fake quantizer.
method extra_repr
python
extra_repr() → str
method forward
python
forward(inputs: Tensor) → Tensor
Apply fake quantization to the input tensor.
Args:
inputs
(torch.Tensor): A tensor to fake-quantize.
Raises:
ValueError
: If the fake quantizer has a negative step size.
Returns:
torch.Tensor
: the fake-quantized tensor if this fake quantizer is enabled, the unchanged input tensor otherwise.
method state_dict
python
state_dict(*args, **kwargs) → OrderedDict[Any, Any] | dict[str, Any]
Updated: 2024-06-13T23:42:42