Manual Compression

Upload Model

upload_model(self, input_model_path: str, input_shapes: List[Dict[str, int]] | None = None, framework: Framework = Framework.PYTORCH) → ModelBase

Upload a model for compression.

Parameters:
- input_model_path (str) – The file path where the model is located.
- input_shapes (List *[*Dict *[*str , int ] ] , optional) – Input shapes of the model. Defaults to [].
- framework (Framework) – The framework of the model.
Raises:
e – If an error occurs while uploading the model.
Returns:
Uploaded model object.
Return type:
ModelBase

Details of Parameters

Framework

class Framework(value)

An enumeration.

Available Framework

Name	Description
TENSORFLOW_KERAS	TensorFlow-Keras
PYTORCH	PyTorch GraphModule
ONNX	ONNX

Example

from netspresso.enums import Framework

FRAMEWORK = Framework.PYTORCH

📘
Note

ONNX (.onnx)

Supported version: PyTorch >= 1.11.x, ONNX >= 1.10.x.

If a model is defined in PyTorch, it should be converted into the ONNX format before being uploaded.

How-to-guide for the conversion of PyTorch into ONNX.

PyTorch GraphModule (.pt)

Supported version: PyTorch >= 1.11.x.

If a model is defined in PyTorch, it should be converted into the GraphModule before being uploaded.

The model must contain not only the status dictionary but also the structure of the model (do not use state_dict).

How-to-guide for the conversion of PyTorch into GraphModule.

TensorFlow-Keras (.h5, .zip)

Supported version: TensorFlow 2.3.x ~ 2.8.x.

Custom layer must not be included in Keras H5 (.h5) format.

The model must contain not only weights but also the structure of the model (do not use save_weights).

If there is a custom layer in the model, please upload TensorFlow SavedModel format(.zip).

Input Shapes

📘
Note

For input shapes, use the same values that you used to train the model.

If the input shapes of the model is dynamic, input shapes is required.

If the input shapes of the model is static, input shapes is not required.

For example, batch=1, channel=3, height=768, width=1024.

input_shapes = [{"batch": 1, "channel": 3, "dimension": [768, 1024]}]

Currently, only single input models are supported.

Example

from netspresso import NetsPresso


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
model = compressor.upload_model(
    input_model_path="./examples/sample_models/mobilenetv1.h5",
    input_shapes=[\{"batch": 1, "channel": 3, "dimension": [224, 224]\}],
)

Select Compression Method

select_compression_method(self, model_id: str, compression_method: ~netspresso.enums.compression.CompressionMethod, options: ~netspresso.clients.compressor.v2.schemas.compression.base.Options | None = Options(reshape_channel_axis=-1, policy=<Policy.AVERAGE: 'average'>, layer_norm=<LayerNorm.STANDARD_SCORE: 'standard_score'>, group_policy=<GroupPolicy.AVERAGE: 'average'>, step_size=2, step_op=<StepOp.ROUND: 'round'>, reverse=False)) → ResponseSelectMethod

Select a compression method for a model.

Parameters:
- model_id (str) – The ID of the model.
- compression_method (CompressionMethod) – The selected compression method.
- options (Options , optional) – The options for pruning method.
Raises:
e – If an error occurs while selecting the compression method.
Returns:
The compression information for the selected compression method.
Return type:
ResponseSelectMethod

Details of Parameters

Compression Method

class CompressionMethod(value)

An enumeration.

Available Compression Method

Name	Description
PR_L2	L2 Norm Pruning
PR_GM	GM Pruning
PR_NN	Nuclear Norm Pruning
PR_SNP	Structured Neuron-level Pruning
PR_ID	Pruning By Index
FD_TK	Tucker Decomposition
FD_SVD	Singular Value Decomposition
FD_CP	CP Decomposition

Example

from netspresso.enums import CompressionMethod

COMPRESSION_METHOD = CompressionMethod.PR_L2

❗️
Warning

Nuclear Norm is only supported in the Tensorflow-Keras framework.

Structured Neuron-level is only supported in the PyTorch and ONNX frameworks.

📘
Note
Click on the link to learn more about the information. [Compression Method]

Options

class Policy(value)

An enumeration.

class LayerNorm(value)

An enumeration.

class GroupPolicy(value)

An enumeration.

class StepOp(value)

An enumeration.

Example

from netspresso.enums import Policy, LayerNorm, GroupPolicy
from netspresso.clients.compressor.v2.schemas import Options

OPTIONS = Options(
    policy=Policy.AVERAGE,
    layer_norm=LayerNorm.TSS_NORM,
    group_policy=GroupPolicy.COUNT,
    reshape_channel_axis=-1
)

📘
Note
Click the link for more information. [Pruning Options]

📘
Note

This parameter applies only to the Pruning Method (PR_L2, PR_GM, PR_NN, PR_SNP).

Details of Returns

Example

from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, Policy, LayerNorm, GroupPolicy
from netspresso.clients.compressor.v2.schemas import Options


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
compression_info = compressor.select_compression_method(
    model_id="YOUR_UPLOADED_MODEL_ID",
    compression_method=CompressionMethod.PR_L2,
    options=Options(
        policy=Policy.AVERAGE,
        layer_norm=LayerNorm.STANDARD_SCORE,
        group_policy=GroupPolicy.AVERAGE,
        reshape_channel_axis=-1,
    ),
)

Output

>>> compression_info
CompressionInfo(
    compression_method="PR_L2",
    available_layers=[
        AvailableLayer(name='conv1', values=[""], channels=[32]),
        AvailableLayer(name='layers.0.conv2', values=[""], channels=[64]),
        AvailableLayer(name='layers.1.conv2', values=[""], channels=[128]),
        AvailableLayer(name='layers.2.conv2', values=[""], channels=[128]),
        AvailableLayer(name='layers.3.conv2', values=[""], channels=[256]),
        AvailableLayer(name='layers.4.conv2', values=[""], channels=[256]),
        AvailableLayer(name='layers.5.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.6.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.7.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.8.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.9.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.10.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.11.conv2', values=[""], channels=[1024]),
        AvailableLayer(name='layers.12.conv2', values=[""], channels=[1024])
    ],
    options=\{'reshape_channel_axis': -1, 'policy': 'average', 'layer_norm': 'tss_norm', 'group_policy': 'average'\}
    original_model_id="YOUR_UPLOADED_MODEL_ID",
    compressed_model_id="",
    compression_id="",
)

Set Compression Params

Details of Parameters

Values of available layer

Compression Method	Number of Values	Type	Range
PR_L2	1	Float	0.0 < ratio < 1.0
PR_GM	1	Float	0.0 < ratio < 1.0
PR_NN	1	Float	0.0 < ratio < 1.0
PR_SNP	1	Float	0.0 < ratio < 1.0
PR_ID	(Num of Out<br/>Channels - 1)	Int	0 ≤ channels < Num of Out Channels
FD_TK	2	Int	0 < rank ≤ (Num of In Channels or<br/>Num of Out Channels)
FD_CP	1	Int	0 < rank ≤ min(Num of In Channels or<br/>Num of Out Channels)
FD_SVD	1	Int	0 < rank ≤ min(Num of In Channels or<br/>Num of Out Channels)

Example

for available_layer in compression_info.available_layers:
   available_layer.values = [0.2]

Output

>>> compression_info
CompressionInfo(
   compression_method="PR_L2",
   available_layers=[
      AvailableLayer(name='conv1', values=[0.2], channels=[32]),
      AvailableLayer(name='layers.0.conv2', values=[0.2], channels=[64]),
      AvailableLayer(name='layers.1.conv2', values=[0.2], channels=[128]),
      AvailableLayer(name='layers.2.conv2', values=[0.2], channels=[128]),
      AvailableLayer(name='layers.3.conv2', values=[0.2], channels=[256]),
      AvailableLayer(name='layers.4.conv2', values=[0.2], channels=[256]),
      AvailableLayer(name='layers.5.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.6.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.7.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.8.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.9.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.10.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.11.conv2', values=[0.2], channels=[1024]),
      AvailableLayer(name='layers.12.conv2', values=[0.2], channels=[1024])
   ],
   options=\{'reshape_channel_axis': -1, 'policy': 'average', 'layer_norm': 'tss_norm', 'group_policy': 'average'\}
   original_model_id="YOUR_UPLOADED_MODEL_ID",
   compressed_model_id="",
   compression_id="",
)

Compress Model

compress_model(self, compression: ResponseSelectMethod, output_dir: str, dataset_path: str | None = None) → CompressorMetadata

Compress a model using the provided compression information.

Parameters:
- compression (CompressionInfo) – The information about the compression.
- output_dir (str) – The local path to save the compressed model.
- dataset_path (str , optional) – The path of the dataset used for nuclear norm compression method. Default is None.
Raises:
e – If an error occurs while compressing the model.
Returns:
Compress metadata.
Return type:
CompressorMetadata

Example

compressed_model = compressor.compress_model(
    compression=compression_info,
    output_dir="./outputs/compressed/graphmodule_manual",
)

Full Example

from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, GroupPolicy, LayerNorm, Policy
from netspresso.clients.compressor.v2.schemas import Options


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# 1. Declare compressor
compressor = netspresso.compressor_v2()

# 2. Upload model
model = compressor.upload_model(
    input_model_path="./examples/sample_models/graphmodule.pt",
    input_shapes=[\{"batch": 1, "channel": 3, "dimension": [224, 224]\}],
)

# 3. Select compression method
compression_info = compressor.select_compression_method(
    model_id=model.ai_model_id,
    compression_method=CompressionMethod.PR_L2,
    options=Options(
        policy=Policy.AVERAGE,
        layer_norm=LayerNorm.STANDARD_SCORE,
        group_policy=GroupPolicy.AVERAGE,
        reshape_channel_axis=-1,
    ),
)

# 4. Set params for compression(ratio or rank)
for available_layer in compression_info.available_layers[:5]:
    available_layer.values = [0.2]

# 5. Compress model
compressed_model = compressor.compress_model(
    compression=compression_info,
    output_dir="./outputs/compressed/graphmodule_manual",
)