dmx.compressor.advanced_recipe.DmxSLaNCHyperparams

class dmx.compressor.advanced_recipe.DmxSLaNCHyperparams(position: str | None = None, mlp_type: str | None = None, device: device | None = None, prev_ln_weight: Module | None = None, fc1: Module | None = None, fc2: Module | None = None, up_proj: Module | None = None, down_proj: Module | None = None, gate_proj: Module | None = None, v_proj: Module | None = None, o_proj: Module | None = None)

SLaNC hyperparamters with default values

__init__(position: str | None = None, mlp_type: str | None = None, device: device | None = None, prev_ln_weight: Module | None = None, fc1: Module | None = None, fc2: Module | None = None, up_proj: Module | None = None, down_proj: Module | None = None, gate_proj: Module | None = None, v_proj: Module | None = None, o_proj: Module | None = None) → None

Methods

__init__([position, mlp_type, device, ...])

Attributes

device
down_proj
fc1
fc2
gate_proj
mlp_type
o_proj
position
prev_ln_weight
up_proj
v_proj