#local inference
- Huawei Open-Sources MACaQ: 122B Model Quantized in 64 Minutes
Huawei's research arm has open-sourced MACaQ, a model-agnostic quantization toolkit that can compress a 122B-parameter model in 64 minutes using 32.5 GB of memory.
Huawei's research arm has open-sourced MACaQ, a model-agnostic quantization toolkit that can compress a 122B-parameter model in 64 minutes using 32.5 GB of memory.