Search Results - RepositoryStats

FlexLLMGen FMInference

565

9.3k

apache-2.0

111

Running large language models on a single GPU for throughput-oriented scenarios.

opt gpt-3 offloading deep-learning high-throughput machine-learning large-language-models

This repository has been archived (exclude archived)

Created 2023-02-15

107 commits to main branch, last one 4 months ago

mixtral-offloading dvmazur

230

2.3k

mit

28

Run Mixtral-8x7B models in Colab or consumer desktops

llm pytorch offloading google-colab quantization deep-learning colab-notebook language-model mixture-of-experts

Created 2023-12-15

86 commits to master branch, last one about a year ago

ao pytorch

232

1.9k

bsd-3-clause

43

PyTorch native quantization and sparsity for training and inference

mx brrr cuda llama dtypes float8 pytorch sparsity training inference optimizer offloading transformer quantization

Created 2023-11-03

1,145 commits to main branch, last one a day ago

QECO ImanRHT

40

193

mit

11

A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning (DRL) for Mobile Edge Computing (MEC) | This algorithm captures the dynamics of the MEC environment by integrating ...

dqn mdp mec d3qn ddqn offloading lstm-networks deep-q-network edge-computing resource-management network-optimization performance-evaluation markov-decision-processes deep-reinforcement-learning

Created 2023-07-31

193 commits to main branch, last one 18 hours ago