Code

modelinference refactor#762

Code

Merged

hiworldwzj merged 55 commits intomainModelTC/lightllm:mainfrom mm_refactorModelTC/lightllm:mm_refactorCopy head branch name to clipboardMar 11, 2025

Commits

Commits on Mar 5, 2025

• refactor mm

  

  shihaobai

  committed
  969cba6

  Copy full SHA for 969cba6

• update

  

  shihaobai

  committed
  85330a6

  Copy full SHA for 85330a6

• update

  

  shihaobai

  committed
  e47dbfc

  Copy full SHA for e47dbfc

Commits on Mar 6, 2025

• Merge branch 'main' into mm_refactor

  

  shihaobai

  committed
  bfdfed0

  Copy full SHA for bfdfed0

• refacor llama

  

  shihaobai

  committed
  b613878

  Copy full SHA for b613878

• fix quantization

  

  shihaobai

  committed
  d4644c9

  Copy full SHA for d4644c9

• fix datatype

  

  shihaobai

  committed
  487850b

  Copy full SHA for 487850b

• refactor bloom

  

  shihaobai

  committed
  9d4dfa9

  Copy full SHA for 9d4dfa9

• refactor cohere

  

  shihaobai

  committed
  63c637f

  Copy full SHA for 63c637f

• remove unused import for chatglm

  

  shihaobai

  committed
  82c147a

  Copy full SHA for 82c147a

• refactor gemma

  

  shihaobai

  committed
  d871dbb

  Copy full SHA for d871dbb

• remove unused import for internlm

  

  shihaobai

  committed
  bea2c34

  Copy full SHA for bea2c34

• remove unused import for internlm2

  

  shihaobai

  committed
  1267e49

  Copy full SHA for 1267e49

• remove unused import for minicpm

  

  shihaobai

  committed
  1fb57bb

  Copy full SHA for 1fb57bb

• refactor mixtral

  

  shihaobai

  committed
  52de3f3

  Copy full SHA for 52de3f3

• remove unused import for qwen

  

  shihaobai

  committed
  b98388d

  Copy full SHA for b98388d

• remove unused import for qwen2

  

  shihaobai

  committed
  37ac3ed

  Copy full SHA for 37ac3ed

• refactor starcode

  

  shihaobai

  committed
  18f1802

  Copy full SHA for 18f1802

• refactor starcoder2

  

  shihaobai

  committed
  7cb7df7

  Copy full SHA for 7cb7df7

• feat: refactor transformer_layer_weight.py in deepseek2

  

  niushengxiao

  committed
  de4067b

  Copy full SHA for de4067b

• refactor vit and fix

  

  shihaobai

  committed
  6db6b55

  Copy full SHA for 6db6b55

• update common

  

  shihaobai

  committed
  25dbffc

  Copy full SHA for 25dbffc

• fix vit test

  

  shihaobai

  committed
  26244b3

  Copy full SHA for 26244b3

• update cli

  

  shihaobai

  committed
  d3bf115

  Copy full SHA for d3bf115

• feat: split fused_moe_weight

  

  niushengxiao

  committed
  fa5bcc6

  Copy full SHA for fa5bcc6

• fix: remove tp_rank and world_size params

  

  niushengxiao

  committed
  83abc5f

  Copy full SHA for 83abc5f

Commits on Mar 7, 2025

• remove tp_rank for llama

  

  shihaobai

  committed
  d95e0d0

  Copy full SHA for d95e0d0

• feat: refactor BMMWeight

  

  niushengxiao

  committed
  e4b0d26

  Copy full SHA for e4b0d26

• fix: remove tp_rank and world_size

  

  niushengxiao

  committed
  973f56f

  Copy full SHA for 973f56f

• bloom

  

  shihaobai

  committed
  264f044

  Copy full SHA for 264f044

• remove tp_rank

  

  shihaobai

  committed
  4b53f0f

  Copy full SHA for 4b53f0f

• update server

  

  shihaobai

  committed
  d7b9f26

  Copy full SHA for d7b9f26

• fix: bmm can't run fp8

  

  niushengxiao

  committed
  2e635f7

  Copy full SHA for 2e635f7

• rename world_size

  

  shihaobai

  committed
  9bca5f7

  Copy full SHA for 9bca5f7

• rename layer_name

  

  shihaobai

  committed
  77d3972

  Copy full SHA for 77d3972

Commits on Mar 9, 2025

• feat: add w8a8b128 quant in the mm_weight

  

  niushengxiao

  committed
  d79f35d

  Copy full SHA for d79f35d

Commits on Mar 10, 2025

• rename

  

  shihaobai

  committed
  ea6d74c

  Copy full SHA for ea6d74c

• fix deepseek v2

  

  shihaobai

  committed
  e396797

  Copy full SHA for e396797

• fix: fix bugs in the quant about v3

  

  niushengxiao

  committed
  250b80c

  Copy full SHA for 250b80c

• xfix: continue fix

  

  niushengxiao

  committed
  59e1a5d

  Copy full SHA for 59e1a5d

• fix: continue fix

  

  niushengxiao

  committed
  bfb50d6

  Copy full SHA for bfb50d6

• fix: continue fix

  

  niushengxiao

  committed
  7b36448

  Copy full SHA for 7b36448

• refactor

  shihaobai

  committed
  a9fd68a

  Copy full SHA for a9fd68a

• update

  shihaobai

  committed
  7ea3bb4

  Copy full SHA for 7ea3bb4

• merge

  shihaobai

  committed
  d31b122

  Copy full SHA for d31b122

• fix

  shihaobai

  committed
  7d18bdc

  Copy full SHA for 7d18bdc

• reformat

  shihaobai

  committed
  6bc5c8d

  Copy full SHA for 6bc5c8d

• reformat

  shihaobai

  committed
  a1b0803

  Copy full SHA for a1b0803

• update dist

  shihaobai

  committed
  a5f850d

  Copy full SHA for a5f850d

Commits on Mar 11, 2025

• fix: fix a bug

  

  niushengxiao

  committed
  92df35b

  Copy full SHA for 92df35b

• fix

  hiworldwzj

  committed
  33f704f

  Copy full SHA for 33f704f

• fix

  hiworldwzj

  authored
  f45233c

  Copy full SHA for f45233c

• fix

  hiworldwzj

  committed
  f8ca85b

  Copy full SHA for f8ca85b

• fix

  hiworldwzj

  committed
  9482356

  Copy full SHA for 9482356

• fix

  hiworldwzj

  authored
  a901018

  Copy full SHA for a901018