Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New GEMM kernels for weight-only quantization #2090

Merged
merged 225 commits into from
Aug 19, 2024
Merged

New GEMM kernels for weight-only quantization #2090

merged 225 commits into from
Aug 19, 2024

Commits on Mar 8, 2024

  1. preprocess for kv-int8

    @lzhangzz
    lzhangzz committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    0e546d7 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. working kv-int8

    @lzhangzz
    lzhangzz committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    33a2726 View commit details
    Browse the repository at this point in the history

  2. minor

    @lzhangzz
    lzhangzz committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    9d089db View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. working kv-int4

    @lzhangzz
    lzhangzz committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    a52d6ec View commit details
    Browse the repository at this point in the history

  2. optimize kv-int4

    @lzhangzz
    lzhangzz committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    db61d42 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. optimize kv-int4

    @lzhangzz
    lzhangzz committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    082e49f View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. optimized SIMT f16/u8/u4 decoding

    @lzhangzz
    lzhangzz committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    7ee4388 View commit details
    Browse the repository at this point in the history

  2. fix tc decoding

    @lzhangzz
    lzhangzz committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    48ef596 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. int8 tc decoding

    @lzhangzz
    lzhangzz committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    77cfc98 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. int4 tc decoding

    @lzhangzz
    lzhangzz committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    9957801 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. minor

    @lzhangzz
    lzhangzz committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    dc2eb32 View commit details
    Browse the repository at this point in the history

  2. optimize

    @lzhangzz
    lzhangzz committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    41a7c0d View commit details
    Browse the repository at this point in the history

  3. optimize tc kv-int4/int8

    @lzhangzz
    lzhangzz committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    86f572d View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. fix sm_75/sm_70

    @lzhangzz
    lzhangzz committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    373386f View commit details
    Browse the repository at this point in the history

  2. simplify

    @lzhangzz
    lzhangzz committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    cc456f1 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. bf16+kv4/8

    @lzhangzz
    lzhangzz committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    5f25c0c View commit details
    Browse the repository at this point in the history

  2. support more mma instruction

    @lzhangzz
    lzhangzz committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    ff8c439 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    cd0f266 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. dispatching

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    2d2ed7d View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into online-quant

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    99864f6 View commit details
    Browse the repository at this point in the history

  3. integration

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    a4238b8 View commit details
    Browse the repository at this point in the history

  4. remove offline kv params

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    798f39d View commit details
    Browse the repository at this point in the history

  5. fix msvc build

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    0269951 View commit details
    Browse the repository at this point in the history

  6. fix msvc build

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    6250739 View commit details
    Browse the repository at this point in the history

  7. fix lint

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    b225059 View commit details
    Browse the repository at this point in the history

  8. fix lint

    @lzhangzz
    lzhangzz committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    fb281f2 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. fix cmake

    @lzhangzz
    lzhangzz committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    1fe6a61 View commit details
    Browse the repository at this point in the history

  2. fix lint

    @lzhangzz
    lzhangzz committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    4bef81d View commit details
    Browse the repository at this point in the history

  3. fix lint

    @lzhangzz
    lzhangzz committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    8118cb1 View commit details
    Browse the repository at this point in the history

  4. minor

    @lzhangzz
    lzhangzz committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    3509ec2 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    7929d76 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. gemm baseline

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    14aff92 View commit details
    Browse the repository at this point in the history

  2. optimize

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    2e8cda2 View commit details
    Browse the repository at this point in the history

  3. minor

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    6756a67 View commit details
    Browse the repository at this point in the history

  4. tb swizzle

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7593d8f View commit details
    Browse the repository at this point in the history

  5. minor

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    d4e9a13 View commit details
    Browse the repository at this point in the history

  6. tune

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    5ba9c88 View commit details
    Browse the repository at this point in the history

  7. minor

    @lzhangzz
    lzhangzz committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    86fd759 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. wip

    @lzhangzz
    lzhangzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    b679f48 View commit details
    Browse the repository at this point in the history

  2. minor

    @lzhangzz
    lzhangzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    8d903c4 View commit details
    Browse the repository at this point in the history

  3. fp16 transcription

    @lzhangzz
    lzhangzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    521a013 View commit details
    Browse the repository at this point in the history

  4. optimize

    @lzhangzz
    lzhangzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    bc19536 View commit details
    Browse the repository at this point in the history

  5. tune

    @lzhangzz
    lzhangzz committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    a6d0d33 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. adjust layout

    @lzhangzz
    lzhangzz committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    0074ca5 View commit details
    Browse the repository at this point in the history

  2. optimize

    @lzhangzz
    lzhangzz committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    d14d8a4 View commit details
    Browse the repository at this point in the history

  3. tune

    @lzhangzz
    lzhangzz committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    5c4b7ce View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    1d7dec1 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    c0bdaca View commit details
    Browse the repository at this point in the history

  2. f16xs4/8 gemm

    @lzhangzz
    lzhangzz committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    8aeb310 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    aed8c47 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. dequant

    @lzhangzz
    lzhangzz committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    08dfb1c View commit details
    Browse the repository at this point in the history

  2. fix Q

    @lzhangzz
    lzhangzz committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    bce922a View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. fix Q

    @lzhangzz
    lzhangzz committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    cb7b14c View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. end-to-end test

    @lzhangzz
    lzhangzz committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    510e349 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. optimize Q

    @lzhangzz
    lzhangzz committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    9463260 View commit details
    Browse the repository at this point in the history

  2. pack Q

    @lzhangzz
    lzhangzz committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    a450d76 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. tune

    @lzhangzz
    lzhangzz committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    95cbfae View commit details
    Browse the repository at this point in the history

  2. split-k

    @lzhangzz
    lzhangzz committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    d8707f2 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. sliced-k

    @lzhangzz
    lzhangzz committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    4ee0636 View commit details
    Browse the repository at this point in the history

  2. fix Q

    @lzhangzz
    lzhangzz committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    4ceab20 View commit details
    Browse the repository at this point in the history

  3. add transpose_m8n8_b32

    @lzhangzz
    lzhangzz committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    4da07a4 View commit details
    Browse the repository at this point in the history

  4. tune gemm

    @lzhangzz
    lzhangzz committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    283db23 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. predicate support

    @lzhangzz
    lzhangzz committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    77d4592 View commit details
    Browse the repository at this point in the history

  2. tune

    @lzhangzz
    lzhangzz committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    0b0f44b View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. dispatch

    @lzhangzz
    lzhangzz committed May 1, 2024
    Configuration menu
    Copy the full SHA
    ff47c14 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. dispatch v2

    @lzhangzz
    lzhangzz committed May 6, 2024
    Configuration menu
    Copy the full SHA
    f7e8d11 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. automatic tuning

    @lzhangzz
    lzhangzz committed May 8, 2024
    Configuration menu
    Copy the full SHA
    11cfb78 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. nvbench

    @lzhangzz
    lzhangzz committed May 9, 2024
    Configuration menu
    Copy the full SHA
    9ea1d83 View commit details
    Browse the repository at this point in the history

  2. better API

    @lzhangzz
    lzhangzz committed May 9, 2024
    Configuration menu
    Copy the full SHA
    2d48543 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. GPU metrics

    @lzhangzz
    lzhangzz committed May 10, 2024
    Configuration menu
    Copy the full SHA
    3502922 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2024

  1. update cost model

    @lzhangzz
    lzhangzz committed May 11, 2024
    Configuration menu
    Copy the full SHA
    f71a397 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. add simt impl

    @lzhangzz
    lzhangzz committed May 14, 2024
    Configuration menu
    Copy the full SHA
    cd93d1c View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. add 16816

    @lzhangzz
    lzhangzz committed May 15, 2024
    Configuration menu
    Copy the full SHA
    1d43c68 View commit details
    Browse the repository at this point in the history

  2. add 884

    @lzhangzz
    lzhangzz committed May 15, 2024
    Configuration menu
    Copy the full SHA
    d41a812 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed May 17, 2024
    Configuration menu
    Copy the full SHA
    07ab492 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. smem copy

    @lzhangzz
    lzhangzz committed May 20, 2024
    Configuration menu
    Copy the full SHA
    8053a29 View commit details
    Browse the repository at this point in the history

  2. minor

    @lzhangzz
    lzhangzz committed May 20, 2024
    Configuration menu
    Copy the full SHA
    c949982 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. NT & NN

    @lzhangzz
    lzhangzz committed May 21, 2024
    Configuration menu
    Copy the full SHA
    16ea28f View commit details
    Browse the repository at this point in the history

  2. transformation

    @lzhangzz
    lzhangzz committed May 21, 2024
    Configuration menu
    Copy the full SHA
    fc17896 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed May 24, 2024
    Configuration menu
    Copy the full SHA
    7bb684a View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed May 27, 2024
    Configuration menu
    Copy the full SHA
    938a1a7 View commit details
    Browse the repository at this point in the history

  2. add UV

    @lzhangzz
    lzhangzz committed May 27, 2024
    Configuration menu
    Copy the full SHA
    a641f1c View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. refactor testbed

    @lzhangzz
    lzhangzz committed May 28, 2024
    Configuration menu
    Copy the full SHA
    59b270d View commit details
    Browse the repository at this point in the history

  2. working fp16 packing

    @lzhangzz
    lzhangzz committed May 28, 2024
    Configuration menu
    Copy the full SHA
    76a0784 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. update

    @lzhangzz
    lzhangzz committed May 29, 2024
    Configuration menu
    Copy the full SHA
    55e0db4 View commit details
    Browse the repository at this point in the history

  2. use (m, k) & (n, k)

    @lzhangzz
    lzhangzz committed May 29, 2024
    Configuration menu
    Copy the full SHA
    5babfa6 View commit details
    Browse the repository at this point in the history

  3. simplify

    @lzhangzz
    lzhangzz committed May 29, 2024
    Configuration menu
    Copy the full SHA
    0e4ca34 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. dispatch for conversion

    @lzhangzz
    lzhangzz committed May 30, 2024
    Configuration menu
    Copy the full SHA
    35cca77 View commit details
    Browse the repository at this point in the history

  2. refactor

    @lzhangzz
    lzhangzz committed May 30, 2024
    Configuration menu
    Copy the full SHA
    0f02252 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed May 31, 2024
    Configuration menu
    Copy the full SHA
    cd245db View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. refactor

    @lzhangzz
    lzhangzz committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    0e615ff View commit details
    Browse the repository at this point in the history

  2. simplify

    @lzhangzz
    lzhangzz committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    f6bd704 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. refactor quantization

    @lzhangzz
    lzhangzz committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    64bb2cb View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2024

  1. quantization

    @lzhangzz
    lzhangzz committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    48e2559 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. fix pack_cnt_m/k

    @lzhangzz
    lzhangzz committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c0d5119 View commit details
    Browse the repository at this point in the history

  2. uint8_t

    @lzhangzz
    lzhangzz committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    69485e3 View commit details
    Browse the repository at this point in the history

  3. uint4_t

    @lzhangzz
    lzhangzz committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    d01e6b8 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. symmetry

    @lzhangzz
    lzhangzz committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    d031219 View commit details
    Browse the repository at this point in the history

  2. refactor

    @lzhangzz
    lzhangzz committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    3968337 View commit details
    Browse the repository at this point in the history

  3. large pack

    @lzhangzz
    lzhangzz committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    328fdf3 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. fix SmemCopy for packed inputs

    @lzhangzz
    lzhangzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    e4c8c23 View commit details
    Browse the repository at this point in the history

  2. tune

    @lzhangzz
    lzhangzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    3ba464a View commit details
    Browse the repository at this point in the history

  3. SIMT

    @lzhangzz
    lzhangzz committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    c2a16b9 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. SIMT packing

    @lzhangzz
    lzhangzz committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    1212973 View commit details
    Browse the repository at this point in the history

  2. SIMT int8

    @lzhangzz
    lzhangzz committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    502d818 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. SIMT int4

    @lzhangzz
    lzhangzz committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    8588c65 View commit details
    Browse the repository at this point in the history

  2. fix group size

    @lzhangzz
    lzhangzz committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    ad9fe51 View commit details
    Browse the repository at this point in the history

  3. mma.m8n8k4

    @lzhangzz
    lzhangzz committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    72e73b5 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. clean-up

    @lzhangzz
    lzhangzz committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ffe7f37 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. refactor epilogue

    @lzhangzz
    lzhangzz committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c91725b View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. fix smem layout for C

    @lzhangzz
    lzhangzz committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    64d6831 View commit details
    Browse the repository at this point in the history

  2. tune epilogue

    @lzhangzz
    lzhangzz committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    483dfcc View commit details
    Browse the repository at this point in the history

  3. TN

    @lzhangzz
    lzhangzz committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4369c65 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. optimize

    @lzhangzz
    lzhangzz committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e60b4a3 View commit details
    Browse the repository at this point in the history

  2. fix _src_step_k

    @lzhangzz
    lzhangzz committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    128b797 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. use raked partition

    @lzhangzz
    lzhangzz committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d0bdc5e View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. fix Tiled_MMA_v2 & optimize smem copy

    @lzhangzz
    lzhangzz committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    b742cad View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into gemm2_refactor

    @lzhangzz
    lzhangzz committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    f44ba31 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. working w4a16

    @lzhangzz
    lzhangzz committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    c0e4849 View commit details
    Browse the repository at this point in the history

  2. add missing

    @lzhangzz
    lzhangzz committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    11bda59 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. fuse up and gate

    @lzhangzz
    lzhangzz committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    b56b3f7 View commit details
    Browse the repository at this point in the history

  2. fused silu

    @lzhangzz
    lzhangzz committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    006dd5a View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. sm75 and sm70

    @lzhangzz
    lzhangzz committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    b513389 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. cache policy

    @lzhangzz
    lzhangzz committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    ff518bc View commit details
    Browse the repository at this point in the history

  2. remove unused

    @lzhangzz
    lzhangzz committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    ca5c3b5 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. col major output

    @lzhangzz
    lzhangzz committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    4d87165 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2024

  1. fix tiling of C

    @lzhangzz
    lzhangzz committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    74a9e8f View commit details
    Browse the repository at this point in the history

  2. wip

    @lzhangzz
    lzhangzz committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    efa4747 View commit details
    Browse the repository at this point in the history

  3. wip

    @lzhangzz
    lzhangzz committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    909f358 View commit details
    Browse the repository at this point in the history

  4. wip

    @lzhangzz
    lzhangzz committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    5ab9deb View commit details
    Browse the repository at this point in the history

  5. fix iterator

    @lzhangzz
    lzhangzz committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    4f05ad3 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. update

    @lzhangzz
    lzhangzz committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    1dc4238 View commit details
    Browse the repository at this point in the history

  2. update kernel signature

    @lzhangzz
    lzhangzz committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    e3c7f51 View commit details
    Browse the repository at this point in the history

  3. fix packing

    @lzhangzz
    lzhangzz committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    b01d8cb View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. update

    @lzhangzz
    lzhangzz committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    6a732e2 View commit details
    Browse the repository at this point in the history

  2. refactor

    @lzhangzz
    lzhangzz committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    a0d6a1b View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. update

    @lzhangzz
    lzhangzz committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2addff2 View commit details
    Browse the repository at this point in the history

  2. update

    @lzhangzz
    lzhangzz committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    64ab2dc View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. update

    @lzhangzz
    lzhangzz committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    53fabcc View commit details
    Browse the repository at this point in the history

  2. alpha beta

    @lzhangzz
    lzhangzz committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    fb18189 View commit details
    Browse the repository at this point in the history

  3. set beta

    @lzhangzz
    lzhangzz committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    243a8b9 View commit details
    Browse the repository at this point in the history

  4. fix & clean-up

    @lzhangzz
    lzhangzz committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    d4c3b1c View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. check max splits & add qwen

    @lzhangzz
    lzhangzz committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    ddd3a45 View commit details
    Browse the repository at this point in the history

  2. add tp

    @lzhangzz
    lzhangzz committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    6ccf1cb View commit details
    Browse the repository at this point in the history

  3. refactor LlamaLinear

    @lzhangzz
    lzhangzz committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    b2c8744 View commit details
    Browse the repository at this point in the history

  4. share linear layer

    @lzhangzz
    lzhangzz committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    eaf8dea View commit details
    Browse the repository at this point in the history

  5. tuning interface

    @lzhangzz
    lzhangzz committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    ac0c8c9 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. update

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    47b5f9a View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    7de1d83 View commit details
    Browse the repository at this point in the history

  3. skip nvbench for MSVC

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    caa379a View commit details
    Browse the repository at this point in the history

  4. define uint when needed

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    8b80eb6 View commit details
    Browse the repository at this point in the history

  5. fix

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    e20af20 View commit details
    Browse the repository at this point in the history

  6. fix

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    fedbb3c View commit details
    Browse the repository at this point in the history

  7. fix

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    5563c67 View commit details
    Browse the repository at this point in the history

  8. update

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    8c1ca2d View commit details
    Browse the repository at this point in the history

  9. disable large kernels

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    2002747 View commit details
    Browse the repository at this point in the history

  10. fix

    @lzhangzz
    lzhangzz committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c73670b View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. refactor model conversion

    @lzhangzz
    lzhangzz committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    9990af3 View commit details
    Browse the repository at this point in the history

  2. fix lint

    @lzhangzz
    lzhangzz committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    433ff3a View commit details
    Browse the repository at this point in the history

  3. simplify target model

    @lzhangzz
    lzhangzz committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    99ae4ae View commit details
    Browse the repository at this point in the history

  4. refactor model import

    @lzhangzz
    lzhangzz committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    601869c View commit details
    Browse the repository at this point in the history

  5. minor

    @lzhangzz
    lzhangzz committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    6a2fe32 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. pad inter_size for tp

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    350517e View commit details
    Browse the repository at this point in the history

  2. refactor

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    7553d82 View commit details
    Browse the repository at this point in the history

  3. skip sm_80 and sm_90 on MSVC

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    6d2d4e8 View commit details
    Browse the repository at this point in the history

  4. fix msvc build

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    a9a09ea View commit details
    Browse the repository at this point in the history

  5. fix msvc build

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    a0812db View commit details
    Browse the repository at this point in the history

  6. fix msvc build

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    13efc6e View commit details
    Browse the repository at this point in the history

  7. fix msvc build

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f27efa9 View commit details
    Browse the repository at this point in the history

  8. fix msvc build

    @lzhangzz
    lzhangzz committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    0ec184e View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. fix cu12 / sm90 build

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    092ea09 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    24ea3bb View commit details
    Browse the repository at this point in the history

  3. fix

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    cb71e48 View commit details
    Browse the repository at this point in the history

  4. fix ut

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    1f75bab View commit details
    Browse the repository at this point in the history

  5. fix missing include

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3d9785d View commit details
    Browse the repository at this point in the history

  6. support GPTQ models

    @lzhangzz
    lzhangzz committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    faf55d9 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. fix ut

    @lzhangzz
    lzhangzz committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    2aff617 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. parse tuning args

    @lzhangzz
    lzhangzz committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    7e473b8 View commit details
    Browse the repository at this point in the history

  2. minor

    @lzhangzz
    lzhangzz committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    3f40e02 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    1a0c43a View commit details
    Browse the repository at this point in the history

  4. minor

    @lzhangzz
    lzhangzz committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    3d0653e View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. add missing header

    @lzhangzz
    lzhangzz committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ba5ec11 View commit details
    Browse the repository at this point in the history

  2. add missing headers

    @lzhangzz
    lzhangzz committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    82fc482 View commit details
    Browse the repository at this point in the history

  3. fix converter

    @lzhangzz
    lzhangzz committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    8bdea82 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. fix internvl reader initializer

    @lvhan028
    lvhan028 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    20ddd86 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #3 from lvhan028/fix-internvl 

    fix internvl reader initializer
    @lzhangzz
    lzhangzz authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    dc1fa7e View commit details
    Browse the repository at this point in the history

  3. fix

    @lvhan028
    lvhan028 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    3141b82 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #4 from lvhan028/fix-xcomposer2 

    fix xcomposer2
    @lzhangzz
    lzhangzz authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    fe7b721 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    a9ece3f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'gemm2_dev' of github.com:lzhangzz/lmdeploy into gemm2_dev

    @lzhangzz
    lzhangzz committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    772b2ba View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. tuning

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    8e08c06 View commit details
    Browse the repository at this point in the history

  2. remove unused

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    52b5a7c View commit details
    Browse the repository at this point in the history

  3. tuning

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    cbaa811 View commit details
    Browse the repository at this point in the history

  4. minor

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    0cb05d3 View commit details
    Browse the repository at this point in the history

  5. fix lint

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    325f7fe View commit details
    Browse the repository at this point in the history

  6. fix lint

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    7598b7a View commit details
    Browse the repository at this point in the history

  7. fix lint

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    dc3a105 View commit details
    Browse the repository at this point in the history

  8. minor

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    57b942f View commit details
    Browse the repository at this point in the history

  9. fix lint

    @lzhangzz
    lzhangzz committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    69f90ae View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. fix baichuan2-7b, deepseek-vl and xcomposer2d5-4bit

    @lzhangzz
    lzhangzz committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    bf33b6d View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. tune sm_70

    @lzhangzz
    lzhangzz committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    83eacc7 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. optimize sm70 & fix converter

    @lzhangzz
    lzhangzz committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    610f4e6 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. optimize v100

    @lzhangzz
    lzhangzz committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    d5960a9 View commit details
    Browse the repository at this point in the history

  2. fix lint

    @lzhangzz
    lzhangzz committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ddc4f3a View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    afe55c0 View commit details
    Browse the repository at this point in the history

  4. RTX 4090

    @lzhangzz
    lzhangzz committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    0f78e2b View commit details
    Browse the repository at this point in the history

  5. fix lint

    @lzhangzz
    lzhangzz committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    1831e4d View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. refactor & batch_dim support

    @lzhangzz
    lzhangzz committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    f2866b8 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. A100

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b6c7ebb View commit details
    Browse the repository at this point in the history

  2. TuningParams

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    38cfab3 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into gemm2_dev

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b835420 View commit details
    Browse the repository at this point in the history

  4. lint

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    7855310 View commit details
    Browse the repository at this point in the history

  5. lint

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    ab424d5 View commit details
    Browse the repository at this point in the history

  6. minor

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    cac6e51 View commit details
    Browse the repository at this point in the history

  7. switch to m-major MMA for sm70

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    945214b View commit details
    Browse the repository at this point in the history

  8. recognize GPTQ models

    @lzhangzz
    lzhangzz committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    d43becb View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. RTX 2080 & GTX 1660

    @lzhangzz
    lzhangzz committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    1110fa9 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. fix missing return

    @lzhangzz
    lzhangzz committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    1eb0585 View commit details
    Browse the repository at this point in the history

  2. fix cu12 build for sm90

    @lzhangzz
    lzhangzz committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    a11a580 View commit details
    Browse the repository at this point in the history

  3. fix ptr of operand C

    @lzhangzz
    lzhangzz committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    51d85be View commit details
    Browse the repository at this point in the history

  4. disable cache eviction policy on sm_90

    @lzhangzz
    lzhangzz committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    5a1711a View commit details
    Browse the repository at this point in the history

  5. fix lint

    @lzhangzz
    lzhangzz committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    8404e22 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2024

  1. add refs

    @lzhangzz
    lzhangzz committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    d71edd5 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. fix lint

    @lzhangzz
    lzhangzz committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    50454b0 View commit details
    Browse the repository at this point in the history

  2. lint

    @lzhangzz
    lzhangzz committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    fcdc6cb View commit details
    Browse the repository at this point in the history