Compile bug: convert.cu

[pwilkin]

Git commit

221c0e0

Operating systems

Linux

GGML backends

CUDA

Problem description & steps to reproduce

My daily compilation of llama.cpp from newest source failed on convert.cu. Can confirm reverting to 18f3b5f fixes it.

First Bad Commit

07a19e2

Compile command

cmake -B build -DGGML_CUDA=ON -DGGML_CUDA_FORCE_CUBLAS=OFF -DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_CUDA_F16=ON -DGGML_BLAS=ON -DGGML_BLAS_VEND
OR=OpenBLAS
cmake --build build --config Release -j 8

Relevant log output

[  9%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-f16-instance-ncols1_32-ncols2_1.cu.o
/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(34): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + 0] = v.x;
                 ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 764

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(35): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + y_offset] = v.y;
                        ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 764

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(34): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + 0] = v.x;
                 ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 766

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(35): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + y_offset] = v.y;
                        ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 766

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(34): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + 0] = v.x;
                 ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 768

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(35): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + y_offset] = v.y;
                        ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 768

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(34): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + 0] = v.x;
                 ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 770

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(35): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + y_offset] = v.y;
                        ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 770

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(34): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + 0] = v.x;
                 ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 772

/devel/tools/llama.cpp/ggml/src/ggml-cuda/convert.cu(35): error: more than one operator "=" matches these operands:
            function "__nv_bfloat16::operator=(float)" (declared at line 305 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(short)" (declared at line 530 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned short)" (declared at line 534 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(int)" (declared at line 538 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned int)" (declared at line 542 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(long long)" (declared at line 546 of /usr/include/cuda_bf16.hpp)
            function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 550 of /usr/include/cuda_bf16.hpp)
            operand types are: nv_bfloat16 = __half
      y[iy0 + y_offset] = v.y;
                        ^
          detected during:
            instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 474
            instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 772

[pwilkin]

Oh yeah, this might be relevant too:

ilintar@LinuksowaJaskinia:/devel/tools/llama.cpp$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0

[Calorado]

This same compilation error also happens on Windows with CUDA 12.8 and Clang 19.1 in Visual Studio

[EverchangerL]

Same issue on Windows with the latest CUDA / MSVC / Windows SDK

F:\LLM\llama.cpp>cmake --build build --config Release -j 6
MSBuild 17.14.14+a129329f1 for .NET Framework

  Compiling CUDA source file ..\..\..\..\ggml\src\ggml-cuda\convert.cu...

  F:\LLM\llama.cpp\build\ggml\src\ggml-cuda>"F:\LLM\Apps\Cuda\bin\nvcc.exe"  --use-local-env -ccbin "C:\
  Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\bin\HostX64\x64" -x cu
    -I"F:\LLM\llama.cpp\ggml\src\ggml-cuda\.." -IF:\LLM\llama.cpp\ggml\src\..\include -IF:\LLM\Apps\Cuda
  \include -IF:\LLM\Apps\Cuda\include     --keep-dir ggml-cuda\x64\Release -use_fast_math -maxrregcount=
  0    --machine 64 --compile -cudart static -std=c++17 -arch=native -extended-lambda -compress-mode=siz
  e -Xcompiler="/EHsc -Ob2"   -D_WINDOWS -DNDEBUG -DGGML_BACKEND_BUILD -DGGML_BACKEND_SHARED -D_CRT_SECU
  RE_NO_WARNINGS -DGGML_SCHED_MAX_COPIES=4 -D_XOPEN_SOURCE=600 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGGM
  L_CUDA_USE_GRAPHS -DGGML_CUDA_F16 -DGGML_SHARED -D"CMAKE_INTDIR=\"Release\"" -Dggml_cuda_EXPORTS -D_WI
  NDLL -D_MBCS -D"CMAKE_INTDIR=\"Release\"" -Dggml_cuda_EXPORTS -Xcompiler "/EHsc /W1 /nologo /O2 /FS
  /MD /GR" -Xcompiler "/Fdggml-cuda.dir\Release\vc143.pdb" -o ggml-cuda.dir\Release\convert.obj "F:\LLM\
  llama.cpp\ggml\src\ggml-cuda\convert.cu"
F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(34): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + 0]        = v.x;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q4_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 764

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(35): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + y_offset] = v.y;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q4_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q4_0, dst_t=nv_bfloat16]" at line 764

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(34): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + 0]        = v.x;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q4_1, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 766

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(35): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + y_offset] = v.y;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q4_1, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q4_1, dst_t=nv_bfloat16]" at line 766

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(34): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + 0]        = v.x;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q5_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 768

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(35): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + y_offset] = v.y;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q5_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q5_0, dst_t=nv_bfloat16]" at line 768

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(34): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + 0]        = v.x;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q5_1, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 770

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(35): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + y_offset] = v.y;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=2, dequantize_kernel=dequant
  ize_q5_1, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  2, dequantize_kernel=dequantize_q5_1, dst_t=nv_bfloat16]" at line 770

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(34): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + 0]        = v.x;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=1, dequantize_kernel=dequant
  ize_q8_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 772

F:\LLM\llama.cpp\ggml\src\ggml-cuda\convert.cu(35): error : more than one operator "=" matches these ope
rands: [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
              function "__nv_bfloat16::operator=(float)" (declared at line 113 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(short)" (declared at line 191 of F:\LLM\Apps\Cuda\inclu
  de\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned short)" (declared at line 192 of F:\LLM\Apps\C
  uda\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(int)" (declared at line 193 of F:\LLM\Apps\Cuda\include
  \cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned int)" (declared at line 194 of F:\LLM\Apps\Cud
  a\include\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(long long)" (declared at line 195 of F:\LLM\Apps\Cuda\i
  nclude\cuda_bf16.hpp)
              function "__nv_bfloat16::operator=(unsigned long long)" (declared at line 196 of F:\LLM\Ap
  ps\Cuda\include\cuda_bf16.hpp)
              operand types are: nv_bfloat16 = __half
        y[iy0 + y_offset] = v.y;
                          ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(__nv_bflo
  at16 &&)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:/LLM/Apps/Cuda/include\cuda_bf16.h(4432): note #3326-D: function "__nv_bfloat16::operator=(const __n
  v_bfloat16 &)" does not match because argument #1 does not match parameter
    struct __declspec(align(2)) __nv_bfloat16 {
                                ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(107): note #3326-D: function "__nv_bfloat16::operator=(const vo
  latile __nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const volatile __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(106): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &) volatile" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline volatile __nv_bfloat16 &__nv_bfloat16::operat
  or=(const __nv_bfloat16_raw &hr) volatile { __x = hr.x; return *this; }
                                                                                                  ^
  F:\LLM\Apps\Cuda\include\cuda_bf16.hpp(105): note #3326-D: function "__nv_bfloat16::operator=(const __
  nv_bfloat16_raw &)" does not match because argument #1 does not match parameter
        __declspec(__host__) __declspec(__device__) inline __nv_bfloat16 &__nv_bfloat16::operator=(const
   __nv_bfloat16_raw &hr) { __x = hr.x; return *this; }
                                                                                         ^
            detected during:
              instantiation of "void dequantize_block<qk,qr,dequantize_kernel,dst_t>(const void *, dst_t
   *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t) [with qk=32, qr=1, dequantize_kernel=dequant
  ize_q8_0, dst_t=nv_bfloat16]" at line 474
              instantiation of "void dequantize_block_cuda<qk,qr,dequantize_kernel,dst_t>(const void *,
  dst_t *, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, cudaStream_t) [with qk=32, qr=
  1, dequantize_kernel=dequantize_q8_0, dst_t=nv_bfloat16]" at line 772

  10 errors detected in the compilation of "F:/LLM/llama.cpp/ggml/src/ggml-cuda/convert.cu".
  convert.cu
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\CU
DA 12.9.targets(801,9): error MSB3721: выход из команды ""F:\LLM\Apps\Cuda\bin\nvcc.exe"  --use-local-en
v -ccbin "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\bin\HostX64\
x64" -x cu   -I"F:\LLM\llama.cpp\ggml\src\ggml-cuda\.." -IF:\LLM\llama.cpp\ggml\src\..\include -IF:\LLM\
Apps\Cuda\include -IF:\LLM\Apps\Cuda\include     --keep-dir ggml-cuda\x64\Release -use_fast_math -maxrre
gcount=0    --machine 64 --compile -cudart static -std=c++17 -arch=native -extended-lambda -compress-mod
e=size -Xcompiler="/EHsc -Ob2"   -D_WINDOWS -DNDEBUG -DGGML_BACKEND_BUILD -DGGML_BACKEND_SHARED -D_CRT_S
ECURE_NO_WARNINGS -DGGML_SCHED_MAX_COPIES=4 -D_XOPEN_SOURCE=600 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGG
ML_CUDA_USE_GRAPHS -DGGML_CUDA_F16 -DGGML_SHARED -D"CMAKE_INTDIR=\"Release\"" -Dggml_cuda_EXPORTS -D_WIN
DLL -D_MBCS -D"CMAKE_INTDIR=\"Release\"" -Dggml_cuda_EXPORTS -Xcompiler "/EHsc /W1 /nologo /O2 /FS   /MD
 /GR" -Xcompiler "/Fdggml-cuda.dir\Release\vc143.pdb" -o ggml-cuda.dir\Release\convert.obj "F:\LLM\llama
.cpp\ggml\src\ggml-cuda\convert.cu"" с кодом "1". [F:\LLM\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.v
cxproj]