Skip to content

Conversation

@ggerganov
Copy link
Member

The precision for Q4_0 has degraded since #1508
Use Q4_1 instead

ref #1551

The precision for Q4_0 has degraded since #1508
Copy link
Contributor

@stsydow stsydow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested for Q5.1 and it seem to work as well.

@cebtenzzre
Copy link
Collaborator

Could we merge this? I was going to fix the use of abs instead of std::abs in the delta calculation and add a few missing static qualifiers, but there's no point if the example doesn't even run successfully.

@ggerganov ggerganov merged commit d119c04 into master Sep 20, 2023
@ggerganov ggerganov deleted the fix-benchmark-matmult branch September 20, 2023 07:02
pkrmf pushed a commit to morlockstudios-com/llama.cpp that referenced this pull request Sep 26, 2023
The precision for Q4_0 has degraded since ggml-org#1508
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants