[C++][Compute] Implement non-hash count_distinct aggregate kernel

[asfimport]

ARROW-12728 added a hash_count_distinct hash aggregate kernel, but there is no non-hash count_distinct aggregate kernel.

Reporter: Ian Cook / @ianmcook
Assignee: Percy Camilo Triveño Aucahuasi / @aucahuasi

Related issues:

• [C++][Compute] Implement count_distinct/distinct hash aggregate kernels (relates to)
• [R] Binding for n_distinct() with no grouping (is depended upon by)

PRs and other links:

• GitHub Pull Request #11257

_{Note: This issue was originally created as ARROW-14035. Please see the migration documentation for further details.}

[asfimport]

Percy Camilo Triveño Aucahuasi / @aucahuasi:
Can you please elaborate more about this requirement?

1. Do we need to compute the same thing of hash_count_distinct but without using the hash table from the hash group?

2. Are we going to offer non hash version for all hash_x functions too? (hash_distinct, hash_count, hash_sum)

   cc @ianmcook  @lidavidm

[asfimport]

Ian Cook / @ianmcook:

1. Do we need to compute the same thing of hash_count_distinct but without using the hash table from the hash group?
   Yes. @lidavidm  if I am missing any nuance here, please let me know :)
   Are we going to offer non hash version for all hash_x functions too? (hash_distinct, hash_count, hash_sum)
   Yes I think we should aim for that (or nearly that; there might be a few exceptions where it does not make sense.) Comparing the lists of aggregation functions and hash (grouped) aggregation functions in compute.rst, they are mostly the same already, with just a few differences. I think this issue and ARROW-13309 are the most important two additions to bring these two lists closer to parity.

[asfimport]

Percy Camilo Triveño Aucahuasi / @aucahuasi:
Thanks @ianmcook, another question: What is the difference between value_counts and count_distinct?

https://github.com/apache/arrow/blob/master/docs/source/cpp/compute.rst#associative-transforms

 

[asfimport]

David Li / @lidavidm:
value_counts gives you a histogram where the x-axis are the distinct values and the y-axis is the number of occurrences of that value. count_distinct is  just COUNT(DISTINCT *).

Also, value_counts is a vector kernel whereas this should be a scalar aggregate kernel.

[asfimport]

Percy Camilo Triveño Aucahuasi / @aucahuasi:
Thanks David!

[asfimport]

Percy Camilo Triveño Aucahuasi / @aucahuasi:
Draft PR #11257

[asfimport]

Percy Camilo Triveño Aucahuasi / @aucahuasi:
Related https://issues.apache.org/jira/browse/ARROW-14158

[asfimport]

Neal Richardson / @nealrichardson:
Issue resolved by pull request 11257
#11257