Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 SparseAssociation

Source Notebook

Create a rectangular data structure that behaves like a SparseArray indexed by string keys

Contributed by: Sjoerd Smit

ResourceFunction["SparseAssociation"][array,{key1,key2,…}]

creates a data structure in which an element at position {i,j,…} can be accessed with the string key sequence {keyi, keyj, …}.

ResourceFunction["SparseAssociation"][array,{{key1,1,key1,2,…},{key2,1,…},…}]

creates a data stucture that uses the keys keyi,j at level j of the array.

ResourceFunction["SparseAssociation"][array,keys,default]

uses default as the default value for the underlying SparseArray.

ResourceFunction["SparseAssociation"][{keys1→val1,keys2→val2,…},Automatic,…]

specifies the data from array rules.

ResourceFunction["SparseAssociation"][{keys1,keys2,…}→ {val1,val2,…},Automatic,…]

can also be used.

ResourceFunction["SparseAssociation"][assoc,Automatic,…]

specifies the data from a nested Association.

Details and Options

ResourceFunction["SparseAssociation"] acts like a rectangular data structure of nested associations but is more memory efficient because the keys are not duplicated at each level. In addition, ResourceFunction["SparseAssociation"] offers the possibility to specify beforehand what default value should be returned when trying to access empty parts of the array, whereas Association returns Missing[…] in that situation.
ResourceFunction["SparseAssociation"] is a constructor-container function like SparseArray and Association. A ResourceFunction["SparseAssociation"] can be constructed from a regular array that satisfies ArrayQ, a nested Association or a list of rules of the form {{keyi,1,keyi,2,…} → valuei,…} in the same way as a SparseArray.
Only string keys are supported currently.
The underlying SparseArray can be extracted with Values. The keys used at different levels can be extracted with Keys. The function Normal converts the data to a nested Association.
A correctly constructed ResourceFunction["SparseAssociation"] matches ResourceFunction["SparseAssociation"][_Association?(#["ValidatedQ"]&)].
If a ResourceFunction["SparseAssociation"] is created from a nested Association, all Missing[…] values at any level will be dropped and replaced with the default value. The corresponding keys will still be included.
Like SparseArray, default is 0 if not specified.
A ResourceFunction["SparseAssociation"] can be converted to SparseArray with Values, to a list of rules with ArrayRules and to a Dataset of nested associations with Dataset.
Dataset[sparseAssoc, "IncludeDefaultValues"→False] can be used to create a ragged Association with all of the default values dropped. This also works with Normal.
A ResourceFunction["SparseAssociation"] can be queried like an Association using sparseAssoc[key1,key2,…] or using Part.
Accessing elements with Part can be done using string keys or integer indices.
Map is supported but only for operations that leave the dimensions of the array unchanged. For example, mapping Total over a matrix is currently not supported.

Examples

Basic Examples (14) 

Convert a regular array into a SparseAssociation:

In[1]:=
array = BlockRandom[ SeedRandom[1]; RandomInteger[2, {4, 3, 2}] ]
Out[1]=
Image
In[2]:=
sparseAssoc = ResourceFunction["SparseAssociation"][array, {"a", "b", "c", "d"}]
Out[2]=
Image

Extract elements from it:

In[3]:=
sparseAssoc["d", "b", "a"]
Out[3]=
Image

This is equivalent to:

In[4]:=
array[[4, 2, 1]]
Out[4]=
Image

Unlike Association, you cannot extract keys that haven’t been specified beforehand:

In[5]:=
sparseAssoc["e", "f", "g"]
Image
Out[5]=
Image

SparseAssociation works with Part:

In[6]:=
sparseAssoc[["d", "b", "a"]]
Out[6]=
Image

Extracting multiple elements with Part yields another SparseAssociation of reduced size:

In[7]:=
sparseAssoc[[{"a", "c"}, All, "a"]]
Out[7]=
Image

Indexing by position is also possible:

In[8]:=
sparseAssoc[[{1, 3}, All, 1]]
Out[8]=
Image

Convert the SparseAssociation to a SparseArray:

In[9]:=
Values[sparseAssoc]
Out[9]=
Image

Further convert the SparseArray to a nested list using Normal:

In[10]:=
Normal[%]
Out[10]=
Image

Convert the SparseAssociation to array rules:

In[11]:=
ArrayRules[sparseAssoc]
Out[11]=
Image

Convert the sparse association directly to a nested Association:

In[12]:=
Normal[sparseAssoc]
Out[12]=
Image

Convert it to a ragged dataset with the default elements omitted:

In[13]:=
Dataset[sparseAssoc, "IncludeDefaultValues" -> False]
Out[13]=
Image

Convert the sparse association to a nested Association with the default elements omitted:

In[14]:=
Normal[sparseAssoc, "IncludeDefaultValues" -> False]
Out[14]=
Image

Reassemble the SparseAssociation from the nested Association (up to permutation of the keys):

In[15]:=
ResourceFunction["SparseAssociation"][%, Automatic]
Out[15]=
Image

Scope (6) 

Construct a SparseAssociation from a SparseArray:

In[16]:=
ResourceFunction["SparseAssociation"][ SparseArray[{{1, 1} -> 1, {2, 2} -> 2, {3, 3} -> 3, {1, 3} -> 4}], {"a", "b", "c"} ]
Out[16]=
Image

Define a SparseAssociation using an array of rules:

In[17]:=
rules = {{"a", "a"} -> 1, {"a", "c"} -> 4, {"b", "b"} -> 2, {"c", "c"} -> 3}; ResourceFunction["SparseAssociation"][rules, Automatic]
Out[18]=
Image

Use Keys to extract the keys used at the different levels. The ordering of the keys depends on their appearance in the rules:

In[19]:=
Keys[%]
Out[19]=
Image

Specify the ordering of the keys in advance:

In[20]:=
Keys@ResourceFunction["SparseAssociation"][rules, {"a", "b", "c"}]
Out[20]=
Image

Specify different orderings at different levels:

In[21]:=
Keys@ResourceFunction["SparseAssociation"][ rules, {{"a", "b", "c"}, {"c", "b", "a"}}]
Out[21]=
Image

Use a default element other than 0 and specify keys that do not appear in the list of rules:

In[22]:=
ResourceFunction[ "SparseAssociation"][{"a" -> 2}, {"a", "b", "c", "d"}, 1]
Out[22]=
Image

Keys without a specified value will yield the default value:

In[23]:=
%["c"]
Out[23]=
Image

Specify different keys at different levels:

In[24]:=
sparseAssoc = ResourceFunction[ "SparseAssociation"][{{"a", "f"} -> 2}, {{"a", "b", "c", "d"}, {"e", "f", "g"}}]
Out[24]=
Image

The functions Length, Dimensions, ArrayDepth, MatrixQ, VectorQ and ArrayQ are supported:

In[25]:=
sparseAssoc = ResourceFunction[ "SparseAssociation"][{{"a", "d"} -> 2}, {"a", "b", "c", "d"}]
Out[25]=
Image
In[26]:=
Through[{Length, Dimensions, ArrayDepth, MatrixQ, VectorQ, ArrayQ}@ sparseAssoc ]
Out[26]=
Image

Map is supported for operations that preserve the dimensions of the array:

In[27]:=
Map[f, sparseAssoc, {2}]
Out[27]=
Image
In[28]:=
Normal@Values[%]
Out[28]=
Image

Construct a SparseAssociation from nested associations:

In[29]:=
ResourceFunction[ "SparseAssociation"][<|"a" -> <|"f" -> 1|>, "b" -> <|"f" -> 2, "g" -> 3|>|>, Automatic]
Out[29]=
Image

Missing values at any level will be dropped, but the keys will be retained:

In[30]:=
ResourceFunction[ "SparseAssociation"][<|"a" -> <|"f" -> 1, "g" -> Missing[]|>, "b" -> <|"f" -> 2, "h" -> 3|>, "c" -> Missing[]|>, Automatic]
Out[30]=
Image
In[31]:=
Keys[%]
Out[31]=
Image

Applications (1) 

A SparseAssociation is more memory efficient than a nested Association because the keys do not have to be stored multiple times on each level:

In[32]:=
sparseAssoc = BlockRandom[SeedRandom[1]; ResourceFunction["SparseAssociation"][ RandomVariate[BernoulliDistribution[1/10], {26, 26, 26}], Alphabet[]] ]
Out[32]=
Image
In[33]:=
{ ByteCount[sparseAssoc], ByteCount[ Normal@sparseAssoc], (* Full nested associations that store the default values as well *) ByteCount[ Normal[sparseAssoc, "IncludeDefaultValues" -> False]] (* Nested associations that only store the 1's *) }
Out[33]=
Image

Properties and Relations (4) 

A SparseAssociation can be reconstructed from its array rules and its keys:

In[34]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ RandomInteger[5, {4, 2}], {{"a", "b", "c", "d"}, {"e", "f", "g"}}, 1]
Out[34]=
Image
In[35]:=
ResourceFunction["SparseAssociation"][ArrayRules[sparseAssoc], Keys[sparseAssoc]] === sparseAssoc
Out[35]=
Image

It can also be reconstructed from its underlying SparseArray and its keys:

In[36]:=
ResourceFunction["SparseAssociation"][Values[sparseAssoc], Keys[sparseAssoc]] === sparseAssoc
Out[36]=
Image

It can be reconstructed from the corresponding nested Association if the correct default value is provided:

In[37]:=
ResourceFunction["SparseAssociation"][Normal[sparseAssoc], Keys[sparseAssoc], 1] === sparseAssoc
Out[37]=
Image

Test if a SparseAssociation has been constructed correctly:

In[38]:=
MatchQ[sparseAssoc, ResourceFunction[ "SparseAssociation"][_Association?(#["ValidatedQ"] &)] ]
Out[38]=
Image

Possible Issues (4) 

It is not possible to mix keys and position specifications at one level:

In[39]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ RandomInteger[5, {4, 3}], {"a", "b", "c", "d"}]; sparseAssoc[[{"a", 2}]]
Image
Out[40]=
Image

This is analogous to the same limitation of Association:

In[41]:=
<|"a" -> 1, "b" -> 2|>[[{"a", 2}]]
Image
Out[41]=
Image

Use only strings or only integers instead:

In[42]:=
sparseAssoc[[{"a", "b"}]]
Out[42]=
Image
In[43]:=
sparseAssoc[[{1, 2}]]
Out[43]=
Image

At different levels, different specifications can be used:

In[44]:=
sparseAssoc[[{1, 2}, {"a", "b"}]]
Out[44]=
Image

Neat Examples (3) 

Excel files often have row and column headers. SparseAssociation provides an easy mechanism to create a lookup table from such data:

In[45]:=
sheet = Import["ExampleData/elements.xls", {"Data", 1}][[ All, {2, 1, 3, 4}]]; sheet // TableForm
Out[46]=
Image
In[47]:=
sparseAssoc = ResourceFunction["SparseAssociation"][ sheet[[2 ;;, 2 ;;]], { sheet[[2 ;;, 1]], (* row headers *) sheet[[1, 2 ;;]] (* column headers *) } ]
Out[47]=
Image

The data can now be queried like an Association:

In[48]:=
sparseAssoc["He", "AtomicWeight"]
Out[48]=
Image

Convert to a Dataset:

In[49]:=
Dataset@sparseAssoc[[{"H", "N"}, {"Name", "AtomicNumber"}]]
Out[49]=
Image

Publisher

Sjoerd Smit

Version History

  • 3.0.0 – 01 October 2020
  • 2.0.0 – 24 August 2020
  • 1.0.0 – 26 March 2020

Related Resources

License Information