This blog post proclaims the Python package “ChernoffFace” and outlines and exemplifies its function chernoff_face that generates Chernoff diagrams.
The design, implementation strategy, and unit tests closely resemble the Wolfram Repository Function (WFR) ChernoffFace, [AAf1], and the original Mathematica package “ChernoffFaces.m”, [AAp1].
This post proclaims and briefly describes the Python package, SparseMatrixRecommender, which has different functions for computations of recommendations based on (user) profile or history using Sparse Linear Algebra (SLA). The package mirrors the Mathematica implementation [AAp1]. (There is also a corresponding implementation in R; see [AAp2]).
The package is based on a certain “standard” Information retrieval paradigm — it utilizes Latent Semantic Indexing (LSI) functions like IDF, TF-IDF, etc. Hence, the package also has document-term matrix creation functions and LSI application functions. I included them in the package since I wanted to minimize the external package dependencies.
The package includes two data-sets dfTitanic and dfMushroom in order to make easier the writing of introductory examples and unit tests.
Remark: “SMR” stands for “Sparse Matrix Recommender”. Most of the operations of this Python package mirror the operations of the software monads “SMRMon-WL”, “SMRMon-R”, [AAp1, AAp2].
Workflows
Here is a diagram that encompasses the workflows this package supports (or will support):
Here is narration of a certain workflow scenario:
Get a dataset.
Create contingency matrices for a given identifier column and a set of “tag type” columns.
Examine recommender matrix statistics.
If the assumptoins about the data hold apply LSI functions.
For example, the “usual trio” IDF, Frequency, Cosine.
Do (verify) example profile recommendations.
If satisfactory results are obtained use the recommender as a nearest neighbors classifier.
Monadic design
Here is a diagram of typical pipeline building using a SparseMatrixRecommender object:
Remark: The monadic design allows “pipelining” of the SMR operations — see the usage example section.
This package is based on the Python package SSparseMatrix, [AAp5].
The package LatentSemanticAnalyzer, [AAp6], uses the cross tabulation and LSI functions of this package.
Usage example
Here is an example of an SMR pipeline for creation of a recommender over Titanic data and recommendations for the profile “passengerSex:male” and “passengerClass:1st”:
Remark: More examples can be found the directory “./examples”.
Related Mathematica packages
The software monad Mathematica package “MonadicSparseMatrixRecommender.m” [AAp1], provides recommendation pipelines similar to the pipelines created with this package.
Here is a Mathematica monadic pipeline that corresponds to the Python pipeline above:
The package SMRMon-R, [AAp2], implements a software monad for SMR workflows. Most of SMRMon-R functions delegate to SparseMatrixRecommender.
The package SparseMatrixRecommenderInterfaces, [AAp3], provides functions for interactive Shiny interfaces for the recommenders made with SparseMatrixRecommender and/or SMRMon-R.
The package LSAMon-R, [AAp4], can be used to make matrices for SparseMatrixRecommender and/or SMRMon-R.
Here is the SMRMon-R pipeline that corresponds to the Python pipeline above:
The project repository “Scalable Recommender Framework”, [AAr1], has documents, diagrams, tests, and benchmarks of a recommender system implemented in multiple programming languages.
This Python recommender package is a decisive winner in the comparison — see the first 10 min of the video recording [AAv1] or the benchmarks at [AAr1].
Code generation with natural language commands
Using grammar-based interpreters
The project “Raku for Prediction”, [AAr2, AAv2, AAp6], has a Domain Specific Language (DSL) grammar and interpreters that allow the generation of SMR code for corresponding Mathematica, Python, R, and Raku packages.
Here is Command Line Interface (CLI) invocation example that generate code for this package:
> ToRecommenderWorkflowCode Python 'create with dfTitanic; apply the LSI functions IDF, None, Cosine;recommend by profile 1st and male'
obj = SparseMatrixRecommender().create_from_wide_form(data = dfTitanic).apply_term_weight_functions(global_weight_func = "IDF", local_weight_func = "None", normalizer_func = "Cosine").recommend_by_profile( profile = ["1st", "male"])
NLP Template Engine
Here is an example using the NLP Template Engine, [AAr2, AAv3]:
Concretize["create with dfTitanic; apply the LSI functions IDF, None, Cosine;recommend by profile 1st and male",
"TargetLanguage" -> "Python"]
(*
"smrObj = (SparseMatrixRecommender()
.create_from_wide_form(data = None, item_column_name=\"id\", columns=None, add_tag_types_to_column_names=True, tag_value_separator=\":\")
.apply_term_weight_functions(\"IDF\", \"None\", \"Cosine\")
.recommend_by_profile(profile=[\"1st\", \"male\"], nrecs=profile)
.join_across(data=None, on=\"id\")
.echo_value())"
*)
Python for Prediction 
