Are there any plans to continue maintenance of the library?

[sofeikov]

Hi there,

I’ve recently been using the library in some non-standard settings (e.g., adapting it to generate counterfactual explanations for hetero GNNs). It’s been a fun exercise, but I did run into a few quirks along the way, which I’ve had to patch ad hoc.

I’ve considered submitting a PR, but it looks like the project may no longer be actively maintained. I’m debating whether it makes more sense to fork it instead.

In particular, I’d love to see support for pandas>=2.0.0, as well as compatibility with newer numpy versions and a few algorithmic improvements.

Appreciate your time,
Konstantin

[amit-sharma]

@sofeikov thanks for your message. I haven't been able to find time recently, but yes, very much committed to maintaining the library. Your proposed extensions sound very interesting.

A new release has been pending for a while. Can you share the requirements apart from pandas? I can take a look and aim to release a version based on it.
Adding @gaugup

[sofeikov]

@amit-sharma thanks for getting back to me!

Yeah, more modern numpy and pandas would be a great start, just so it can better support more modern workflows and repos.

Apart from that, there was this thing with gradient methods. First of all, it seems to be ignoring user set thresholds by doing this in dice_pytorch.py

        self.stopping_threshold = stopping_threshold
        if self.target_cf_class == 0 and self.stopping_threshold > 0.5:
            self.stopping_threshold = 0.25
        elif self.target_cf_class == 1 and self.stopping_threshold < 0.5:
            self.stopping_threshold = 0.75

So whatever arbitrary threshold I pass into it, it gets overridden. I will explain why this might be important below.

After that the best candidates are cached based on their proximity to the requested threshold

                if ((self.target_cf_class == 0 and all(i <= self.stopping_threshold for i in test_preds_stored)) or
                   (self.target_cf_class == 1 and all(i >= self.stopping_threshold for i in test_preds_stored))):
                    avg_preds_dist = np.mean([abs(pred[0]-self.stopping_threshold) for pred in test_preds_stored])
                    if avg_preds_dist < self.min_dist_from_threshold[loop_ix]:
                        self.min_dist_from_threshold[loop_ix] = avg_preds_dist
                        for ix in range(self.total_CFs):
                            self.best_backup_cfs[loop_ix+ix] = copy.deepcopy(temp_cfs_stored[ix])
                            self.best_backup_cfs_preds[loop_ix+ix] = copy.deepcopy(test_preds_stored[ix])

While I understand that this is done in the spirit of obtaining minimally viable CF examples, that just pass the threshold, supposedly going through minimal changes, I think there might be a place for a couple of more things:

1. Optionally, target the responses that gave the largest probability for the desired class(as opposed to those closest to the threshold)
2. Optionally, store and return best-effort examples, if reaching threshold is impossible

The reason I think all of the above is useful is the following one. Sometimes classifiers are used to assign probabilities of certain events and their softmax outputs are used as such, for example during a Monte Carlo simulation. In this case it might desirable to answer the following questions:

1. what changes do I need to make to increase probability of a desired outcome by, say, 7%. I do not think it is currently possible. This would be an example of arbitrary threshold usage.
2. What is the most I can squeeze out of this particular outcome probability, given the loss penalty constraints? This is an example of best effort CFs.

With this it would make it possible to use Dice in more probabilistic settings, beyond those where just passing a threshold is important.

Dice already has all the needed machinery to help with those and in my experience, those are very real practical applications.

Let me know if you’d be open to these ideas or if I can help in any way.

[amit-sharma]

Thanks for these detailed comments @sofeikov Yes, these suggestions make sense. I also like your proposals below.

what changes do I need to make to increase probability of a desired outcome by, say, 7%. I do not think it is currently possible. This would be an example of arbitrary threshold usage.
What is the most I can squeeze out of this particular outcome probability, given the loss penalty constraints? This is an example of best effort CFs.

As I said, my bandwidth is limited, so it will be great if you can contribute and issue PRs for these extensions. Meanwhile, let me work on getting a release with support for the latest numpy and pandas.

[sofeikov]

@amit-sharma awesome, I could contribute a couple of those small changes, one by one, once there is a new release with newer pandas and numpy. Appreciate a quick turn around 👍

I'm subscribed to the releases notifications, so once it's out, i will know

[sofeikov]

Btw, another thing I was thinking about was to implement a possibility to prohibit feature increase/decrease. For example, if you working with someone's skill level, counterfactuals telling a skill needs to be decreased does not make sense, but it might be desirable to recommend an increase there.

I guess it could be implemented through an additional loss term, penalising non-symmetric deviations from an additional skill lower/upper bound.

It currently does not seem to be possible, as the private data interface treats feature ranges as those reflecting their valid range, not permitted range for optimisation. Let me know if I'm wrong.

Let me know what you think!

[amit-sharma]

Technically, the range of the private data interface can be used to enforce the range over which the CFs should be generated, but yeah, an explicit variable will be better.

[amit-sharma]

@sofeikov I have updated the library code to support py3.12 and latest version of pandas and numpy. Can you take a look and try out the code in the main branch? If everything looks good, I can release a new version of DiCE in the next couple of days.

[sofeikov]

@amit-sharma lgtm! Would be good to get it released. The only thing I was thinking of, if it will be easier to require tensorflow >= 2.0, because anything below I think it super dated, but then again I have not used TF in years, so not really sure as to the state of that ecosystem.

[amit-sharma]

for backwards compatibility, I thought it may be good in case someone is using tf1. The code supports both versions, so it should be okay.

[jlopezpena]

Do people still use TF1? Last version in the 1.x branch only has wheels for python <= 3.7 which has been EOL'd for over a year

[amit-sharma]

v0.12 is now released! https://github.com/interpretml/DiCE/releases/tag/v0.12

look forward to your contributions @sofeikov

@jlopezpena will now work to remove tf1 and other older dependencies for the next release.