Counterfactual explanation at will, with zero privacy leakage

While counterfactuals have been extensively studied as an intuitive explanation of model predictions, they still have limited adoption in practice due to two obstacles: (a) They rely on excessive access to the model for explanation that the model owner may not provide; and (b) counterfactuals carry information that adversarial users can exploit to launch model extraction attacks. To address the challenges, we propose CPC , a data-driven approach to counterfactual. CPC works at the client side and gives full control and right-to-explain to model users, even when model owners opt not to. Moreover, CPC warrants that adversarial users cannot exploit counterfactuals to extract models. We formulate properties and fundamental problems underlying CPC , study their complexity and develop effective algorithms. Using real-world datasets and user study, we verify that CPC does prevent adversaries from exploiting counterfactuals for model extraction attacks, and is orders of magnitude faster than existing explainers, while maintaining comparable and often higher quality.


INTRODUCTION
Machine Learning (ML) is increasingly prevalent in decision-making across high-stakes fields such as financial analysis [39], healthcare diagnosis [63] and policy making [27].This increase underscores the importance and necessity of interpreting complex ML models, to ensure that stakeholders can thoroughly understand model decisions before taking any actions [74,85,96].One particularly useful interpretation is counterfactual explanation [16,31,38,85,93,114].
Counterfactual explanation indicates how features of a given instance need to be changed to achieve a different outcome, usually from an undesired prediction to a desired one [54,85,113].It has been heavily studied [17,54,88,109] as it offers actionable and plausible feedback to the customers.
Example 1: Banks often collaborate with third-party companies e.g., Upstart [9] and ZestAI [10] that offer ML-based credit assessment and decision-making of loan applications from their customers.The ML model  is owned by the third party and considers features of customers of the bank, e.g., income, credit score (cscore), request loan amount (lamount).As an ML service user, the bank submits its customer Alice's information -income ($5K), cscore (fair), lamount ($100K) -to the third party.However, her loan is unfortunately denied by .To assist the bank in helping Alice make the necessary adjustments needed for approval, the third party may provide a counterfactual   and x ′  : x ′  suggests an increase in income by $1K and an improvement in cscore to 'good', while alternatively x ′  proposes to reduce lamount from $100K to $20K.□ Open challenges.While there has been a host of work on computing high-quality counterfactual [31,37,38,64,70,71], there are however open challenges that impede their applications.
Challenge I: Right-to-explain.Not all ML systems, in particular those hosted in the cloud as a remote ML service [6,110,117], support counterfactual explanation.Instead, it demands both dedicated cooperation from model owners and engineering effort from service providers.More specifically, to compute counterfactual explanations for a given instance x, existing approaches [52,89,114] work by generating multiple candidate instances close to x and invoke the model to predict on them, to find out those that are valid counterfactual explanations for x.This relies on multiple additional accesses to the model other than the inference on x, which is completely at the discretion of the model owners and service providers.As a result, not all ML systems and services support counterfactual explanations, failing to uphold "the right to explanation" requested by the European Union General Data Protection Regulation (GDPR) [80,104].
[Open question].A pressing ethical open question is whether we can allow all the users of ML services to exercise their right to access counterfactual explanations for arbitrary ML models, at all times, even when the model owners or service providers opt not to?Challenge II: Privacy leakage.Another factor that hinders the adoption of counterfactual explanation in especially those remote ML services is the associated privacy leakage risks, as already observed in prior studies [12,90,105,117].Counterfactual explanations are essentially purposefully generated instances and their predictions.Hence, they carry information that is not part of the model inference.As a result, providing them to the users of the ML service can unintentionally reveal sensitive information about the ML model and the training set that the model owners may want to protect.
Example 2: Continue with Example 1, the third-party companies typically aggregate customer information from various banks to form a more extensive training set for developing a high-performing loan assessment model .In such scenario, it is crucial for the third-party to ensure 's privacy [26,57] against potential misuse by the other malicious ML service users, e.g., rival banks.Counterfactuals x ′  and x ′  generated in Example 1 may come from the entire feature space or even directly from the training set of , unintentionally revealing sensitive information about  to the bank.□ user can reconstruct a surrogate model  ′ with a high fidelity to the original model , e.g., up to 98.6% for extracting a typical DNN model [12,117].
[Open question].Hence, a critical open question is whether there exists a counterfactual explanation that leaks no privacy information, thereby preventing model extraction attacks over counterfactuals.
Client-centric privacy-preserving counterfactuals.To address the challenges, we present CPC (Client-centric Privacy-preserving Counterfactual explanation), a framework for computing counterfactual explanations with the following guarantees: (a) For model users, it permits access to counterfactual of their predictions at will, even when the model owners or ML service providers opt not to.(b) For model owners, it assures that the counterfactuals do not leak any information about the model and training data, precluding model extraction attacks over counterfactuals.
Guarded counterfactuals.The enabling idea of CPC is guarded counterfactuals.Unlike traditional counterfactuals that are artificially generated from the entire feature space and are "open-ended", guarded counterfactuals are from a designated universe U of instances and are "closed-ended".
Counterfactuals guarded by a privacy-preserving universe U would prevent adversaries from exploiting counterfactuals to extract ML models.A natural choice for such universe, denoted by U  , is inference instances that are predicted during model serving and are also those to be explained via counterfactuals.With U  , guarded counterfactuals strictly reveal nothing during explanation.
To make it work in practice, a fundamental problem is to compute guarded counterfactuals while optimizing their quality.We consider two popular quality measures of counterfactuals, namely diversity [85] and succinctness [54].We show that the problem is intractable.Indeed, optimizing either diversity or succinctness while bounding the other is already Np-Complete.Despite the intractability, we present efficient Ptime approximation algorithms with provable bounds, for both measures.
Client-centric explanation.In contrast to existing approaches that rely on the model owner to generate counterfactual explanations, CPC operates completely at the client side, controlled by the user that receives predictions from, e.g., a remotely hosted black-box ML model with restricted access.
To do this, CPC uses a client-side cache to collect inference instances as U I during model serving.It then computes counterfactuals guarded by U I .For those clients that do not have local memory to hold all the relevant inference instances or U I is an online stream, CPC can operate in a streaming explanation mode, under which it treats U I as a stream of instances coming online and it incrementally refines explanation as each new inference instance passes, without the need to store U I .
We develop two streaming algorithms for computing guarded counterfactuals in such a mode, while optimizing their diversity and succinctness, respectively; when optimizing one quality measure, we bound the other.For both cases, our algorithms are provably competitive when compared to the optimal batch explanation algorithm that knows the entire U I in advance (Theorem 4).
Effectiveness.We implement CPC and evaluate its effectiveness.We find the following.
(1) We first show that, with CPC, users can still have access to counterfactual explanation when using e.g., Amazon SageMaker [6] service that comes with no support for counterfactual explanation.This also applies to any other ML system, be it remote or local.
(2) We validate the benefit of guarded counterfactuals in defending against model extract attacks.By comparing with 4 state-of-the-art counterfactual methods over real-life datasets, explanations generated by CPC completely prevent model extraction attacks from exploiting counterfactuals, while existing methods contribute to 14.7% to 30.6% of privacy leakage exploited by the attacks.
(3) Despite unique benefits of (1) and ( 2), guarded counterfactuals produced by CPC are consistently comparable to and often better in quality than those of state-of-the-art methods, e.g., on average 35.5% higher in diversity and 39.5% more succinct over the 4 datasets.
(4) Furthermore, on average CPC is 205.7 times faster than existing counterfactual explanation methods, up to 3 orders of magnitude.
(5) In the streaming explanation mode, CPC takes only 1.7% of the memory cost of that in the batch explanation mode with just 17.8% and 14.5% of dip in diversity and succinctness, respectively.
Contributions.In summary, we make the following contributions: • We identify open challenges in counterfactual explanation and propose guarded counterfactuals as a response (Sections 2-3).• Based on it, we propose CPC, a framework for client-centric privacy-preserving counterfactual explanation.We study key problems underlying CPC and settle the complexity (Section 4).• We present approximation algorithms for computing guarded counterfactuals (Section 5).
• We also develop competitive online algorithms for CPC to operate in streaming mode (Section 6).
• We implement CPC and empirically verify its effectiveness for enabling user-controlled counterfactual explanation that can ward off model extraction attacks over explanation (Section 7).
Remark.CPC offers contextual explanability, by operating on a relation of model input and output during inference as a context.This continues with previous efforts towards the vision of in-database model explanation [15], as CPC can in principle be deployed on or incorporated into a typical DBMS (see more in Section 4.1).It also serves as a response from our data management community to address the recent call for actions regarding "explainability for everyone" instead of model owners [23].
Related Work.We categorize the related work as follows.
Explainable machine learning.There is a vast literature on explanation techniques to interpret blackbox ML models [17,52,74,96,97,119].They can be classified into two classes: local explanations and global explanations.Local explanations interpret the prediction of an individual instance, and are in contrast to global explanations, which aim to address how the model's overall behavior affects decisions.Among these, local explanations have gained more traction as they are more tailored and comprehensible to the specific contexts and cases at hand [84,96,97].Current families of mainstream local explanations include feature importance, rule-based and instance-based methods.Feature importance methods, such as LIME [96] and SHAP [74], quantify the contribution of each feature to a single prediction.This can be accomplished using a proxy or by computing Shapley values.Moving on to rule-based methods, e.g., Anchor [97], they aim to generate human-friendly explanations in the form of dependencies between feature values and the prediction of a targeted instance [59].Instancebased explanations, also known as counterfactuals [31,93,114], offer a more tangible understanding of model decisions by illustrating what changes in the features could alter the prediction of an instance and lead to a desirable outcome.In this work, we focus on counterfactual explanation.
Counterfactual explanation.There has been a host of work on counterfactual explanation of ML predictions [16,31,38,52,70,71,85,93,109,114,119].Conceptually, they work by formulating the task as an optimization problem where the objective function captures the quality of candidate instances as counterfactuals for the targeted instance.Such function is often a weighted combination of multiple measures of counterfactuals, e.g., a linear combination [37] of diversity [85], closeness [54], sparsity [31].While they share the overall structure of their objective functions, the key differences lie in their strategies in searching the optimal solutions in the entire feature space [16,38,52,71,85,109] or the training set [31,93,119], by adopting gradient descent [85,109,114] or heuristics [52,71].There has also been a vast and rich literature on the causality [14,48,49,58,62,65,87,91,92,99,113,122] Proc.ACM Manag.Data, Vol. and fairness [66,68,95,99,103] of counterfactuals.By employing given or learned probabilistic causal models, these approaches aim to ensure that counterfactuals follow natural cause-effect laws and address potential unfairness or bias in certain subpopulations e.g., a particular race or gender.Our work differs from existing methods in the following.(1) Existing methods assume unlimited access to the model or the training set, which CPC does not.A key implication of this difference is that existing approaches have to be deployed and supported by the model owners in order to provide counterfactual explanation, which raises the open Challenge I.In contrast, CPC works for and is operated by the model users, irrelevant to the model owners.(3) CPC supports inference time counterfactual explanation and monitoring, which are not supported by existing systems.(4) Built upon simple and intuitive algorithms, CPC is up to 3 orders of magnitude faster than existing approaches with comparable or even higher quality on public datasets.
Privacy attack via counterfactuals.In the realm of ML security, adversaries commonly launch model extraction attacks [12,36,60,117] against ML models, in particular those deployed in remote cloud servers.The attack aims to train a substitute model that approximates the original model well in terms of accuracy and fidelity.While there has been a host of work on both counterfactual computation and privacy attacks against ML models separately, only recently attention has started been paid to model attacks via counterfactuals.A recent study [12] firstly showed how models can be easily stolen by treating counterfactuals as training instances.Based on the intuition that counterfactuals are close to the decision boundary of an ML model, [117] subsequently proposed a more effective model extraction attack that additionally uses the counterfactuals of counterfactuals, which they proved is able to extract a linear model with perfect fidelity.
In contrast to existing work that focuses on attacks that exploit counterfactuals, we aim to defend such attacks for explanations.CPC computes counterfactuals guarded by inference instances that naturally ward off such attacks, without even affecting the performance of explanation and the quality of counterfactuals.While CPC handles both model extraction attack and membership attack, in this work we focus on the former as it is more complex and robust.
Data selection.In essence, CPC reduces the problem of explaining model predictions to a data selection task within the explanation (inference) universe.While CPC adopts variants of minimum cover and maximum coverage algorithms [18,19,76,112] as proof-of-concept, in principle it can work with a more diverse range of data selection techniques [24,25,29,61,86].Intuitively, data selection refers to selecting a subset from a large set of candidate data that meets specific conditions, while optimizing e.g., fairness [11,86], diversity [24,86], among others [25].In particular, package queries [28,29,29,30,75] have been proposed as a generic approach to supporting data selection atop a typical RDBMS.We remark that they can be well incorporated into CPC and extend the latter with additional objectives and constraints that apply to counterfactuals, e.g., fairness [68], potentially opening an avenue towards a vision of in-database support of model explanation.

PRELIMINARIES
We review the basics of counterfactual explanation and justify the associated open challenges.
ML basics.Let  1 , . . .,   be  features, where each   ( ∈ [1, ]) draws values from a domain dom(  ).The feature space is denoted by X = dom( 1 ) × • • • × dom(  ), and a tuple x in X is called an instance.x[  ] is the projection of x on feature   .Consider a blackbox ML model  in the context of supervised learning.For any instance x ∈ X,  returns a prediction (a.k.a.label)  (x) ∈ Y, where Y is the label space. is typically trained by a supervised ML algorithm using a set of pre-labeled instances from X; these instances are called training instances and the process is referred to as model training.Once  is trained, it can be used to generate predictions for new unknown instances.This is known as model inference, and such new instances are known as inference instances.
Counterfactuals.Given a model  and an instance x ∈ X to be explained, a counterfactual x ′ refers to an instance from X such that  (x ′ ) ≠  (x).In practice, counterfactuals may also have to meet application-specific conditions, e.g., validity [54], feasibility [113], sparsity [31] and similarity [37].For example, validity assures that  (x ′ ) of counterfactual x ′ should be a desired or targeted outcome.
For simplicity of exposition, we assume that the counterfactuals have passed the checks of those application-specific conditions; however, as will be shown in the experimental study (Section 7), our approach is generic and does support all these additional properties.
Counterfactual explanation.Given model  and instance x, a counterfactual explanation for x is a set C of counterfactuals for x [85].There are up to exponentially many counterfactual explanations for a given instance and we often want to compute those of high-quality.Two commonly used quality measures for counterfactual explanations are diversity [37,85] and succinctness [54,113].
Diversity.The diversity of a counterfactual explanation C for x, denoted by diverse(C), is defined as | x ′ ∈C  (x ′ , x)|, where  (x ′ , x) is the set of features that differ between x ′ and x.Intuitively, a counterfactual explanation with larger diversity carries more possible actions that can be taken to change the prediction of x [54].
Succinctness.The succinctness of counterfactual explanation C, denoted by succinct(C), is the number of counterfactuals in C. Intuitively, succinct explanation C is more digestible as it contains less instances and requires less effort to interpret [82].
An ideal counterfactual explanation should be both diverse and succinct [54,64].However, there is an inherent trade-off between the two measures [31,113].Intuitively, an explanation C with more counterfactuals might naturally boost diversity, but this can compromise succinctness, and vice versa.Moreover, it is not clear whether having multiple counterfactuals, each focusing on a single feature modification, is superior to fewer counterfactuals with changes over more diverse features.The interplay of these two properties suggests that neither dominates the other.The choice to prioritize one property over the other depends on the specific task at hand [31].
Model serving architecture.We consider the cloud-based model serving architecture, which is ubiquitous in real-life applications.As shown in Fig. 1, there are three different roles in such a workflow: model owner, model users and end customers.The model owner owns the ML model, which is deployed and hosted by some model serving service provider in the cloud.The service provider exposes a controlled interface to the ML model so that model users can access at the  Model users may use the predictions with some decorations to serve end customers, so that customers do not need to access model directly.In Example 1, the model owner is Upstart [9] and ZestAI [10] that assist banks with loan-decisions.The bank is a model user and Alice is a customer.
Challenge: right-to-explain.While almost all major cloud providers offer ML model hosting service [6-8], only few of them provide counterfactual explanations [119].Despite the active research on counterfactual explanation [31,37,38,64,85,88,109], we are still in great need of an approach applicable to these remotely hosted ML models even when service providers have decided not to.[36,60,110] to steal ML models that are typically proprietary.

Model extraction attacks. Driven by the desire to circumvent training costs, adversarial users often launch model extraction attacks
Model extraction attacks aim to extract a functionally equivalent surrogate model  ′ that behaves similarly to the targeted model .Specifically, the adversary queries  with  instances x 1 , . . ., x  , obtaining  predictions  (x 1 ), . . .,  (x  ) from , just as normal users.It then trains a replica model  ′ on the dataset {(x 1 ,  (x 1 )), . . ., (x  ,  (x  ))} to minimize the loss over an evaluation set , where  is the indicator function.Challenge: attacks over counterfactuals.Since counterfactuals tend to align closely with the manifold of training instances [37,54,113], ML systems supporting counterfactual explanation may be more susceptible to exposure.In particular, counterfactuals can be exploited by the adversaries as additional training data of high quality to learn  ′ .By asking the service provider counterfactual explanations C 1 , . . ., C  for inference instances x 1 , . . ., x  , an adversary now have a set  ∈ [1, ] C  of high quality "training instances" that assists its model extraction.Using counterfactual explanation, it has been shown that the attack can "recover" a typical DNN model with fidelity up to 98.6% [12,117].

GUARDED COUNTERFACTUAL
To address the challenges, we propose guarded counterfactual explanation with the below guarantees: (g1) it enables counterfactual explanation for model users even if model owners opt not to; and (g2) it is privacy-preserving such that it cannot be exploited for model extraction attacks.
In contrast to prior methods that draw instances from the entire feature space or training set as counterfactuals, we consider counterfactuals guarded by a privacy-preserving explanation universe.
Explanation universe.Consider an ML model  with feature space X.An explanation universe U for  is a subset of X.Consider a set I ⊆ X of inference instances processed during model serving and a set P ⊆ X of instances that the owner of  wants to protect.
Universe U is privacy-preserving for  w.r.t.I and P if for any ML architecture M defined over X, the trained model of M over I, denoted by  I , and that trained over I ∪ U, denoted by  U , satisfy the following (assuming the same training algorithm): where Pr( I (x) =  (x)) is the probability that  I and  make the same prediction for x; similarly for Pr( U () =  (x)).Intuitively, when U is privacy-preserving, exposing instances from U and their predictions by  to the model users does not give any advantage to them to extract sensitive information about  specified by P, i.e., the predictions of instances from P made by .
Guarded counterfactuals.Let x be an inference instance in I.A counterfactual x ′ of x is guarded by universe U if x ′ is in U. Since, nevertheless, inference instances in I and their predictions will need to be provided to model users, by definition counterfactuals guarded by a privacy-preserving universe U will naturally give zero advantage to adversarial users to extract predictions of  over instances protected by P. Hence, they completely ward off model extraction attacks over counterfactuals.
Inference universe and counterfactuals.A natural choice of privacy-preserving universes is the inference set I itself or a subset thereof, which we refer to as an inference universe and denote by U I .Note that for any set U I ⊆ I, any P of instances to be protected and any x ∈ P, we have Pr( That is, U I is privacy-preserving for  w.r.t.I and P.
Counterfactuals of instance x for  guarded by an inference universe U I for  are referred to as inference counterfactuals of x.In addition to the capability of preserving privacy, computing inference counterfactuals, i.e., the instances and associated predictions, requires only knowledge of I and predictions of  over I, which is already available for model users during model serving.As a result, this naturally confirms: Proposition 1: Explanation composed of inference counterfactuals upholds guarantees g1 and g2.□ Remark.In practice, an adversarial model user can perform model extraction attacks using only the inference set [60,83]; however, such attacks can be made much more effective when the adversary has access to counterfactuals [12,117], as they carry abundant additional information about the model.Guarded counterfactuals aim to prevent adversaries from exploiting counterfactuals to assist in model extraction.Orthogonal to this, privacy-preserving ML [34,117,121] can help defend attacks against the inference set.

CLIENT-CENTRIC EXPLANATION
While Proposition 1 tells us that we can address challenges raised earlier with inference counterfactuals for explanation, we have however not discussed how to compute such explanations.To this end, we present CPC, a framework for computing high-quality explanations via inference counterfactuals from model user's perspective.

Framework
As shown in Fig. 1b, CPC co-locates with the client-side model interface that sends inference instances to the remote model and receives corresponding predictions.It collects a relation of model input and output that the user sends and receives during model serving.CPC uses such relation as the explanation universe U I and computes counterfactuals guarded by U I , without any access to the remote model.Depending on how CPC reads U I , it can operate under two modes: batch explanation mode and streaming explanation mode.Batch explanation.In this mode, CPC analyzes inference instances and their associated predictions in batches, decided and collected via a local cache, e.g., a buffer pool or a Redis cache.For a given batch as the universe U I and a targeted instance x to be explained, it computes an explanation C for x with counterfactuals guarded by U I , while optimizing its quality, i.e., diversity and succinctness.
CPC offers a fine-grained and firm control over the quality of explanations, by allowing users to purposefully optimize one quality measure while strictly bounding the other.This is based on the study of the following two problems underlying CPC.
Input an instance x ∈ X( 1 , , . . .,   ), a batch U I of inference instances from X and their associated predictions by a (blackbox) ML model , an integer .
Output explanation C of counterfactuals guarded by U I for x.
Intuitively, SBEP is to identify the most diverse explanation C with up to  counterfactuals for x.
Diversity-bounded explanation.Similarly, the diversity-bounded explanation problem, denoted by DBEP, is to compute the most succinct explanation C (i.e., maximize succinct(C)) with inference counterfactuals that has diversity above a given lower bound .
Streaming explanation.In practice, the data input to an ML system may come in a streaming manner, e.g., a stream of transactions from an online bank [56] or measurements from a sensor network [107].These datasets are often too large to fit into memory, particularly for users with limited resources, e.g., no memory for caching the entire U I .However, it is often necessary to take into account all relevant instances when explaining, to provide explanation of high quality globally.
To this end, CPC offers to operate in the streaming explanation mode.Under this mode, instead of collecting and batching the inference universe U I , CPC treats U I as a stream with inference instances coming online one by one.Upon the arrival of each new instance x in U I , it incrementally refines the explanation it computes before the arrival of x as U I grows to U I ∪ {x}, while maximizing diversity (for SBEP) or minimizing succinctness (for DBEP).
Note that in this mode CPC does not need to "remember" the inference universe as it does when operating in the batch mode.It requires less memory overhead and maintains explanations in real-time.However, due to the memoryless streaming access to the inference universe U I , the explanation quality can be lower than those found by CPC in the batch mode.This said, as will be shown in Sections 4.2 and 6, we give streaming algorithms for both sSBEP and sDBEP that are competitive to their counterparts in the batch mode.
Benefits.CPC brings several benefits.(1) As discussed in Section 3, with inference counterfactuals CPC provides privacy-preserving explanations at the disposal of model users.(2) CPC avoids accessing ML models, significantly improving the efficiency of explaining (see Section 7.4).(3) By varying U I , CPC can provide users with timely, dynamic and contextual explanations, offering more insight (see Section 7.2).(4) CPC reduces the problem of explaining into a data-driven task over U I , opening new possibilities of further extensions, e.g., fair [24,86] and causality-aware [48,49,99] explanation.
Assumptions and limitations.We assume the following.
(a) On model users.CPC targets model users with long-term high demand for 3rd party ML inference services, e.g., banks that use credit assessment models [9,10].Such users often rely on predictions from the 3rd party models to serve large groups of customers (source of U I ) who may require reliable and real-time explanations that the model owners do not provide due to cost or privacy concerns.CPC cannot serve ad-hoc model users with limited inference instances.
(b) On explanation universe U I .To make practical use of CPC, we assume that instances in U I and their predictions by  are causally consistent [58,99], such that CPC could align well with realworld cause and effect specified by, e.g., causal models [49,91].This may be done by incorporating causality-aware explanations [48,49] and exploiting domain knowledge [65,87,92], to capture and enforce causal dependencies between features of instances in U I .
In addition, instances in U I are assumed to be relevant and close to those to be explained.This is to assure that counterfactuals guarded by U I can meet application-specific criteria, e.g., sparsity, similarity (recall Section 2).Moreover, for counterfactuals to be actionable and realistic [64], U I shall follow the same or a similar distribution of the training data [113], or come from the data manifold [62].We remark that this is often the case as inference set and training set are supposedly close in typical learning setup [22,110].
Although CPC by default uses inference universe U I , it is not the only option for CPC to function; application-specific, privacy-preserving universes picked by the users are also a viable option.2) CPC may suffer from a cold-start when there are no sufficient instances in U I .To remedy this, one may fall back on traditional methods for start.(3) Instances may get explanations of varying quality depending on their "representatives" in U I , which can lead to e.g., unfairness [40].One way to improve upon this is to incorporate fairness data selection [11,86] into CPC, or augment U I with selected instances sampled from feature space to improve quality and fairness.(4) While CPC prevents model user (e.g., a bank) from extracting 3rd party models via counterfactuals, the explanation that an end customer of the user receives may still reveal information about other peers in the same U I .One can mitigate this by anonymizing [20,46] U I or using differential privacy [41,42] as a post-hoc step.

Complexity of Client-Centric Explanation
We next settle the complexity of key problems underlying CPC.
Complexity.Both SBEP and DBEP are intractable when CPC operates in batch explanation mode.(1) We show that SBEP is Np-Hard by a reduction from the maximum coverage (MCR) problem, which is Np-Complete [50].Given an instance of MCR, i.e., a set  of elements  1 , . . .,   , a collection F of subsets  1 , . . .,   of  , and integers  and , MCR is to decide whether there exists an -coverage of  , i.e., at least  elements of  can be covered by  sets in F .Given  , F ,  and , we construct an inference universe U I over  features  1 , . . .,   that consists of  + 1 instances x 1 , . . ., x  and x: We show that there exists C ⊆ U I with diverse(C) ≥  and succinct(C) =  for x if and only if the MCR instance has an -coverage.
(2) We next prove that DBEP is Np-Hard by reduction from minimum set cover (MSC), a classic Np-Complete problem [50].Given an MSC instance, namely, a set  of elements  1 , . . .,   , a collection F of subsets  1 , . . .,   of  , and an integer , MSC is to decide whether there exists a -cover, i.e.,  sets in F that together cover all  elements of  .Given  , F and , we build the same U I as in (1).We then prove that there exists C ⊆ U I with succinct(C) =  and diverse(C) =  for x if and only if the MCR instance has a -cover.□ Approximability.In light of Theorem 2, any practical algorithm for SBEP or DBEP has to be approximate or heuristic.Nonetheless, we show that, in the batch mode, there exist approximation algorithms for both SBEP and DBEP with performance guarantees.Following standard notion of approximation ratios [112], we have: Theorem 3: (1) There exists a polynomial-time   − 1 -approximation algorithm for problem SBEP.
(2) There is a polynomial-time ln -approximation for DBEP.□ We will prove Theorem 3 by giving such algorithms in Section 5.
Competitiveness.Using competitiveness analysis [13], we show that there exist algorithms for streaming SBEP and DBEP with bounded competitiveness against the optimal batch explanations.
More specifically, we say that an algorithm A  for streaming SBEP is -competitive if for any batch algorithm A  , any input to SBEP,  • diverse(C  ) ≥ diverse(C  ), where C  is the explanation computed by A  that incrementally maintains C  when the universe U I is revealed one by one as a stream; C  is computed by A  that reads the entire U I as a batch; similarly for streaming DBEP.

Theorem 4: (1) There exists a 4-competitive algorithm for the streaming version of SBEP.
(2) There exists a 4 √ 2-competitive algorithm for streaming DBEP assuming  ≥  where  is the number of features.□ This assures that even if the client has no memory to batch U I , users can still employ CPC to obtain guarded counterfactual explanations; moreover, the explanation quality is guaranteed close to the optimal that they would have hoped for in the batch mode with infinitely large memory to collect U I .

BATCH CLIENT-CENTRIC EXPLANATION
As a constructive proof of Theorem 3, we present algorithms for both SBEP and DBEP in this section.
Algorithm dSBC.We start with an algorithm for problem SBEP, denoted by dSBC and shown in Algorithm 1.Given an instance x 0 to be explained, an inference universe U I = {x 1 , . . ., x  } and integer , dSBC returns a diverse set C of at most  counterfactuals guarded by U I as the explanation for x 0 .For the ease of presentation, we assume that each instance in U I is a qualified counterfactual for x 0 ; in practice this can be done by a linear scan of U I which filters out those instances that do not comply with application-specific constraints on counterfactuals, e.g., validity, sparsity or similarity.
Similar to the classic MCR algorithm [112], algorithm dSBC works by iteratively picking instances in U I as counterfactuals in C until C grows to the limit specified by .In each iteration, it selects one that has fewest identical features with x 0 .More specifically, it first identifies an initial counterfactual (lines 1-3) and then iteratively picks more counterfactuals from U I one by one (lines 4-6).Denote by A[x  = x 0 ] the set of features where x  and x 0 share the same value.Each time dSBC picks x  among all the instances from U I that are not yet in C, such that is the minimum among all (line 5).The set C is returned once it contains  counterfactuals (line 7).
Algorithm sDBC.We next present an algorithm for DBEP, denoted by sDBC, to generate a counterfactual explanation C with diversity bounded by  while minimizing its succinctness.It works the same as dSBC except the termination condition of the iterations (line 4 of dSBC in Algorithm 1), similar to the classic MSC algorithm [112].Specifically, the iterations now terminate when | x  ∈ A[x  ≠ x 0 ] |≥, where A[x  ≠ x 0 ] represents the set of features on which x  and x 0 disagree.Intuitively, once counterfactuals in C differ from x 0 on  distinct features, sDBC terminates.
Example 4: Continue with Example 3. Assume that applications 2-4 are valid counterfactual candidates for application 1. (1) Given  = 2, dSBC initially picks application 2, as it shares only three features with application 1. dSBC further selects application 3 via lines 5-6, since it shares the fewest features with applications 1-2.Hence, applications 2 and 3 are returned as counterfactual explanation C for application 1. (2) Along the same lines, sDBC returns applications 2 and 3 as the explanation for application 1 when  = 3. □ Analysis.Both dSBC and sDBC can be implemented in  ( 2 )-time, where  is the number of instance in U I and  is the number of features of an instance.The algorithms warrant the following.

Lemma 5:
(1) For any , the diversity of the optimal explanation is at most (1 + 1  −1 )-times of that of C returned by dSBC.
(2) For any , sDBC always returns an explanation C with a succinctness that is at most ln -times of that of the optimal explanation.□ Remarks.
(1) One might be tempted to "reverse" the reduction in the proof of Theorem 2 to reduce SBEP to MCR, and use classic MCR algorithm [112] for SBEP.We remark that this is indeed a valid approach to SBEP; However, it can incur higher cost than dSBC as the reduction, if "reversed", would involve computing the complement of A[x  = x 0 ] (corresponds to line 2 and 5 of dSBC), which can be expensive if the number of instances relevant to x 0 is high; sDBC is also similar.This said, both algorithms and the proof of Lemma 5 (omitted due to space limit) do largely follow the idea of MCR and MSC [112], but without explicitly implementing the reductions.
(2) CPC is extensible.It can be extended with alternative solutions to SBEP and DBEP, e.g., data selection [24,86], to take into account additional factors like fairness [11,86], or package queries [29,75] which would add the support of CPC in a typical database system.

STREAMING EXPLANATION
As a proof of Theorem 4, we give algorithms for streaming SBEP and DBEP, for CPC to operate in the streaming explanation mode.Unlike batch mode in Section 5, the approach to the streaming mode follows the "reverse" of reductions in the proof of Theorem x  ← arg min MCR [18,76,123] and MSC [19,44] for CPC.For completeness, we first describe the algorithms and then show that they implement the reductions, thus verifying Theorem 4.
Algorithm dSBC.We start with streaming SBEP.We give an algorithm, denoted by dSBC and shown in Algorithm 2, that computes a counterfactual explanation C guarded by a streaming universe U I It works by incrementally maintaining C (initially empty) upon the arrival of each inference instance in U I (lines 2-3).It does not memorize arrived instances due to lack of local memory in streaming mode, and returns C after the stream ends (line 4).The goal is to maximize the diversity of C while subject to a succinctness bound .Upon the arrival of x  , if C contains less than  instances, dSBC adds x  to C (line 2).Otherwise, it decides whether x  should be added to C and if so which existing instance needs to be evicted from C to maintain succinctness bound  (via procedure uCdiv; line 3).
More specifically, uCdiv first identifies the existing instance x  in C that contributes least to diverse(C) (line 6), measured as is the set of features where x  and x 0 differ.Intuitively, this counts the number of "private" features in C that are exclusively present in x  , i.e., no other instances in C differ from x 0 over such features.With this, dSBC then replaces x  with x  only if the contribution of x  to C satisfies where   refers to the set of total features in which the instances in C differ from x 0 (lines 7-9).Otherwise, C remains unchanged.
Algorithm sDBC.We next present an algorithm, denoted by sDBC (Algorithm 3), for the streaming version of DBEP.Similar to dSBC, sDBC works by maintaining and updating an explanation C as instances of U I pass online.It only updates C with new instances that bring benefits to the succinctness of C while maintaining diversity lower bound .To measure the benefits, it maintains an integer effectiveness for each feature   , denoted by e[  ], of which the value changes when a new instance x  arrives.sDBC identifies a set  of features from A[x  ≠ x 0 ] for x  , each with an effectiveness strictly less than ⌈log | |⌉.It then refreshes C according to  .
More specifically, C is empty initially and the effectiveness of each feature is set to a minimum value (lines 1-3).Upon the arrival of x  , sDBC computes its  with largest cardinality (via procedure uCsuc; line 6).If  is empty, it keeps C intact and waits for the next instance of the stream (line 7).Otherwise, it updates C by adding x  and sets the effectiveness of each feature of  to ⌈log | |⌉ ALGORITHM 3: Algorithm sDBC Input: U I as a stream with x 1 , x 2 , . . .arriving online, an instance x 0 , integer .Output: a counterfactual explanation C when U I ends.
(lines 8-10).Intuitively, by line 6, the effectiveness e[] of each feature  in  is increased at least by 1 since  is at least twice as the size of  ′ in the last iteration that included and updated .It adds x  to C only when its  is non-empty, i.e., at least one feature of which the effectiveness needs to increase.To make C more succinct, sDBC removes redundant instances from C in two phases.Initially, it discards those that only differ from x 0 over the features in  (lines [11][12].This is because these features are essentially embodied by the most recent x  that yields a greater benefit.In phase two, it iteratively deletes instances from C with the fewest "private" features (recall dSBC) until the diversity of C reaches the lower bound  (lines 13-16).
Analysis.Upon the arrival of each x  , dSBC and sDBC take  ()-time and  ( 2 )-time, respectively, to maintain the counterfactual explanation C, where  is the number of features.The space complexity of dSBC and sDBC is only  () and  ( 2 ), respectively.Note that both  and  are quite small in practice.The negligible time and space complexity justifies that they can support CPC well for real-time applications with stringent resource constraints.Besides the practicality, they also correctly implement the reductions and offer performance guarantees for streaming CPC.
(1) We clarify that our main goal is to show that there exist effective algorithms to support CPC in the streaming mode, and we do this by "reversing" the reductions in the proof of Theorem 2 to adapt and optimize classic streaming MCR [18,76] and MSC [19,55] algorithms for CPC.

EXPERIMENTS
In this section, we experimentally evaluate the effectiveness of CPC.Below we first specify the settings (Section 7.1).We then evaluate the capability of CPC to find high quality counterfactual explanations while defending model extraction attacks (Section 7.2-7.6).

Experimental Setting
Datasets.We used 4 real-world datasets from the Kaggle contest portal [3] and the UCI repository [5] that are commonly used for evaluating explanations [31,54,85] (see Table 1 for details).
For each dataset, we designated non-actionable features to ensure that counterfactuals do not alter their values.Specifically, the non-actionable features are set as follows: {age, sex, race} for Adult, {age} for Credit, {age, sex, race} for Compas and {gender} for Loan.
Compared methods.As shown in Table 2, we compared with 11 state-of-the-art methods.
(1) Counterfactual explanation methods.We tested 4 popular counterfactual explanation methods, implemented using the Carla library [88] with the default configurations recommended by Carla.Among them, DICE is capable of generating multiple counterfactuals as an explanation for a query instance.In contrast, GS, FACE and CCHVAE only produce explanation of singleton counterfactual.
To address this, we repeatedly executed these three methods multiple times with varied random seeds for their parameters; this yields explanation of multiple counterfactuals, consistent with DICE.
(2) CPC variants.We also tested 7 variants of CPC where we use alternative methods for SBEP and DBEP in CPC, as follows.
(a) Data selection.CPC MaxMin and CPC MaxSum are two variants of CPC that employ MaxMin and MaxSum, respectively, for selecting a diverse explanation from U I , where MaxMin (resp.MaxSum) is a recent data selection method that is shown effective in maximizing the minimum (resp.total) distance among the chosen data [24,86].
(b) Package queries.We also employed package queries [29,75]   GSV [76], GOPS [123] and DR [55], respectively, for solving SBEP and DBEP when CPC operates in streaming explanation mode.Both GSV and GOPS are single-pass streaming algorithms for approximating maximum -set coverage, while DR targets minimum set cover.The first two are viable alternatives to dSBC and DR is a practical substitute for sDBC.
(d) Optimum.CPC OPT is a CPC variant that uses brute-force enumeration to find optimal solutions to both SBEP and DBEP in CPC.
We remark that all these 4 counterfactual methods rely on the ML model  from model owners or service providers to deduce counterfactual explanations, i.e., they are not client-centric and users have no control of whether and how they can obtain the explanation.CPC and all its variants run on explanation universe U I that consists of instances already pass the application-specific constraints (e.g., sparsity, similarity) on individual counterfactuals.CPC MaxMin and CPC MaxSum focus exclusively on optimizing diversity and operate in batch mode.On the other hand, CPC GSV , CPC GOPS and CPC DR are tailored for streaming mode, with the first two focusing on maximizing diversity and the last one on optimizing succinctness.
Measures.We assessed the efficiency of each method by the average time it takes to generate the explanation for a single instance.
In addition, we also evaluated the quality of counterfactual explanations computed by all methods and the degree of privacy leakage in their counterfactuals w.r.(1) Measuring explanation quality.We used four measures to quantify the quality of counterfactual explanations.Consider a counterfactual explanation C for an instance x.The first two measures assess the quality of C as a whole.(a) Diversity [37,85].the diversity of C is the total number of distinct features of counterfactuals in C that differ from those of x.(b) Succinctness [54,113]: the number of counterfactuals in C.
We also evaluated the quality of each individual counterfactual in the explanation.Consider a counterfactual x ′ ∈ C for x.(c) Similarity: the similarity between x and x ′ is measured by the Gower distance [37], which is defined as 1 ).The diversity of a counterfactual explanation should not exceed the number of actionable features in the dataset; similarly for the sparsity of an individual counterfactual.The similarity varies between 0 and 1.For the succinctness, similarity and sparsity, smaller values are preferable; conversely, the larger the better for diversity.
(2) Measuring privacy leakage.We start with the attacks.
Attack strategy.We adopted dualCF [117], a recent model extraction attack strategy that utilizes counterfactuals.It uses both the counterfactual and its subsequent counterfactual (referred to as CCF) as training pairs for training the substitute model  ′ .Specifically, dualDF first sends a query instance x to the counterfactual method and retrieves a counterfactual x ′ .Following this, x ′ itself is used as a new query to derive its CCF, x ′′ .Using this procedure, dualDF can generate a set of counterfactuals and CCFs, which, together with the inference set, serve as a training set to create  ′ .Intuitively, the counterfactual and CCF sit on opposite sides of the decision boundary of the original .Furthermore, both the counterfactual and CCF naturally have similar distances to the boundary as they are derived from the same counterfactual method.Together with inference set, this assists adversaries in training  ′ that closely resembles .
Privacy metrics.To measure the privacy leakage of counterfactual explanations against model extraction attacks, we compare the performance of the surrogate model  ′ with that of the original model  over an evaluation set X  ⊆ X.We use two metrics [12,117].
where  is the indicator function, i.e., 1 if  (x) =  ′ (x).It measures the prediction similarity between  and  ′ over X  .A higher agreement indicates that  ′ behaves more similarly to .
(b) Discrepancy: the discrepancy of  ′ measures how different its predictions are to those of  over X  .It is defined as where  is the true label for .Intuitively, discrepancy is an overall assessment of the quality of  ′ without requiring  ′ and  to make the same prediction for every instance.Instead, this specific requirement is exactly what the agreement measures.A higher agreement or lower discrepancy signifies a more successful model extraction attack by the adversary.
Configuration.We deployed CPC on a machine with an Intel Core i7-6700HQ CPU @2.6GHz and 32 GB of RAM as the client, and used AWS EC2 m5.2xlarge instance to host the remote model .
For each dataset, a 60/30/10 ratio was used to split it into 3 disjointed sets: training, inference All compared methods are tested with their default parameters.Moreover, all existing counterfactual explanation methods have unlimited access to the target model  and its training set, which CPC has none.In the tests, we set the similarity threshold for each counterfactual to 0.3, and sparsity to 4 for Credit and 2 for the other three datasets.The default succinctness and diversity bounds are 5 and 4, respectively, for all datasets.Each experiment was run three times, and the average is reported.

Case Study and User study
We first illustrated the benefits of CPC over the four counterfactual explainers via a case study.We then conducted a user study to evaluate how humans perceive their explanations in practice.
Case study.We start with a case study.
Right-to-explain.CPC gives users the right to access counterfactual explanation even when the model owner does not.As an example, we hosted the original model  over Loan dataset on Amazon SageMaker [6] as a remote model inference service that charges by the number of queries to the remote model.SageMaker provides neither counterfactual explanation nor APIs for computing  3 at the client side, incurring no cost from SageMaker.In contrast, all other counterfactual methods require  to be open to "explanation queries" that invoke  over candidate counterfactuals, which also incurs additional cost even if the owner permits.
Privacy preservation.Since CPC ensures that its counterfactuals come solely from the inference set that is already known to the users, it does not reveal any extra information about the ML model.In contrast, counterfactuals from DICE, FACE, GS and CCHVAE can be seen as "new" training instances.Adversaries could incorporate these new instances into their existing inference set in their attacks, creating a surrogate model with high accuracy (see Section 7.3).
Quality.We further illustrate the quality of explanations for instance x 0 over Loan that has a denied decision.Assume a user wants to generate explanation C with 3 counterfactuals and the sparsity of each is no more than 2. To allow all methods to explain correctly, we deployed the model at the client-side and allowed unlimited model access for all methods except for CPC.Table 3 shows x 0 and its explanations by each method; only changed features are shown.
(1) Controllability.CPC offers strict controls on the validity of the counterfactuals.For instance, FACE, GS and CCHVAE do not meet the sparsity bound while CPC does.Moreover, FACE and CCHVAE modify immutable features, e.g., Gender, which is clearly not viable.
(2) Relevance.CPC draws counterfactuals from real instances predicted during inference, which are more relevant than methods that generate counterfactuals from the feature space, which are artificial and unrealistic.For example, counterfactuals of GS and CCHVAE are less feasible since in financial transactions [39], LoanTerm cannot be arbitrarily specified, especially not as a random decimal.
(3) Quantification.The diversity of the explanation by CPC is 4, higher than 3 for DICE. of FACE, GS and CCHVAE are invalid as they do not satisfy user specific requirements on sparsity.
User Study.We further conducted a user study to evaluate CPC and 4 counterfactual explanation methods regarding (1) practicality: which method produces counterfactuals that are easier for users to understand and accept, and if they are causally consistent, and (2) contextuality: how do users perceive counterfactuals that may vary over time or be dependent of user-specified contexts?To this end, we designed an online questionnaire 1 using Google Forms.The first page (page 1) of the survey asked about the participants' professional background.Considering that counterfactual methods should be accessible to non-technical users without ML expertise, we recruited 38 participants, half familiar with ML terminologies and the other half less so.Page 2 introduced the basics of ML models and counterfactuals.We randomly selected 10 instances with denied ML decision from Loan dataset and presented explanations generated by each counterfactual method, similar to those in the case study.Each survey evenly and randomly included 3 instances.To ensure impartiality, the methods were anonymized so that participants were unaware of the specific methods used.
Design.We next introduce the remaining part of the survey in detail.
As shown in Table 4, Questions 1-4 (Q1-Q4) are measured by a 5-point Likert scale (higher scores indicate better performance), and Question 5 is a single-choice query.Q1-Q4 were presented without direct comparison between different methods.We first evaluated explanations computed by CPC, then asked the same questions for explanations by DICE, FACE, GS and CCHVAE, in that order.This forms pages 3-7 of the survey.Page 8 included Question 5, which directly asked respondents to choose their preferred method.
(2) Contextuality.We then assessed how users perceive contextual counterfactuals by CPC for the same instance with different contexts as the explanation universe.To create these contexts, we categorized instances based on different time and features.For time-dependent contexts, we evenly divided Loan sorted in ascending order of Income into four non-overlapping batches (assuming that people's income increases over time).For the feature-grouped contexts, we created different batches by specifying multiple expected ranges for Income and LoanAmount.Following [69,123], as shown in Table 4, we designed 5 questions from various aspects including acceptability, trustworthiness, stability, guidance and insightfulness.Q6-Q9 and Q10 were on page 9 and 10 of the survey, respectively.All the questions were evaluated using the 5-point Likert scale.
Results.From the gathered feedback, we found the following.
(1) Overall, out of the 38 answers, a significant 28 users chose CPC as their preferred method (Q5).For Q1, on average both CPC and FACE scored over 4 on a 5-point Likert scale, while DICE only received a score of 3.4, and GS and CCHVAE scored even less than 2. This indicates that users generally find counterfactuals generated by CPC and FACE to be relevant and meaningful.However, FACE scored only 1.7 in Q2, whereas CPC and DICE scored much higher, at 3.9 and 3.7 respectively.This highlights the importance of sparse explanations for practical user actions.Moreover, in Q3 and Q4, 89.5% of users felt that CPC provided a sufficiently diverse range of feature options (i.e., scoring ≥ 4), and 86.1% of users believed these counterfactuals were causally consistent.These findings demonstrates that among all compared counterfactual explanation methods, CPC offers the most realistic and practical suggestions.
(2) When CPC's counterfactuals vary over time, on average the scores for Q6-Q8 were 4.1, 4. 3.5, respectively.This indicates that most participants still accept and trust the system in such scenarios, even though some find CPC's explanations not somewhat stable.More importantly, a notable 71.1% of users believed the potential pattern reflected in explanations at different times could be beneficial in improving their future strategy (Q9).
(3) Users recognized the benefit of CPC's contextual counterfactuals that are tailored to scenarios with specific ranges of picked features.Specifically, almost all participants believed that these contextual counterfactuals delivered more useful information and deeper insights, with 35 out of 38 responses scored over 4 on Q10.

Defending Attacks over Counterfactuals
We next evaluated the effectiveness of all counterfactual methods in defending against model Compared to 4 existing counterfactual methods, on average across all datasets CPC is 40.1, 110.8, 9.4, and 662.7 times faster than DICE, FACE, GS and CCHVAE, respectively.This is because these methods need to access the ML model for counterfactual candidates, which is not required by CPC and its variants.Moreover, CPC is on average 30.9 times faster than its variants, up to 197 times.This shows that CPC can efficiently compute high-quality counterfactual explanations on-the-fly, making it ideal for real-time scenarios.
Impact of |U I | and #-.We further examined CPC's efficiency by varying (1) the inference universe size |U I | from 60% to 100% of the inference set; and (2) the feature number #- (rounded to the nearest whole number) from 60% to 100% of the total feature number of each dataset.The results are shown in Figures 3m-3n.As expected, the running time of CPC increases when |U I | or #- grows, but the increment is minor and acceptable.For instance, over the Adult dataset, the time

x
(a) Prior approaches: Counterfactuals are computed by model owner.Adversaries can exploit counterfactuals to extract a surrogate  ′ with a high fidelity to .CPC computes counterfactuals at client without model access.It also prevents adversaries from exploiting counterfactuals.

Fig. 1 .
Fig.1.State-of-the-art counterfactual explanation Vs.CPC explanation with x ′  and x ′  : x ′  suggests an increase in income by $1K and an improvement in cscore to 'good', while alternatively x ′  proposes to reduce lamount from $100K to $20K.□

( 2 )
This further implies that existing counterfactuals carry information about the model or training set, making them vulnerable to model extraction attacks as described in open Challenge II.Instead, CPC computes counterfactuals guarded by inference instances, averting model extraction attacks from exploiting counterfactuals.

Fig. 2 .
Fig. 2.An example inference universeclient-side, e.g., providing an instance x for inference and obtaining a prediction  (x) as a response.Model users may use the predictions with some decorations to serve end customers, so that customers do not need to access model directly.In Example 1, the model owner is Upstart [9] and ZestAI[10] that assist banks with loan-decisions.The bank is a model user and Alice is a customer.

Example 3 :
Continue with Example 1.Consider the inference set I in Fig 2 with 8 loan applications and predictions by .Applications 2-4 and 6-8 could serve as inference counterfactuals for application 1. Application 5 cannot as it was also rejected, i.e., having the same prediction as application 1. □ Proc.ACM Manag.Data, Vol. 2, No. 3 (SIGMOD), Article 130. Publication date: June 2024.
(c) On deployment.Due to the client-centric design, CPC has limitations on use.(1) It assumes that model predictions are deterministically determined by features provided by the model user.If the model also uses internal variables, their values must be passed to the client user along with the predictions of inference instances during model serving, so that CPC can reason properly.(

Theorem 2 :
Both SBEP and DBEP are Np-Complete.□ Proof sketch: SBEP is in Np because one can verify in polynomial time whether a set of inference counterfactuals C satisfies diverse(C) ≥  and succinct(C) ≤ .Clearly, the same also applies to DBEP.We next show that both SBEP and DBEP are Np-Hard.

Table 1 .
Datasets and their prediction tasks these steps do not affect the effectiveness of features, thereby maintaining the performance bounds.
[24]: select  instances and maximize the minimum pairwise distance CPC MaxSum MaxSum[24]: similar to MaxMin, but maximize the sum of pairwise distances CPC variant with Package Queries [29] method description CPC PQ use package queries [29] for solving SBEP and DBEP in CPC CPC variants with Max-Cover [76, 123] and Set-Cover [55] methods description CPC [55]]lect explanations from U I in CPC.A package query returns a set of instances meeting predefined single and global constraints; it scales by employing ILP solvers structurally via query decomposition.(c)Max-cover&Set-cover.Denote by CPC GSV , CPC GOPS and CPC DR the CPC variants that use GSV GSV[76]: a single-pass streaming algorithm for approximating maximum -set coverage, using the "guess, subsample, and verify" framework CPC GOPS GOPS[123]: a streaming MCR algorithm with the aid of computer simulation CPC DR DR[55]: a streaming MSC algorithm based on iterative dimensionality reduction CPC variant with optimal search method description CPC OPT optimal answers to SBEP and DBEP in CPC via brute-force enumeration

Table 2 .
Compared explanation methods and CPC variants t. model extraction attacks.

Table 3 .
Counterfactual explanations in the case study and evaluation sets.The training set is used only for training the target model .Specifically, we trained a Multilayer Perceptron (MLP)  for each dataset using the Adam optimizer to minimize the binary cross-entropy loss.For the Adult and Credit datasets,  comprises six hidden layers with 512, 288, 144, 72, 36, 9 and 3 neurons, while for the Compas and Loan datasets,  has three hidden layers with 36, 9 and 3 neurons.The inference set contains instances that were explained, and also those used by the adversary to extract the surrogate model  ′ via dualCF attacks.The evaluation sets are used to gauge the accuracy of both  and  ′ , and to measure how well  ′ is aligned with .Unless explicitly stated otherwise, CPC is equipped by default with dSBC and uses the complete inference set as the explanation universe.

Table 4 .
Proc.ACM Manag.Data, Vol. 2, No. 3 (SIGMOD), Article 130. Publication date: June 2024.Questions in the user study counterfactuals.Despite this, CPC still generates counterfactual explanations, e.g., for x 0 of Table of DICE exceeds that of CPC by 33.3%.This is because DICE cannot flexibly optimize diversity while also bounding similarity; it works by using a higher weight to diversity in its objective function when maximizing diversity, which results in reduced similarity.(2)Diversityandsuccinctness.CPC consistently outperforms all the compared methods on all datasets in diversity and succinctness.As depicted in Fig3g, on average the diversity of CPC exceeds that of DICE and CPC variants, CPC MaxMin , CPC MaxSum and CPC PQ , by 33.7%, 5.1%, 23.3% and 9.4% across all datasets respectively.Moreover, its diversity gap from the optimal solution CPC OPT is negligible, only 2.4% on average.On the other hand, as shown in Fig3h, CPC (sDBC) is also the best when users prioritize succinctness.We remark that CPC MaxMin and CPC MaxSum are not suitable for optimizing succinctness.Compared to the strongest competitor, CPC PQ , CPC's explanations are 12.1%, 6.9%, 16.7% and 18.2% more succinct over Adult, Compas, Credit and Loan respectively.Similarly, CPC is comparable to CPC OPT in optimizing succinctness.(3)Impact of  and .We also evaluated the impact of the succinctness bound (desired counterfactuals ) and the diversity bound (desired changed features ) on the performance of CPC.Fig 3i depicts the diversity of CPC (dSBC) on Compas when varying  from 1 to 5. Fig3jshows the succinctness of CPC (sDBC) when  ranges from 2 to 6 on Loan.We observe that for all values of , CPC consistently outperforms all the competitors in diversity, and the benefit of CPC becomes even more evident when  increases.For instance, the gap between CPC and DICE grows from 0.4 to 0.8 when  varies from 2 to 5. Similarly, CPC is the most succinct.On average, the succinctness of CPC surpasses DICE and CPC PQ by 27.0% and 18.5% respectively.This verifies that CPC can reliably deliver high-quality explanations, catering to various values of  and .(4)Impact of |U I |.Finally, we evaluated how the size |U I | of inference universe U I impacts the quality of counterfactuals of CPC.Varying |U I | from 60% to 100% of the inference set of all datatsets, we tested the diversity and succinctness.As shown in Fig 3k-3l, we observe that (1) with the increase of |U I |, both diversity and succinctness improve as expected; (2) even when |U I | is at 60%, on average CPC still outperforms all compared methods in diversity and succinctness by 12.7% and 22.1% respectively (Figures 3g-3h); and (3) CPC is robust against varying |U I |.For instance, on average the diversity decreases by only 4.6% when |U I | is reduced from 100% to 60%.Efficiency.We also tested the efficiency.The average time (ms) for explaining an instance over all datasets is shown in the table below.