Cite as: Buechner, J., Simon, J., and Tavani, H.T. (2014). “ Re-Thinking Trust and Trustworthiness in Digital Environments.” In Autonomous Technologies: Philosophical Issues, Practical Solutions, Human Nature: Proceedings of the Tenth International Conference on Computer Ethics Philosophical Enquiry: CEPE 2013. (Edited by E. Buchanan, et al.). Menomonie, WI: INSEIT, pp. 65-79. Re-Thinking Trust and Trustworthiness in Digital Environments Jeff Buechner Rutgers University Judith Simon University of Vienna, Department of Philosophy, Universitätsstr. 7, 1010 Vienna, Austria: judith.simon@univie.ac.at & Institute of Technology Assessment and Systems Analysis, Karlsruhe Institute of Technology, Karlstr. 11, 76133 Karlsruhe, Germany: judith.simon@kit.edu Herman T. Tavani Rivier University, 420 Main St., Nashua, NH 03060 USA htavani@rivier.edu Keywords Trust, Trust Relationships, Trustworthiness, Rational Trust, Justified Trust, Artificial Agents Abstract In this paper, we examine the concept of trustworthiness and the role that it plays in trust relationships, both within and outside digital environments. First, we consider and reject the traditional notion of trustworthiness (TW), where TW is a characteristic or property applicable only to the trustee in a trust relationship. We then defend a novel concept of TW, which we argue applies both to the trustor and trustee. Next, we describe a scheme for evaluating the TW of trustors and trustees, both from an epistemic and a moral perspective, via criteria based on rationality and evidence. Finally, we look at the question of TW in the context of AAs. We argue that while humans are capable of entering into various degrees of trust relationships with AAs (depending on factors such as the level of autonomy of the AAs involved), current AAs (no matter how autonomous or sophisticated they may be) are not capable of qualifying as TW in a moral sense but only in an epistemic sense. INTRODUCTION The inspiration for the present paper came from some questions generated by claims made in two 1 previous works on trust in the context of artificial agents (AAs). These works defended an account of trust (in general) in which a trust relationship between two human agents (HAs) – A and B – is one in which the following five conditions obtain: I. II. III. IV. V. A has a normative expectation (which may be based on a reason or motive) that B will do such-and-such; B acknowledges that A has this normative expectation, and B is responsible for what it is that A normatively expects her to do; A has the disposition to normatively expect that B will do such and such responsibly; A’s normative expectation that B will do such-and-such can be mistaken; [Subsequent to the satisfaction of Conditions (i) – (iv)] A develops a disposition to 2 trust B. Because of the nature of the normative expectations involved, a trust relationship affecting A and B is more than a mere reliance that A has on B. A normative expectation is in place in the following kind of situation: When A relies on B to do what B should, A does not only expect that B will do it, A expects it of B. Here, A can normatively expect that B will do X, even when A does not expect that B will do it. (That is, A does not predict that B will do it, but A still expects it of B.) So a trust relationship cannot be reduced to mere reliance or to any non-normative expectations. We further argued that trust relationships between HAs and AAs are also possible in virtue of various contexts or zones of trust in which HAs and AAs typically interact, i.e., contexts along the lines of what Margaret Urban Walker (2006) calls “zones of default trust” and “zones of diffuse, default trust.” Building on Walker’s insights, we argued that HAs are capable of entering into trust relationships with a wide range of AAs, including multi-agent systems that, in turn, can be diffusely distributed across various elements in a system or network of agents. Also, we further argued that both the kinds and degrees of possible HA-AA trust relationships in these zones can vary, depending on the level of autonomy of the AAs involved. So, for example, while HAs can enter into indirect (and minimal) trust relationships with AAs in “diffuse default” zones of trust (e.g., with some types of multi-agent systems), HAs are also capable of having a much more direct, transparent, and robust trust relationships with certain kinds of “functionally autonomous AAs” (FAAAs). We elaborate on some of these relevant distinctions in the final section of the paper. First, however, we note that one important question that has been raised about this model of trust, which had not been addressed previously, has to do with issues concerning the 1 2 See Buechner and Tavani (2011) and Tavani and Buechner (forthcoming). Note that Condition II in this list is a slightly revised version of the one included in the original framework in Buechner and Tavani (2011). We are grateful to Lloyd Carr for his critique of that framework, which led us to modify Condition II. 3 trustworthiness of the AAs involved. The remainder of this paper is dedicated to examining the concept of trustworthiness. TRUST AND TRUSTWORTHINESS Despite the proliferation of recent scholarly work on trust (especially the literature on e-trust), we believe that there is a glaring lacuna—the nature of the relationship between trust and trustworthiness. It seems that theoreticians either focus on trust or on trustworthiness (with a much stronger focus on the former term), while the relational character of trust and trustworthiness remains implicit at best. It is this relationship between trust and trustworthiness that will be the central focus point of this contribution. In philosophical accounts, there has been a stronger emphasis on spelling out definitions of trust and a basic contentious issue in theories of trust concerns the question whether trust is a belief, an expectation, an attitude, or an emotion (cf. Simon forthcoming). Depending on their answers to this question different theoreticians have proposed cognitive (Gambetta 1988), will-based or affective accounts of trust. In particular definitions of trust often aim at setting trust apart from mere reliance and different philosophers have proposed various factors X to distinguish trust from mere reliance : Hardin (2002) argues that it is encapsulated interest that distinguishes trust from mere reliance, while Baier, a strong opponent of cognitive accounts (Baier 1986 & Baier1991), proposes a willbased account of trust arguing that trust depends not merely upon expectations, but on beliefs about the competence and the good-will of the trustee. Holton (1994) proposes a participant stance as what distinguishes trust from mere reliance, while Jones (1996) and Lahno (2001) both challenge that trust is a form of belief altogether and propose affective accounts of trust. In most of these accounts, trustworthiness remains rather implicit, although it should be noted that several theoreticians on trust have developed accounts of trustworthiness as well that align with the respective cognitive, will-based or affective accounts. Departing from a cognitive account of trust, Gambetta & Hamill relate game theory to signalling theory in order to analyze how taxi drivers in Belfast and New York assess the trustworthiness of their prospective clients while Hardin (1996, 2002) assesses the dispositions, motivations and constraints of trustworthiness based upon his encapsulated interest model. Potter (2002) and Daukas (2006) both offer normative accounts of trustworthiness based upon virtue ethics, while Pettit (1995) introduces the notion of trustresponsiveness as a more neutral alterative to virtue-theoretical accounts of trustworthiness. 3 For example, Judith Simon, at the Conference on Evolutionary Robotics, Organic Computing, and Adaptive Ambience (Karlsruhe Institute of Technology, Germany, October 2001) questioned whether this account of trust has anything to do with the “characteristics” of the AAs involved, and if so, whether it would “make sense to argue for the trustworthiness of agents in the first place.” A series of email exchanges on this topic ensued between the authors, resulting in the present paper. With respect to trust in digital environments, it is illuminating to take a look at research on trust in multi-agent systems. Interestingly, the center of attention is almost reverse: it is trustworthiness formalized as reputational differences between agents that is in focus while trust describes the relations, the links between the agents characterized by different degrees of trustworthiness or reputation (cf. Sabater & Sierra 2005). Our claim therefore is not that the relationship between trust and trustworthiness has never been addressed. Rather, we argue that we need to understand trust and trustworthiness as relational concepts and that any attempt to understand one without the other will lead to impoverished understandings of our practices of trusting. Moreover, assessing practices of trusting as rational or justified, presupposes relating trust to trustworthiness, because the justification or rationality of an act of trusting is tied to the question of whether a trustee is trustworthy, resp. whether the trustor has sufficient evidence for the trustee’s trustworthiness. Hence, the first goal of this paper is to shed some more light on this fundamental but often neglected relationship between trust and trustworthiness. Secondly, and this is a crucial shift in understanding the relational nature of trust and trustworthiness, we propose that trustworthiness must be understood not merely as a characteristic of the trustee, but also of the trustor. Hence, we distinguish between the trustworthiness of a trustor (TWtrustor) and the trustworthiness of a trustee (TWtrustee). While the latter refers to trustworthiness as commonly conceived, expanding the concept of trustworthiness to the trustor is the core novelty we propose in this paper. While the trustworthiness of the trustee usually is considered to comprise of epistemic and moral components (i.e. competence and honesty), the trustworthiness of a trustor is usually not addressed in classical account on trustworthiness at all. We propose a remedy to this lopsidedness and argue that the trustworthiness of a trustor also has epistemic and moral components related to the skills a trustor possesses in evaluating trustees’ trustworthiness and in an understanding and willingness to comply to the normative force of trust relationships. Crucially, we claim that for a trust relationship and act of trusting to be justified or rational both trustors and trustee need to the trustworthy. A third aspect that we wish to underscore in our contribution is that we consider both TWtrustor and TWtrustee to be context-sensitive and state-like rather than general and trait-like. I may be a trustworthy trustor with respect to trusting a fellow philosopher’s opinion on the philosophy of trust, but I am less skilled in assessing the trustworthiness of my car mechanic or physician. Similarly, I may be trustworthy as a trustee with respect to giving someone directions in Vienna, but much less so for directions in Bangkok. Hence, despite the fact that maybe some agents (both as trustors or as trustees) may be more trustworthy than others (either because they have a higher disposition to be trustworthy trustees or because they have better skills at detecting trustworthy trustees thereby raising their trustor TW), the examples above nonetheless should make evident that both types of trustworthiness are context-specific competencies and dispositions. Finally, these examples also elucidate another characteristic of trustworthiness, namely that it has different components. According to John Hardwig’s seminal paper on the role of trust for knowledge, (Hardwig 1991) trustworthiness (or TWtrustee in our terminology) depends on the trustee’s honesty, competence, and his/her adequate assessment of the limits of his/her competence - as a second-order competence, but also as a form of honesty. In the case of giving directions in Bangkok, I simply lack the competence. There may be cases, however, in which I lack the willingness to be honest and am hence my lack of trustworthiness is a due to a moral failure, not an epistemic one. TWtrustor also has different components, albeit different ones. A trustworthy trustor is not only skilled in differentiating trustworthy from untrustworthy trustees (the epistemic component), he is must also be able to understand the normative nature of trust relations (i.e. the moral component, including the normative expectations involved). Imagine a trustor who conveys the impression that a task he/she entrusts the trustee to do is for the trustee’s benefit and easy to fulfill. If the trustee engages in this “trust” relationship and fulfils the entrusted act, which turns out to be neither beneficial for her nor easy, she has been exploited by an untrustworthy trustor. JUSTIFICATION AND RATIONALITY CONDITIONS FOR EVALUATING AGENTIVE TRUSTWORTHINESS JUDGMENTS We start with two intuitions about the nature of the relationship between trust and trustworthiness. The first intuition is that it is rational for an agent to trust those who are trustworthy and irrational for an agent to trust those who are not trustworthy. But agents may get it wrong about which agents are, and which agents are not, trustworthy. The second intuition about the relation between trust and trustworthiness is that where an agent has sufficient evidence to believe of an agent that she is trustworthy, that agent is justified in his belief that the other agent is trustworthy. Notice that it might be rational for an agent to trust another agent even though there is not sufficient evidence for believing that the agent is trustworthy. And it might happen that an agent has sufficient evidence for believing that another agent is trustworthy, yet it is not rational to trust that agent. Thus we distinguish two distinct kinds of trust—rational trust and justified trust. These two features of the relation are necessary conditions—they are invariant features of that relationship. There are two different ways in which A would not be rational to believe that B is trustworthy even though A has sufficient evidence to justify the belief that B is trustworthy. The first sort of case will involve A having false beliefs. For instance, A might falsely believe that tomorrow B will succumb to some psychological illness that will make her untrustworthy even though, today, there is sufficient evidence that B is trustworthy. The second sort of case does not involve A having any false beliefs. A has ample evidence that agent B is trustworthy in delivering notes to other people, so A is justified in believing that B is trustworthy in that way. A asks B to deliver a note (in a sealed envelope) to B’s wife. The note contains a declaration-of-A’s-love for B’s wife. There is a slight chance the envelope will open while being handled. B, though trustworthy in delivering notes to other people, is quite a jealous fellow. If B discovered the contents of the letter, not only would he not deliver it to his wife, but A’s life would be in jeopardy. So A is not rational to trust B to deliver the letter. Notice that the small probability the envelope will open does not diminish in any way A’s justification for believing that B is trustworthy in delivering notes to other people, but it does suggest that it would be irrational for A to trust B to deliver the letter. We can also find cases where it is rational for A to believe that B is trustworthy even though A does not have sufficient evidence to justify her claim that B is trustworthy. Here, too, the cases divide into two. The first sort of case is one in which A has false beliefs. For instance, A falsely believes that B will engage on a trustworthiness building program tomorrow, even though, today, there is not sufficient evidence to warrant A in believing that B is trustworthy. The second sort of case is one in which A does not have any false beliefs (about B), but has needs which require that B perform some task for A. This need creates a compelling reason for A to believe that B is trustworthy even though A does not have sufficient evidence to justify that belief. What is the importance of distinguishing between judgments of trustworthiness that are rational, but not justified, and judgments of trustworthiness that are justified, but not rational? The importance of such a distinction is that it will allow more flexibility and in applying our conception of trustworthiness, as well as broaden the scope of the application of our conception. There will be cases in which A judges B to be trustworthy in doing such-and-such, because it is rational to do so, even though A does not have sufficient evidence for that judgment, as well as cases in which A judges B to be trustworthy in doing such-and-such, because A has sufficient evidence for the judgment, even though it might not be rational to do so. The difference between the two judgments can be explained in terms of the failure of justification conditions for trustworthiness (with respect to competence or skill A) to align with rationality conditions for trustworthiness (with respect to competence or skill A). We will make one additional remark, which provides the basis for a transition to the final section of our paper. Consider a case in which an agent is judged to be trustworthy, where the judgment is not justified in terms of available evidence, but is rational to make, in terms of what the trustor expects of the agent who is the trustee. Here is an example. Jack judges that Phil is trustworthy in giving directions in Thai. However, Jack does not have enough evidence to arrive at this judgment, since the only available evidence he does have is that Phil has started a course in Thai, and would like to become conversant in Thai. However, Jack knows that if he, now, does not take Phil to be trustworthy in giving directions in Thai, that Phil will become despondent (or worse), and will not only drop the course in Thai, but will compromise Jack in several ways that Jack will come to regret. So it is rational for Jack to judge that Phil is trustworthy with respect to giving directions in Thai, even though Phil has not yet achieved full competence in giving directions in Thai. We do not want to dismiss rational judgments of trustworthiness which are not justified by the evidence. One way to accommodate such judgments is to posit degrees of trustworthiness, which we will discuss in detail in the last section of our paper. WHY TRUSTORS (NECESSARILY) NEED TO BE TRUSTWORTHY On the relational conception of being trustworthy, an agent is related (in the appropriate way) to a context-specific and context-determined competence that has a normative dimension. On our view, an agent can be trustworthy with respect to one competence package, but not trustworthy with respect to another competence package. For instance, agent A can be trustworthy with respect to giving directions in German, but not trustworthy with respect to giving directions in Thai. The reason why the package has a normative dimension is that competence failures are not the only reason to reject an agent being trustworthy. An agent can be competent, but fail to have the proper normative properties. For instance, agent A might be fluent in Thai, but dislike Thai people, and so not be trustworthy in giving directions in Thai to Thai people, even though the same agent is trustworthy with respect to giving directions in Thai to German speaking people who also speak Thai. This view differs from the standard view in the literature, which takes trustworthiness to be a either a disposition or a property/characteristic of the agent such that the agent is trusted in any context, to do such-and-such. On the standard view, a skill or a competence is conceptualized as a property (or set of properties) of the agent. These properties are shared by everyone who is defined to be trustworthy—so it is unlikely that specific or context-sensitive and context-determined skills and competences would show up in a definition of trustworthines on the standard view. Only the properties which are shared by everyone who is trustworthy with respect to anything at all will be included in a definition of trustworthiness. On the standard view, a definition of trustworthiness will apply equally to someone who is trustworthy with respect to giving directions in Thai (but not in German) and to someone who is trustworthy with respect to giving directions in German (but not in Thai). This view is too strong (and, so, the properties which define trustworthiness are too weak)—what conditions would guarantee that an agent is trustworthy to that extent—that is, to the extent that all agents who are trustworthy share those properties? On our view, there are different context-specific and context-determined competence packages, and thus there is no need to worry about a general set of features which would allow for trustworthiness across the board. For instance, the agent who is trustworthy with respect to giving directions in Thai is also sincere, and will not deceive someone when giving directions in Thai. A deceiver who is competent in giving directions in Thai could not be trustworthy in giving directions in Thai. On our view, it is necessary that a trustor be trustworthy in the following way: that this trustor be able to determine in a reliable way when other agents are trustworthy (with respect to some contextsensitive and context-determined package). For instance, agent A is a trustor who cannot reliably determine that another agent is trustworthy with respect to fixing his car. It might not be either rational or justifiable for another agent to be entrusted by that trustor to fix his car. Here is one way in which it would not be rational for that agent to be entrusted to fix the trustor’s car: the auto mechanic agent is incompetent, but the trustor takes him to be competent. However, if the auto mechanic does attempt to fix (and consequently ruins) the trustor’s car, he will be faced with a devastating lawsuit, as well as word of mouth destruction of his reputation and his automobile repair business. Here is another way in which it is not rational for the mechanic to be entrusted with the trustor’s car. Although the auto mechanic is competent in repairing automobiles, and the trustor reliably determines this to be so, the trustor does not understand the normative aspects of trust, and so reacts in normatively strange ways to the perfectly good repair work which the auto mechanic does to the trustor’s car. It would be irrational for the trustee auto mechanic to agree to repair the trustor’s car, since there are too many volatile problematic ways in which the trustor might strangely react to that work, which would make life miserable for the trustee. More generally, a trustor needs to be trustworthy, otherwise he might exploit a trustee in various ways. Secondly, where a trustor is not good at determining how to apply the concept of trustworthiness in specific cases, normatively strange things can occur. That lack of ability could be attributed to various faults in the trustor, one of which is an inability to understand the normative components of the trust relation. If the trustor does not understand what it is to have a normative expectation, then she will not be able to make either justified or rational judgments of trustworthiness in others. If so, she fails to be trustworthy. It is important to see that the trustor’s ability to determine the trustworthiness of a trustee is contextsensitive and context-determined. That is, the trustor does not have a magical ability to determine the trustworthiness of any agent for any kind of activity to which that agent might be entrusted (as it would be on the standard view—which is another reason we have for rejecting the standard view). Rather, the trustor must be reliable in determining the context-sensitive and context-determined package with respect to which the trustee is trustworthy. This means that the trustor must be competent in determining that an agent has a particular skill or competence for which he is considered trustworthy, and that the trustor is competent in determining that the agent is, say, sincere, and not a deceiver. Two important consequences follow from this condition on our relational view of trustworthiness. The first is that there is reciprocity between trustor and trustee. Call the context-sensitive and context-determined competency package (with respect to which a trustee is trustworthy) C. A trustor must be able to reliably determine C. For instance, the trustor’s judgment that the trustee has C can be justified (by epistemic evidence) or rational (with respect to life goals a trustor might have). The reliable determination of C by the trustor is itself a context-sensitive and contextdetermined competency package which is the reciprocal of C. Call it C*. That both C and C* have this reciprocal structure allows us to solve a vexing problem which has been posed by Karen Jones: how can a trustee identify herself to a trustor as someone who is trustworthy? Unless there is reciprocation between trustor and trustee respect to trustworthiness, there will be no non-circular means of identification. We believe that our way of avoiding the trap of circularity is a significant contribution to the literature on trust and another reason to think that our relational conception of being trustworthy is superior to the standard conception. The second important consequence is that the trustor is evaluated with respect to a context-sensitive and context-determined competence package, and not some general (and possibly hard to specify and epistemically act upon) set of psychological and normative features (which we would have in the standard view of trustworthiness). This means that the epistemic reliability of trustworthiness judgments is epistemically easier to establish than in the cases which would occur under the standard conception of trustworthiness. Consider: a trustworthiness judgment is made by a trustor who takes an agent to be (or not to be) trustworthy. Jack judges that Phil is NOT competent in giving directions in Thai, because Jack is fluent in Thai and successfully tried to teach Phil Thai. So Jack’s judgment that Phil is not trustworthy in giving directions in Thai is epistemically reliable. Why not simply say that Jack’s judgment is reliable? To add that it is epistemically reliable is to say that the reliability of Jack’s judgment had good epistemic grounds—had good evidence for being true. The epistemic reliability of a judgment of being trustworthy can get factored into determining the competence or skills of an agent and into determining the normative aspects (such as moral values) of an agent. There is also the question of what a competence for determining competences should look like, but that would take us well beyond the scope of the present paper, and we leave it for another paper. ARTIFICIAL AGENTS, TRUSTWORTHINESS, AND DEGREES OF TRUST Thus far, we have focused our discussion on trustworthiness mainly with regard to human agents (HAs), where these agents could be evaluated as either trustworthy or untrustworthy in virtue of their satisfying or failing to satisfy the required epistemic and moral conditions for trustworthiness (both for the trustor and the trustee). But we have not yet explicitly considered questions concerning trustworthiness in the context of artificial agents (AAs). So, in this section, we ask whether (and if so, how) AA’s could be evaluated as trustworthy agents. Initially, it might seem plausible to think of some kinds of AAs, but not others, in terms of trustworthiness/untrustworthiness. For example, one might conjecture that an autonomous AA, or what we have elsewhere referred to as a functionally autonomous AA (FAAA), could be capable of being trustworthy, while a less sophisticated AA (and especially a diffuse and widely distributed multi-agent system) would not able to satisfy the conditions required for trustworthiness. Or, alternatively, one might assume that in the case of all AAs, questions pertaining to trustworthiness are not applicable, one way or another. We begin our analysis by recalling two claims regarding the concept of trust vis-à-vis AAs that were briefly mentioned in the introductory section of this paper: (1) HAs can enter into trust relationships with AAs—at minimum, they can enter into indirect (and minimal) trust relationships with AAs in “diffuse-default zones of trust” (e.g., zones or contexts that can include multi-agent systems comprising both AAs and HAs); and (2) HAs are capable of having a much more direct, transparent, and robust trust relationships with certain kinds of FAAAs. Since various “degrees of trust” are possible in HA-AA trust relationships, or what Carr (2013) refers to as “inter-agential trust relationships,” one could reasonably ask whether it is also possible to speak of degrees of trustworthiness involving AAs. Before answering this question, however, we first briefly describe how degrees of trust in HA-AA trust relationships are possible and how they can be correlated with levels of (ethical) agency. In distinguishing among these various levels or degrees of trust, it is instructive to appeal to a model of (ethical) agency advanced by Moor (2006), which differentiates four levels of ethical agents. Agents, in Moor’s scheme, are organized into the following four categories: (i) ethical impact agents, (ii) implicit ethical agents, (iii) explicit ethical agents, and (iv) full ethical agents. Whereas (i) – the “weakest” sense of (ethical) agent in Moor’s scheme – will have ethical consequences to their acts, (ii) have some “ethical considerations” built into their design and “will employ some automatic ethical actions for fixed situations.” However, Moor notes that (iii) will have, or at least act as if they have, “more general principles or rules of ethical conduct that are adjusted and interpreted to fit various kinds of situations.” Finally, (iv) “can make ethical judgments about a 4 wide variety of situations” and in many cases can “provide some justification for them.” Moor 4 Moor illustrates the first two categories of agent with some specific examples. In the case of an ethical-impact agent, he uses the example of a “robotic camel jockey” (a technology used in Qatar to replace young boys as jockeys, and thus freeing those boys from slavery in the human trafficking business). Two examples illustrating his implicit-ethical-agent category include an airplane’s automatic pilot system and an ATM (automatic teller machine); Moor points out that both technologies have built-in programming code designed to prevent harm from happening – in these two instances, one is designed to prevent physical harm to the passengers and crew onboard an airplane, and the other to prevent ATM customers from being short-changed in financial transactions. describes agents in category (iv) as having the kinds of ethical features that we typically attribute to full-blown ethical agents (or what he describes as “normal human adults”). Moor notes that AAs have not yet (and, for that matter, may never) achieve (iv); he also notes that there is still some debate about whether AAs will ever qualify as (iii). Nonetheless, Moor’s model illustrates is an interesting spectrum of agency with some useful gradients that can also help us to understand distinct levels of trust in HA-AA trust relationships (as they relate to his four levels of ethical agents). 5 We further believe that the strength or degree of trust in HA-AA trust relationships can also be analyzed both in terms of (a) the level of autonomy of the individual AAs involved, and (b) the kinds of interactions (direct vs. indirect) that occur between the HAs and AAs in the trust environment. Regarding (a), we have already suggested that the more autonomous (and more sophisticated) the AA, the higher the level of trust that an HA can accord the AA. With respect to (b), the more direct the physical relation between the two agents, the stronger the trust relationship can be. An additional variable or element affecting degrees of trust in HA-AA trust relationships has to with the kinds of stakes involved in a particular situation. For example, the stakes could range from mere inconvenience for the HAs involved, at one end of the spectrum (i.e., low stakes), to potential loss of life for HAs at the other end (high stakes). To see how degrees of trust can vary in the case of HA-AA trust relationships, consider the following examples involving two very different kinds of AAs: a CareBot (whom we will call “Charlie”), who is dedicated to assisting my 93-year-old father-in-law (Ralph); and a multi-agent system (“Agent X”) responsible for regulating the street lights in the neighborhood in which I live. First, consider the different kinds of interactions that HAs are capable of having with the two AAs. Whereas Ralph (and members of his family) can interact directly with Charlie, HAs are typically able to interact only indirectly with Agent X (for example, via the electric company that is responsible for regulating the street lights in my neighborhood). Additionally, Charlie has autonomy in a functional sense (i.e., “he” is an FAAA) and also has some human-like characteristics built into “his” design, while X is merely a non-autonomous system of software programs that also happens to be “disembodied” (in the sense that it is not physically identifiable as a distinct entity). Next, consider the kinds of stakes for the HAs who are involved in trust relationships with the two very different kinds of AAs. If Charlie fails to perform “his” responsibilities correctly, Ralph could be at risk of not receiving his proper dosage of medication at the appropriately scheduled intervals. If, on the other hand, Agent X fails to turn on my 5 In a separate work (Tavani 2012), these degrees of trust are worked out in more detail. The summary of Moor’s model in this section draws from an analysis included in H. Tavani. Ethics and Technology: Controversies, Questions, and Strategies for Ethical Computing. 4th ed. Hoboken, NJ: John Wiley and Sons, 2013. neighborhood street lights at dusk tonight, I may experience some minor inconvenience (for example, I might be unable to take an evening walk in my neighborhood). In the case of the stakes involving Charlie and Ralph, we have a potential life-threatening situation – Ralph could die (high stakes) as a result of his Charlie’s failure to perform “his” important medical duties. But in the case of the street lights failing to come on at the prescribed time, I (and possibly some of my neighbors as well) may be inconvenienced by having to cancel our evening walks (low stakes). Next we turn to the specific question of trustworthiness for the two agents. (Note that whereas Agent X is an ethical-impact agent, in Moor’s scheme, Charlie would likely qualify as either an implicit-ethical agent or an explicit-ethical agent). We have seen that an HA can have a much stronger trust relationship with Charlie than with X. But is Charlie, who is a far more sophisticated AA than X, also more trustworthy than X? Applying the model of trustworthiness articulated in earlier sections of this paper – one in which trustworthiness was shown to apply both to the trustor and the trustee (in a trust relationship) – we need to take into consideration not only Charlie’s possible trustworthiness/untrustworthiness but also Ralph’s. For example, if Ralph fails to understand some of the normative expectations in trust relationships involving caregivers, or of normative expectations in trust relationships in general, Ralph (as the trustor) may fail to satisfy the conditions of trustworthiness required in our framework. However, since Ralph is an HA, and since trustworthiness issues affecting HAs were discussed in the previous sections, we do not need to focus our analysis here on questions about his trustworthiness. Instead, the important question for us in this section is whether Charlie (or X, or for that matter, any AA) can qualify as a trustworthy agent – i.e., satisfy the requirements specified in our scheme. Recall our earlier distinction involving two components of trustworthiness: (i) trustworthiness in an epistemic sense and (ii) trustworthiness in a moral sense. Both Charlie and X would seem capable of being evaluated in terms of trustworthiness in an epistemic sense, i.e., in virtue of each agent’s reliability and “competencies.” But can either agent also be trustworthy/untrustworthy in a moral 6 sense ? First, we note that neither Charlie nor X could qualify as a full moral agent (in Moor’s fourfold scheme). Also, consider that while both Charlie and Agent X can “let me down,” or can “disappoint me,” neither AA can – i.e., has the ability to – “betray” my trust. So, it would seem odd to say that either Charlie or X could qualify as trustworthy/untrustworthy in the moral sense. But while neither Charlie nor X are full moral agents, they are nonetheless “normative agents,” whose epistemic trustworthiness can be evaluated in terms of competence, reliability, etc. 7 6 Note that by “moral sense” here, we mean moral vs. non-moral. So “moral” is not necessarily equated with “morally good,” but rather means being able to be evaluated by moral categories as either “morally good” or “morally bad.” 7 Many of the claims made in this section have benefited from discussion with Lloyd Carr. See Carr (2013) for his accounts of trust and trustworthiness. Initially, we might have assumed that Charlie would be capable of being more trustworthy than X (even if only in an epistemic sense), because Charlie is a more sophisticated AA, who also possess some human-like features, and since HAs would be capable of having a stronger trust relationship with Charlie than with X. However, we have seen that in the case of all AAs, there are no corresponding degrees of trustworthiness (in general) that correlate with the degrees of trust that we articulated. So we can infer that there is no necessary connection between the sophistication of an AA and its degree of trustworthiness. The degree or level of trustworthiness for any AA will only be able to be evaluated in an epistemic sense, i.e., in terms of the AA’s reliability and competence. So, it turns out that, epistemically, Agent X can be more trustworthy than Charlie even though HAs can establish a stronger trust relationship with Charlie than they can with X. For example, consider that X could be part of a vast multi-agent network (say, a power company) that includes both AAs and HAs, and which, over time, has proven far more reliable in performing its functions than the generation of Carebots (that include Charlie) have been in performing their tasks. Based on our analysis, one could reasonably conclude that there is only a minimal (or “impoverished”) sense of trustworthiness in trust relationships involving AAs and HAs. As normative agents, AAs can clearly satisfy the epistemic component of trustworthiness; but as “nonmoral normative agents,” they cannot satisfy the requirements for the moral component. Only full moral agents – i.e., only humans (at least at this point) – are capable of satisfying those requirements and thus being trustworthy in both an epistemic and a moral sense. In closing, it is perhaps worth briefly speculating as to why some HAs might easily be fooled by sophisticated AAs that may sometimes seem to “elicit” trustworthiness. Turkle (2011) refers to this phenomenon as the “Eliza effect” (in light of the “Eliza” software program developed by Joseph Weizenbaum at MIT in the 1960s, which “elicited trust” by tricking users into thinking that they were interacting with a human). Turkle suggests that when sophisticated AAs such as Charlie exhibit characteristics that seem sufficiently human-like, those AAs can sometimes “trick” humans into thinking that they are interacting with other humans and this this, in turn, may influence their 8 thinking that these AAs can be trustworthy agents. However, this kind of reaction on the part of 8 Turkle notes that when an AA (as a “relational artifact”) appears to be interested in HAs, it can cause “people to respond as if they were in a relationship.” This is especially apparent in the case of sophisticated AAs that exhibit facial expressions, such as Kismet (developed in MIT’s AI Lab). Turkle suggests that because AAs can be designed in ways that make people feel as if an AA cares about them (as in the case of Paro, a carebot designed to comfort the elderly), people can develop feelings of trust in, and attachment to, those AAs. For example, she notes that Cynthia Breazeal, one of Kismet’s designers, had also developed a “maternal connection” with this AA while she was a student at MIT, and that Breazeal had a difficult time separating from Kismet after graduation. In Turkle’s view, this kind of “attachment” raises questions having to do with trust and authenticity – and perhaps we could add questions having to do with “trustworthiness” as well! some HAs would seem to have more to do with possible psychological/affective states in humans than it does with the logic of trustworthiness. CONCLUSION In this paper, we have argued for a novel account of trustworthiness in trust relationships. We saw that in many conventional accounts, trust is viewed as relationship between a trustor and a trustee, while trustworthiness is considered a property or characteristic of the trustee only. In our account, however, trustors must also satisfy conditions affecting both epistemic and moral criteria in order to be judged trustworthy. So, trustworthiness can also be viewed in a relational way that involves two entities and that includes both epistemic and moral requirements that must be satisfied on the part of the trustor as well as the trustee. With respect to inter-agential trust relationships involving HAs and AAs, on the other hand, we argued that AAs are capable of satisfying only the epistemic component of trustworthiness and thus cannot be evaluated as being either trustworthy or untrustworthy in a moral sense. Even though AAs do not qualify as moral agents, we saw how they nevertheless qualify as normative agents that can be evaluated in terms of their competence and reliability. SHORT BIOGRAPHIES: Jeff Buechner is a member of the Department of Philosophy, Rutgers University-Newark and a Fellow of the Saul Kripke Center, CUNY, The Graduate Center. He is the author of Gödel, Putnam and Functionalism, MIT Press, 2008. Judith Simon is senior researcher at the Karlsruhe Institute of Technology and the University of Vienna. She is also associate postdoctoral fellow at the Institut Jean Nicod in Paris and has held visiting positions in the US (Stanford), Italy (Trento) and Catalonia (CSIC-IIIA Barcelona). Currently, she is researching the relationship between trust, knowledge and ICT as Principle Investigator of the project “Epistemic Trust in Socio-Technical Epistemic System” funded by the Austrian Science Fund. Herman Tavani is Professor Emeritus of Philosophy at Rivier University and a visiting scholar at the Harvard School of Public Health. He is the author of Ethics and Technology (Wiley), now in its fourth edition, and the editor or co-editor of four books on ethical and social aspects of information technology. ACKNOWLEDGEMENTS This research was supported by the Austrian Science Fund (FWF): P 23770-G17. REFERENCES Baier, A. C. (1986). "Trust and Antitrust." Ethics 96(2): 231-260. Baier, A. (1991). "Trust and Its Vulnerabilities & Sustaining Trust". Tanner Lectures on Human Values, Salt Lake City, University of Utah Press. Buechner, J. and H. Tavani (2011). "Trust and multi-agent systems: Applying the ‘diffuse, default model’ of trust to experiments involving artificial agents." Ethics and Information Technology 13:39-51. Carr, L. (2013). "Trust: An Analysis of some Aspects." Available at http://www.rivier.edu/faculty/lcarr/Trust%20%20an%20analysis%20of%20some%20aspects.pdf. Castelfranchi, C., R. Falcone, et al. (2006). "Being Trusted in a Social Network: Trust as Relational Capital." iTrust 2006 Lecture Notes on Artificial Intelligence (LNAI) 3986: 19-32. Daukas, N. "Epistemic Trust and Social Location." Episteme 3.1-2 (2006): 109-24. Gambetta, D. and H. Hamill (2005). Streetwise. How taxi drivers establish their customer's trustworthiness. New York, Russell Sage Foundation. Hardwig, J. (1991). "The Role of Trust in Knowledge." The Journal of Philosophy 88(12): 693-708. Hardin, J. (1996). "Trustworthiness." Ethics 107: 26-42. Hardin, R. (2002). Trust and Trustworthiness. New York, Russell Sage Foundation. Holton, R. (1994). "Deciding to trust, coming to believe." Australasian Journal of Philosophy 72(1): 63-76. Jones, K. (1996). "Trust as an Affective Attitude." Ethics 107: 4-25. Jones, K. (2012). "Trustworthiness." Ethics 123(1): 61-85. Lahno, B. (2001). "On the Emotional Character of Trust." Ethical Theory and Moral Practice 4(2): 171-189. Moor, J. H. (2006). "The Nature, Difficulty, and Importance of Machine Ethics." IEEE Intelligent Systems 21 (4): 18-21. Parfit, D. (2011). On What Matters. Volume 1. Oxford, Oxford University Press. Pettit, P. (1995). "The cunning of trust." Philosophy and Public Affairs 24: 202-225. Potter, N. (2002). How Can I Be Trusted? A Virtue Theory of Trustworthiness. Lanham: Rowman & Littlefield. Sabater, J. and C. Sierra (2005). "Review on Computational Trust and Reputation Models." Artificial Intelligence Review 24: 33-60. Simon, J. (2010). "The entanglement of trust and knowledge on the Web." Ethics and Information Technology 12(4): 343-355. Simon, J. (forthcoming). Trust, In: Pritchard, D. (Ed.): Oxford Bibliographies in Philosophy. New York: Oxford University Press. Taddeo, M. (2010). "Modelling Trust in Artificial Agents, a First Step toward the Analysis of eTrust." Minds and Machines 20(2): 243-257. Tavani, H. (2012). "Degrees of Trust and Levels of Moral Agency (in the Context of Machine Ethics)." Paper presented at the Second Annual International Symposium on Digital Ethics, Loyola University, Chicago, Illinois, October 29. Tavani, H. & J. Buechner (forthcoming). "Autonomy and Trust in the Context of Artificial Agents." In M. Decker and M. Gutmann, eds. Evolutionary Robotics, Organic Computing, and Adaptive Ambience. Berlin, Germany: Verlag LIT. Turkle, S. (2011). "Authenticity in the Age of Digital Companions." In M. Anderson and S. L. Anderson, eds. Machine Ethics. New York: Cambridge University Press, 62-78. Walker, M. U. (2006). Moral repair: Reconstructing moral relations after wrongdoings. Cambridge, MA, Cambridge University Press.