Handling Working Memory Knowledge Through a Consultant-Level Resource Management Strategy

Working memory is an important component of cognition that influences key cognitive processes, such as language. As such, working memory should play a key role in cognitive models for language-capable robots. The ways in which working memory buffers are organized within a robot's architecture can inform processes such as Referring Expression Generation. Thus, it is important to understand how information and resources within working memory may be organized to lead to human-like robotic language. Previous work on the DIARC cognitive architecture described an entity-level, feature-based working memory framework in which each known entity had its own dedicated working memory buffer. This paper expands on that framework and proposes a new resource management strategy in which sets of entities that belong to the same type share a single working memory buffer. We end the paper with a brief discussion of how this novel strategy compares to the previously implemented entity-level strategy.


INTRODUCTION AND MOTIVATION
Cognitive models for language-capable robots may borrow inspiration from human cognition in order to promote more natural, human-like dialogue.Working memory is a pervasive component of human cognition [3], infuencing key cognitive processes, including language [cf. 1, 2, 6, 9-11, 14, 18].Given its importance to cognition, working memory is commonly featured in robot cognitive architectures and plays a major role on the intrinsic decision-making processes that guide an agent's behavior.In this paper, we directly build upon recent work on one of these architectures, DIARC [15], which has historically implemented cognitively-inspired models of working memory to facilitate the process of Referring Expression Generation [16,22].For example, activated working memory contents may be prioritized by a robot speaker when generating referring expressions to facilitate hearers' understanding.
One of the main characteristics of working memory is its limited capacity, which is usually estimated to range from 4 to 9 items [5,13].This low storage capacity and the volatility of working memory suggest that its contents are subject to constant updates.Therefore, it is important to understand how items leave working memory in order to give space to more salient items.As such, DIARC's working memory models are centered around the mechanism of forgetting, which performs the maintenance of working memory through two cognitively-inspired strategies.The frst strategy is decay [4,8], in which items within working memory are removed from bufers over time.The second strategy is interference [7,19], in which older working memory items are replaced with newer entries if not rehearsed or reiterated.
Finally, instead of storing salient entities, DIARC's working memory bufers store the features of such entities to prioritize the quality over the quantity of working memory representations [12].This feature-based perspective introduces diferent structural options for how working memory bufers are distributed across the architecture.In previous work, Williams et al. [22] described an entity-level, feature-based framework in which each entity known by the architecture has a dedicated working memory bufer that will hold salient features.In this work, we formally defne a consultant-level framework (see Section 2.1) that instead assigns working memory bufers to diferent entity types.These bufers are shared by all entities that belong to their corresponding type.For example, if a robot has information about three humans in a knowledge base, these three entities would share the same "people" working memory bufer.More details about DIARC's knowledge bases and working memory are provided in the next section.
After formalizing and introducing our new strategy for working memory bufer organization (Section 3), we perform a discussion of how our model compares to the entity-level framework described in previous work (Section 4).Finally, we state our concluding remarks and a few directions for future work (Section 5).

THE DIARC ARCHITECTURE
In this section, we briefy introduce how DIARC's knowledge bases and working memory component work.This information is necessary to understand how our new resource management strategy functions and informs robot decision-making.

DIARC's Knowledge Bases and Consultants
In DIARC, world knowledge is not handled by a single knowledge base, but is rather split across diferent Distributed Heterogeneous Knowledge Bases (DHKBs) [23], allowing each diferent type of knowledge to be handled in the way that suits it best.This decentralized knowledge organization creates the need for a special type of architectural component called consultants [20].Consultants flter the domain-specifc information from these DHKBs into frst-order logic statements that can be understood and used by any other component within the architecture.Thus, when other architectural components require certain information from a DHKB, a consultant will only share the necessary domain-independent information in frst-order logic format with them.For example, in a task where a robot needs to formulate a description for an object that it knows as object_1, the consultant responsible for storing the features of object_1 may be queried by other DIARC components and return a list of frst-order logic statements that can be used in a description (e.g., object_1 = [blue(X), mug(X)]).

DIARC's Working Memory Manager
The Working Memory (WM) Manager is a component responsible for implementing forgetting and maintaining working memory bufers across the architecture.Previous work [22] has described an entity-level, feature-based implementation for the WM Manager.In this work, we expand this concept with a formal defnition of the WM Manager component that includes a consultant-level resource management strategy.
Our formal defnition of the WM Manager introduces two new features to this component.First, the parameter , which specifes the active resource management strategy within the architecture.Second, in the initial framework the interference parameter specifed the size limit for each entity's working memory bufer.With the introduction of the consultant-level strategy, now specifes the upper bound for the number of total working memory properties per consultant.As such, when = { }, each consultant ∈ will have a dedicated working memory bufer ⊆ such that * We use instead of to avoid confusions with previously established notation.
| | ≤ for managing constraints that span all entities in .On the other hand, when = {}, each entity within a consultant will have a dedicated working memory bufer ⊆ such that | | ≤ ⌈ ⌉ for managing salient constraints.

CONSULTANT-LEVEL FRAMEWORK
In this section we describe how features are encoded and removed from Working Memory bufers at a consultant-level.

Encoding
In a consultant-level working memory model, features are added to a consultant's working memory bufer when they are used to describe an entity within that consultant's domain.The pseudocode for this process is listed in Algorithm 1.
Algorithm 1 Consultant-level encoding for all ∈ do 4: if ∈ and ∈ then else if ∈ and ∉ then 8: push( , ) Ð When a referring expression ⊆ is generated ∈ or processed by the robot architecture (line 1), for each consultant ∈ (line 2), each constraint ∈ (line 3) is added to 's working memory bufer if it is part of the list of constraints that can be handled by (i.e., ∈ ).If is already in the working memory bufer, then its recency is updated (lines 4-6).Otherwise, it is added to the working memory bufer (lines 7-8).

Removal
The removal of constraints from working memory bufers is governed by the forgetting parameters specifed in = [, ].When decay is in use (i.e., ≠ ∞), the least recent property from a bufer is removed every seconds.When interference is in use (i.e., ≠ ∞), the least recent constraints are removed from a bufer that exceeds the maximum capacity .Algorithm 2 provides the pseudocode for this process.
Algorithm 2 Consultant-level removal  for all ∈ do  For each consultant ∈ (line 2), if seconds have passed and there exist properties in , then the least recent property is removed from (lines 3-4).In addition, if the size of exceeds the maximum working memory bufer size , the least recent properties are removed until bufer capacity matches (lines 5-6).

DISCUSSION
We will now briefy discuss how the consultant-level strategy compares to the previously proposed entity-level strategy in terms of their spatiotemporal needs and cognitive plausibility.While the consultant-level strategy seems to carry a better computational efciency and a structure that is closer to the psychological defnition of working memory, there are a few downsides of using it instead of the entity-level strategy.

Spatiotemporal Needs
The spatial and temporal needs of the consultant-level working memory resource management strategy may be signifcantly lower than those of the entity-level strategy.This is observed because in the latter strategy one working memory bufer will be assigned to each entity known by the architecture (see Figure 1a).As such, in situations where the number of entities known by a consultant is considerably high, the processes of encoding and removal may take a while longer to be completed.Furthermore, a consultant-level strategy reduces the number of required working memory bufers according to the entity types that are known by the architecture (see Figure 1b).Depending on the level of abstraction between diferent entity types, the total working memory capacity for the entire architecture may be known in advance.For example, if an architectural model adopts the taxonomy for core knowledge proposed by Spelke and Kinzler [17], there will be at most fve active consultants within it (i.e., people, locations, objects, actions, numbers).Thus, the total working memory capacity for the architecture will have an upper bound of 5 elements.

Cognitive Plausibility
A consultant-level resource management strategy is closer to cognitive psychology models of human working memory in terms of storage than the entity-level strategy.Specifcally, the entitylevel resource management strategy is more likely to bring overall storage capacity above the speculated human limit even with low bufer sizes, and requires storage of some information about every known entity in working memory, which is simply not cognitively plausible.While a consultant-level strategy is not dependant on the number of known entities and signifcantly reduces the total working memory storage that is required, it still may overshoot the stipulated working memory capacity for humans, depending on the number of consultants, and on whether traditional human working memory limitations are assumed to hold at the entity or feature level.For example, an architectural model with fve consultants will already surpass the stipulated human capacity if = 2.With this in mind, future work may investigate the feasibility of a global resource management strategy for working memory, in which only one working memory bufer is assigned to all entity types.

Disadvantages of Using the Consultant-Level Strategy
Using a consultant-level strategy in situations where the total number of known entities by the architecture is not high may not be ideal, as it may make referring expression generation less consistent.Since working memory bufers are shared by entities of the same type, the way in which a robot chooses to formulate descriptions of objects will not be as specifc as that of an entity-level strategy.This is due to the fact that in an entity-level strategy, the bufer for each entity will hold the most recent constraints that were used to describe it and thus will promote the repetition of the same properties in subsequent robot-formulated utterances.For example, if an object is described to a robot as "the blue mug," the properties blue(X) and mug(X) will remain in that object's working memory bufer independently of how many other objects are referenced afterwards, and the robot will reuse blue(X) and mug(X) to describe the object when necessary.In contrast, a consultant-level strategy will update the bufer for each entity type whenever an entity of that type is mentioned, and thus may not guarantee a consistent referring expression generation process, as multiple entities may share the same constraints.For instance, if an object is described to a robot as "the blue mug, " blue(X) and mug(X) may be overwritten by the features of other objects that are referenced after the blue mug.This limitation is important as a less consistent process of referring expression generation may hurt human interlocutors' ability to perform reference resolution, i.e., understand which entity the robot is referring to, and may decrease the quality of interaction.

CONCLUSION
In this work, we introduced a consultant-level resource management strategy for the working memory models implemented in the DIARC cognitive architecture.We formally proposed a defnition for the Working Memory Manager component that adds the need to specify which resource management strategy is guiding working memory.Furthermore, we performed a high-level discussion of the consultant-level strategy in terms of spatiotemporal needs and cognitive plausibility compared to the entity-level strategy.We observed that the consultant-level strategy likely promotes better computational efciency both in terms of space and time needed to run the encoding and removal algorithms.In addition, the consultant-level strategy has a higher cognitive plausibility than the entity-level strategy, as less working memory bufers need to be active simultaneously.
The introduction of this model opens opportunities for subsequent work to investigate how a consultant-level resource management strategy impacts important natural language processes within DIARC.For example, future work may assess how the referring expressions of an agent that is using this strategy to guide its working memory are perceived by human interlocutors.An experiment can be designed to compare the utterances generated by an entity-level strategy, a consultant-level strategy, and a baseline framework for Referring Expression Generation that does not use working memory.The experimental results will help identify the best structural choices for robotic working memory models targeted at enabling better language-based interactions with humans.Finally, future work may also consider the implementation of a global resource management strategy that limits working memory information to a single architecture-wide bufer, an organization that will be more cognitively plausible than the currently existing strategies and that may be more appropriate for use in certain social contexts and interactions.
}} represents the active working memory resource management strategy within the architecture.When = {}, an entity-level strategy is active within the architecture.Similarly, when = { } * , a consultant-level strategy is active within the architecture.• = [, ] represents the list of parameter values for Decay and Interference that guide how Working Memory functions.The parameter represents the decay factor associated with Decay and the parameter represents the maximum Working Memory bufer size associated with Interference; • = { 1 , . . ., } represents the set of active consultants
In an entity-level strategy, each entity known by the architecture will have a dedicated working memory bufer.Properties in working memory bufers are used to describe only their corresponding entity.(b)In a consultant-level strategy, all entities of the same type share one single working memory bufer.Properties in a consultant's bufer may apply to diferent entities.

Figure 1 :
Figure 1: Architectural overview of the entity-level and consultant-level strategies with two example consultants.