2.1 Human-autonomy interaction

In teams of human and autonomous agents, we consider three entities: human agent, autonomous agent, and the application. The application is the entity that is subject to shared control or shared influences of both agents. In contrast to others, we do not only consider the application to be a technical system [16], but a broader entity, also covering non-physical applications (e.g., home assistant) or processes potentially featuring non-mechanical, lifeless objects (e.g., manufacturing a part). In many domains the application is equated with the autonomous agent, whereas we understand autonomy as a feature that has a physical embodiment (see, e.g., [17], [18]), e.g., a robot or a car, through which the autonomous agent acts. This leads to a mutual split in between the two agents and the shared control system defined by the application on which the agents act. The interaction in a shared control system may be described as between the autonomous and human agent with the application as a mediator. Hence, the interaction reduces formerly to a dyad between autonomous agent and human constrained by the application. The interaction and shared action or control decision in the shared application, is mediated by means of arbitration [16]. In the legislation domain, Snijders [19] discusses a potential change from human arbitrators to “robiters” (AI arbitrators) in legal practices. He comes to the conclusion, that it is against practical law and technically impossible to hand over legal arbitration to only AI (“it would be a mortal sin” [55, p.242]). In purely technical systems, arbitration is already used either as consensual decision of human and autonomous agent [16] or by employing an AI mediator [20]. In recent work, Mandischer et al. [21] argue that it is impossible for an autonomous agent to arbitrate a decision or at all interact to find a consensus if they are unaware of the human agents capabilities – and vice versa. Figure 1 depicts an arbitration process including the initial perception of capabilities.

Figure 1:

Arbitration between an autonomous agent and a human to control a shared application, analogous to [21]. Arbitration is solving an initial dissent of intents to a consensual control decision. To arbitrate, agents perceive the application and each other – particularly to get insights on the other’s capabilities – and then interact to solve the dissent. The depicted partially autonomous car is an example for such applications subject to shared decision making. There are many other potential applications.

The interaction of the two agents is treated separately to the arbitration itself. Freire et al. [22] propose a cognitive architecture which enables adaptive HRC with the aim of seamless interaction, i.e. intuitive and organic. Seamless interaction is also strongly connected to the flow theory [23], in which the person is absorbed into the (work) task. Recently, Prajod et al. [24] and Chen et al. [25] proposed frameworks which shall allow a human-robot or human-autonomy team to reach the flow state. However, both are more on a conceptual level but indicate an improvement to modern human-autonomy teaming if implemented.

2.2 Assistants for people with disabilities

Already in 1999, Newell and Gregor [26] emphasized that it is important to consider “extraordinary” human-machine interaction (HMI) in the sense of HMI with people with disabilities^[4] (PwD). In many applications, the performance of PwD may be raised by using PwD-centered systems design. They also highlighted, that PwD often develop special skills which may be exploited in customized HMI. In few applications, PwD are already assisted by autonomous agents in work processes. However, most methods address a specific type of disability or work, hence, are less or not adaptive at all. Thus, it requires expert knowledge to adopt such methods into work, which is currently lacking in the industry [27]. Mondellini et al. [8] analyze the behavior of people with autism spectrum disorder (ASD) in collaborative manufacturing processes. They observe deviant behavior patterns from neurotypical work persons, indicating a need for specialized interaction and assistance patterns. Kremer analyzes the participation of PwD in work processes through the use of robots [7] and the allocation of work [28] between them in virtual scenarios, while focusing on learning disabilities. Miralles et al. [29] describe an improvement in performance in PwD through the use of line production. The results can also be transferred to parts of the primary labor market, where processes are typically cycle time bound. Berreta et al. [4] investigate implications for the design process of human-AI workplaces, in particular needs, skills and job identity. Wilkens et al. [9] give recommendations for the implementation of human-AI workplaces in industry. The last two articles are part of the competence center HumAIne, which deals with human-centric AI in the world of work and also considers participation. Nevertheless, there is little applied research on HRC with PwD in work processes. Chutima and Khotsaenlee [30] propose a method for planning and balancing line work incorporating people with and without disabilities as well as robots. However, all agents are spatially isolated. There is no real HRC. Kildal et al. [6] use a robot in scenarios with people with cognitive disabilities to highlight components in work steps using a laser. Again, there is no direct collaboration, but the robot must be able to interpret the work context and progress. Weidemann et al. [31] assist PwD with collaborative robots in workshops for PwD. While their interaction design method is adaptive to person’s individual capabilities, the implementation of the interaction is static, allowing no in-situ adaptation. Further, tasks are assigned to either robot or human, resulting in a non-cooperative process. To this end, the task of adaptive and seamless assistance of PwD by means of collaborative robots remains unsolved. In the following, we will showcase an improved way to design autonomous assistants: through the usage of capability estimation.

3.1 A philosophical perspective

In philosophy, there exist two common perspectives on abilities: the conditional logic and the modal theory. All are more or less semantic expressions of relations between an objective (e.g., do a task) and a condition (e.g., has the ability). In teaming, philosophical analysis of abilities – or in our case capabilities – helps to understand the requirements for abilities to be present. This complements the otherwise technical approaches, we ought to implement in systems engineering. Particularly, the conditional logic helps to better understand and implement relations between capabilities and their reflectors in the actual world.

The conditional logic centers about the logical relation “if a then b” (written a → b) and its inferences used for reasoning on the terms. Conditionals enable Boolean operations and mathematical proofs [33]. However, in material conditionals, inverting the initial statement is not easily done, not only on a language and grammatical but also on a factual level. This leads to, e.g., the paradox of material implications: Some true statements that are intuitively false. In material conditional logic a statement is false only if within the conditional “a then b” a is true and b is false. Thus if a vacuous truth is used, i.e. a conditional of which the antecedent cannot be satisfied. Given the absence of a condition a, which is required for b to make intuitive sense, b is always true as both b and ¬b exist. For example, “if someone else would have written this paper, it would be about autonomous driving”. This statement is trivially true as both possible consequents b and ¬b are plausible – or true. Therefore, instead of binary conditionals, i.a. de Finetti [34] proposes trivalent conditionals, in which the state value of a and b may become unknown. In this case, “a then b” is true only if both a and b are true, and false only if a is true and b is false. All other combinations are undefined. The so-called de Finetti tables (see Table 1) are also used in Bayesian approaches to explain and explore human reasoning models [35].

Adams [36] and Lewis [37] propose two ways of dealing with probabilistic conditionals, mainly based on conditional probabilities. Adams defines the probabilistic conditional P(a → b) for the two Booleans a and b as

In other words: the probability of the statement “a then b” is the same as the conditional probability of b given a. This means, that probabilistic conditionals are fully based in Bayesian probabilities, which will later enable them for further exploitation in our framework. Lewis further proposes a triviality theorem:

i.e. given a compatibility of a with both consequents b and ¬b, the probability of a → b degrades to the unconditioned probability of b. Probabilistic conditionals are further extended to compound conditionals, which feature nested probabilities, e.g., P(a → (b → d)) [38]. These were also applied to de Finetti tables [33].

The modal theory is centered about the possibility of an agent to perform an action, i.e. “[…] for [an agent] x to have an ability a it is necessary, but not sufficient, that it be possible that x does b”^[5] [39]. Hence, also in modal analysis, abilities are subject to possibility – or, more technical, probabilities. These are reflected by Possible Worlds (e.g., Stalnaker [40]). If the statement is true in any possible (i.e. thinkable) world, it is possible. The specific possible world then states an extra condition for the statement: “x has the ability to b only if x does b in some world satisfying d”, where d are the conditions of the possible world that enables P(b|a ∧ d) > 0, and a is the ability that defines the “ability to b” [39]. The sentence states the modal analysis of ability. In fact, conditional logic may be coined a sub-class of the modal theory. Both may be combined into the statement: “x has the ability to b only if x does b in a world in which x tries b that is otherwise maximally similar to the actual world” [39]. The actual world is one of the possible worlds that equals the real world [40].

3.2 An occupational perspective

To give a less abstract perspective on actual work, our methodology mainly incorporates standards from occupational medicine and analysis. The IMBA documentation procedure [11] defines a set of 70 top-level capabilities sorted into nine categories. Some top-level capabilities have subordinate capabilities, e.g., trunk movement has the subordinates rotation movements while sitting, rotation movements while standing, and bending/straightening. Hence, capabilities are representatives of elemental abilities of the human. IMBA defines a scale {0, 1, 2, 3−, 3+, 4, 5} on which the human’s individual capabilities and the requirements in the work task are rated. Larger values indicate better fulfillment of the capability. If the person’s capability equals or exceeds the defined requirement for each capability involved in a work task, the person is able to fill in the according job position. The distribution of values is based on the normal distribution, hence, 3− and 3+ characterize the average worker. The evaluation of a person’s capabilities is depending on direct influencing factors (DIs), framework conditions (FCs), and degrees of freedom (DoF). DIs are derived from the work task and indicate mostly physical work characteristics, e.g., frequency and duration of a task. FCs indicate which opportunities for variation the work person has within the work process and DoF indicate which of these opportunities the person can utilize. These dependencies are depicted in Figure 2a. IMBA is used to indicate in which work tasks a person has insufficient capabilities. Therefore, it may be used to allocate tasks between the human and an autonomous agent, as demonstrated by Hüsing et al. [5]. “Capability profile for the integration of people with disabilities into work”^[6] (MELBA) [42] is a variant of IMBA, which focuses on key qualifications. The documentation procedure employs a progressive scale {1, 2, 3, 4, 5} to evaluate its 29 capabilities. Achterberg et al. [43] studied the inter-rater reliability of physicians applying MELBA. They observed good reliability for most capabilities. These results are also indicators for the related standard IMBA.

Figure 2:

Influences on the human’s performance according to IMBA and ICF. Arrows point in direction of influence, e.g., framework conditions influence DOFs. (a) Influences on a capability in IMBA. Usually a framework condition defines the opportunity for a DOF which may then be used by the work person. (b) Influences on the health condition in ICF. A core concept of ICF is that the majority of factors influence each other indicated by bidirectional influences. The model is based on the biopsychosocial model [41].

ICF is a standardized classification procedure issued by the WHO [10]. ICF is more generalist than IMBA and MELBA. It indicates how a person’s performance is influenced by body functions, activities, and participation. These again, are influenced by personal factors and the environment (see Figure 2b). Therefore, ICF accounts for more influences from indirect factors. IMBA is more focused on work-related influences loosing the broader scope on a person’s social and societal environment, and living conditions. In contrast to IMBA and MELBA, ICF is open source and developing into a de facto standard on the classification of people with disabilities. Hennaert et al. [44], [45] proposed a matching of IMBA onto ICF, in which all physical capabilities were found to have a representation between standards. Note that ICF operates on a linear decreasing scale {0, 1, 2, 3, 4}, in which 0 is a regular capability and there is no indication of better then average performance. In fact, ICF does not declare the rating of a capability but the severity of an impairment, i.e. 0 indicates no impairment on a capability.

4.1 Capability framework

Both standards discussed in Section 3.2 have pros and cons regarding the estimation of capabilities. On the one hand, ICF defines broad influences also taking into account meta-data that enriches the decision making of a potential AI method. The relation between influences and health is indicated but not described quantifiably. The more than 1,400 health indicators are too complex to properly be modelled, whereas only a subset is really relevant for capability estimation in the work context. In addition, ICF’s scale is incapable of modelling better than regular capabilities, which looses the nuancing in more demanding work processes. IMBA, on the other hand, has less, more work-focused capabilities, a more nuanced scale, and better description of the relation between influences and capabilities. However, still the majority of influences are not quantifiable and assessment is based on the experience of the occupational physician. In addition, IMBA is closed source, which prevents a widespread usage. Therefore, in a capability estimation model, both standards shall be combined. The lack in quantifiability, which is subject to both standards, is mitigated by the use of machine learning. We theorize, that the observed features of the target human and the rating by the occupational physician can be correlated if sufficient data is available. Hence, a machine learning method will learn to imitate the reasoning of the physician. However, we do not want to train such AI end-to-end, but use as much predetermined dependencies and influences as possible. Therefore, our aim is to combine the standards with machine learning. Figure 3 shows the semantic influences between aspects of the capability framework used for this cause. Within the semantic model, there are five main components:

• Resources: The data that defines the baseline or rationale of the process and capacity. It is taken from ICF and indicates how the person’s environment and opportunities for participation influence the framework of work. The activity indicates how motivated the person is in taking potential opportunities in the framework. Body functions is the functioning of human body parts, including limbs and organs. Resources are less measurable by means of sensors, but meta-data defined prior to capability estimation.

• Process: Analogous to IMBA as indicated in Figure 2a.

• Capacity: The capability of the human isolated from any external restrictions or aids established by the interaction context.

• Performance: The capability of the contextualized human under influence of external factors. The performance may be raised by using a robot as assistive device.

• Reflectors: The reflectors of the human’s performance within observable features by means of sensing, e.g., gait. It may be required to build chains to infer performed capabilities from observed features as not all are directly observable. For example: strength is not directly observable by means of just RGB cameras, but may be inferred from semantic object relations, human posture, or even facial expressions of stress. Note, that many reflectors are the inversion of the DIs in IMBA.^[7]

Figure 3:

Capability Framework combining the ICF and IMBA standard and indicating, that the robot may influence the humans performance as part of the interaction context.

This structure promotes an interesting observation: While the capacity is bound to the individual agent, the performance is an amalgamation of diverse contextual factors including other agents. We could now be tempted to account the performance directly to the team. However, the context defined in the process may be different for each agent. The same accounts for the resources. For example: an aid placed at a work station may not be accessible for a robot or not contribute to its capabilities. The autonomous agent will also not be subject to societal factors regarding the use of aids and, particularly, not peer-pressured into, e.g., refraining to use aids. On the contrary, once we define another agent as part of the context, the performance is not separable for the agents anymore. It is possible to assess the ratio of contribution put into the shared performance [16], [21], but the outcome is the performance of the team. Thus, if we apply the structure in Figure 3, we need to account for contextual influences (process and resources without influences on capacity) of both agents, but indicate a shared performance. This interaction symbiosis is depicted in Figure 4. The reflectors observed from the team performance are separable by agent, but influenced by the context. Hence, it will be hard to assess isolated human capabilities in case, that the robot is already part of the shared action, as it influences the observed data. In addition, in a team the individual agents may take back their individual performance and act less than possible based on their capacity [21].

Figure 4:

Influences on the agents in a teaming context. The interaction of the agents influences the reflectors. The agents interact as discussed in Section 2.1 and Figure 1.

4.2 Bayesian modelling of capabilities

In the following, we use the notation c _j for a capability, where j refers to a specific capability according to a standard, e.g., c_1.02 is the second capability in the first category of the IMBA standard, i.e. Standing. Capabilities are closely connected to capacity (person isolated from context) and performance (person in context) according to the ICF standard (see Section 3.2). We, define the quantification of a capability, i.e. the rating of the capability according to an agent, as capacity c j i or performance c ̂ j i , accordingly. The specific agent is denoted by the superscript i. Based on the framework introduced in Section 4.1, we show the example modelling of the capability Standing. Table 2 lists all capabilities in the complex “body posture” according to the IMBA handbook [11], including their FCs, DoF, and DIs. The capability Standing is denoted c_1.02. Other capabilities are listed to give a better grip on the differences within a capability complex.

In contrast to how Bayesian reasoning is usually used, we neither try to find a state on the start or end of a chain, but an intermediate state within the Bayesian network (see Figure 3). The graph is composed of nodes representing features, capabilities, and resources. Capability nodes carry seven states 0, 1, 2, 3−, 3+, 4, 5 according to IMBA. The states of other nodes depend on the specific node type. Most are binary with annotated unknown state, i.e. {1, unknown, 0}. For such extended binary nodes, the probability tables from capability philosophy are applicable. These define the transition of inputs to outputs. Given the state X_k−1 of the predecessor node n_k−1, node n _k may be assigned a value X _k based on the de Finetti table (see Table 1), where X _k is the value of node k. Assume the conditional X_k−1 → X _k or X_k−1 → ¬X _k as a defined precondition, then the probability according to de Finetti with unknown state is

where X _k is the vectorized form of the most probable state, i.e.

For example: to have an option to support the body weight, an aid needs to be present (compare influences 22 and 24 on c_1.02 in Table 2). Therefore, we can define the precondition X i 24 → X i 22 , read “if aids are present, then there is an option to lean or support the body or parts of the body”. If no other influences on X i 24 are present, the conditional probability P ( X i 22 | X i 24 ) of the precondition is according to Equation (3). Defining parts of the graph this way lowers the overall number of parameters to be trained, with the downside that these parts need to be defined beforehand and become static. In addition, this is only applicable on binary nodes. In case that the binary node does not carry an unknown state, the probability of the unknown state is divided among the 1 and 0 states. However, we advise against removing the unknown state as this would implicate a system without uncertainties despite being subject to uncertainty internally (e.g., within the parameters) and externally (e.g., sensors, learning data).

4.2.1 IMBA components

Purely based on the dependencies described in the IMBA handbook [11] (see Table 2), it is possible to construct a Bayesian network for a capability. As indicated in Figures 2a and 3, framework conditions influence the capability, a DoF, or both. Likewise, reflectors^[8] influence the capability directly. Missing influences are supplemented through logical reasoning, e.g., we argue that the movement space (i₁₈) has significant influence on posture variations (i₂₀) and changes (i₂₁). If the movement space degrades, there are fewer options for different postures while standing. The strength of the influence is learnt in the form of the conditional probabilities within the Bayesian network. We also model the input modalities that are needed to measure or estimate the state values within the nodes. Note that a state value may be unknown and can be inferred within the Bayesian network if sufficient other states are known. The modelled input modalities indicate one option to set the state values. For some states there are multiple options. We choose the modalities in hindsight of using fewest sensors and with an optical camera as main sensor. Figure 5 depicts the model of the capability Standing. All FCs and DoF are subject to meta-data. Some nodes’ state values may be measured as an additional modality or as an alternative data source, e.g., movement space may be taken from the work documentation or a CAD, or be measured in-situ by a (3D) camera. Feeding all DoF and FCs with only meta-data is not reasonable, as this would result in a rather static model, which may be completely unable to detect dynamic capability changes. Reflectors are typically not enriched by meta-data as they tend to change more rapidly than FCs and DoF. For example: a person refraining from using an aid is a slow process compared to fast changes in the duration of a (partial) work task. In prior work, Mandischer et al. [46] modelled this aspect as a Langevin system, in which the uncertainties of the slow system are superimposed by the fast system part. Similar behavior is expected here. As the chains to the fast part (i.e. reflectors) are shorter, less uncertainty is expected due to error evolution. Consequently, the fast system has less intra-model uncertainties. Combined with the Langevin system properties, this may help to reduce the overall uncertainty in the model, i.e. by modelling the fast part with uncertainties, while the slow part is subject to less or no uncertainties in the model.

Figure 5:

Bayesian network of capability c_1.02 (Standing) based on the semantics depicted in Table 2 including input modalities (meta-data and perceived/processed sensor data). Circles depict nodes, rectangles are input data. Flat-back arrows are influences pointing from source to effect. Blue flat-back arrows are de Finetti-style influences. Green acute-back arrows are data streams from sensors, input models, or process meta-data to nodes. Nodes ordered by FCs, DoF, capability, DIs as reflectors (top-down).

In c_1.02, many influences are binary with annotated unknown state: i₀₁, i₀₂, i₀₃, i₀₄, i₁₉, i₂₀, i₂₁, i₂₂, i₂₄. The other nodes may be characterized by discrete sets or discrete ranges (i₀₇, i₁₁, i₁₄, i₁₈, i₂₇). While i₀₇ and i₂₇ are categorized according to the subject, e.g., i₂₇ according to the categories in ISO 20345:2021 [47], and i₁₈ may be categorized in {none, limited, unlimited}, i₁₁ and i₁₄ need to be categorized into value ranges. These ranges are dependent on the capability and task as the time scales may vary harshly. Optionally, these ranges may be converted in qualitative ranges, e.g., {slower, on par, faster}. An overview of node types and number of state values is listed in Table 3. We also annotated the configuration used in the exploration in Section 5.2. The categories shall be selected as detailed as needed, but as few as possible. A reasonable count shall be lower than the categories of the capabilities (here: s c 1.02 = 7 ). This means, that in the suggested case, the node c_1.02 already requires a conditional probability matrix P ( X c 1.02 | X i 01 , X i 02 , X i 03 , X i 04 , X i 07 , X i 19 , X i 20 , X i 21 , X i 22 , X i 27 ) with the size (7 × 229, 635) as there are

(6) ∏ k ∈ N − ( n c 1.02 ) ( s k ) ≤ 229,635

permutations for the predecessor nodes’ (N⁻) state values. However, these may be evaluated with only

(7) ∑ k ∈ N − ( n c 1.02 ) ( s k ) ⋅ s c 1.02 ≤ 252

parameters, which is the agglomerated size of each individual conditional probability matrix P ( X c 1.02 | X k ) , where k ∈ N − ( n c 1.02 ) . This shows that a Bayesian network analyzing only one capability^[9] may be trained with reasonable training data, but a network analyzing multiple capabilities simultaneously may fastly become very complex and demanding to train. Note, that we are interested in an intermediate state (capability). This state’s value has to be inferred.

Table 3:

Types of nodes used for modelling capability c_1.02, annotated with the suggested number of state values and the configuration used later in the exploration (Section 5.2). Binary nodes are extended by an unknown state, range nodes carry ranges of continuous values split into ≤ 7 categories, and discrete nodes have a pre-defined set of discrete/qualitative values.

	Suggestion		Exploration			Suggestion		Exploration
Node	Type	s k	Type	s k	Node	Type	s k	Type	s k
i 01	Binary	3	Binary	3	i 18	Discrete	3	Discrete	3
i 02	Binary	3	Binary	3	i 19	Binary	3	Binary	3
i 03	Binary	3	Binary	3	i 20	Binary	3	Binary	3
i 04	Binary	3	Binary	3	i 21	Binary	3	Binary	3
i 07	Discrete	≤ 7	Discrete	3	i 22	Binary	3	Binary	3
i 11	Range	≤ 7	Discrete	4	i 24	Binary	3	Binary	3
i 14	Range	≤ 7	Discrete	4	i 27	Discrete	5	Discrete	3
c 1.02	IMBA capability			7

4.2.2 ICF components

We have not yet included aspects of ICF into the Bayesian network. Features in ICF may help to overcome some of the limitations of the pure IMBA-based reasoning. If we would only base the estimation of the DoF states on the de Finetti influences, the network would assume that, e.g., if an aid is present, the human would always use it. This is a flaw in deterministic modelling as already pointed out in Section 3.1. The motivation of the human to use the opportunity of using an aid is part of the activity (i.e. motivation) in ICF and may be modelled accordingly. However, the standard does not indicate a quantifiable way to assess the activity of a person. This aspect needs to be learned as part of the conditional probability within the according influence. The need to model and compute motivation in a quantifiable manner has already been discussed in literature [48], [49], [50]. Vijayaraghavan and Roy [51] propose a transformer-based network to track mental states in a conversation which is trained with weakly annotated data. We consider the motivation in a work task to be detectable by means of another language than vocalization: the observable human behavior. If the motion of the human body is considered the semantic language of capabilities including mental states, it is reasonable to use transformer or GPT (generative pre-trained transformer) models to detect states such as the activity or motivation within a work task. The activity may then be considered as the resource that allows a human to take opportunities and manifest their capacity (see Figure 3). Figure 6 depicts the adaptation of one chain in Figure 5 towards ICF. We have omitted other chains for the sake of clarity. The participation is not directly measurable but input by means of meta-data, e.g., from the work documentation or legal documents. The motivation is detected based on the body movement as reflector. The participation hereby is a major influence, as it would otherwise be unclear if the person does not want to, or is not allowed or unable to partake in the work task. The capacity c_1.02 is based on the isolated individual. Therefore, only meta-data on the person (e.g., from initial medical examinations), the presence body parts (e.g., missing limbs), and the body movements (e.g., limping) is required to assess the capacity. The capacity again is the baseline to the performance c ̂ 1.02 (individual in context), which influences the reflectors. It becomes obvious that ICF adds a new level of complexity on (a) the network itself and (b) the methods generating the input for the network. However, particularly the introduction of capacity leads to less variance in the network leading up to the performance. Further, the addition of motivation gives the DoF better clues on the nature of the person’s behavior in taking opportunities. This may later be an effective tool to perform diagnostics on the Bayesian network. The Bayesian network’s property to infer state values gives the method more transparency, which is a required feature if the networks shall be used in a medical product or together with a human.

Figure 6:

Adaptation of the Bayesian network for c_1.02. The depiction focuses on the influences on c_1.02 and nodes n i 22 and n i 22 . The additions cover the ICF resources participation, activity, and body functions. c_1.02 is the capacity, while c ̂ 1.02 is the performance.

5.1 Challenges in data collection

There are two main data-related challenges when working with the IMBA standard: the availability of data and the quality of data. We discuss both aspects in the following and give an example for the basic population required in the Bayesian network.

5.1.2 Bias in capability profiles

Bayesian networks, besides data on the dependencies within the network, require a statistical population within the state to be inferred (here: the capabilities). These are generated from IMBA capability profiles. These are, again, generated by occupational experts. However, as IMBA and similar standards are rather subjective, there is a certain variance between evaluators. Further, IMBA profiles are made in relation to a work process. When testing people according to IMBA, tests become increasingly time-consuming. Therefore, an occupational physician commonly starts with the easier tests and if the worker is successful, they employ more elaborate testing. If there exist no relevant work processes that require a higher capability rating, the evaluator may choose to not test for it. For example: if there are no processes requiring a 4 or higher in a specific capability, it is sufficient to test for 3+ and lower to cover all possible work positions for the worker. This results in distorted populations. Figure 7 depicts a basic population for c_1.02 computed from 290 samples.

Figure 7:

Population for capability c_1.02 compared to a normal distribution with μ = 3 and σ = 1 scaled to the sum of samples n = 290 (lines interpolated). Samples taken from a rehabilitation clinic covering people from different companies, focused on manual labor. Data from multiple evaluators.

The samples are taken from a rehabilitation clinic and feature individuals from diverse companies at the end of their stay. The data was generated as part of the rehabilitation process. The publication of the data is in accordance with the data owner. While the population approximates the normal distribution reasonably well, 3− is overpopulated compared to 3+. Further, there are no samples in the 0 and 5 categories. The former is expected, as the data covers mostly people who shall be reintegrated into work. However, this highlights an issue in data collection: Usually capability profiles are made in situations where an impairment is expected (e.g., rehabilitation) or when employing people with disabilities. Both groups lead to a left shift in the normal distribution and violate the IMBA assumption of capability ratings being distributed according to the standard normal distribution (c.f. Section 3.2). In addition, the extended scale^[11] in IMBA is comparably new, with 3 being the old mean. In the new scale, 3− is the “aesthetic” mean, i.e. the center value of a septet, which may promote the usage of 3− instead of 3+. In conclusion, we will (a) require more and more diverse data to build a good estimate on the statistical population – potentially also capability profiles generated explicitly on regular work personnel – and (b) cover a wide range of applications, particularly also featuring the ratings 0 and 5.

5.2 Persona-based exploration

Due to the reasons stated in Section 5.1, there is no sufficient real data available to train our methods, yet. Training the network on artificial and/or simulated scenarios would just validate the functioning of the Bayesian network, but not the applicability of the method in real scenarios. Therefore, we refrain from validating the end-to-end Bayesian network on artificial data, but we still want to validate some core aspects of the methodology: the input modalities, the applicability of de Finetti influences, and the interconnection of the influences towards the capability (here: for Standing). We see the exact Bayesian network in Section 4.2 as an example of how the general methodology and architecture is applied. The capability Standing is interchangeable by any other capability in IMBA. To give a qualitative validation of parts of the methodology, we use a persona-based exploration, which is discussed in the following.

5.3 Outlook on capability-based autonomous teaming

In recent work, Mandischer et al. [21] introduced the concept of capability deltas, that offer a quantifiable source to assess the gap between a human’s individual performance and the fulfillment of task requirements r j k . This results in an n-dimensional vector Δ^i,k of individual deltas δ j i , k , where j is the capability, k the task identifier, and i is a specific person. The objective of teaming is now to minimize the vector norm

by applying any norm. The team delta is subject to capability-individual agglomeration rules

where B ^k is the index set of relevant capabilities in a task and superscript A and T refer to the automation and team, accordingly. Note, that task fulfillment may be reached by different combinations of capabilities, e.g., in a task to reach forward, an impaired arm’s reach may be compensated by bending the torso more. Therefore, the simple minimization of Equation (8) by means of c ̂ j A j ∈ B k will just reach one feasible solution but not necessarily the best one. In fact, it is not necessary to compensate all capabilities with δ j i , k > 0 , but only the ones that are not yet compensated by the human agent themselves. The best possible policy for minimizing the team delta will need to be subject to exploration. We theorize that there is a tolerable leftover error that still allows to fulfill the requirements on the team (similar to how experts apply IMBA) while still promoting satisfaction of the human.

It is hard to foresee the best possible solution to the stated problem. Even though we consider a “best” solution to exist, there is a realistic possibility of equally suited ambiguous solutions. We assume that the human is motivated to work. Hence, the team shall promote the independence of the human within the work process. To this end, the automation has to support as less as possible:

The consequential problem is that there is no clear definition of least support. As stated before, we can rearrange the problem such that some capabilities are less challenged while others are more [21]. It is unclear if supporting two human capability deltas δ j i , k = 1 , where δ j i , k = r j k − c ̂ j i , is to be considered fewer support than supporting a single δ j i , k = 2 , or vice versa. We can argue with an analogy from human behavior: an objective function used in human movement simulation is to minimize muscle activation and effort (e.g., Veerkamp et al. [62]). Hence, one possible “best” solution to the teaming problem – at least in physical tasks – is the one that simultaneously minimizes the total support by the automation and the human muscular effort.