We are witnessing a growing need for the evaluation of an enlarged universe of minds beyond human intelligence, including a variety of newfangled machines, enhanced humans, hybrids and collectives. The tools from human psychometrics, comparative psychology, animal cognition and artificial intelligence are insufficient to cope with this challenge —if not simply inconsistent when put together. Most of the precepts of cognitive measurement from all these areas must be overhauled and rebuilt upon more solid foundations, using a universal perspective as a guide, but also as a refutation principle. For that purpose, a new universal psychometrics is presented.

This monograph focuses on primary research, by formulating new scientific questions and hypotheses, and introducing a whole range of definitions and results. It integrates the fundamentals and most significant research developments in the past two decades, with the aim of providing a unified view of the universal evaluation of cognitive abilities and personality traits. The intended audience includes researchers and academics in cognitive science, artificial intelligence, psychometrics, comparative psychology, intelligence research, psychology and animal cognition. Given this diversity of fields, a great effort has been made to render the terminology and mathematical content accessible to a wider readership.
Major questions addressed in the book:

1. How can behavioural features be measured in the machine kingdom?

2. How universal and adaptive can a behavioural test be?

3. Are IQ tests valid for any machine and what do they measure?

4. How can abilities be identified, through task breadth or similarity?

5. Is intelligence one or many, objective or subjective?

6. Can we formalise task difficulty without particular populations?

7. Is there a common component for all abilities, a universal g factor?

8. Can general intelligence be independent of the task distribution?

9. Will scales fully range from minimal cognition to supercapacities?

10. Can intelligence be defined solely with computational principles?

11. Should collectives and hybrids be evaluated unlike their individuals?

12. Is intelligence useful for adaptive testing and self-assessment?

13. Can social abilities be measured through situated tests?

14. What kind of ability can capture communication and language skills?

15. How can potential abilities be measured and how do they develop?

16. Can any universal machine and human become arbitrarily intelligent?

17. How can cognitive abilities and personality traits develop in general?

18. Are behavioural features sufficient to characterise personhood?

19. What are the limits and consequences of supercapacities?

20. Can human, animal and AI evaluation benefit from an integration?

PART I A long-pondered outfit

1 Extended nature
2 Mind the step: scala universalis

PART II The evaluation discordance

3 The evaluation of human behaviour
4 The evaluation of non-human natural behaviour
5 The evaluation of artificial intelligence
6 The boundaries against a unified evaluation

PART III The algorithmic confluence

7 Intelligence and algorithmic information theory
8 Cognitive tasks and difficulty
9 From tasks to tests
10 The arrangement of abilities
11 General intelligence

PART IV The society of minds

12 Cognitive development and potential
13 Identifying social skills
14 Communication abilities
15 Evaluating collective and hybrid systems

PART V The kingdom of ends

16 Universal tests
17 Rooting for ratiocentrism
18 Exploitation and exploration

(c) José Hernández Orallo and Cambridge University Press, 2017.