We are a highly interdisciplinary research group working at the intersection among cognitive, social and computational sciences. LABSS is based at the Institute of Cognitive Sciences and Technologies (ISTC) of the National Research Council of Italy (CNR) and aims to foster an explorative approach to Agent Based Modeling and Simulation.
New paper on peer review from LABSS and UniValencia: Mechanism change in a simulation of peer review: from junk support to elitism.
Our honest, totally unbiased, objective evaluation of this work is: reading it will change your life. You will sleep better. A sense of clarity will ensue. The pictures will spring up your imagination. The only paper you really need to read this year.
Ahem. Well maybe we’re a little bit overplaying it. Ok, here’s the abstract:
Peer review works as the hinge of the scientific process, mediating between research and the awareness/acceptance of its results. While it might seem obvious that science would regulate itself scientifically, the consensus on peer review is eroding; a deeper understanding of its workings and potential alternatives is sorely needed. Employing a theoretical approach supported by agent-based simulation, we examined computational models of peer review, performing what we propose to call redesign, that is, the replication of simulations using different mechanisms. Here, we show that we are able to obtain the high sensitivity to rational cheating that is present in literature. In addition, we also show how this result appears to be fragile against small variations in mechanisms. Therefore, we argue that exploration of the parameter space is not enough if we want to support theoretical statements with simulation, and that exploration at the level of mechanisms is needed. These findings also support prudence in the application of simulation results based on single mechanisms, and endorse the use of complex agent platforms that encourage experimentation of diverse mechanisms.
We are glad to point at the special issue of CNR “Almanacco della Scienza” on the winners of the Ricercat@mente prize : in particular our Giulia Andrighetto with her research on norms (in Italian).
Abstract:
Punishment plays a crucial role in favoring and maintaining social order. Recent studies emphasize the effect of the norm-signaling function of punishment. However, very little attention has been paid so far to the potential of group punishment. We claim that when inflicted by an entire group, the recipient of punishment views it as expressing norms. The experiments performed in this work provide evidence that humans are motivated not only by material incentives that punishment imposes but also by normative information that it conveys. The same material incentive has a different effect on the individuals’ future compliance depending on the way it is implemented, having a stronger effect when it also conveys normative information. We put forward the hypothesis that by inflicting equal material incentives, group punishment is more effective in enhancing compliance than uncoordinated punishment, because it takes advantage of the norm-signaling function of punishment. In support of our hypothesis, we present cross-methodological data, that is, data obtained through agent-based simulation and laboratory experiments with human subjects. The combination of these two methods allows us to provide an explanation for the proximate mechanisms generating the cooperative behavior observed in the laboratory experiment.
In the cycle of the aCrossSocial seminars, we announce for Jan. 22 Francesca Giardini:
What goes around comes around: How reputation can support cooperation in natural and artificial societies.
Discussant: Laura BARCA
il 13 gennaio 2014, ore 17, presso la Sala delle Colonne dell’Università LUISS, Via Pola 12, Roma, si terrà la presentazione del libro Minding Norms. Mechanisms and Dynamics of Social Order in Agent Societies.
“Norms are prescribed conducts applied by the majority of
people. Getting across cultures and centuries, norms
evolved to rule all human relationships, from the most
formal to the most intimate. Impinging on any sphere of
life, from religious to political, norms affect social, moral,
and even aesthetical behaviours. They are enforced
through centralized sanctions or distributed control, and
originate through deliberate acts of issuing or from spon-
taneous interaction in informal settings. Despite ubiquity
and universality, norms are still awaiting for a general
comprehensive theory, simultaneously doing justice to
three intuitions: that, under variable contents, norms cor-
respond to a common notion; that, once brought about,
norms feedback on their producers, affecting their con-
ducts; and finally that before and in order to drive the
behaviours of individuals, norms must affect their beliefs
and goals: people must detect and accept norms before
converting them into observable behaviours.”
Giulia Andrighetto has been awarded the Ricercat@mente Prize!
Jointly sponsored by the Consiglio Nazionale delle Ricerche and the Accademia Nazionale dei Lincei, the prize is for the best researcher under the age of 35 in the field of Humanities, Social Sciences and Cultural Knowledge. It aims to give young researchers greater visibility and promote talent and excellence in Italian Science.
Andrighetto received her award at a ceremony on 18 Novemeber, at the headquarters of the Consiglio Nazionale delle Ricerche in Rome.
The introduction to the CMOT special issue with papers from ICORE 2011 is out! Yay!
Read the introduction here
This is how it starts:
Reputation is commonly defined as “the beliefs or opinions that are generally held about someone” . As such, reputation is the prototypical representative of social artifacts that we use to make sense out of social complexity. It shares with norms, responsibility, power, and trust – to cite our own favorite examples – the characteristic of making sense only in a social context. Reputation places itself in the most abstract position among these social artifacts.
Why is it so? Because when properly defined, reputation loosely connects with the object-level actions of the individual. First, in order to distinguish reputation in the proper sense, one must refer to some kind of majority rule – since it must be “generally held”. Second, reputation needs a model where the agent that elaborates and reasons on it must be endowed with a mind able to hold different levels of beliefs, since this agent must not confuse reputation with experience or evaluation. Third, while norms, power, trust and responsibility have all more or less direct connection to an action at the object level – violate or punish, exert, perform an uncertain transaction, gain attribute for an event – reputation only connects directly with the action of gossiping.
….
The interdisciplinary workshop on Gossip and Reputation at the Lorenz Center in Leiden has been co-organized by Francesca Giardini
Gossip and reputation management are essential features of our world.
Their investigation is part of the frontiers of research in at least three scientific domains: the social, the natural and the computational sciences. Understanding the dynamics, evolution and change of social information transmission requires a truly inter-disciplinary scientific effort.
Aim of this workshop is to define an “atlas” of research on gossip and reputation, in which mainland, i.e., shared concepts that overcome disciplinary boundaries, and detached territories, i.e., field-specific aspects, are explored and defined. The resulting map will be used to lay the foundation for new models and theories of gossip and reputation management, and to test their validity across boundaries between disciplines.
The workshop will be highly interactive and it will be structured around working groups which are expected to work towards the definition of “work packages” which will cover the different levels (individual, inter-individual, group, intergroup, societal), the relation between the levels (micro-macro links), as well as data, methods and tools.
As research in Multi-Agent Systems (MAS) has been expanding its focus from from the individual, cognitive focussed, agent models to models of socially situated agents, MAS researchers have been showing rising interest in social theories. Particular attention has been given to normative concepts because it is expected that norms could play as key a role in articulating agent interactions as the one norms play in human social intelligence. Thus, the label of “normative multi-agent system” has been attached to systems where individual and collective behaviour is affected by norms. This book is not a state of the art of normative multi-agent systems, nor a systematic description of the key concepts, or a compendium of the most salient challenges. However, the reader will find in its chapters something of each of these three contents because Normative Multi-Agent Systems is an effort to clarify the ideas behind the label and to put in perspective the work that is being done in this area.
In evidence:
We are proud to announce:
FRANCISCO GRIMALDO and MARIO PAOLUCCI, Advs. Complex Syst. DOI: 10.1142/S0219525913500045
A SIMULATION OF DISAGREEMENT FOR CONTROL OF RATIONAL CHEATING IN PEER REVIEW
Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities — the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted (“rational cheating”). We analyze a homogeneous scenario, where all conferences aim to the same level of quality, and a heterogeneous scenario, in which conferences request different qualities, showing how this affects the update mechanism proposed. We also present a first step toward an empirical validation of our model that compares the amount of disagreements found in real conferences with that obtained in our simulations.
Keywords: Artificial social systems; peer review; agent-based simulation; trust reliability and reputation
Francesca Giardini, Rosaria Conte, Mario Paolucci
Reputation
Why Read This Chapter?
To understand the different conceptions underlying reputation in simulations up to the current time and to get to know some of the approaches to implementing reputation mechanisms, which are more cognitively sophisticated.
Abstract
In this chapter, the role of reputation as a distributed instrument for social order is addressed. A short review of the state of the art will show the role of reputation in promoting (a) social control in cooperative contexts – like social groups and subgroups – and (b) partner selection in competitive contexts, like (e-) markets and industrial districts. In the initial section, current mechanisms of reputation – be they applied to electronic markets or MAS – will be shown to have poor theoretical backgrounds, missing almost completely the cognitive and social properties of the phenomenon under study. In the rest of the chapter a social cognitive model of reputation developed in the last decade by some of the authors will be presented. Its simulation-based applications to the theoretical study of norm-abiding behaviour, partner selection and to the refinement and improvement of current reputation mechanisms will be discussed. Final remarks and ideas for future research will conclude the chapter.
