METHODOLOGICAL ADVANCEMENTS FOR GENERALIZABLE INSIGHTS INTO COMPLEX SYSTEMS (MAGICS)

For the past decade or more, there has been an assumption and hope that the explosion of digital data streams (e.g., social media, purchase patterns, traffic dynamics, etc.) combined with powerful machine learning tools would usher in a new era of research in complex, dynamic, evolving systems. It was widely thought that this powerful combination would enable better understanding of how large-scale systems respond to changes - such as how regional economies adapt to new conditions, or how population-level dynamics shift in response to demographic changes. Despite many attempts, results have failed to meet expectations. Progress has stalled because current statistical methods cannot create models that remain valid when applied to evolving, open, time varying, recursive, reactive, non-ergodic systems. The limitations of current methods for modeling human systems have revealed fundamental constraints on the ability to model and forecast human behavior in complex systems, and addressing these challenges requires overcoming several significant challenges that large data sets and ML do not address. A partial list includes: unstable mappings between latent constructs and observable data, insufficient methods to apply ideographically derived principles to aggregate behavior in non-ergodic systems, uncertainty in determining optimal sampling strategies, and lack of metatheoretical frameworks to support flexible application of relevant theories across contexts and domains of behavior. This list is not exhaustive, and it is likely that other challenges will also play a critical role in understanding human behavior in open systems. These must be identified and addressed to improve our ability to anticipate human behavior.

Addressing these gaps requires entirely new thinking about how to derive meaning from given sociotechnical data sets, including new techniques, theoretical insights, and understanding of the fundamental limits of inference possible from available data. By e