Latent Communication in Artificial Neural Networks

Luca Moschella (Ph.D. Student)

As neural networks permeate various scientific and industrial domains, understanding the universality and reusability of their representations becomes crucial. At their core, these networks create intermediate encodings or representations of input data and subsequently leverage them to perform specific downstream tasks. This research centralizes on the question of the universality and reusability of these representations. Do the latent representations crafted by a neural network remain exclusive to a particular trained instance, or can they generalize across models, adapting to factors such as randomness during training, model architecture, or even data domain. This adaptive quality introduces the notion of latent communication — a phenomenon that describes when representations can be unified or reused across different neural spaces. A salient observation from our research is the emergence of similarities in latent representations, even when these originate from distinct or seemingly unrelated neural networks. By exploiting a partial correspondence between the two spaces that establishes a semantic link, we found that these representations can either be projected into a universal representation (Moschella et al., 2023), coined as relative representations, or be directly slated from one space to another (Maiorca, Moschella, et al., 2023). Intriguingly, this holds even when the transformation relating the spaces is unknown (Cannistraci, Moschella, Fumero, et al., 2023) and when the semantic bridge between them is minimal (Cannistraci, Moschella, Maiorca, et al., 2023). Latent communication allows for a bridge between independently trained neural networks, irrespective of their training regimen, architecture, or the data modality they were trained on – as long as the data semantic content stays the same (e.g. images and their captions). This holds true for both generation, classification and retrieval downstream tasks; in supervised, weakly-supervised, and unsupervised settings; and spans various data types including images, text, audio, and graphs --showcasing the universality of the latent communication phenomenon. From a practical standpoint, our research offers the potential to repurpose and reuse models, circumventing the need for resource intensive retraining, enables the transfer of knowledge across them and allows for performance evaluation directly in the latent space. Reflecting its significance, latent communication has been a focal point in the workshop UniReps: Unifying Representations in Neural Models at NeurIPS 2023, co-organized by our team.

Primary Host:	Emanuele Rodolà (Sapienza University of Rome)
Exchange Host:	Francesco Locatello (IST Austria)
PhD Duration:	01 November 2020 - 01 April 2024
Exchange Duration:	01 October 2023 - 01 May 2024 - Ongoing