A growing consensus in the field is that many machine learning problems possess symmetries and that neural networks which respect those symmetries – are equivariant – outperform models that are not. The goal of the PhD project is to generalize the used mathematical toolkit of groups and representations to more general structures applicable to new domains. Already, we have generalized a symmetry group to a symmetry groupoid in the local symmetries of graphs. This required the generalization of equivariance to the concept of natural transformation from category theory. Going forward, we plan on studying category theoretical formulations of causality and exploring how invariances and equivariances arise in causal models.