Foundation models are self-learned models that capture broad expectations about a particular domain, e.g. text or multimedia, and allow to generate completions in this domain, e.g. completions of texts or generations of pictures. Knowledge graphs describe facts and refer to ontological definitions of concept and relations. In this thesis, we research opportunities of acquiring foundation models for knowledge graphs from heterogeneous fact representations. Core are means of entity and concept linking, the representation of domain constraints, and the possibility to learn both factual and conceptual representations. Fundamental questions concern the mixture of representations involving textual definitions and graph structures as well as methods for self-learning from existing knowledge graphs.