One of the main characteristics of human intelligence is the ability to reason over relationships among various entities. Multi-relational graphs, specifically knowledge graphs, are an excellent way to capture such relational knowledge. Good representation learning over multi-relational graphs fosters downstream applications like knowledge graph completion and question answering. While traditional factorization-based models (FM) have been the off-the-shelf choices, they fall short in inductive reasoning and incorporating entity features. On the other hand, there is a growing interest in applying graph neural networks (GNN) and parametric knowledge in pretrained language models (PLM) for this task. Our project aims to establish the link between FM, GNN, and PLM, of which the insights can be used to design better representation learning algorithms on multi-relational graphs.