At the beginning of the Republic of China, higher education faced two kinds of reformations under the dramatic social changes: one is making itself change from traditional models to modern models; the other is to set up graduate education based on the development of undergraduate education. Through the investigation of the Science Institute in Peking University and Tsinghua Academy of Chinese learning, this paper focuses on the initial creation of university research institutions, scientific research and the institutionalization of graduate education by the ways of analyzing related historical materials and empirical approach. With the development of education, the foundation of graduate education is an important issue faced by every country, such as the establishment of Berlin University in Germany and that of Johns Hopkins University in America. Each country has its own pattern and characteristic, including China. China under the semi-colonial and semi-feudal society needed to learn the existing models of Western countries in order to develop its own. The first physics students, who graduated from Peking University in 1916 demanded the graduate education. A year after Cai Yuanpei became the president of Peking University and established the Science Institute in the University in the same year following the Germany education idea. In 1925, Tsinghua tried to be an independent university from a preparatory school for those students who were sent by the government to study in the United States. Therefore, it decided to found its graduate school called Tsinghua Academy of Chinese Learning, which combined the Chinese traditional academic models with Western university system. Although both Science Institute in Peking University and Tsinghua Academy of Chinese Learning did not last for a long time, they could be taken as an inevitable outcome during the foundation of new institution. Both of them had the critical significance for the further development of graduate schools in China.