Mammalian embryonic differentiation is intimately linking concurrent establishment of lineage specific transcriptional programs with buildup of matching epigenetic landscapes supporting them. Recently, single cell RNA-seq facilitated the construction of detailed transcriptional atlases describing gene expression programs in all embryonic lineages. Comparable models of the gene regulatory machineries driving differentiation toward these lineages are missing, and our understanding of the different epigenetic layers that enable and repress the associated gene expression modules is still limited. Here we perform single cell Hi-C on mouse embryonic tissues immediately at the end of gastrulation and develop a new parametric mixture model for describing replication dynamics in S-phase cells. We then use clusters of S-phase cells to anchor clustering of single Hi-C contact matrices, allowing for the first time robust de-novo analysis of scHi-C maps from in-vivo populations with control over cell cycle confounding factors. We unexpectedly discover distinct chromosome structure in primitive erythrocyte, some of which involve dramatic disruption of TAD structure out of the context of any differentially expressed locus. Further dissection of the embryo proper into clusters with mesodermal and ectodermal chromosomal conformations indicate broad pattern of hierarchical germ-layer chromosomal differentiation. Such structure is evident in E9 embryo cells, at least 2.5 days after the initial setup of the germ-layer programs. In summary, embryonic chromosomal conformation and the associated replication dynamics suggest a basis for commitment in terminally differentiated cells like erythrocytes, and for tissue specific gene regulation in mesoderm and ectoderm precursor cell populations.