We study the integral quaternions and the integral octonions along the combinatorics of the $24$-cell, a uniform polytope with the symmetry $D_{4}$, and the Gosset polytope $4_{21}$ with the symmetry $E_{8}$. We identify the set of the unit integral octonions or quaternions as a Gosset polytope $4_{21}$ or a $24$-cell and describe the subsets of integral numbers having small length as certain combinations of unit integral numbers according to the $E_{8}$ or $D_{4}$ actions on the $4_{21}$ or the $24$-cell, respectively. Moreover, we show that each level set in the unit integral numbers forms a uniform polytope, and we explain the dualities between them. In particular, the set of the pure unit integral octonions is identified as a uniform polytope $2_{31}$ with the symmetry $E_{7}$, and it is a dual polytope to a Gosset polytope $3_{21}$ with the symmetry $E_{7}$ which is the set of the unit integral octonions with $\operatorname {Re}=1/2$.