The morphology of galaxies is one of the most basic observations and provides crucial constraints on their formation history. However what we observe is light which does not perfectly reflect the underlying stellar mass, the more fundamental tracer of a galaxy’s structure. The ratio between mass and light varies as a function of age, metallicity, dust and many other factors. Importantly this ratio varies within a galaxy meaning the distribution of light we observe is biased away from the underlying stellar mass distribution. The variations in the mass-to-light ratio can be observed as radial gradients in the color of a galaxy that have been observed in local galaxies for decades. During my thesis I studied the evolution of color gradients within galaxies since cosmic noon, the peak age of star-formation in the Universe, and its effect on the galaxy morphology we observe. I will describe a new method imcascade, which is a more flexible representation of a galaxy’s light distribution to better measure the complex structure of galaxies. imcascade allows more accurate measurements of color gradients at cosmic noon. I then apply this method to study the structure of galaxies at redshifts one to two using data from the Hubble Space Telescope. These results show that even in their early formation these galaxies show strong gradients in their color. This leads to the average half-light radius increasing with time whereas the half-mass radius stays constant. This markedly different evolution may change the established theories of the structural evolution of galaxies. I will end by showing recent results using early data from JWST. The longer wavelength coverage allows us to study radial variations in stellar populations to understand the physical cause of color gradients.