At the macroscopic (big) level, light does not DEfract INSIDE a prism, rather it REfracts at the SURFACES of a prism. But at the atomic level each tiny part of the beam is diffracting from the atoms making up the prism. The total effect of all parts of the beam diffracting and interfering results in a change of beam direction, meaning refraction, at that surface. So the overall effect of diffraction and interference at a plane, transparent surface is called refraction.
There is a similar effect at the other surface of the prism, where the beam exits the prism although the bending (refraction) is in the opposite direction.
For a glass prism in air, a light beam in air strikes the first surface of the prism, bends towards the normal (normal = line perpendicular to the surface), passes through glass to the next surface, and bends away from the norm as is passes into air. At the first surface the atoms in the glass absorb the light photons (light beam is made up of light photons) and emit them again, a short time later. The delay is very, very short, but enough to have the effect of changing the direction of the beam. This is because as the group of atoms near the surface absorb and emit photons, the photons are in phase, and undergo interference in such a way that the light wavelets from all the emitting atoms superpose and interfere constructively in the new direction. Only the direction of overall constructive interference results in the light beam continuing on in the glass. Other directions have overall negative interference, extinguishing the beams in those directions. The angle between the impinging (striking beam, coming from the air) determines in what direction constructive interence takes place. When the angle is 90 degrees (the light beam strike the glass surface straight on), there in no change of angle in the path of the light beam as is crosses into the glass. When the angle is oblique (different from 90), the atomic interence effects result in a change of path. The absorption-delay-emssion also means the light photons travel more slowly in glass than in air, and this results in diffraction, the same diffraction, just another way of considering it. A crude way of seeing this is to draw a beam of finite width, imagining that edge get slowed down, while the farther edge travels fast to the nearby glass surface, then it gets slowed down. Sketching this out shows an change in beam direction.
(Note: If the glass is full of impurities or tiny bubbles of air, then there truly is diffraction in the pure sense, without refraction, INSIDE the glass prism, but I do not think this is what the questions refers to.)
You did not ask about this point, but of course a prism separates the colors in a light beam. All of the above diffration-interference-resulting in refraction depends on the wavelengths of the light (of the photons). White light is composed of photons having a wide range of wavelengths. Of course, beams (photons) of different wavelengths are seen as different colors. The amount of bending at each surface is different for each color (more bending for short wavelength blue, less bending for long wavelength red, in between for in between colors). Equivalently, the speed of light INSIDE THE GLASS is slightly different for each color. The first surface bends and separates the colors some, and the second surface does it more, so that the light beams leving the prism are strongly separated by color.