Nitrogen is said to be responsible for the yellow or orange hues in a diamond