Glassy materials are characterized by a plethora of metastable states. This thesis present a theoretical study of these states in the framework of finite dimensional spin glasses - one of the paradigms of disordered systems in statistical mechanics - using simple models, phenomenological approaches and numerical computations involving combinatorial optimization. Of particular interest is the structure of the energy landscape, the nature of the phase diagram and the putative presence of a chaotic temperature dependence. Our results strongly suggest that the energy landscape is complex and that it does exist large scale low energy excitations as predict by mean field theories, corresponding to spongy clusters with non trivial topology. However, in contrast to the mean field case, the phase diagram seems to be trivial since no evidence for a spin glass phase under a magnetic field, or for a mixed phase with both ferromagnetic and spin glass ordering, are found. A scenario called TNT for Trivial - Non Trivial, for which these properties are expected, is presented and seems compatible with the known numerical results. The presence of temperature chaos is illustrated in two models : a mean field like spin glass under Curie-Weiss approximation and a solvable Random energy Random entropy model. These models give strong evidences for such a chaotic temperature dependence in real systems. Finally, general properties of ground states in disordered systems are studied both analytically and numerically. Nature of excitations, finite size effects, sample to sample fluctuations, universality and lower critical dimensions are discussed.