Many statistical models are defined in terms of polynomial constraints, or in terms of polynomial or rational parametrizations. In such algebraic statistical models, there is often an intimate connection between the geometry of parameter spaces and the behavior of statistical procedures. This talk will exemplify such connections for classical methods of statistical inference such as likelihood ratio and Wald tests. The focus will be on the problem of testing hypotheses with singularities, which arise in particular in so-called hidden or latent variable models.