Bayesian inference has been used effectively to estimate parameter values from observed data for a wide range of scientific models. This type of inference is preferred for many applications, as it allows us to quantify the uncertainty in our beliefs about the parameter values, but exact methods are typically intractable, so approximations are used instead. A common class of algorithms used to perform these approximations are Markov-Chain Monte Carlo (MCMC) methods. Recent work has seen the development of MCMC-based tools that allow scientists to infer parameters easily and efficiently without needing to be experts in Bayesian inference. Despite these improvements, there are many types of model which still present challenges for Bayesian inference. We propose to investigate two such types in this project, motivated by two applied inference problems. The first is the problem of determining parameters of differential equations, used to model many physical and biological processes. The second is the problem of inferring the parameters of astronomical models using data recorded from charge-coupled devices (CCDs). We will research new methods capable of performing efficient Bayesian inference on these models, starting by adapting existing MCMC methods to work well with our applied problems.