In this dissertation, we develop improved estimation of average treatment effect on the treatment (ATT) which achieves double robustness, local efficiency, intrinsic efficiency and sample boundedness, using a calibrated likelihood approach. Moreover, we consider an extension of two-group causal inference problem to a general data combination problem, and develop estimators achieving desirable properties beyond double robustness and local efficiency. The proposed methods are shown, both theoretically and numerically, to be superior in robustness, efficiency or both to various existing estimators. In the first part, we review existing estimators on average treatment effect (ATE), mainly based on Tan (2006, 2010). This review provides a useful basis for improved estimation of average treatment effect on the treated (ATT). In the second part, we propose new methods to estimate the average treatment effect on the treated (ATT), which is of extensive interest in Econometrics, Biostatistics and other research fields. This problem seems to be often treated as a simple modification or extension of that of estimating overall average treatment effects (ATE). But the propensity score is no longer ancillary for estimation of ATT, in contrast with estimation of ATE. We study the efficient influence function and the corresponding semiparametric variance bound for the estimation of ATT under three different assumptions: a nonparametric model, a correct propensity score model and known propensity score. Then we construct Augmented Inverse Probability Weighted (AIPW) estimators which are locally efficient and doubly robust. Furthermore, we develop calibrated regression and likelihood estimators that are not only doubly robust and locally efficient, but also intrinsically e cient and sample bounded. Two simulations and real data analysis on a job training program are provided to demonstrate the advantage of our estimators compared with existing estimators. In the third part, we extend our methods to a general data combination problem for moment restriction models (Chen et al. 2008). Similarly, we derive augmented inverse probability weighted (AIPW) estimators that are locally efficient and doubly robust. Moreover, we develop calibrated regression and likelihood estimators which achieve double robustness, local efficiency and intrinsic efficiency. For illustration, we take the linear two-sample instrumental variable problem as an example, and derive all the relevant estimators by applying the general estimators in this specific example. Finally, a simulation study and an Econometric application on a public housing project are provided to demonstrate the superior performance of our improved estimators.

State space model is a class of models where the observations are driven by underlying stochastic processes. It is widely used in computer vision, economics and financial data analysis, engineering, environmental sciences and etc. My thesis mainly addresses the parameter estimation problem of state space model and the applications of it. This thesis starts with a brief introduction and the motivation for studying the problems in the first chapter. The second chapter follows the first one by covering the main tools used to study the topics in the thesis. The general framework of state space models and its related filtering methods, Kalman Filtering for linear Gaussian models and sequential Monte Carlo for other cases, are introduced. The information criteria, as a tool for model selection, are also covered in this chapter. The parameter estimation problem is mainly discussed in the third chapter. Two algorithms under the general framework of Stochastic Approximation methods are proposed. These two algorithms attain much faster convergence rate and less computational cost by variance reduction techniques which utilize the property of sequential Monte Carlo methods. Two numerical examples are examined to compare the performance. Another contribution of Chapter 3 is the application of sequantial Monte Carlo methods in modeling and predicting the bond yield curve with regime-switching Dynamic Nelson-Siegel model. The fourth chapter, which is a joint work with Hao Chang, develops a state space model with regime switching to detect periodically collapsing rational bubbles in stock price. The present-value stock-price model is expressed in a state space form and the bubble process is modeled as a conditional dynamic linear system. The asset-bubble system is estimated by a novel sequential Monte Carlo based method, Mixture Kalman Filter (MKF). The efficacy of the proposed method is examined by simulated observations and real stock index of the US market. Another application of state space model with regime switching is discussed in the fifth chapter, in which real-time Blood Glucose Monitoring problem is addressed using a conditional dynamic linear system modeling. A study with a biostatistical dataset, Star 1 dataset, has shown the advantage of the proposed novel estimation framework. In the sixth chapter, a nonparametric regression model, l1 trend filtering method is discussed. Two trend filtering models out of state space representation, both of which have similar property as l1 trend filtering, are proposed. With the implementation of sequential Monte Carlo methods as well as a greedy Viterbi algorithm, both trend filtering models can operate on-line rather than just on batch data. To better emphasize the two models' improvement in on-line trend filtering, a real world econometrics topic is introduced. The econometric example shows the competence of trend filtering as well as the efficiency of the proposed models.

Protein folding has been a challenging puzzle for decades but it is still not fully understood. One important way to gain insights of the mechanism is to study how kinetics in the unfolded state affects protein folding. The answer to this fundamental issue hinges on the time scale to equilibrate the unfolded state and the energy landscape of the unfolded state. We construct Markov state models (MSMs) of several mini-proteins to study the kinetics of their unfolded state ensemble and find that the folding kinetics are two-state even though there are multiple folding pathways with nonuniform barriers, which are the direct consequences of rapid mixing within the unfolded state. Also, we introduce a time integral of a proper correlation function, namely relaxation time, to characterize the time scale of equilibration within the unfolded state. However, the mean first passage times (MFPTs) between different regions of the unfolded state are observed to be orders of magnitude longer than the folding time. This seeming paradox is solved by the derivation of a simple relation that shows the mean first passage time to any state is equal to the relaxation time of that state divided by its equilibrium population. This simple relation explains why MFPTs among unfolded states can be very long but the energy landscape can still be smooth (minimally frustrated). As a matter of fact, when the folding kinetics is two-state, all of the unfolded state relaxation times are faster than the folding time. This result supports the well-established funnel-like energy landscape picture and resolves an apparent contradiction between this model and the recently proposed kinetic hub model of protein folding. Markov state model is a powerful tool but we seek for alternative ways of studying kinetics when MSM does not work very well. For example, diffusion maps of dimensionality reduction and discrete transition-based reweighting analysis method, are very useful in determining a geometrical measure that preserves intrinsic dynamics and in fully utilizing enhanced sampling simulation data.

This dissertation contains two parts. In the first part, we develop backfitting algorithms for doubly penalized additive and ANOVA modeling using total variation penalties carefully extended to multivariate functions. In the second part, we develop subsampling strategies to accelerate large-scale multi-class logistic regression with big data. Additive and ANOVA modeling aim to capture nonlinear relationship between response and covariates. First, we develop a backfitting algorithm to implement a doubly penalized method for additive models using total-variation and empirical-norm penalties. Total-variation penalty leads to an automatic knots selection for each component function, whereas empirical norm penalty can result in zero solutions for component functions and hence facilitates component selection in high-dimensional settings. We present numerical experiments to demonstrate the effectiveness of the proposed algorithms for linear and logistic additive modeling. Next we construct a new class of total variation penalties, called hierarchical total variations, which measures total variations at different levels including main effects and multi-way interactions for multivariate functions. We derive suitable basis functions for multivariate splines and then extend our back fitting algorithm to implement functional ANOVA modeling with double penalties. We demonstrate potential advantages of our proposed method compared with other alternatives including MARS, tree boosting, and random forest on simulated data and real data in both linear and logistic regression scenarios. A major challenge for statistical modeling in the big data era is that the available data volume far exceeds the computational capability. A common approach for solving this problem is to employ a subsampled dataset that can be handled by available computation resources. We propose a general subsampling scheme called local uncertainty sampling to accelerate the computation of large-scale logistic regression and examine the variance of the resulting estimator. We show that asymptotically, the proposed method always achieves a smaller variance than that of the uniform random sampling. Moreover, when the classes are conditionally imbalanced, significant improvement over uniform sampling can be achieved by our proposed subsampling strategy. Empirical performance of the proposed method is compared to other methods on both simulated and real-world datasets, and the results confirm our theoretical analysis.

This dissertation focuses on developing new statistical methods for analyzing and modeling financial time series. The first part of this dissertation discusses modeling of functional and distributional time series, assuming the series are driven by a finite dimensional underlying feature process. Functional time series are commonly observed in finance and difficult to model mainly due to high dimensionality. We instead focus on a low-dimensional latent feature process and connect it with the original functional time series through generalized state-space models. A state-space model assumes that observations are driven by an underlying dynamic state process and is widely used in many fields. We propose a generalized two-stage Sequential Monte Carlo (SMC) joint estimation framework to model functional time series driven by its feature process through state-space models and perform on-line estimations and predictions. In order to improve computation efficiency, we also implement parallel computing and re-design computation algorithms to integrate with non-linear optimization and SMC calculations. Two financial applications are presented to demonstrate the robustness and efficiency of our proposed framework. The first application aims to extract and model the daily implied risk neutral densities from observed call option prices. We view the underlying risk neutral density as a functional time series driven by its feature process and model it with a parametric mixed log-normal distribution through a state-space model. We conduct both simulation and empirical studies and compare prediction performance of our models with that of random walk models. Empirically, the proposed models improves prediction performance significantly. The second financial application studies daily cross-sectional distribution of 1000 largest market capitalization stock returns from year 1991 to year 2002. Similarly we view cross-sectional distribution as a functional time series driven by its feature process and model it with a four-parameter generalized skewed t-distribution. Using proposed two-stage SMC joint estimation framework, we build models separately for different market conditions, including the dot-com crisis. In both bearish and bullish markets, prediction performances of our models gain substantial improvement comparing with random walk models. The second part of dissertation presents a new portfolio optimization strategy for minimum variance portfolios with constraints on short-sale and transaction costs. Unlike traditional mean-variance theory, our method minimizes only the portfolio risk and uses analysts' consensus ratings to pre-screen stocks. An empirical study of S&P 500 stocks from year 1990 to year 2009 is conducted to demonstrate effectiveness. We show that portfolios constructed and optimized using our strategy deliver considerable improvement of performance in terms of Sharpe ratio comparing with benchmark portfolios and portfolios in past literatures.

Protein evolution is governed by a balance of mutation and selection that act on an ensemble of protein sequence variants. The acquisition of a mutation and the effect it has on a variant's prevalence in this ensemble depends on networks of interactions with the rest of the protein sequence, a phenomenon known as epistasis. Evidence of this epistatic interaction network can be extracted from protein sequence alignments in the form of pairwise correlations. In this dissertation, we focus on building models derived from statistical physics of correlated mutation patterns and from them, extracting information that describes the protein fitness landscape and the interdependency of specific mutation patterns with drug-resistance. Using HIV-1 protease as our model system, we extract the pair correlations present in conventional multiple protein sequence alignments to build maximum entropy Potts Hamiltonian models of the drug-experienced HIV-1 protease mutational landscape. We demonstrate that these models are able to predict higher order sequence statistics and the fitness effects of multiple simultaneous mutations. Using the Hamiltonian to score individual protease sequences, we are able to identify the mutation patterns that responsible for promoting protease inhibitor resistance. As Hamiltonian models of sequence covariation is a growing field, we provide a first-order analytic framework to quantify the error in the model's predictions and provide a retrospective error analysis on the models published in the literature. Lastly, we provide an overview of related research efforts to study drug resistance within HIV-1 protease and its substrate, and to computationally screen putative drug candidates targeting HIV-1 integrase.