System identification of biological processes poses numerous challenges in each stage of the identification exercise due to inherent complexity, cost, difficulty in conducting experiments, and complexity in the model structure (high-dimensional parameter space). Experiment design (data), model selection, and parameter estimation are three crucial stages of a system identification exercise. This work aims to address a crucial set of challenges in each stage of the identification exercise. Firstly, the precision of the parameter estimates is highly dependent upon the information contained in the data; The loss of practical identifiability and sloppiness in the model structure are the major challenges in estimating parameters precisely and closely related to the information contained in the data. Therefore, quantifying information is an essential step in data- driven modelling. In this work, we introduce a new way of quantifying information gain in the Bayesian framework using the complement of Bhattacharyya coefficient (leads to squared Hellinger distance). It is seen that the bounds of the coefficient have an insightful interpretation naturally in terms of information gained on the parameter of interest. Further, we apply the proposed information gain index to design optimal experiments and model selection in the Bayesian work. Secondly, assessing the goodness of the model structure is necessary for large-scale nonlinear system identification. Nonlinear model structures frequently suffer from loss of identifiability, anisotropic sensitivity in the parameter space, partial observability, and bifurcation. Several analytical and numerical methods are proposed to detect loss of identifiability in nonlinear model structures; however, nearly all of them suffer either from a lack of scalability or generalizability to nonlinear functions. In this work, we revisit the existing measure of sloppiness and address certain critical unanswered questions concerning sloppiness, particularly related to its quantification and practical implications in various stages of system identification. Further, we systematically examine sloppiness at a fundamental level and formalise two new theoretical definitions. We establish a mathematical relationship between the parameter estimates’ precision and sloppiness in linear predictors using the proposed definitions. Further, we develop a novel computational method and a visual tool to assess the goodness of a model around a point in parameter space by identifying local structural identifiability, sloppiness and finding the most sensitive and least sensitive parameters for non-infinitesimal perturbations. We demonstrate the working of our method in benchmark systems biology models of various complexities. Lastly, we propose a novel framework for efficient grey-box identification. The framework identifies the source of significant uncertainty in the predictions and parameter estimates.