Estimation of Model Parameters Challenge
Definitions
Pvalue for Parameter Predictions
Call N_{p} the number of parameters to predict. Each model has a different number of parameters. Models 1, 2 and 3 have N= 34, 41 and 57 parameters to predict. Let’s denote as v_{i}^{pred} and v_{i}^{real} the predicted and actual parameters used in the simulations where i runs between 1 and N_{p}. The “distance” between predicted and real parameters can be taken to be:
A null model was be created from the distance between estimated and known parameters, based on the predictions of all the participants. If there are M participants, we chose at random one of the M predictions for v_{1}^{pred}, then at random one of the M predictions for v_{2}^{pred}, …, and finally one of the M predictions for v_{Np}^{pred}. The resulting value of D_{j}^{param} represents one possible random choice of predictions amongst all the participants. By doing the same process a large number of times, we generated a distribution of D_{j}^{param} between known and estimated parameters, for which a pvalue can be estimated for the actual predictions. That pvalue denoted as p_{j}^{param} for model j is tabulated for each team for models 1, 2 and 3 in the the table below.
Pvalue for TimeCourse Predictions
Let’s denote as p_{k}^{pred}(t_{i}) and p_{k}^{sim}(t_{i}) the predicted and simulated levels of protein k at times t_{i}. Because the initial conditions were given, the real challenging predictions take place after some time has elapsed from t=0. We took that time to be 5. Therefore the squared distance between predicted and measured protein abundances for model j was taken to be:
Note that the squared difference terms are normalized with the model of noise that was implemented in the data provided.A null model was created from this distance, based on the predictions of all the participants. If there are M participants, we chose at random one of the M predictions for p_{k}^{pred}(t_{11}), then at random one of the M predictions for p_{k}^{pred}(t_{12}), …, and finally one of the M predictions for p_{k}^{pred}(t_{40}). The resulting value of D_{j}^{prot }represents one possible random choice of predictions amongst all the participants. By doing the same process a large number of times, we generated a distribution of distance squares, for which a pvalue can be estimated for the actual predictions. That pvalue denoted as p_{j}^{prot} for model j is tabulated for each team for models 1, 2 and 3 in the the table below.
Overall score
For each of the three models, each team obtained a pvalue for the timecourse predictions and a pvalue for the parameter predictions. The overall score is the product of all the pvalues:
Teams that participated in the challenge can login to deanonymize their identity.
pvalue for parameter predictions

pvalue for protein timecouse predictions 
OverallScore 

Team 
Model 1 
Model 2 
Model 3 
Model 1 
Model 2 
Model 3 
All Models 
orangeballs 
3.7E03 
3.7E07 
5.1E04 
1.0E23 
2.8E08 
1.1E08 
116.6 
crux 
2.8E05 
3.4E03 
8.4E02 
3.9E22 
1.2E01 
5.0E02 
73.1 
Team #71 
4.8E03 
1.2E03 
1.2E02 
3.0E15 
6.4E04 
1.2E01 
59.4 
Team #136 
2.7E03 
1.2E01 
5.6E02 
1.3E06 
6.6E08 
3.6E06 
53.5 
Team #105 
6.6E03 
2.7E04 
6.0E02 
7.0E07 
6.4E08 
1.0E+00 
46.8 
Team #76 
2.6E02 
1.6E01 
8.2E01 
7.9E15 
8.7E01 
1.0E+00 
38.3 
Team #39 
2.5E01 
5.6E01 
5.3E01 
1.3E08 
2.3E06 
9.8E01 
33.7 
Team #150 
2.6E02 
9.6E01 
1.0E+00 
1.8E04 
1.0E+00 
4.0E06 
24.7 
Team #194 
2.2E01 
2.1E02 
1.1E01 
1.0E+00 
1.6E03 
6.7E05 
23.6 
Team #228 
8.4E04 
1.3E01 
4.4E02 
2.2E04 
1.0E+00 
1.0E+00 
20.7 
Team #109 
1.0E+00 
9.9E01 
9.9E01 
1.0E+00 
4.4E04 
1.6E05 
18.8 
Team #23 
4.4E01 
3.4E02 
3.7E01 
1.0E+00 
2.3E01 
1.3E05 
17.9 
Team #199 
1.9E03 
2.4E01 
2.9E02 
5.2E03 
4.4E01 
1.0E+00 
17.3 
Team #77 
2.2E01 
4.9E01 
7.9E03 
1.0E+00 
1.0E+00 
9.9E04 
14.0 
Team #135 
1.0E+00 
9.4E01 
6.7E01 
1.0E+00 
7.3E01 
7.6E05 
10.3 
Team #157 
2.6E01 
3.3E01 
5.2E01 
1.0E+00 
5.8E02 
8.0E01 
6.2 
Team #89 
4.1E01 
2.5E01 
2.0E01 
1.0E+00 
4.0E01 
1.0E+00 
4.8 
Team #129 
7.0E01 
1.0E+00 
9.3E01 
1.0E+00 
1.0E+00 
8.9E02 
2.8 
Team #88 
8.7E01 
5.6E01 
2.6E01 
1.0E+00 
1.0E+00 
8.8E01 
2.2 