Skip to main content

Table 3 Average estimates of haplotype frequencies for a 25 loci case

From: An EM algorithm based on an internal list for estimating haplotype distributions of rare variants from pooled genotype data

    k=2    k=4
Position of ‘1’ TRUE   CDMLE EML EM- CDL EM-ATCDL EM-TCDL   CDMLE EM-CDL EM-ATCDL EM-TCDL
       (a) n=100       
None 0.7995   0.7973 0.8279 0.8283 0.8003 0.8067   0.7961 0.8535 0.7957 0.8119
    (0.0232) (0.0169) (0.0170) (0.0215) (0.0192)   (0.0204) (0.0118) (0.0179) (0.0144)
1 0.0509   0.0508 0.0412 0.0412 0.0477 0.0477   0.0502 0.0344 0.0457 0.0494
    (0.0155) (0.0115) (0.0115) (0.0119) (0.0112)   (0.0152) (0.0083) (0.0093) (0.0086)
2 0.0034   0.0036 0.0019 0.0019 0.0031 0.0024   0.0032 0.0008 0.0028 0.0020
    (0.0040) (0.0022) (0.0022) (0.0025) (0.0027)   (0.0044) (0.0012) (0.0023) (0.0029)
3 0.0436   0.0441 0.0353 0.0353 0.0425 0.0408   0.0435 0.0277 0.0396 0.0411
    (0.0139) (0.0104) (0.0104) (0.0112) (0.0107)   (0.0144) (0.0077) (0.0097) (0.0080)
5 0.0034   0.0035 0.0019 0.0019 0.0031 0.0027   0.0031 0.0008 0.0028 0.0015
    (0.0039) (0.0021) (0.0021) (0.0028) (0.0029)   (0.0043) (0.0011) (0.0017) (0.0024)
6 0.0034   0.0027 0.0017 0.0016 0.0029 0.0022   0.0038 0.0013 0.0030 0.0019
    (0.0035) (0.0021) (0.0021) (0.0028) (0.0030)   (0.0050) (0.0017) (0.0022) (0.0026)
9 0.0073   0.0092 0.0056 0.0056 0.0085 0.0087   0.0073 0.0029 0.0074 0.0079
    (0.0065) (0.0039) (0.0039) (0.0046) (0.0052)   (0.0066) (0.0026) (0.0045) (0.0062)
11 0.0034   0.0041 0.0022 0.0022 0.0036 0.0029   0.0032 0.0008 0.0027 0.0016
    (0.0048) (0.0025) (0.0025) (0.0029) (0.0031)   (0.0046) (0.0013) (0.0021) (0.0025)
15 0.0034   0.0039 0.0021 0.0021 0.0032 0.0026   0.0034 0.0009 0.0029 0.0017
    (0.0052) (0.0028) (0.0028) (0.0032) (0.0033)   (0.0050) (0.0013) (0.0021) (0.0025)
19 0.0068   0.0069 0.0039 0.0039 0.0061 0.0055   0.0075 0.0022 0.0058 0.0051
    (0.0056) (0.0031) (0.0031) (0.0040) (0.0044)   (0.0066) (0.0020) (0.0031) (0.0042)
20 0.0068   0.0073 0.0041 0.0041 0.0061 0.0058   0.0080 0.0023 0.0057 0.0051
    (0.0060) (0.0035) (0.0035) (0.0042) (0.0045)   (0.0065) (0.0020) (0.0028) (0.0040)
21 0.0034   0.0038 0.0020 0.0020 0.0032 0.0029   0.0041 0.0011 0.0033 0.0022
    (0.0041) (0.0022) (0.0022) (0.0027) (0.0031)   (0.0051) (0.0014) (0.0023) (0.0028)
22 0.0102   0.0117 0.0070 0.0070 0.0095 0.0099   0.0110 0.0043 0.0096 0.0091
    (0.0075) (0.0047) (0.0047) (0.0053) (0.0054)   (0.0085) (0.0033) (0.0046) (0.0056)
23 0.0034   0.0032 0.0018 0.0018 0.0028 0.0024   0.0038 0.0010 0.0029 0.0019
    (0.0039) (0.0023) (0.0023) (0.0028) (0.0030)   (0.0045) (0.0012) (0.0021) (0.0026)
24 0.0102   0.0096 0.0060 0.0060 0.0095 0.0098   0.0114 0.0043 0.0098 0.0118
    (0.0065) (0.0041) (0.0041) (0.0051) (0.0057)   (0.0076) (0.0028) (0.0047) (0.0057)
1, 3 0.0040   0.0048 0.0039 0.0039 0.0045 0.0043   0.0071 0.0049 0.0061 0.0052
    (0.0049) (0.0037) (0.0037) (0.0041) (0.0040)   (0.0070) (0.0040) (0.0046) (0.0048)
1, 9 0.0029   0.0030 0.0021 0.0021 0.0023 0.0018   0.0051 0.0023 0.0028 0.0017
    (0.0035) (0.0024) (0.0024) (0.0033) (0.0033)   (0.0055) (0.0021) (0.0031) (0.0032)
6, 7 0.0204   0.0210 0.0150 0.0148 0.0215 0.0195   0.0203 0.0104 0.0210 0.0219
    (0.0098) (0.0072) (0.0072) (0.0075) (0.0077)   (0.0105) (0.0043) (0.0068) (0.0060)
3, 14 0.0034   0.0035 0.0021 0.0021 0.0020 0.0028   0.0031 0.0012 0.0015 0.0021
    (0.0037) (0.0022) (0.0022) (0.0032) (0.0031)   (0.0038) (0.0015) (0.0024) (0.0026)
3, 6, 7 0.0034   0.0037 0.0028 0.0029 0.0025 0.0030   0.0047 0.0028 0.0022 0.0016
    (0.0047) (0.0036) (0.0036) (0.0044) (0.0047)   (0.0053) (0.0025) (0.0033) (0.0035)
1, 6, 7, 24 0.0034   0.0031 0.0021 0.0021 0.0006 0.0006   0.0036 0.0018 0.0006 0.0000
    (0.0033) (0.0023) (0.0023) (0.0023) (0.0021)   (0.0037) (0.0018) (0.0017) (0.0004)
1, 12, 13, 22, 25 0.0034   0.0037 0.0024 0.0024 0.0026 0.0029   0.0038 0.0016 0.0005 0.0025
    (0.0038) (0.0025) (0.0025) (0.0031) (0.0029)   (0.0035) (0.0015) (0.0018) (0.0027)
Sum of remaining   0.0376 0.0248 0.0248 0.0117 0.0124   0.0893 0.0366 0.0255 0.0109
haplotype probabilities            
Sum of probabilities   0.0247 0.0241 0.0244 0.0218 0.0286   0.0296 0.0285 0.0179 0.0367
of missed haplotypes            
Sum of squared errors   0.00166 0.00186 0.00189 0.00110 0.00106   0.00201 0.00415 0.00091 0.00089
Length of list   26.77 116.28 26.77 19.06 18.25   45.23 45.23 25.38 15.62
SD of length   (3.26) (81.30) (3.26) (3.03) (2.18)   (3.94) (3.94) (4.78) (2.40)
       (a) n=200       
None 0.7995   0.7979 0.8248 0.8250 0.7981 0.8009   0.7990 0.8451 0.7970 0.8040
    (0.0150) (0.0117) (0.0117) (0.0148) (0.0132)   (0.0154) (0.0086) (0.0133) (0.0125)
1 0.0509   0.0514 0.0433 0.0433 0.0492 0.0503   0.0502 0.0387 0.0462 0.0507
    (0.0103) (0.0082) (0.0082) (0.0089) (0.0088)   (0.0121) (0.0069) (0.0077) (0.0080)
2 0.0034   0.0035 0.0020 0.0020 0.0033 0.0031   0.0037 0.0011 0.0030 0.0024
    (0.0032) (0.0018) (0.0018) (0.0024) (0.0026)   (0.0035) (0.0011) (0.0013) (0.0021)
3 0.0436   0.0430 0.0362 0.0362 0.0435 0.0426   0.0441 0.0322 0.0406 0.0426
    (0.0092) (0.0075) (0.0075) (0.0082) (0.0074)   (0.0105) (0.0054) (0.0071) (0.0065)
5 0.0034   0.0033 0.0018 0.0018 0.0032 0.0028   0.0034 0.0011 0.0030 0.0023
    (0.0028) (0.0016) (0.0016) (0.0020) (0.0023)   (0.0035) (0.0011) (0.0013) (0.0021)
6 0.0034   0.0038 0.0023 0.0023 0.0033 0.0031   0.0033 0.0014 0.0028 0.0022
    (0.0031) (0.0019) (0.0020) (0.0021) (0.0025)   (0.0035) (0.0013) (0.0014) (0.0019)
9 0.0073   0.0080 0.0054 0.0054 0.0079 0.0088   0.0081 0.0036 0.0070 0.0092
    (0.0038) (0.0026) (0.0026) (0.0032) (0.0037)   (0.0043) (0.0019) (0.0026) (0.0031)
11 0.0034   0.0032 0.0018 0.0018 0.0030 0.0027   0.0032 0.0010 0.0029 0.0023
    (0.0026) (0.0016) (0.0016) (0.0022) (0.0024)   (0.0031) (0.0011) (0.0015) (0.0021)
15 0.0034   0.0035 0.0019 0.0019 0.0030 0.0028   0.0038 0.0012 0.0031 0.0028
    (0.0033) (0.0018) (0.0018) (0.0023) (0.0025)   (0.0031) (0.0010) (0.0015) (0.0021)
19 0.0068   0.0063 0.0039 0.0039 0.0062 0.0061   0.0066 0.0026 0.0056 0.0057
    (0.0039) (0.0025) (0.0025) (0.0029) (0.0034)   (0.0041) (0.0015) (0.0018) (0.0026)
20 0.0068   0.0068 0.0042 0.0042 0.0062 0.0063   0.0063 0.0026 0.0054 0.0059
    (0.0039) (0.0025) (0.0025) (0.0028) (0.0030)   (0.0038) (0.0015) (0.0022) (0.0026)
21 0.0034   0.0038 0.0022 0.0022 0.0035 0.0032   0.0037 0.0012 0.0031 0.0025
    (0.0035) (0.0020) (0.0020) (0.0023) (0.0026)   (0.0034) (0.0011) (0.0015) (0.0021)
22 0.0102   0.0105 0.0071 0.0071 0.0097 0.0103   0.0112 0.0052 0.0086 0.0098
    (0.0058) (0.0037) (0.0037) (0.0041) (0.0041)   (0.0052) (0.0022) (0.0030) (0.0029)
23 0.0034   0.0039 0.0022 0.0022 0.0031 0.0030   0.0035 0.0011 0.0030 0.0025
    (0.0027) (0.0015) (0.0016) (0.0021) (0.0022)   (0.0030) (0.0010) (0.0015) (0.0021)
24 0.0102   0.0105 0.0069 0.0069 0.0107 0.0114   0.0106 0.0049 0.0088 0.0115
    (0.0050) (0.0033) (0.0033) (0.0043) (0.0042)   (0.0059) (0.0023) (0.0036) (0.0043)
1, 3 0.0040   0.0046 0.0037 0.0037 0.0041 0.0040   0.0052 0.0041 0.0049 0.0047
    (0.0041) (0.0032) (0.0032) (0.0033) (0.0034)   (0.0048) (0.0029) (0.0030) (0.0031)
1, 9 0.0029   0.0036 0.0026 0.0026 0.0030 0.0022   0.0030 0.0018 0.0027 0.0010
    (0.0027) (0.0019) (0.0019) (0.0026) (0.0028)   (0.0031) (0.0015) (0.0022) (0.0020)
6, 7 0.0204   0.0191 0.0148 0.0146 0.0215 0.0199   0.0206 0.0124 0.0212 0.0223
    (0.0067) (0.0048) (0.0047) (0.0056) (0.0054)   (0.0073) (0.0037) (0.0048) (0.0046)
3, 14 0.0034   0.0030 0.0019 0.0019 0.0020 0.0027   0.0036 0.0015 0.0025 0.0029
    (0.0029) (0.0018) (0.0018) (0.0026) (0.0025)   (0.0026) (0.0011) (0.0020) (0.0018)
3, 6, 7 0.0034   0.0039 0.0029 0.0029 0.0021 0.0027   0.0038 0.0025 0.0024 0.0019
    (0.0035) (0.0025) (0.0025) (0.0029) (0.0031)   (0.0041) (0.0022) (0.0028) (0.0029)
1, 6, 7, 24 0.0034   0.0039 0.0028 0.0028 0.0007 0.0006   0.0039 0.0020 0.0002 0.0000
    (0.0025) (0.0019) (0.0019) (0.0019) (0.0018)   (0.0026) (0.0014) (0.0009) (0.0002)
1, 12, 13, 22, 25 0.0034   0.0035 0.0025 0.0024 0.0027 0.0032   0.0033 0.0017 0.0004 0.0031
    (0.0022) (0.0016) (0.0017) (0.0023) (0.0023)   (0.0026) (0.0013) (0.0014) (0.0020)
Sum of remaining   0.0340 0.0227 0.0227 0.0102 0.0074   0.0703 0.0310 0.0255 0.0076
haplotype probabilities            
Sum of probabilities   0.0103 0.0097 0.0100 0.0132 0.0173   0.0131 0.0118 0.0111 0.0200
of missed haplotypes            
Sum of squared errors   0.00077 0.00125 0.00126 0.00059 0.00054   0.00101 0.00281 0.00055 0.00048
Length of list   39.65 152.30 39.65 22.08 19.63   71.24 71.24 29.29 19.81
SD of length   (3.90) (71.65) (3.90) (4.20) (3.70)   (5.02) (5.02) (5.82) (5.22)
  1. Average estimates of haplotype frequencies for a 25 loci case based on 100 simulations of n pools of k individuals each using CDMLE (collapsed data MLE), EML (EM with combinatorially determined list), EM-CDL (EM with CD list), EM-ATCDL (augmented and trimmed CD list) and EM-TCDL (CD list with trimming and no augmentation), with standard errors in parentheses.