Skip to main content

Table 4 Average estimates of haplotype frequencies for a 32 loci case

From: An EM algorithm based on an internal list for estimating haplotype distributions of rare variants from pooled genotype data

    k=2    k=4
Position of ‘1’ TRUE   CDMLE EML EM- CDL EM-ATCDL EM-TCDL   CDMLE EM-CDL EM-ATCDL EM-TCDL
       (a) n=100       
None 0.7995   0.7979 0.8248 0.8250 0.7981 0.8009   0.7990 0.8451 0.7970 0.8040
    (0.0150) (0.0117) (0.0117) (0.0148) (0.0132)   (0.0154) (0.0086) (0.0133) (0.0125)
1 0.0509   0.0514 0.0433 0.0433 0.0492 0.0503   0.0502 0.0387 0.0462 0.0507
    (0.0103) (0.0082) (0.0082) (0.0089) (0.0088)   (0.0121) (0.0069) (0.0077) (0.0080)
2 0.0034   0.0035 0.0020 0.0020 0.0033 0.0031   0.0037 0.0011 0.0030 0.0024
    (0.0032) (0.0018) (0.0018) (0.0024) (0.0026)   (0.0035) (0.0011) (0.0013) (0.0021)
3 0.0436   0.0430 0.0362 0.0362 0.0435 0.0426   0.0441 0.0322 0.0406 0.0426
    (0.0092) (0.0075) (0.0075) (0.0082) (0.0074)   (0.0105) (0.0054) (0.0071) (0.0065)
5 0.0034   0.0033 0.0018 0.0018 0.0032 0.0028   0.0034 0.0011 0.0030 0.0023
    (0.0028) (0.0016) (0.0016) (0.0020) (0.0023)   (0.0035) (0.0011) (0.0013) (0.0021)
6 0.0034   0.0038 0.0023 0.0023 0.0033 0.0031   0.0033 0.0014 0.0028 0.0022
    (0.0031) (0.0019) (0.0020) (0.0021) (0.0025)   (0.0035) (0.0013) (0.0014) (0.0019)
9 0.0073   0.0080 0.0054 0.0054 0.0079 0.0088   0.0081 0.0036 0.0070 0.0092
    (0.0038) (0.0026) (0.0026) (0.0032) (0.0037)   (0.0043) (0.0019) (0.0026) (0.0031)
11 0.0034   0.0032 0.0018 0.0018 0.0030 0.0027   0.0032 0.0010 0.0029 0.0023
    (0.0026) (0.0016) (0.0016) (0.0022) (0.0024)   (0.0031) (0.0011) (0.0015) (0.0021)
15 0.0034   0.0035 0.0019 0.0019 0.0030 0.0028   0.0038 0.0012 0.0031 0.0028
    (0.0033) (0.0018) (0.0018) (0.0023) (0.0025)   (0.0031) (0.0010) (0.0015) (0.0021)
19 0.0068   0.0063 0.0039 0.0039 0.0062 0.0061   0.0066 0.0026 0.0056 0.0057
    (0.0039) (0.0025) (0.0025) (0.0029) (0.0034)   (0.0041) (0.0015) (0.0018) (0.0026)
20 0.0068   0.0068 0.0042 0.0042 0.0062 0.0063   0.0063 0.0026 0.0054 0.0059
    (0.0039) (0.0025) (0.0025) (0.0028) (0.0030)   (0.0038) (0.0015) (0.0022) (0.0026)
21 0.0034   0.0038 0.0022 0.0022 0.0035 0.0032   0.0037 0.0012 0.0031 0.0025
    (0.0035) (0.0020) (0.0020) (0.0023) (0.0026)   (0.0034) (0.0011) (0.0015) (0.0021)
22 0.0102   0.0105 0.0071 0.0071 0.0097 0.0103   0.0112 0.0052 0.0086 0.0098
    (0.0058) (0.0037) (0.0037) (0.0041) (0.0041)   (0.0052) (0.0022) (0.0030) (0.0029)
23 0.0034   0.0039 0.0022 0.0022 0.0031 0.0030   0.0035 0.0011 0.0030 0.0025
    (0.0027) (0.0015) (0.0016) (0.0021) (0.0022)   (0.0030) (0.0010) (0.0015) (0.0021)
24 0.0102   0.0105 0.0069 0.0069 0.0107 0.0114   0.0106 0.0049 0.0088 0.0115
    (0.0050) (0.0033) (0.0033) (0.0043) (0.0042)   (0.0059) (0.0023) (0.0036) (0.0043)
1, 3 0.0040   0.0046 0.0037 0.0037 0.0041 0.0040   0.0052 0.0041 0.0049 0.0047
    (0.0041) (0.0032) (0.0032) (0.0033) (0.0034)   (0.0048) (0.0029) (0.0030) (0.0031)
1, 9 0.0029   0.0036 0.0026 0.0026 0.0030 0.0022   0.0030 0.0018 0.0027 0.0010
    (0.0027) (0.0019) (0.0019) (0.0026) (0.0028)   (0.0031) (0.0015) (0.0022) (0.0020)
6, 7 0.0204   0.0191 0.0148 0.0146 0.0215 0.0199   0.0206 0.0124 0.0212 0.0223
    (0.0067) (0.0048) (0.0047) (0.0056) (0.0054)   (0.0073) (0.0037) (0.0048) (0.0046)
3, 14 0.0034   0.0030 0.0019 0.0019 0.0020 0.0027   0.0036 0.0015 0.0025 0.0029
    (0.0029) (0.0018) (0.0018) (0.0026) (0.0025)   (0.0026) (0.0011) (0.0020) (0.0018)
3, 6, 7 0.0034   0.0039 0.0029 0.0029 0.0021 0.0027   0.0038 0.0025 0.0024 0.0019
    (0.0035) (0.0025) (0.0025) (0.0029) (0.0031)   (0.0041) (0.0022) (0.0028) (0.0029)
1, 6, 7, 24 0.0034   0.0039 0.0028 0.0028 0.0007 0.0006   0.0039 0.0020 0.0002 0.0000
    (0.0025) (0.0019) (0.0019) (0.0019) (0.0018)   (0.0026) (0.0014) (0.0009) (0.0002)
1, 12, 13, 22, 25 0.0034   0.0035 0.0025 0.0024 0.0027 0.0032   0.0033 0.0017 0.0004 0.0031
    (0.0022) (0.0016) (0.0017) (0.0023) (0.0023)   (0.0026) (0.0013) (0.0014) (0.0020)
Sum of remaining   0.0340 0.0227 0.0227 0.0102 0.0074   0.0703 0.0310 0.0255 0.0076
haplotype probabilities            
Sum of probabilities   0.0103 0.0097 0.0100 0.0132 0.0173   0.0131 0.0118 0.0111 0.0200
of missed haplotypes            
Sum of squared errors   0.00077 0.00125 0.00126 0.00059 0.00054   0.00101 0.00281 0.00055 0.00048
Length of list   39.65 152.30 39.65 22.08 19.63   71.24 71.24 29.29 19.81
SD of length   (3.90) (71.65) (3.90) (4.20) (3.70)   (5.02) (5.02) (5.82) (5.22)
       (b) n=200       
None 0.7113   0.7108 0.7674 0.7677 0.7115 0.7219   0.7110 0.8075 0.7109 0.7374
    (0.0202) (0.0132) (0.0132) (0.0197) (0.0161)   (0.0232) (0.0119) (0.0192) (0.0147)
1 0.0034   0.0031 0.0013 0.0013 0.0032 0.0022   0.0034 0.0005 0.0031 0.0012
    (0.0032) (0.0014) (0.0014) (0.0022) (0.0025)   (0.0054) (0.0007) (0.0015) (0.0018)
3 0.0034   0.0039 0.0016 0.0016 0.0032 0.0028   0.0036 0.0005 0.0030 0.0017
    (0.0034) (0.0014) (0.0014) (0.0022) (0.0025)   (0.0047) (0.0007) (0.0014) (0.0022)
5 0.0034   0.0037 0.0016 0.0016 0.0035 0.0029   0.0036 0.0006 0.0032 0.0017
    (0.0035) (0.0015) (0.0015) (0.0022) (0.0027)   (0.0049) (0.0007) (0.0013) (0.0022)
7 0.0068   0.0066 0.0038 0.0041 0.0061 0.0064   0.0065 0.0027 0.0058 0.0061
    (0.0048) (0.0024) (0.0028) (0.0030) (0.0040)   (0.0070) (0.0024) (0.0024) (0.0053)
9 0.0034   0.0037 0.0015 0.0015 0.0034 0.0029   0.0038 0.0006 0.0031 0.0016
    (0.0038) (0.0016) (0.0016) (0.0024) (0.0029)   (0.0049) (0.0008) (0.0015) (0.0022)
10 0.0102   0.0098 0.0050 0.0050 0.0087 0.0093   0.0103 0.0026 0.0088 0.0085
    (0.0059) (0.0029) (0.0030) (0.0035) (0.0035)   (0.0072) (0.0017) (0.0026) (0.0042)
11 0.0034   0.0038 0.0016 0.0016 0.0032 0.0029   0.0027 0.0004 0.0031 0.0010
    (0.0030) (0.0013) (0.0013) (0.0023) (0.0026)   (0.0044) (0.0007) (0.0014) (0.0019)
14 0.0034   0.0034 0.0014 0.0014 0.0031 0.0023   0.0030 0.0004 0.0029 0.0013
    (0.0037) (0.0016) (0.0016) (0.0022) (0.0027)   (0.0048) (0.0007) (0.0015) (0.0021)
17 0.0034   0.0033 0.0014 0.0014 0.0029 0.0023   0.0031 0.0005 0.0028 0.0014
    (0.0035) (0.0014) (0.0014) (0.0021) (0.0024)   (0.0047) (0.0008) (0.0015) (0.0021)
20 0.0034   0.0037 0.0015 0.0015 0.0030 0.0025   0.0028 0.0004 0.0029 0.0014
    (0.0035) (0.0014) (0.0014) (0.0019) (0.0025)   (0.0042) (0.0006) (0.0014) (0.0021)
21 0.0264   0.0251 0.0164 0.0164 0.0252 0.0252   0.0266 0.0117 0.0257 0.0260
    (0.0089) (0.0051) (0.0051) (0.0064) (0.0063)   (0.0135) (0.0040) (0.0055) (0.0056)
22 0.0068   0.0074 0.0040 0.0040 0.0095 0.0099   0.0067 0.0016 0.0089 0.0074
    (0.0055) (0.0029) (0.0029) (0.0040) (0.0051)   (0.0067) (0.0017) (0.0038) (0.0062)
24 0.0306   0.0307 0.0223 0.0234 0.0289 0.0295   0.0297 0.0194 0.0273 0.0291
    (0.0096) (0.0066) (0.0066) (0.0070) (0.0074)   (0.0146) (0.0049) (0.0058) (0.0061)
25 0.0136   0.0138 0.0075 0.0075 0.0129 0.0129   0.0155 0.0047 0.0139 0.0125
    (0.0072) (0.0036) (0.0036) (0.0040) (0.0036)   (0.0100) (0.0029) (0.0042) (0.0051)
26 0.0034   0.0033 0.0013 0.0013 0.0029 0.0023   0.0037 0.0005 0.0031 0.0014
    (0.0035) (0.0014) (0.0014) (0.0024) (0.0026)   (0.0052) (0.0007) (0.0014) (0.0021)
28 0.0675   0.0661 0.0487 0.0488 0.0615 0.0640   0.0668 0.0390 0.0614 0.0629
    (0.0136) (0.0089) (0.0089) (0.0107) (0.0105)   (0.0162) (0.0075) (0.0090) (0.0086)
30 0.0036   0.0032 0.0017 0.0017 0.0070 0.0064   0.0043 0.0009 0.0077 0.0060
    (0.0029) (0.0015) (0.0015) (0.0039) (0.0056)   (0.0053) (0.0011) (0.0040) (0.0067)
31 0.0034   0.0037 0.0016 0.0016 0.0031 0.0025   0.0031 0.0004 0.0030 0.0014
    (0.0037) (0.0016) (0.0016) (0.0021) (0.0025)   (0.0044) (0.0006) (0.0013) (0.0020)
32 0.0038   0.0034 0.0016 0.0016 0.0045 0.0041   0.0041 0.0007 0.0042 0.0029
    (0.0032) (0.0015) (0.0016) (0.0029) (0.0037)   (0.0060) (0.0010) (0.0021) (0.0038)
2, 25 0.0034   0.0034 0.0015 0.0015 0.0021 0.0026   0.0036 0.0006 0.0013 0.0018
    (0.0033) (0.0015) (0.0015) (0.0029) (0.0028)   (0.0048) (0.0008) (0.0020) (0.0023)
7, 24 0.0510   0.0507 0.0403 0.0386 0.0525 0.0530   0.0523 0.0319 0.0524 0.0519
    (0.0120) (0.0079) (0.0078) (0.0094) (0.0084)   (0.0139) (0.0063) (0.0089) (0.0064)
12, 13 0.0034   0.0031 0.0013 0.0013 0.0016 0.0023   0.0029 0.0005 0.0011 0.0014
    (0.0033) (0.0015) (0.0015) (0.0025) (0.0026)   (0.0049) (0.0009) (0.0020) (0.0022)
21, 23 0.0034   0.0031 0.0015 0.0015 0.0018 0.0023   0.0035 0.0007 0.0016 0.0019
    (0.0035) (0.0016) (0.0016) (0.0027) (0.0026)   (0.0042) (0.0009) (0.0023) (0.0023)
21, 28 0.0009   0.0031 0.0021 0.0021 0.0022 0.0019   0.0035 0.0016 0.0015 0.0010
    (0.0038) (0.0022) (0.0022) (0.0027) (0.0030)   (0.0050) (0.0017) (0.0023) (0.0027)
21, 30 0.0033   0.0035 0.0018 0.0018 0.0022 0.0018   0.0035 0.0008 0.0016 0.0012
    (0.0033) (0.0016) (0.0016) (0.0027) (0.0027)   (0.0044) (0.0010) (0.0024) (0.0023)
22, 30 0.0034   0.0029 0.0014 0.0013 0.0017 0.0017   0.0037 0.0007 0.0014 0.0014
    (0.0031) (0.0015) (0.0015) (0.0024) (0.0027)   (0.0051) (0.0009) (0.0021) (0.0022)
24, 28 0.0034   0.0042 0.0034 0.0033 0.0036 0.0035   0.0064 0.0036 0.0037 0.0040
    (0.0048) (0.0030) (0.0030) (0.0032) (0.0034)   (0.0072) (0.0028) (0.0029) (0.0036)
28, 32 0.0030   0.0034 0.0019 0.0019 0.0022 0.0019   0.0033 0.0011 0.0018 0.0018
    (0.0033) (0.0018) (0.0019) (0.0028) (0.0028)   (0.0037) (0.0011) (0.0021) (0.0026)
4, 7, 24 0.0034   0.0028 0.0015 0.0016 0.0013 0.0024   0.0028 0.0008 0.0008 0.0019
    (0.0025) (0.0014) (0.0014) (0.0022) (0.0022)   (0.0027) (0.0008) (0.0016) (0.0019)
7, 22, 24 0.0034   0.0042 0.0025 0.0026 0.0019 0.0011   0.0029 0.0015 0.0018 0.0012
    (0.0036) (0.0022) (0.0021) (0.0028) (0.0024)   (0.0034) (0.0013) (0.0024) (0.0025)
7, 24, 30 0.0034   0.0034 0.0023 0.0023 0.0019 0.0018   0.0033 0.0014 0.0018 0.0018
    (0.0032) (0.0022) (0.0022) (0.0031) (0.0031)   (0.0035) (0.0013) (0.0024) (0.0029)
Sum of remaining   0.0828 0.0454 0.0453 0.0178 0.0083   0.2194 0.0592 0.0243 0.0160
haplotype probabilities            
Sum of probabilities   0.0278 0.0265 0.0271 0.0263 0.0384   0.0474 0.0462 0.0222 0.0553
of missed haplotypes            
Sum of squared errors   0.00161 0.00451 0.00456 0.00103 0.00100   0.00334 0.01152 0.00092 0.00152
Length of list   62.39 159.68 62.39 30.60 23.57   106.91 106.91 36.11 21.92
SD of length   (4.88) (27.84) (4.88) (6.04) (4.22)   (6.41) (6.41) (7.12) (5.11)
  1. Average estimates of haplotype frequencies for a 32 loci case based on 100 simulations of n pools of k individuals each using CDMLE (collapsed data MLE), EML (EM with combinatorially determined list), EM-CDL (EM with CD list), EM-ATCDL (augmented and trimmed CD list) and EM-TCDL (CD list with trimming and no augmentation), with standard errors in parentheses.