File: test1b.stats.mapsswe

package info (click to toggle)
sctk 2.4.10-20151007-1312Z%2Bdfsg2-3
  • links: PTS
  • area: main
  • in suites: stretch
  • size: 17,332 kB
  • ctags: 4,598
  • sloc: ansic: 25,664; cpp: 13,648; perl: 9,898; sh: 1,566; makefile: 916
file content (70 lines) | stat: -rw-r--r-- 2,713 bytes parent folder | download | duplicates (3)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
      SEGMENTATION REPORT FOR SYSTEMS
              scoring/itword_20.ctm and scoring/itmono_8.ctm
Minimum Number of Correct Boundary words 2

  Number of Segments  3,     1.500 per sentence
  Number of Sentences  2

    # Ref wrds      Err scoring/itword_20.ctm   Err scoring/itmono_8.ctm
---------------------------------------
Totals   13              7           9

Pct err               53.8%       69.2%






                            McNemar test results
                            ====================

                                     scoring/itmono_8.ctm

                                  corr     incorr
      scoring/itword_20.ctm      corr     0          1
	                 incorr     0          2
The NULL Hypothesis:

     The number unique utterance errors are equal for both systems.

Alternate Hypothesis:

     The number of unique utterance errors for both systems are NOT equal.
Decision Analysis:

     Assumptions:
        A1: The distibrution of unique utterance errors
            follows the binomial distribution for N fair coin tosses.

     Rejection criterion:
        Reject the null hypothesis at the 95% confidence level based
        on the following critical values table.  N is the sum of the
        unique utterance errors for both systems being compared and
        MIN(uue) is the minimum number of unique utterance
        foe either system.

          MIN(uue)      P(MIN(uue) | N=  1)
          --------      -------------------
               0               1.000

     Decision:
        There were MIN(uue)=0 unique utterance errors, the probability of
        it occurring is 1.000, therefore the null hypothesis is ACCEPTED
        There is no statistical difference between scoring/itword_20.ctm and scoring/itmono_8.ctm
MTCH_PR_RESULTS (systems: scoring/itword_20.ctm scoring/itmono_8.ctm) (# segs: 3) (Seg per Sent: 1.500) (% Error: 53.85 69.23) (mean: -0.667) (std dev: 0.577) (Z Stat: -2.000) (Stat Diff: Yes)



                COMPARISON MATRIX: FOR THE MATCHED PAIRS TEST
        PECENTAGES ARE MEAN PCT ERROR/SEGMENT.  FIRST # IS LEFT SYSTEM
                  Minimum Number of Correct Boundary words 2


   |-----------------------------------------------------------------------|
   |                       | scoring/itword_20.ctm |  scoring/itmono_8.ctm |
   |-----------------------+-----------------------+-----------------------|
   | scoring/itword_20.ctm |                       | scoring/itword_20.ctm |
   |-----------------------+-----------------------+-----------------------|
   |  scoring/itmono_8.ctm |                       |                       |
   |-----------------------------------------------------------------------|