Raju, van der Linden, and Fleer (1995) developed a parametric procedure called
DFIT (Differential Functioning of Items and Tests) that detects DTF/DIF by comparing
test characteristic curves (TCCs). Linking coefficients, item parameters, and latent
trait scores are required to compute the TCCs. The DFIT method basically identifies
items that have DIF and determines whether DIF cancels or cumulates to produce DTF.
Currently, Raju et al.'s (1995) DFIT method is implemented in the DFITD4 computer program.
DFITD4 identifies DIF items, computes DTF, and when DTF is found, determines which DIF
items, if any, should be removed to establish measurement equivalence. According to
Raju et al., DTF is present when the chi-square statistic is statistically significant
and the DTF index is greater than .006 for two options, .010 for three options, .013
for four options, and .016 for five options.
Conducting DTF Analysis Using DFITD4
Create a folder for storing the following files required for the DFIT analysis.
Put the PARTO3PL.EXE, DFITD4.EXE, and DFITD4.JCL files into that folder, along with the BILOG output files containing item parameters for the reference and focal groups, and theta values (latent trait values) for the reference group (thetas for the focal group are not required). The item parameter files must be named using the extension *.par; the theta files should have the extension *.sco (see how to create *.sco file
here).
Double click on the PARTO3PL.EXE executable file. It will find all the *.par files in that folder, delete the first four lines and subsequent alternating lines, and rename the files using the *.3PL extension. (In our example, focal and reference parameter files are named dfitfoc.3pl and dfitref.3pl, respectively. Similarly, the theta file for the reference group is named dfitref.sco.
Run the ITERLINK program to obtain the linking constants required by DFIT.
Open and modify the file called DFITD4.JCL. This is an input file for the DFITD4 program. It contains information for conducting the DTF analysis as well as descriptive material provided by Raju. Assuming your input files are in the format created by PARTO3PL, you only need to modify the first two lines.
a. Line 1: Enter a title for the DTF analysis.
b. Line 2: Five values, separated by spaces, must be entered on this line, as shown below. The first value specifies the number of items (30 in our example); the second should be 1.0 for the 3PL model; the third sets the criterion for the DTF index (.006 for dichotomous data); the fourth and fifth values are the linking constants, A and K, respectively (1.0307 and -0.3478).
c. Line 3: Enter a sequence of 1s on this line to include every item in your analysis. To exclude an item, leave a blank in the corresponding column.
d. Lines 4 and 5: Specify the FORTRAN format statements for reading your item parameter files. The *.3pl files created above use the format shown.
e. Line 6: Specify the FORTRAN format statement for reading theta values for the reference group. The *.sco files created by BILOG use the format shown. Note, what Raju calls the FOCAL group is actually the REFERENCE group, and vice versa.
AN EXAMPLE FOR THE DFITD4 PROGRAM for o
scale
30 1.0 .006 1.031 -0.348
111111111111111111111111111111
(32X,2F12.6,12x,f12.6)
(32X,2F12.6,12x,f12.6)
(/35x,F12.6)
Running DFITD4
Type DFITD4.EXE at a DOS prompt.
When prompted for the name of the input file, type DFITD4.JCL.
When prompted for the focal group item data name, type DFITREF.3PL. Make sure that you specify the item parameter file for the REFERENCE group (What Raju calls the FOCAL group is actually the REFERENCE group, and vice versa).
When prompted for the reference group item data name, type DFITFOC.3PL.
When prompted for the focal group theta file name, type DFITREF.SCO.
When prompted for the output file name, type DFITD4.OUT
Interpreting DFITD4 Output
At the beginning of the DFITD4.OUT file, you will find reference and focal group parameter, as well as the linking constants.
The next information presented in the output file is a table of DIF results.
This table is not needed when conducting DTF analysis. Instead, refer to the
summary of results at the end of the file. DTF is present if the "PROB" in column 5
is less than .05 and the DTF value in column 3 is greater than Raju's suggested criterion
(in this case, .006, because our data were dichotomous).
In this case, no DTF was found.
As an additional example, the following output was taken from an analysis where DTF was
found. Note, that on runs 1 through 10, the chi-squares were all significant and the
DTF index exceeded the critical value of .006. Consequently, the item with the largest
DIF statistic on each run was removed from the calculations; the removed items are shown
in column 2. On run 11, after eliminating the items shown, DIF cancellation occurred,
as indicated by the DTF index value of .00494.
