Computing Chi-Square Statistics and Fit-Plots Using the MODFIT Program
MODFIT is a Visual Basic for Applications program created using Microsoft Excel 2000. To run MODFIT, Excel 2000
or higher must be installed on your system. The program consists of eight read-only *.xls files. To run the program,
all files must be saved in the same folder. Download MODFIT and a folder containing example data and results here.
MODFIT computes fit plots and chi-squares for item singles, doubles, and triples using the method described by Drasgow et al. (1995). In addition, item/option response functions, information functions, test characteristic curves, test information functions, and conditional standard errors can be computed for several dichotomous and polytomous models. To obtain a copy of MODFIT
for academic research, which is capable of handling more items and larger sample
sizes, please send an e-mail to us, or contact Stephen Stark at University of
South Florida.
Running MODFIT
Assuming you downloaded all the files into a single folder, you can begin running the MODFIT program by opening the main file called MODFIT 4-19-01 for Distribution (web).xls. A message may appear asking whether or not to enable macros; choose "Enable macros." Next, you will be asked whether to open the file as read only; choose "Read only." A splash screen will appear describing the version. It will disappear after about 5 seconds, once the program is loaded into memory.
Next the Introduction and Instructions screen will appear. Press Continue to proceed.
Next, you will be asked to specify either dichotomous or polytomous models. Dichotomous models include the 1PL, 2PL, and 3PL models. Polytomous models include the SGR and BNM. Note, that additional dichotomous and polytomous models are available in the program. The use of dichotomous or polytomous models is exactly the same, with one exception, so, we present an example for the polytomous case involving the SGR model. Thus, click Polytomous to proceed.
Next, you must choose a model, specify the maximum number of response options, and select the type of computations. Note, MODFIT allows items to have different numbers of response options, but you must specify the highest number for plotting and reading response data if examining fit. In your data, responses must be coded consecutively,
beginning with 0; missing data must be coded 9. For example, if you have an inventory using a 7-option Likert format, but some options were collapsed for parameter estimation, you must select 7 as the maximum number of response options, and response data must be coded 0,1,2,3,4,5,6 and 9 for missing.
On the next screen, you must always specify the number of items. If you are examining fit, you must also specify the number of persons in your data file.
Assuming you are examining fit, you must now paste item parameters and response data into the appropriate sheets. MODFIT will activate them automatically. Note, you should use the
Paste Special - Values command in the Excel Edit menu. Once you have pasted the values, click Continue. (For some models, when pasting values, a message box will appear that provides information concerning the parameters and response data. Read carefully, and adjust your values if necessary before proceeding).
A warning message will appear alerting you to potential problems caused by activating other Windows applications while MODFIT is running. In the current version of MODFIT, this issue seems to have been resolved, but running other programs simultaneously may slow the computations substantially.
Next, a message box will appear asking if it is okay to proceed with the calculations. Check the values presented and click "Yes" to continue.
You will be asked where to save the MODFIT results. Note, that MODFIT creates text files, containing details of the calculations, as well as an Excel output file containing response functions, plots, and chi-squares.
Interpreting MODFIT Output
Below are the MODFIT results for the SGR model using the example data accompanying the program. First, click on the ItemParms sheet and make sure that your parameters were read correctly.
Note, that the ORFvals, OIFvals, and IIFvals sheets contain values used for plotting. If you selected the FITPLOTS option, the sheet called EMPvals will contain the empirical response curve values.
To examine fit plots, click on the FITPLOTS sheet. A portion of the sheet is shown below. Note that this sheet contains a fit plot for each option, ordinary option response functions, and the test characteristic curve.
Finally, click the ChiSqr sheet to view the tables of chi-squares for item singles, doubles, and triples, as shown below. In each table, the first column contains the item number; the second contains the sample size for that calculation; the third contains the degrees of freedom (number of cells minus one); the fourth contains the chi-square value; the fifth contains the chi-squares / degrees of freedom ratio statistic; columns six and seven contain the chi-squares adjusted to have a sample size of 3,000. Note, by coincidence, the chi-squares and adjusted chi-squares are the same because a sample of 3,000 was used in this example.
A more useful way of displaying chi-square results is illustrated in the summary tables shown below. The values in each column indicate the number of singlets, doublets, and triplets in each interval. For example, 6 singlets, 4 doublets, and 2 triplets had chi-square to degrees of freedom ratios less than 1, whereas 3 singlets, 1 doublet, and 0 triplets had ratios between 2 and 3. The mean and standard deviation for each item combination are shown in the last two columns. Note, in agreement with the fit plot results, the adjusted chi-square to degrees of freedom ratios are less than 3, indicating excellent fit.
