Estimating Parameters (1PL, 2PL, and 3PL)

AGREEABLENESS CALIBRATION FOR IRT TUTORIAL.
>COMMENT
>GLOBAL DFN='AGR2_CAL.DAT', NIDW=4, NPARM=3, OFNAME='OMIT.KEY', SAVE;
>SAVE SCO = 'AGR2_CAL.SCO', PARM = 'AGR2_CAL.PAR', COV = 'AGR2_CAL.COV';
>LENGTH NITEMS=(10);
>INPUT  SAMPLE=99999;
(4A1,10A1)
>TEST TNAME=AGR;
>CALIB  NQPT=40, CYC=100, NEW=30, CRIT=.001, PLOT=0;
>SCORE  MET=2, IDIST=0, RSC=0, NOPRINT;

If you are using the MS-DOS editor to create or edit this file, it should appear as follows:


Example data and appropriate BILOG files can be downloaded below.

*.PH1
After running BILOG, first examine .ph1 file. This file contains a summary of your input file, the first two response vectors from your data file and a table of classical test theory (CTT) statistics. You should first check that your response data was read correctly and that the CTT statistics match those obtained during the data preparation phase.

In particular, look for items that exhibit extreme p-values (p>.95 or p<.05) or near zero (and of course negative) item-total correlations. These items may cause problems for the convergence of the parameter estimation procedure. Information regarding the IRT parameter estimation procedure is provided in the *.ph2 file.


If you are using the MS-DOS editor to view the agr1_cal.ph1 file, it should appear as follows:


As described above, the last portion of the file provides classical test statistics:

*.PH2
The *.ph2 file contains a summary of the input file specifications concerning parameter estimation, the quadrature points used for parameter estimation (we recommend 40), likelihood values across estimation cycles, and a table of item parameters and standard errors.

It is important to check that the estimation procedure converged; i.e. the largest change in any parameter for the last iteration was below the selected convergence criterion. In this case, a criterion of .001 was used. Lack of convergence is indicated by large changes in the likelihood function up to the maximum number of cycles.

If the estimation procedure did not converge, a number of strategies can be applied. First, one should try increasing the number of cycles. Second, it may be necessary either to specify an alternative prior distribution for one or more parameters (Bays prior distribution). Third, it is often necessary to delete items with low item-total correlations (e.g., < 0.1) as provided in the *.ph1 file's CTT statistics. Note BILOG will not converge if an item with a negative item-total correlation is present.


If you are using the MS-DOS editor to view the agr1_cal.ph2 file, we see that only 14 cycles were necessary to adequately estimate the parameters:


*.PH3
The *.ph3 file contains information concerning theta estimation.


The following output files are created only if explicitly requested using the 'PARM', 'COV' and 'SCO' commands in the BILOG input file.

*.PAR
The *.par file contains item parameters and their standard errors. For each item there are two lines. The first line contains the parameter estimates in the fourth, fifth, and seventh columns. The second line contains the standard errors for these estimates directly under the actual estimates.

In order to rearrange the parameters in a more easy to understand file, we have a program, PARTO3PL, that will read in all *.par files in a directory, delete the first four lines (title information) and the lines that contain the standard errors, leaving only the parameters available for later use in a *.3pl file. Such a file is useful when assessing model-data fit and with examining the presence of differential item functioning.

These parameters can be plotted with the MODFIT program (explained later in this tutorial). Below are the plots for the Agreeableness scale.

If you are using the MS-DOS editor to view the agr1_cal.par file, it should appear as follows:


By using these values with the equation for the 3PL model and varying levels of theta in a spreadsheet, plots can be generated. Later in this tutorial, you will be introduced to the MODFIT program, which can generate plots automatically after being provided with parameters.

Below are the plots for the Agreeableness scale that can be made from the parameters obtained in BILOG.


Plots of Agreeableness items
Previous   Next


Below are the plots for the Conscientiousness scale that can be made from the parameters obtained in BILOG.


Plots of Conscientiousness items
Previous   Next


*.SCO
The *.sco file is created in during Phase 3 and provides ability score information for each respondent. (Note that in the screen capture below, the last two columns are not discussed as they are beyond the scope of this tutorial, please consult your BILOG manual for these special applications, i.e., group fit probability and EAP scoring.)

View of the agr1_cal.sco file in the MS DOS editor:


*.COV
The *.cov file is similar to the *.par file in appearance, but provides parameters as well as the variances/covariances between the parameters. The covariances for the item parameter estimates are necessary for differential item functioning analyses (DIF, explained later in this tutorial).

View of the agr1_cal.cov file in the MS DOS editor:

Note: some of the arrows in the following picture are pointing at the wrong direction (e.g., arrows 5 to 9). Here is the correct format for covariance files:
Lines 1 & 2 contains the title records of the BILOG run that created the covariance file. Afterwards, there are two lines for each item. The first record contains
(1-4) item name,
(5-12) subtest name,
arrow 1: (13-24) slope estimate (a),
arrow 2: (25-36) threshold estimate (b),
arrow 3: (37-48) lower asymptote estimate (c),
arrow 4: (49-60) estimation of error variance for slope (SE_a²), and
arrow 5: (61-72) estimation of error covariance for slope and threshold (Cov_a,b).
The second record contains
(1-12) blank filler,
arrow 6: (13-24) estimation of error variance for threshold (SE_b²),
arrow 7: (25-36) estimation of error covariance for slope and asymptote (Cov_a,c),
arrow 8: (37-48) estimation of the error covariance for threshold and asymptote ((Cov_b,c), and
arrow 9: (49-60) the estimate of the error variance for lower asymptote (SEE_c²).
Each column is formatted as (f12.6).