Skip to main content

Full text of "ERIC ED075383: An Evaluation of School Personnel Utilization Projects. Volume II, An Analysis of the School Personnel Utilization Programs Goals and Objectives with Suggested Revisions."

See other formats


ED 075 383 SP 006 346 
AUTHOR Olson, Margot; Feard, Jacob G. 
TITLE An Evaluation of School Personnel Utilization 

Projects. Volume II, An Analysis of the School 
Personnel Utilization Programs Goals and Objectives 
with Suggested Revisions. 

INSTITUTION Florida State Univ., Tallahassee. Dept. of 
Educational Research. 

SPONS AGENCY Bureau of Educational Personnel Development 

(DHEW/OE) , Washington, D.C. 



NOTE 43p. 

EDRS PRICE MF-$0.65 HC-$3.29 

DESCRIPTORS *Differentiated Staffs; *Fducational Objectives; 
Manpower Utilization; *Program Descriptions; Program 
Evaluation; *School Personnel; *Staff Utilization 


Volume II of the evaluation of Florida State 
University's School Personnel Utilization Program (SPU) is concerned 
with an analysis of goals and objectives with suggested revisions. 
Part I reports an evaluation of the SPU goal and objectives in terms 
of their value as a communication link between SPU program directors 
and local project personnel. Part II reports the procedures used to 
revise the goal and objectives in order to improve their 
communication function. A seven-item bibliography and appendixes of 
research material are included. (Related documents are SP 006 345 and 
SP 006 347.) (MJM) 

ED 075383 



sn a cn — 


An Analysis of the School Personnei Utilization Programs 
Goals and Objectives with Suggested Revisions 

Prepared by 

Margot Olson 
Jacob G. Beard 

The Evaluation Training Center 
The Department of Educational Research 
Florida State University 
Tallahassee, Florida 

The work presented or reported herein was performed 
pursuant to a grant from the U. S. Office of 
Education, Department of Health, Education and Welfare. 


An Analysis of the School Personnel Utilization Programs 
Goals and Objectives with Suggested Revisions 


Part 1. Evaluation of USOE Guidelines 
Introduction. . 
Method, ....- 
Subjects .... 
Data Analysis. . 
Resulte of the Survey... . «6 « + 
2. Revision of the USOE Guidelines 
Introduction. . «+ + ee ee we ee 
Analysis of Unstructured Responses. 
Logical Analysis. ... +. . ses 

The Revised Objectives. .... 

References . 2. «+ see ee eee 

Appendix A... se ee ee ees 

An Analysis of School Personnel Utilization 

Goal and Objectives with Suggested Revisions 


The United States Office of Education (USOE) Bureau of Educaticnal 
Research has developed a statement of the geal and objectives for its 
School Personne] Utilization (SPU) Program and has arranged, as a parr 
cf a larger evaluation effort, for their evaluation and subsequent re- 
vision. The purpose of the study reported ia this volume was to conduct 
such an evaluation and revision of the SPU goal and objectives. 

The general purpose of the stated goal and objectives of the SPU 

program ss to serve as operating guidelines for local SPU project por- 

sonnel. They describe the purpose of the SPU program and communicate 

the specific expectations of the SPU program directors. They are gener- 
ally aimed at achieving "differentiated staffing." 

Part I of this volume reports an evaluation of the SPU goal and 
objectives in terms of their value as a communication link between SPU 
program directors and local project personnel. The guidelines were 
" subjected to a review by a group of educators associated with SPU and 
by a second group who were not associated with SPU projects. Those 
reviewers evaluated the objectives hy means of a modified semantic dif- 
ferential procedure and by recording their general reactions to the 


Part IL of this volume reports the procedure used to revise the 
goal and objectives in order to improve their communication function. 
The opinions of the reviewers collected in Part I were analyzed by 
mombers of the project staff of the Evaluation Training Center (ETC). 
From this analysis a revision of the guidelines was completed. 

A statement of the original goal and objectives as presentad by 
the USOK is given below: 

The goal of the School Personnel Utilization Program is to 

create institutional change by reorganizing the teaching 

and administrative staffs of elementary and/or secondary 


In order to bring this about the School Personnel Utili- 
zation Program has established the following objectives. 

1, ‘lo cheate a number of differentiated staffing patterns 
shich include the following elements: 

(a) Differentiated functions of all school personnel 
: including teachers, administrators, and parapro- 

(b) Differentiated salaries according to functions and 

(c) Flexible instructional time schedules, 
(d) Differentiated instructional modes. 

2, To improve the management, organizations, instructional 
and technological skills and attitudes of professional 


3. To bring about changes in student attitudes and 

4, To increase community participation in the educational 

5. ‘To encourage State Education Agencies to change their 
certification procedures. 


To promote the participation of local teacher organiza- 
tions in major decisions. 

to encourage universities to make changes in inservice 
and preservice programs. 

Fart I: Evaluation of USOE Guidelines 


The opportunity for an independent group to evaluate guidelines 
developed by the USOE for its public service programs is unusual. A 
review of the Literatuce did not reveal any studies directly applicable 
to such an evaluation. Several articles, which will be reviewed in the 
following paragraphs, were found which indirectly related. 

the involvement of program participants in the specification of 
goals and obtective:s:s has been mentioned in several recent publications. 
Metfesse! and Michael (1967), in their paradigm for the evaluation of 
the effectiveness of ‘school programs, suggested involvement of the total 
schoc: sommutty ia the evaluation of programs. The construction of a 
cohesive set of broad goals and specific objectives was also suggested 
within their evaluation framework. Taylor and Maguire (1967) obtained 
value ratings of high school biology objectives from three groups, 
subject-matter experts, curriculum writers, and biology teachers, who 
were interested in and affected by such objectives. Berlak (1970) 
pointed out that public policy issues in education can be questioned 
by any group who feels strongly enough to precipitate a dispute. To 
avoid such disputes, perhaps, those who establish public policy are 
wise in permitting an evaluation based in part on the opinions of in- 
volved persons. 

Scriven (1967) has emphasized the importance of judging the worth 
of educationa! objectives and Stake (1970) has pointed out that objectives 

can be evaluated and that judgment data can be useful in such processes. 

ae ie 

He stated that thea "clarity or confusion" of an objective should be 

acknowledged and that objectives should identify outcomes which eve 

the most "worrhy" from among a vast multitude of possible goals. Stake 

further stared “objectives are judgment data better treated by rules 

that g.verr mass subjictive responses....objectives, like attitudes in 

all their subjectivity, can be collected and scaled objectively [p, 183])." 
in his review of methods for gathering judgment data, Stake identi- 

fied three vypes cf data which an evaluator might collect: 

lL, judgmental responses elicited with a standardized protocol 
which characterize a group of individuals, 

+ structured or unstructuced observations by one or more 
experts, and 


3, analysis of documents by experts to infer values contributing 
to their creation. 

The first of these seems most appropriate for assessing the attitudes 
of program participants towards their program goals and objectives. 

Stake went on to review procedures for collecting judgment data 
from groups and found that surveys, scaling, Q-technique, and semantic 
differential were commonly used. Of these four, the semantic differen- 
tial has certain properties which make it appealing for use, in a modified 
form, as a techinque for assessing attitudes towards objectives established 
for public policy projects. The semantic differential enables the eval- 
uator to gather data which reveals attitudes toward ideas or concepts. 
Objectives can be considered the concept, albeit a complex concept, which 

is explored by using value-laden bipolar adjective pairs. Taylor and 

‘Maguire (1967) used such a technique in their study of biology objectives 

| 4 which was mentioned earlier, 
The purpose of the present. aude was to gather judgment data from 
| program participants about the objectives written by the USOE for 
School Personnel Utilization (SPU) Projects. Groups of teachers, 
| . administrators, and university pexsonnel were selected and asked to 
respond to a semantic differential assessment procedure and to give 
unstructured responses revealing attitudes about the objectives. In 
“addition to evaluating the existing objectives an cffort was made to 

gin information which would be useful in rewriting and improving the 

existing objectives. 


Sixty people throughout the nation were identified as having the 
potential to-make a meaningfu] contribution to the evaluation of the 
USOE objectives for SPU projects. These persons represented six groups 
of people -- teachers, administrators and college personnel who were 
directly associated with SPU projects; and teachers, administrators 
and college personel who had no known association with the projects. 
Of these 60, five were no longer at the same address and another 
nine individuals failed to respond. The numbers of individuals who re- 

sponded, of those in the initial sample, are shown in Figure 1. Seventy- 

Teachers 13 of 18 
Administrators 16 of 20 
College Personnel 17 of 22 
26 of 32 20 of 28 46 of 60 

Figure 1 
Distribution of Subjects Who Returned the Questionnaire 

Compared with the Number in the Intended Sample 

seven percent of the original sample completed and returned the materials 
sent to them. This percentage i3 increased to 84% when only thorn £5 

vho actually received the mater{ils are coneidered. 


A modified version of a semantic differential was developed to 
collect attitudes of program saitied pants and non-participants towards 
the SPU goal] and objectives (henceforth, called objectives only). The 
objectives were listed as 14 different concepts. Those-which containgd 
several ideas were divided into separate objectives. Fach objective 
was placed at the top of a page and followed by 10 sets of bipolar ad» 
jectives. ‘The adjectives were of two types. The first type related 
to worthiness and desirability; the second, to clarity and confusion. 
There were six of the first type and four of the latter. These two 
goups of adjectives were preceded by phrases so that all respondents 
would establish similar response sets in evaluating the objectives, 
Figure 2 presents an example of a page from the instrument. ‘The cover 
letter, instructions, list of objectives, complete instrument, and 
follow-up letter are included in Appendix A. 

The final instrument was preceded by three preliminary forms, each 
one being a refinement-of the preceding form. The first form included 
approximately 25 pairs of bipolar adjectives and was informally adminis- 
tered to a small group (10) of graduate students in educational research. 
From their responses and comments, the list of adjectives was reduced to 

15. A second form was prepared and informally administered to a group 

OBJECTIVE ~- To improve the attitudes of 
professional personnel. 

The idea expressed in this objective is .... 

neneficial eS: a ee: Se SA Se Sa harmful 
‘tppropriate ee Oe ee Se, inappropriate 
timely Eee eel A ee CRE eet ee untimely 
feasihle at eh en Sieg Se infeasible 
logical See: SG LY (ate en eet: illogical 
realistic ee ee SE ARORG ORE Cee OS unrealistic 
The statement of this objective is Awe ; 
., cleac a | SO SS Oe ae ae unclear 
understandable : oo: 3) 3 3 8 confusing 
meaningful ae Sy SO Soe ae (ee meaningless 
adequate See A ait SOW 6 eS ea a inadequate 

QOMMENIS: Please list here any suggestions you have or problems you 
foresee concerning this objective. 

5 Figure 2 
Sample Page from Semantic 

Differential Instrument 

of Florida SPU project directors ard evaluators and State Department. 

of Education personnel, After evaluating their comments, several 

alternative forms were prepared. These were distributed among the 
project personne] at the Evaluation Training Center. From the 
comments of this group, the final. form was composed and prepared 

for distribution to survay participants, 

Data Analysis 

The analysis of the semantic differential was conducted in two 

1. descriptive analysis of the data, and 

2. tests of hypothesis of differences among groups. 
The most favorable response was given the value of seven and the least 
favorable, of one. All omitted responses were coded as neutral (i.e., 
the fourth position). Each participant received a unique identification 
number designed to maintain the distinction of the six groups. 

The unstructured responses were transferred to cards so that the 
comments about each objective could easily be reviewed together. Iden- 
tification of subjects was not retiained. These results will be included 

in Part II which deals with revision of the objectives. 

Results of the Survey - 

Means and standard deviations of response values for each adjective 
by objective classification are presented in Table 1. Since these mean 

responses were al] greater than the neutral response value of four and 

= {0% 

Table 1 

Means and Standard Deviations of Responses to 
Each Objective for Each Bipolar Adjective 

Bipolar Adjectives PSA oars AE Rerer 
de in a pl 5 mo 7 8. ar OE 
‘ : Bene- Appro- Feasi- Logi Real- Under- Mean- Ade- 

ORECEINES __ ficial priate Timely ble cal istic Clear stand ingiul quate _ 
Overall 6.4) 6.52 6.46 5.9) 6.28 5.63 5.52 5.65 5.37 4.52 
Goal AO 66 -82 0.05 %@.05 2.37 1.87 7.48 L.5¢ 2.08 
Differentiated 6.41 6.37 6.33 5,99 6.20 5.85 5.93 5.89 5.87 $4.35 
Functions 83 - 83 .47 2.02 .93 7.25 12.48 1.80 10.384 1.88 
Ditferentiated 6.00 6.11 5.96 5.52 6.290 5.48 6.02 6.13 6.00 3.50 
Salaries Lobe? 880 bas 1.4% 0.97 1.62 0.40 0.4% (0.38 1.77 
4 e 
Flexible 6.26 6.20 6.35 5.91 6.13 5.76 5.96 5.83 5.89 5.4? 
Schedule 1.90 72.45 1.04 2.38 1.41 1.46 1.67 1.88 1.04 12,8? 
Instruction 6.92 5.96 5.91 5.67 5.83 5.65 4.93 4.96 921300 4.72 
Modes 1.27 2.89 20.43 02.383 2.42 Uét 11.89 1.85 7 2,97 
Personne] 6.02 5.91 5.98 5.33 5.65 5.09 5.07 5.20 5.04 4.50 
Management: 1.68 2,59 0.47 2.49 2.73 2.82 2.02 1.69 24.82 3.02 
Professional §586- S:61 5.59 5.26 ‘5.52 5.11 -78 4,87 5.02 4.20 
Organizations 1.338 1.40 1.68 2.89 1.49 67 2.97 1.88 2.76 1,93 
8 ; 
Instructional. 6.72 6.54 6.52 6.28 6.30 6.09 5.98 5 5.87 S52 
Skills w02 “202 “0b “Ret 285 36 --R.57 -EeBE’ TE? 6) 
Personnel 6.15 6.09 6.07 5.30 5.78 5.22 5.41 5.41 5.17 4.79 
Attitudes 1.38 Li d¢ 2.39 2.56 2.62 72.52 2.95 10.84 2.00 2,05 
Student 6.26 6.15 6.09 5.48 5.78 5.43 5.39 5.41 5.25 4,54 
Outcomes 1.27 2ae 88 0.62 2.49 121.87 1.95 12.88 1.82% 2.2% 
Community 6.65 6.61 6.61 6.17 6.46 6.02 6.35 6.33 6.00 5.78 
Participation ae 88 -85 Uta .96 4.13 71.59 2.40 Ue 7,489 
Certification 6.22 6.11 6.1) 5.78 6.00 5.57 5.33 5.54 5.54 5.09 
Procedures 1.92 2.380 2.46 2.55 2.48 12.68 2.20 2.60 2.98 28.22 
Decision 6.09 5.93 6.04 5.54 5.74 5.37 5.89 5.72 5.63 5.24 
Making i190 8087 283 ROD LOL. 477 188 8G BEB T8 
Teacher 6.52 6.46 6.50 5.89 6.52 5.91 5.9 5.80 5.67 5.13 
Training 98 -96 -96 10,68 -94 1.40 71.69 12.75 2.80 2.20 

Note - Standard deviations are in italics. 

Ph ii ae 

several approached the most favorable response value of seven these data 

reveal a generally favorable reaction towards the objectives. As expected, 


the variability of the responses was generally greater when the mean re~ 

sponse was near the mid-point of the scale and was diminished for the extsme 
(high) mean response values. This smaller variability for the high ratings 
might have resulted from greater consensus on these objectives as well as 
by a "ceiling" effect. 

From inspection of Table 1 it might be concluded that the chjectives 
were rated lower on adjectives 7-10, which relate to the clarity with 

which the objectives are expressed, than adjectives 1-6, which relate to 

the worthiness of the objectives. However, caution should be used in 
evaluating mean differences among the adjectives. For example, ir is 
difficult to justify saying that an objective is more logical than it is 
clear. Logic and clarity are different dimensions which can be inter~ 

preted on the same absolute scale only tenuously. This restriction does 

not: apply in the evaluation of objactives within the framework of a single 
adjective. It seems justifiable to conclude, for example, that objockive 
8 is clearer than objective 7. 

A series of statistical tests was made in order to answer the 
following questions: 

1. Did the different groups of respondents rate the objectives 
differently? : 

2. Are the mean ratings for the objectives, across groups and 
adjectives, significantly greater than a neutral response of 

3. If all error variance were removed, would the different objec- 
tives have equal mean ratings? 


The 14 means, across adjectives, for each objective were used as the 
dependent. variables in a multivariate analysis of variance (Morrison, 
1967) and in an analysis of group response profiles (oak. profile 
analysis, Norrison, 1967). Group means used in these tests are pra 
sented in Tables 1 and 2 of Appendix A. The grand means for these 

tests are shown below in Table 2. 

Table 2 

Means and Standard Deviations 
Across Adjectives and Groups for Each Objective 

0 rs ee en ae ee re a ne ee en ae eee. 

OBJECTIVES Mean Standard 
1. Overall Goal 5.83 «2&6 
2. Differentiated Functions 6.02 89 
3. Differentiated Salaries 5.89 1.05 
4. Flexible Schedule 5.938 1.30 
5. Instruction Modes 5.48 1.36 
6. Personnel Management 5.38 1,35 
7. Professional Organizations 5.18 1.25 
8. Instructional Skills 6.18 1.03 
9. Personnel Attitudes 5.53 1,39 
10. Student Outcomes 5.59 1.28 
11. Community Participation 6.30 209 
12. Certification Procedures 5.73 1.41. 
13. Decision Making 5.71 L 3k 
14. Teacher Training, 6.04 1.04 

The results of the analysis of variance are shown in Table 3. There 
were no significant differences between the mean ratings of the objectives 
by those respondents classified as "differentiated staffing" and "non- 

differentiated staffing." There were no significant differences in the 

mean responses related to "position" in the educational system nor with 

= 39:0 

the "position by staffing type" interaction. The hypothesis that. tha 
14 grand means were equal. to the neutral value of four was rejectet 
(p < .01). To further examine this effect, univariate analyses were 
computed to test the hypotheses thac each of the 14 objective means 
was equal. to four, The source table for these tests is reported in 
Table 4, ‘The 14 hypotheses were rejected (p < .01). These results 
substantiate Lhe implication from the descriptive data (see Table 1) 
that the total group of respondents displayed a favorable attitude 

towards each of the objectives. 

Table 3 
Multivariate Analysis of Variance Source 

Table for the 14 Objectives and 
Six Groups 

Grand Means = 4 14,27 46.90% 
Differentiated Staffing (DS) 14,27 0.64 
Position (P) 28,54 0.92 
DS x P 28,54 1,08 
& p< 0] 

The results of the analysis of group response profiles are reported 
in Table 5. No significant differences were found in compar isons of the 
group profiles. However, the hypothesis that the 14 objective means, 
across groups and adjectives, are all equal was rejected. This indicates 
that there were significant Ritiavences in the mean ratings of the 14 


Table 4 
Univariate Analysis of Variance Source 

Table for the Grand Mean Hypotheses of 
the 14 Orjectives 

tee + ce emmee we Hewes +are eeeeoers aeeememenecin 
a a ne ee ee 

nn nn ne nn eee eee eee 

Source Hypochesis SS Error SS df F 
Mean (004. 1--Goal) = 4 150.13 26.51 1,27 152,92% 
Mean (Objective 2) = 4 378.76 33.50 1,27 144 03% 
Mean (Objective 3) = 4 159.06 49,51 1,27 86.74% 
Mean (Objective 4) = 4 175,38 65.44 1,27 72.36% 
Mean (Objective 5) = 4 91.43 77.38 1,27 31,90" 
Mean (Objective 6) = 4 83.26 79.49 1,27 2.8 23% 
Mean (Ubjective 7) = 4 ‘61.31 59.93 1,27 27.62%: 
Mean (Objective 8) = 4 206.40 45,03 1,27 123.75%* 
Mean (Objective 9) = 4 104.17 75.92 1,27 37.05% 
Mean (Objective 10) = 4 116.86. 65.67 1,27 48 05% 
Mean (Ubjective 11) = 4 184,12 43.45 1,27 114,417: 
Mean (Objective 12) = 4 129,61 63.43 L527 53,17" 
Mean (Objective 13) = 4 131,90 69.68 1,27 Shed 
Mean (Objective 14) = 4 234.84 30.90 1,27 297 28: 
* p< .01 
Table 5 
Profile Analysis of Variance 
ot the Six Groups Over the 14 Objective Means 


Grand Means ; 13,28 4.62% 

Differentiated Staffing (DS) 13,28 0.71 

Position (P) 26,56 0.99 

DS x P 26,56 0.90 

* ye 01 

Since the hypothesis that all the objective means are equal (i.e., 

analysis of grand mean profile) was rejected, it seemed that a ranktus of 

the objectives by their mean rating was justified and could facilitate 

the evaluation and revision process. The objectives, beginning with tie 

most lighly rated, are stated in order below. The mean rating is given 

in parenthesis. 

1, (6.30) 



* (6.02) 

8. (5.73) 

9. (5.71) 

10. (5.59) 
11. (5.53) 

12. (5.48) 

To increase community participation in the educational 

Io improve the instructional and technological skills of 
protessional personnel. 

To encourage universities to make changes in insevvice and 
preservice programs. 

To create a number of differentiated staffing patterns 
which include differentiated functions of al] schoo’. 
personnel including teachers, administrators, and 

To create a number of differentiated staffing patterns 
which include fiexible instructional time schedules. 

To create a number of differentiated staffing patterns 
which include differentiated salaries according to 
functions and roles. 

Goal--To create institutional change by reorganizing the 
teaching and administrative staffs of elementary and/or 
secondary schools. 

To encourage State Education Agencies to change their 
certification procedures. 

To promote the participation of local teacher organizations 
in major decisions, 

To bring about changes in student attitudes and achievement. 
To improve the attitudes of professional personnel. 

To create a number of differentiated staffing patterns which 
include differentiated instructional modes. 

13, (%.39) Toe improve the menagement of professionel personne?. 
14. (5,18) To improve the organizations of professfonal. poreonnel, 

It is interesting that the respondents rated the “community partis ?~ 
pation" objective higher than any other in the list. From Table J] it is 
apparent that this objective was considered to be especially worthy 
(adjectives 1-6) and to be clearly stated (adjectives 7-10). Tollorwie-. 
this objective, in order of rating, were those dealing with, “improvesent 
of instructional skills," and "changes in in-service and pre-service 
training pregrans.” Closely following these were those objectives das] Ing 
wich changes in staffing and scheduling. 

Rated lease highly was the objective dealing with the p:ufess‘ton.!) 

organizations. This cbjective was generally rated as leasy wouthy and 

least cicdcly stated, Also ranking near the last were those objectives 

dealing with, "management of professional personnel," "differenviated 

‘and attitudes of students and teachers. 

instructional modes,' 
The last five objectives in the list above received mean ratings o1 
less than five on the “adequacy of statement" dimension. It is quite 
possible that lack of clarity and precision in their statement served to 
lower the ratings of worthiness, i.e., how could an objective be lighiy 
rated if ic is difficult to understand? > 
The comments made on the survey forms by the respondents ada a jieat 

deal of meaning to the rating data and wil] be discussed an tlie uexe 


tl: Revision of the USOF Guidelines 


The revision of the objectives was facilitated by infornatton 
trom a variety of seurces. A logical analysis of the existing chjec- 
tives was performed by project staff in order to examine the extent 
to which the objectives reflect current theories and models of SPN, 
The comments by the survey participants concerning the objectives 
were utilized for ideas and suggestions. The conclusion from the 
semantic differential data that the clarity and adequacy of the obiec- 
tives cevld be improved was emphasized in the revision, Input from 
project staff was encouraged to further refine the objectives. 

The version of the objectives presented in this section of the 

report is not intended as a final form, but only as a step along 2 

continuum of achieving greater value in their role as a communication 

link between federal and local SFU project participants. The fact 
that SPU theories and models are in a constant. state of flux due to 
research, further insight, field experiences, and so forth makes the 
idea of a final version of the objectives unrealistic. It is hoped 
that the chjeetives presented in revised form in this report will be 

evaluaced, criticized, and £urther refined. 

Analysis of tinstructured Responses 

The survey instrument used in evaluating the objectives included 
a request that the respondent "list here any suggestions you have ox 
preblems you foresee ciicnrning this cbjective." Nearly avery respon 
dent made some comments and a few commented on cach objeetive. The 
great majority of the comments were critical, which was expected due 
to the phrasing of the request and sovght due to the nature of this 
evaluation. Relevant comments--those which were expressed by a number 
of subjects or those which revealed a great: deal of insight--were sum- 
marized for each objective and are presented below. 
1. Goal--To create institutional change by reor- 
ginizing the teaching and administrative staffs 
of elementary and/or secondary schools. 
The most £requent comment dealt with the ambiguity expressed bv 
the idea of change. Does the goal express a desire for "change fox 

‘will reorganization result in "significant change," 

the sake of change,’ 
is the idea of improvement necessarily implied? A second group of con= 
ments related to how the change would occur, Will it be done expert- 
mentally first, who does it involve, how will people accept such reforms? 
Several comments expressed the idea that the goal is a sweeping state- 
Ment and is too broad to be meaningful. or evaluated. 
2. Objective--To create a riumber of differentiated 

staffing patterns which include differentiated 

functions of all school personnel including 

teachers, administrators, and paraprofessionals. 

A comment. wis made which expressed the idea that the staffing pat- 

tern expressed in this objective already existed. That is, teachers, 

administrators, and paraprofessionals presently do have different 

- 19 - 

functions within the school. ‘he concept of different finiettons ylehin 
these classifications, vertical differentiation, needs to be expressed. 
Several others stated that the word "function" was confusing and suggest- 
ed that this term be changed or explained, Others asked how or by what 
strategy this objective would be realized. 
3. Objective--To create a number of differentiated 

staffing patterns which include differentiated 

salaries according to functions and roles. 

Again the point was made that one interpretation of this objective 
indicates nothing new. That is, principals are paid more than teachers 
who are paid more than secretaries and so forth. The inclusion of addi- 
tional qualifying phrases is probably necessary to indicate differen- 
tiated salaries within job classifications. One respondent suggested 
that responsibilities is a better term than functions and others indi- 
cated uncertainty concerning how such salary differentiation would be 

4, Objective--To create a number of differentiated 
staffing patterns which include flexible instruc- 
tional time schedules, 

There is some question here about the meaning of flexible instruc- 
tional time schedules and whether it is a necessary part of differen- 
tiated staffing. Does it apply to schedules of students, teachers, or 

Does it include concepts of individualized instruction and/or team 
teaching? One individual commented that such an eb jective seems unreal- 

istic unless computer time is available, 


5. Objecrive--To create a number of differentiated 
statting patterns which include differentiated 
instructional modes. 

Eleven of the 15 comments about this objective revealed confusion 
about the weaning of "differentiated instructional modes." One respon- 
dent: even “checked several of the latest texts in education and could 
not find the term." Thus, unless clarified a variety of interpreta~ 

tions of this objective will probably result. 

6, Objertive--To improve the management: of 
pre fessional personnal. 

A number of comnents implied that the term "management" was too 
ambiguous to be meaningful. Several implied that "improve" was ambi~ 
gious and should include standards for wiatarion: Still other comments 
revecled confusion but were not directed specifically to any part of 
the objective. One respondent replied that truly professional person 
nel can manage themselves. This implies an interpretation that the 
personnel will be wnder very rigid supervision rather than management 
at a higher level which relates more to allocation of. responsibilities 
among staff. Another respondent suggested developing “an organization 
whose operational principles are based upon known research....including, 
the diffusion of power and the self-regulation of staff." 

7. Objective--To improve the organizations of 
professional personnel, . 

A pattern of responses similar to those of the preceding objective 
is evident here. The most frequent comments asked "what organizations" 
and how they would be improved. One respondent mentioned that this 

objective as stated does not seem to be within the scope of SPU. 

. 21 - 

8. Objective--To improve the instructional and 
technological skills of professional personnel. 


The most. frequeac comment concerning this objective related to lack 
of information cr ambiguity, One commenter expressed the idea that this 
objective meant "ler's foster good teachers....if we knew what a ood 
teacher was." Anorher stated that a committee could not improve skil's 
of personne] hut could only provide information or guide, Two pe‘tsons 
commented that seme type cf reward would be necessary to foster motiva- 
tion among personnel to attend worsshops, classes, and so forth. 

9. Objective--To improve the attitudes of professional 

Lack of specifics relating to "what attitudes" and the ambiguity of 

"vas the most frequent criticism of this objective. Are the 

attitudes to be improved specific to differentiated staffing or do thev 
involve more or less? How can the improvement be measured and do atti- 
vides really need to be improved? One respondent asked if the attitudes 
to be improved were related to happiness with low salaries, crowded 

classrooms, autocratic administrators, and unprofessional functions, or 
to making them proud professionals who instruct reasonably sized classes 

at adequate salary and receive respect from their administrators. 

10, Objective--To bring abour changes in student 
attitudes and achievement. 

One respondent stated, "The ol jective is admirable. The problem I 
see, how to achieve the objective." Others asked about the cri- 
teria with which to evaluate attitudes and achievement, some others 

asked if the change to be made was assumed to be positive, and stil] 

others asked about what kinds of changes were implied. One individual. 


sugpested that the objective should be stated in a way which included 
hew the change would be brought about. He suggested that improvement 
of students’ attitudes towards schcol could be facilitated through im- 
proved carcicular offerings and instructional skills. 

11. Objective--To increase communicy participation in 
the educational system. 

With the exception of severa!? responses which indicated more con= 
cise informiticn would improve the objective, the responses were quire 
varied, One individual asked 7£ the ohjective was meant to imply 
either # quantitative increase in participation or a greater variety 
in types of participants or participation. Other comments wera gener- 
aliy favorable and stressed that such participation was important and 
could srchably be measured. 

12. Ohjective--To encourage State Education Agencies 
te change their certification procedures. 

What type of change is implied here? The ambiguity of this state- 
ment, which was criticized by several respondents, is aptly expressed 
by one who wrote, "What do you want? Literacy and 1Q tests added? 

Oral exams? No exams’ Pay your fee and get a license? No depree? 
Mere degrees?" Another respondent suggested that the objective could 
be improved by stating that certification procedures be adapted to 
"provide for new patterns of staffing and for new teaching roles." 

13, Objective--To promote the participation of local 
teacher organizations in major decisions. 

The most often repeated comment related to the ambiguity of the 

term "major decisions." What are these decisions or is this open to 

differences in interpretation as needed? One individual mentioned 

= 94.4 

that a geal of differentiated staffing was the promotion of participa- 
tion in major decisions of everyone concerned, and therefore, whv 
should local teacher organizations be singled out. Three ather quas- 
tions which were raised related to the meaning of promote, how much 
participation, and at what: levels of the school organization this vould 

be emploved, 

14, Ubjeutive--To encourage uwriversities to make 
changes in inservice and preservice programs, 

The rypes or changes implie¢c and how this would be done were the 
comments most: often expressed. Again, as in the statement of the goal, 
:some respondents felce this statement implied change for the sake of 
change. One individual suggested that the implicd changes should at 
least relace to staffing patterns, another mentioned that perhans all 
universities did not need to make changes in their programs, and an- 
other suggested that the term "encourage" was too vague and should be 


Logical Analysis 

In recent months, a number of publications on the concept of dif- 
ferentiated staffing have become available. From these, directors and 
other personnel of SPU projects have arobably developed their indivi- 
dual nctats for SPU. A general, conceptual model of staffing has 
recently been developed by DeBloois (1971). This general model has 

been designed to encompsss the known models of staffing and to be con- 

sistent with curreut organizational theory. DeBloois' staffing model 
g ) 8 

= 2h. 

should, therefore, provide a good reference with which to compare the: 

existing SPU objectives. 

A schematic representation of the conceptual model is shown in 

Figure 4, 

The terminology is operationally defined in DeBloois' paper. 

The exiscing objectives can easily be placed within the categories of 

the model. 

The placement of the objectives within the model ‘s shown 

in Figure 4 by the inclusion of the objective number adjacent to the 

relevant. dimension uf the model, 
the following paragraphs. 



- (GUAL) 
‘lo create institutional change 
by reorganizing the teaching 
and administrative staffs of 
Glenstary and/or secondary 

2. To create a number of differ- 
entiated staffing patterns 
which include differentiated 
functions of all school per- 
somel including teachers, 
administrators, an:| parapro- 

3. To create a number of differ- 
entiated staffing patterns 
which include difterentiated 
salaries according to func- 
tions and roles, 

4. To create a number of differ- 
entiated staffing patterns 
which include flexible in- 
structional time schedules. 

5. ‘To cxeate a number of differ- 
entiated staffing patterns 
which ineinde differentiated 
instructional modes, 



this wiii be further demonstrated in 


Should represent the entire con- 
cept of differentiated staffing. 
1,. general form, as now stated, 
this could possibly be the inter- 

Falls under the category division 
of labor within Perpetuation 

Related specifically to remuner- 

ation within Perpetuation Struc- 

‘Can be included under the cate- 

gory instructional model: schel- 
uling, resource use, 

Can be included under the category 
instructional model: scheduling, 
resource use, curriculum within 
Systems Self—Renewal. 









To improve the management of 
professional personnel. 

To improve the crganizations 
of professional personnel. 

To imscove the instructional 
and technological skills of 
professional personnel. 

To improve the attitudes of 
professional personnel. 

To bring about changes in 
student attitudes and 

To increase community parti- 
cipation in the educational 

To encourage State Education 
Agencies to change their 
certification procedures, 

To promote the participation 

of local teacher organizations 

in major decisions. 

To encourage universities to 
make changes in inservice 
and preservice programs. 


Could conceivably encompass all 
of the Collegiality, Workflow 
Structures, and Perpetuation 
Structures, ies might also in- 

este eee 


Nas no specific referrant within 
the model although it might 
broadly be interpreted as a part 
cf commitment to the profession 
within Professional Disposition. 

Falls within the Perpetuation 
Structures component of the model 
and relates specifically to 

Seems to relate most to the com-~ 
ponent Professional Disposition. 
With a more general interpreta- 

tion it might include, also, 

Individualism and Collegiality. 

Falls within the Accountability 
category and relates specifically. 
to production, 

ls a part of the segment of Work- 
flow Structures entitled philosoph y 

‘concerning r resource use. 

Relates specifically to the cre- 
dentialing category of Perpetuation 

Represents one aspect of decision- 
making within Workflow Structures. 

Relates both to training for im- 
plementation within the category 

Systems Self-Renewal and retraining 
within the category Perpetuation 



INDIVIDUALISM (9) Commitment to the Profession (7) 
Commitment: to the Public Trust 
Commitment to an Area Expertise 

| Commitment to the Student 

| Essential Characteristics 

Exchange with the Organization 
Identification with Organization 

Degree af Indepen.ience / 
Iuterpersona] Competence 
\ Cooperative Exchange with 
\ the Organization | 

| Interaction of Personnel 
Inter-Expections of 

Criteria for Continuing 

Decision-Haking (13) 
Philosophy Concerning 

Resource Use (11) 
Means of Influence 
Assumptions Concerning 
Human Behavior 



Implementation and Fvaluation 

Retraining (8) 

Instructional Model: Scheduling, 
Resource Use, Curriculum (4) (5) 

Philosophy, Product & Process 

Goals, and Objectives ‘ 

Educational Needs Assessment 
Production (student attitudes) (10) 
Efficiency (time and money) 
Maintenance of Structure 
(Morale & drive) 

Remuneration (3) 

Promotion \ 

Division of Labor (2) 

Training for Implemen- 
tation (14) 

Credentialing (12) 


Recruitment. | 

Figure 4: The DeBloois Conceptual. Model for Describing and Comparing 
Patterns of School Personnel Utilization. 

-27 - 

The components of the conceptual model are, for the most part, 
accounted for by the objectives. The range of specificity of the 
objectives in relation to the components of the model is noticeably 
varied. Many of the sub-parts of the main categories are not specifi- 
cally included, which jndicates that many cf the objectives are quite 
generally stated. ‘his generality lends itself to greater ambiguity 
and seems tu be a tenable rationale for the validity of the results 
of the semantic differential data, That is, the generality of the 
objectives is a likely cause of lower evaluations with the adjectives 
related to clarity. 

This analysis points out greater need for specificity in many of 
the revised objectives. However, in a few cases the original objec- 
tives might be considered either too specific or incomplete. For 
example, Objective #13 relates to the decision-making process and 
specifies the invelvement of teacher stgantdations in this process. 
There are other groups, as well, which should be specified in this 
objective. ln cases such as this, the original content of the objec- 
tive needs to be broadened so that the complete concept is expressed. 

This idea relates directly to the adequacy of the objective. 

- 28- 

The Revised Objectives 


The following list of revised objectives is the product of the 
analysis and revision process, ‘The list includes a set of process 
objectives for the USOR SPU program end a set of organizational objec- 
tives for schools perticipating in the SPU program. The organizational 
objectives include some which are related to the organizational struc- 

ture and some related to the organizational process. 

A Proposed Revision of the Goals, Objectives and Guidelines 
of the School Personnel Utilization Program 
The goal of the School Personnel Utilization Program is to jmprove 
the teaching and learning environment in elementary and secondary 
schools by finding more effective ways of organizing their teaching and 
aduinistrative staffs. In order to meet this goal the following objec- 
tives are proposed for the SPU program, 

SPU Program Objectives 

1. Yo orient school personnel and the public to alternative organiza- 
tional structures for schools. 

2. To provide training for school personnel in the managerial, organ- 
izational, and instructional skills required by newly adopted 
organizational structures.” : 

3. To encourage universities to provide inservice and preservice 
training programs in the skills required by alternative organiza- 
tional structures. 

4. To bring into being a credentialing process for professional 

teachers that includes multiple entry and exit points, and non- 
sequential movement to higher certification. 

- 29 - 

To encourage state education agencies to seek legislation and to 
adopt policies which provide for alternative school organizational 

In addition to the SPU Program process objectives, the following, 

guidelines are suggested for project schools funded by the SPU program. 

Guidelines for the Oxganizational 
Structure of Project Schools 

1. The roles of instructional personnel, as well as administrators and 
other personnel, should be differentiated on the basis of the type 
and amount of responsibility assigned. 

Salaries of instructional perscnnel should be differentiated on the 
basis of their roles and responsibilities. 

Classroom instructional personnel should be provided promotional 
incentives which allow them to advance in responsibility and pay 
while remaining teachers, These promotional incentives should be 
equivalent to those provided administrators and other non-instruc- 
tional personnel. 

4. Instructional time schedules should be flexible. 

Guidelines for the Organizational 
Process of Project Schools 

5. Decisions should be made at the levels in the school's organiza- 
tional structure where the most information exists. 

6. The schoo] staff should engage in group problem solving. 

7. The school should utilize a nunber of instructional strategies and 
should provide a wide variety of resources to students for facili- 
tation of their learning. 

8. The community should participate in the implementation of the in- 
structional program. 

9. The school's in-service training programs should be designed to 
facilitate the achievement of school goals. 

10. ‘the assignment of responsibilities within the school organization 
should be based in part on the individual differences of its mem- 
bers, their different strengths and weaknesses, and their varied 
personal goals. 

- 30 - 

Teachers, administrators, and other school personnel should parti- 
cipate as peers in the school's organizational structure, though 
their responsibilities differ as to type and amount. 

School personnel should recognize that they must be mutually de- 
pendent if organizational goals are to be achieved. 

The school organizational structure should encourage its staff 
members to be self-actualizing. 

The community should participate in the setting of goals for the 
instructional program. 

The school should have a system for objectively evaluating: 

a. Staff, pupli, and community attitudes. 

b. Cost-effectiveness of specific elements of the school program. 
c. Success of its graduates, transfers, and drop-outs. 

d. School program relevancy. 

e. Pupil achievement in terms of school objectives. 

An attempt has been made to state the objectives clearly. However, 

in trying to achieve a second criterion of conciseness, their communi- 

cation value may have been decreased. Further discussion of the con- 
cepts underlying the organizational objectives may be found in DeBloois' 

(1971) conceptualization of the school personnel utilization program. 


Berlak, H. Values, goals, public policy and educational evaluation. 
Review of Educational Research, 1970, 40, 261-270. 

Delloois, M. A conceptual model of school personnel utilization: A 
developmental. spectrum for evaluation purposes. Tallahassee, 
Florida: Evaluation Training Center, 1971. 

Morrison, D. Multivariate statistical methods. New York: McGraw- 

Hill, 1967. 

Metfessel, N. S. & Michael, W. B. A paradigm. involving multiple 
criterion measures for the evaluation of the effectiveness of 

school programs. [Educational and Psychological Measurement, 
1967, 27, 931-43. 

Scriven, M. The methodology of evaluation. Perspectives of Curricu- 
lum Evaluation. AERA Monograph Series on Curriculum Evaluation, 

1967, No. 1. 

Stake, R. E. Objectives, priorities, and other judgment data. Review 
of Educational Research, 1970, 40, 181-212. 

Taylor, P. A., & Maguire, T. A. Perceptions of some objectives for a 
science curriculum. Science Education, 1967, 51, 488-93. 




Cover Letter for the Initial Goal Evaluation . . 

Instructions for the Initial Goal Survey... . 

School Personnel Wrilization Objectives. 

Goal Fvaluation Instrument .... 6. 1. ee we we we 

Follow-up Letter for the Goal Evaluation ..... 

Table 1: 

Table 2: 

Means and Standard Deviations Across 
Adjectives for Each Group and Objective. 

Means and Standard Deviations Across 
Adjectives for Each Group and Adjective. 

= 93> 


» 35 

- 36 








Department of Educational Research 
College of Fducation Sept ember 4 » 1970 

Dear H 

| The United States Office of Education, through its Bureau 
of Educational Personnel Development, has established a program 
te help our nation's schools utilize more effectively their 
iustructional] ‘and administrative personnel. A preliminary set 
of objectives has been prepared by this "School Personnel 
Utilization" (SPC) program staff. They have now asked this 
departisent for an independent evaluation and refinement of the 
initial statement of objectives, 

We are asking approximately sixty active educators having 
specific professional roles to evaluate the original objectives 
| and to suggest changes. If revision is called for, and that is 
likely, you will receive copies of the revised objectives for 
additional comment before a final craft is submitted to the 
Office of Education. 

The SPU program staff has taken a significant step in sub- 
mitting their goals and objectives to professional and public 
appraisal. I hope that you will take this opportunity to parti- 
cipate in the appraisal, The enclosed instrument will facilitate 
your evaluation of the objectives. Comments which suggest 
specific revisions would be especially helpful. Would you please 
return your response by September 23. 

Thank you. : 


Jacob G. Beard 
Associate Professor 


- 34 - 

om we pares ee 



« + ~ 


On the following pages, you will find a statement of a goal or 
chjective and a group of adjectives. Evaluate the statement as 
a nationi coal or objective for the SPU program by the follow- 
ing procedures: ; 


Check an fetiewe sf vou fee! that the statement is 
very choredy reloted co one ond of the scale, 

say Bene ee OOS a a GL OOEE 
or ; 
s pial ce : wee et 2 compte te Ne loose 

Check ae follows if you feed that the statement is 
anite closets related to ene ond of the scale, 

wed Bi NE 1 Re OR EKONE 
ar t 
Weeyy') oo ot ae se Rens Se aM 3 strong 
Ch; ay fedlevs if you feel that the statement is 

ontiy slightly related to one end of the scale, 

To) ee er en oe, ae 7?) narrow 
or , 
broad ot Boe a hte Ss SPOS a) st MAT TOW. 

Check as follows {£ you feel that the statement is 
equally associated with both ends of the scale or 

if the seale is completely irrelevant. a 
proper os oe ee ee ee improper 

Place your check-marks in the middles of the spaces, not on 
the boundaries (i.e., t_y/73, not va Vs 

Check cvery seale for every statement--do not omit_any. 
Never put more than one check-mark on a single scale. 
You should be able to vork at a fairly high speed through 

this questionnaire, but do not be careless because we want 
i ? 
your tine opinion. 

The complete ser of objectives is presented on the next page. 
After stedving them carefully, proceed to the following pages 
and bepia the arsessinene task as presented above. 


1. Objectives : 

The yoal of the School Personnel Utilization Program is to create 
institutional change by reorganizing the teaching and administrative 
staffs of elementary and/or secondary schools. In order to bring this 
about, the School Personnel Utilization Program has the following 
| objectives, 

1. To create a number of differentiated staffing patterns which 
include the following elements: 

a. differentiated functions of all school personnel including 
teachers, administrators, and paraprofessionals, 

b. differentiated salaries according to functions and roles 
c. flexible instructional time schedules 
d. differentiated instructional modes 

2. To improve the management, organizations, instructional and 
technological skills and attitudes of professional personnel. 

3. To bring about changes in student attitudes and achievement, 
4. To increase community participation in the educational system. 

5.. To encourage State Education agencies to change their certifica- 
tion procedures. 

6. To promote the participation of local teacher organizations in 
major decisions. : 

7. To encourage universities to make ¢hanges in inservice and pre- 
service programs. 

Goal Evaluation Instrument 

GOAL--To create institutional change by reor- 
ganizing the teaching and administrative staffs 
of elementary and/or secondary schools.* 

The idea expressed in this objective is ....... 

beneficial 0 OS, Ne te ee armel 
appropriate too) inappropriate 
timely oo Mt tinely 
feasible Bae ie 2 infeasible 
logical Bo et oe eb ogicall 
realistic : : : : : : unrealistic 

The statement of this objective is . 1. 6 sw ew ew wes 

clear — So A nelear 
understandable :_ 3: 3 so? 33s confusing 
meaningful toss meaningless 
adequate fot Ss inadequate 

COMMENTS: Please list here any suggestions you have or problems you 
foresee concerning this objective. 

* For the sake of brevity,’the semantic differential scale below each 
goal has not been reproduced for the next 23 objectives. It is to 
be understood that the instrument sent to the respondents consisted 
of 23 pages, one goal/page, with the full scale included for each 

- 37 - 


so oreere eee — Pee ot eee eeeen teen me tee 

Department of Educational Research 
College of Fducution October & , 1970 

Dear : 

Your participation was requested in a survey of the United 
States Office of Education objectives for School Personnel 
Utilization (SPU) which was mailed September 4, 1970. To date, 
we have not received your x;esponsé to the questionnaire. As 
stated in the initial letter, the sample of participants was 
incentionally kept small. Thus, your response is extremely 
importent in order to make a valid analysis of the results. 

It is rare that the U. S. 0. E. permits such evaluation of 
: its goals and objectives. You are in a position of making a 
rare and valued contribution to the establishment of the 
national SPU goals and objectives. We hope that you will re- 
evaluate your decision not to participate in this survey and 
return your resporses to us immediately. 

Thank vou for your consideration. 


Jacob G. Beard 
Associate Professor 

JGEs eg 

- 38 - 

Teble 1 

Means and Stendard Deviations 
Across Adjectives for Each Group and Objective 

Objective Respondent _. Teachers Administrators 
~ eee eet A Seen ee! 2. 

1 N83 6.26 og 6.29 56 

Overali Goal Not D.S. $396 0..75 5.82 0.96 

Differenciated ° DS. 6.20 .&: 6.10 .72 

Funet ie ns Not D.S. 5.68 Lear 5.82 1.72 


Differentiated n.S. 6.08 .98 5.76 12.02 

Salaries Not. DS. 5676 “0.98 5.98 1.03 


Flexible D.S. 6.64 .42 5.80 1.24 

Schedule Not D.S. 6.31 0.87 6.26 1.13 


Instruction D.S. Seis 2.82 5.64 1.42 

Modes Not N.S. 6.04 2.22 5.18 12.49 


Personus 1 D.S. 5.48 12.20 4.99 1.46 

Management Not D.S. 5.62 0.86 5.42 2.09 

: 7 

Professional DiSs 5.54 09 5.18 1,27 

Organizations Not D.S. Sele. L683: 5.19 12.80 


Instructional D.S. 6.00 68 6.49 .69 

Skills Not D.S. 6,39 0.88 5,74 12.81 


Personnel D.S. 5.66 1.04 6.05 1.25 

Attitudes Not D.S. 6.19 0.91 5.45 2.09 


Student D.S. 6.14 8&8 6.08 2.02 

Outcomes Not D.S. §.91 0.98 5.30 2.06 


Community D.S. 6.72 .4¢ 6.60 .5?7 

Participation Not D.S. 5.98 2.30 6.41 .84 

12 : 

Certification N.S. 6.18 2.27 6.49 .55 

Procedures Not D.S. 6.11 2.75 4.68 1.89 


Decision D.S. 6.28 1.50 5.76 .74 

Making Not D.S. 6.25 0.92 5.54 1.58 


Teacher D.S. 6.38 58 6.10 .72 

Training «42 



ne Prof. 


. 80 
- 84 

Table 2 

Means and Standard Deviations 
Across Objeetives for Fach Group and Adjective 

Adjectives Respondent __ Teachers Administrators College Prof. 
x 0G x 0 =. Q 

Beneficial D.S. 6.54 .46 6.31 .380 5.93 .34 
Not D.S. 6.39 .54 6.09 .69 6.29 .68 
Appropriate D.S. 6.47 .52 6.21 .42 5.81 .65 
Not D.S. 6.34 .56 5.99 .62 6.31 .57 
Timely D.S. 6.36 .69 6.25 .387 5.80 .é54 
Not D.S. 6.30 .64 6.04 .52 6.31 .54 
Feasibie D.S. 6.19 .64 5.72 .84 5.48 .56 
Not D.S. 5.79 .74 5.62 . .92 5.67 .64 
Logical D.S. 6.50 .67. 6.12 .62 5.54 .665 
: Not D.S. 6.17. .6é9 5.76 .98 6.16 .82 
Realistic D.S. 6.14 .72 5.65 .8&l 5.26 .38 
Not D.S 5.70 .80 5.28 1.00 5.63 .78 
Clear D.S. 5.64 .89 5.99 .70 4.85 1.28 
Not D.S. 6.12 12.02 Behl Bets 5.54 .74 
Unders .andable D.S. 5.61 .89 5.98 .64 4.80 2.27 
Not D.S. 6.18 .97 5.46 12.30 5.59 77 
Meaningful D.S. 6.03 .&2 5.78 .74 4.61 2.26 
Not D.S. 6.81 72.12 5.53 12.14 5.54 .79 
Adequate D.S. 5.04 2.62 5.49 .90 4.18 12.37 
Not D.S. 5.46 1.34 4.84 1.74 5.01 .90 

- 40 -