DOCUMENT
RESUME
EA 002 382
ED 030 967
By -Case, C. Mar stork
The Application of PERT to Large-Scale Educational Research and Evaluation Studies.
Educational Testing Service, Princeton, N J
Report No- RM -69- 14
Pub Date 8 Feb 69
Note 9 16p, Paper presented at the Annual Meeting of the Amer. Educ. Res. Assn. (Los Angeles, Calif., Feb. 8,
EDRS Price MF-S0.25 HC-S0.90
Descriptors -* Critical Path Method, Educational Researchers, Manpower Utilization, Networks, *Ob|ectives,
Program Coordination * Program Evaluation, * Program Planning * Research Projects, Resource Allocations^
Task Analysis, Work Simplification
Identifiers -PERT, * Program Evaluation and Review Technique
The application of a PERT system to a large-scale pro|ect will increase the
probability of accomplishing project objectives by providing greater visibility of (1)
the project objectives, (2) the relationships among the parts of the projects, and (3)
the relationships of those component parts to the project objectives. Consequently,
educational researchers should find that a PERT system will increase their collective
ability to coordinate planning and evaluation efforts. A review of a PERT system
applied to a study of disadvantaged school children indicates that in order to
achieve optimal utility, the system should be implemented well ahead of the project
starting time, and must be responsive to program developments through continued
updating procedures. (JH)
o
rm-69-1^
[N*
vO
O
l!*\
o
Q
LU
RESEARCH
MEMORANDUM
THE APPLICATION OF PERT TO LARGE-SCALE EDUCATIONAL
RESEARCH AND EVALUATION STUDIES
C. Marston Case
U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE
OFFICE OF EDUCATION
THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROri THE
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION
POSITION OR POLICY.
Paper presented at the meetings of the American
Educational Research Association, Los Angeles,
California, February 8, 19^9 •
04
00
CO
Educational Testing Service
04
Princeton, New Jersey
o
o
May 1969
| E
r
ERIC
THE APPLICATION OF PERT TO LARGE-SCALE EDUCATIONAL
RESEARCH AND EVALUATION STUDIES
This morning I would like to cover four points related to the manage-
ment of large scale studies. First, I will discuss the need for what I
call "project visibility." Second, I will note some basic features of a
PERT chart using the example in the handout. As some of you already know,
PERT is an acronym standing for Project Evaluation and Review Technique.
Third, I will use a very simple application as a basis for making some
recommendations about implementing PERT. Finally, I will describe an
application of PERT to a large scale project at Educational Testing
Service. In this last item I will refer to the reports in the handout,
which come from one of IBM’s PERT- like computer packages.
The success of any project depends on whether the participants actu-
ally contribute to its accomplishment. This may seem obvious, but one often
finds that the participants in a large-scale project, including the manager,
are vague about how their efforts are related to the project objectives.
Such projects often fail to meet their objectives; sometimes the objectives
have to be changed so that they conform with what the people on the project
have been doing. But we try to avoid that. Usually when one finds that the
efforts of the people working on the project are not well related to the
project objectives one will also find there is a lack of project visibility.
I feel that this visibility can easily determine the difference between a
highly successful project and a not so successful or an abandoned one.
Three things need to be made visible: (l) the project objectives them-
selves, (2) the relationships among the parts of the project, and (3) the
relationships of those component parts to the project objectives.
- 1 -
- 2 -
Figure 1 in your handout is an example of how we can go about improving
project visibility. It looks rather like a flow chart, but strictly speak-
ing it is not one. It is a work item network. Each box represents an item
of work. The leader of a certain part of a project estimated that this set
of 23 work items was necessary to the completion of his part of the project.
In a work item network, a key relationship among the work items is made visible.
This key relationship is precedence. The arrows in the network mean that
the work item following the arrow cannot begin until the work item preceding
the arrow is finished. Thus one sees from a work item network how the work
items are related to each other in terms of their succession in time. Our
objective is the completion of all of the work items, but since the prece-
dence relations all flow into the last work item, we may as well say that
our objective is the completion of that last work item.
Thus far we have defined precedence relations among the work items .
Nov let us make this network into a resource allocation scheme where the
resource to be allocated is time. We can do that by assigning time durations
to the work items. The number at the lower right corner of each work item
box is the expected number of days needed to do the given work item.
Now let us see how PERT fits into the picture. We have in the network
diagram of Figure 1 all the information contained in any PERT chart .
1. It has work items (the boxes).
2. It shows the precedence relations among the work items (the lines
and arrows ) .
3. It shows expected time durations of work items (the numbers in the
lower right corners).
ERJC
- 3 -
PERT is a method of shoving proposed resource allocations; in using it one
develops a time allocation scheme, and perhaps a scheme for allocating other
resources, such as cost or manpower. It has been successfully employed in
managing large-scale projects in the construction industry and in the Depart-
ment of Defense. It was iirst used in the development of the Navy's Polaris
missile.
Nov a full-blown PERT treatment of a project involves the use of a com-
puter to analyze the network before the project starts, and then periodically
during the execution of the project. In small projects, however, worthwhile
benefits can be gained with the use of a PERT-like diagram, without getting
into more sophisticated analyses. I will give you an example. Recently my
wife agreed to the use of a PERT chart for cleaning the house and getting
ready for guests later in the day . We drafted a network diagram which gave
a clear view of the day's tasks and the precedence relations among them.
Rooms had to be put in order before they were dusted and then dusted before
they were vacuumed. Operations in the various rooms could run along roughly
parallel except in our little boy's room, which was done last, so he could
play there while we were busy elsewhere. The dishes could not be washed
until all the cooking was done. The roast was to go in the oven at a certain
time, and so on.
The chart worked quite well. It not only kept us at relevant tasks in
their proper order, but it became an unobtrusive rallying point; we worked
in reference to the chart, with no need to ask or tell the other what to do.
Now, this unobtrusiveness is an important point in coordinating research
projects, because most people, especially researchers, do not like to be
told what they are to do, in any direct way. This is one reason I think
-k-
that educational researchers will find that a PEET chart makes it
easier to coordinate efforts with others on large-scale projects.
We have been noticing the fact that the PERT chart, a detailed and
explicit document, is a help in maintaining momentum toward project goals
and that project morale is benefited by such momentum. However, the rERT
chart can be a product of management alone, with little representation of
the desires of those who are to follow its dictates. On the other hand,
a PERT chart can be the product of the joint efforts of project management
and the other participants. If it is a joint product there will be a spirit
of coordination and cooperation; done in this spirit, PERT helps to avoid
that obtrusiveness sometimes ascribed to management.
Wow, people are usually able to clean house without the benefit of a
PERT chart. However, these sorts of simple applications of PERT are invalu-
able in giving one a sense of how PERT works. I recommend that prospective
PERT users make a few such simple applications before they attempt the use
of PERT in a large-scale project.
A PERT system provides a method to make plans before a project begins
and then to review and revise the plans while the project is being carried
out. In the house cleaning, for example, we jointly prepared the PERT chart
and then referred to it when each work item was done in order to choose the
next thing to do. During the process of doing these tasks we noted down the
times each work item was started and finished and compared them with our
planning estimates to see whether we would have to work harder or would have
time to spare before the guests arrived.
This reviewing and revising while the project is going on is crucial in
keeping the project on schedule, largely because it keeps the relationship of
-5-
everyone's efforts to project objectives visible to all concerned. This
process, known as updating, should be a regularly-scheduled event during
the lifetime of the project. I think updating is the most demanding part
of keeping a large PERT system going, because the people whose work is
represented on the charts often have to be reminded of the need for infor-
mation about their progress. You can use various devices to prod the par-
ticipants for this information; you can set up a reporting schedule and
provide progress report forms; you can write memos, make phone calls, and
visit the project participants. Such special efforts are frustrating!^
inefficient but it will not be necessary to resort to them so often if the
project is well planned early enough and with all the people in the project
participating in the planning.
Now let us turn to an example of the application of PERT to a large
ppoj ect . A group of people at ETS have embarked on a large-scale longi-
tudinal study of disadvantaged preschool children. Another person and I
were attached to the project as advisors and implementers of a PERT approach
to the coordinating of the project. We came into this role more because
of our ability to bring a computer to bear on the problem than because of
management skill or experience. We have used an IBM PERT— type computer
package called Project Control System, or PCS, which runs on the IBM 360
computers .
At the outset, the project had seven component parts called task forces,
with a task force leader designated for each. Each task force was to provide
a way to assess a different aspect of the preschool child or his environment.
The main objective of the project is to identify the components of early
- 6 -
education in preschoo] and primary grades that are associated vith children's
cognitive, perceptual and personal-social development. We first worked with
the task force leaders in setting up work item networks. Figure 1 is one of
the early task force work item networks. In the initial seven task force
networks there were about 220 work items. About a month was required to
gather this ?Ln format ion and get it into the computer.
Once we had this initial version in the computer, we were able to pro-
vide schedules and graphs corresponding to each task force's proposed plans.
Page 2 of your handout shows an example of a schedule report corresponding
to the network in Figure 1. This tabulates the work items and the earliest
and latest they can start and finish. It also indicates the float or slack
associated with such scheduling; this is the amount of leeway one has in
doing the given work item providing everything else goes according to
schedule. Notice that in some cases this is a negative number, which indi-
cates inconsistencies in the proposed plan. Page 3 shows a graph of some
of the same information. We do not need to examine all the details of
these reports here although you may want to study them later.
In this project we are estimating manpower requirements for each work
item. We are doing this in order to avoid excessive manpower demands in
peak demand periods which would be costly or impossible to meet . The IBM
program prepares reports showing manpower requirements versus time. Pages
h and 5 of the handout are examples of these. Page h , for instance, shows
a tabulation of manpower requirements of various work items by days, with
the total requirement of the project for each day at the bottom. On page
5 the resource utilization report shows, with asterisks, the total require-
ment for each week and, with C's, the cumulative requirements of the
project over the term of the entire project.
- 7 -
There are three problems we have encountered in this implementation
of PERT which seem to be of sufficient importance to mention here. First,
we did not have the initial version of the project (the planning
version) on the computer and ready to go at the time the project started.
This meant that people were working by plans, some of which were later
shown to be inconsistent. That is, they could not use as much time as
they were alloting themselves and still meet certain deadlines. This is
a perennial problem: the problem of gaining time enough before the project
starts to prepare a plan which is at once consistent and acceptable to all
concerned.
The second problem we have encountered to some extent derives from the
first; at least it is aggravated by the presence of the first. This is the
problem I mentioned earlier, keeping the PERT system up to date. As soon as
the project is started, updating should begin. This means that periodically,
say every week or two, the network should be reviewed and revised estimates
of resource requirements should be provided. If any work items have been
started or finished since the last updating, they should be reported. When
update information is put into the computer system, work status reports such
as the one shown on page 6 of the handout can be provided. This report
tabulates information on the work items which have not been completed. It
includes the original estimate of time required and the present estimate of
remaining time needed to finish the work item, plus other information. In
our project updating has been somewhat thwarted by the fact that other
things received priority over the updating activities.
The third problem, which is not independent of the first two, is
that the PERT materials are not sufficiently used as a basis for reviewing
- 8 -
and revising project plans. Now this is the one key use for which PERT was
designed. The main reason PERT has not been used in reviewing and revising
plans (aside from the problems of a late start and weak updating) is that
the project staff has had to contend with a variety of unexpected events.
These repaired major revisions in project plans, such as personnel changes
and changes in the subjects to be studied. As a result the staff has often,
in critical times, fallen back on ad hoc measures to revise project plans.
In this project we are continuing to work on bringing the status of
the plans up to date. We expect the PERT system to be of more benefit as
the project progresses, plans crystalize, and the PERT system catches up
with the plans .
I believe that a PERT system, implemented well ahead of project start
and kept responsive to project developments, provides a certain visibility
of the structure of the project not available by other means. Several
benefits derive from such visibility. Basically these stem from the fact
that everyone concerned sees how the pieces of the project fit together in
support of the project objectives. With this readily available grand view
of the project, ramifications of any changes are quickly assessed and
massive changes are more easily handled. The visibility aids in maintaining
communication up and down and across project organizational lines, with a
PERT network acting as an impersonal entity to represent the wills of all
project participants. My conclusion is that such visibility, in an other-
wise well-conceived project, will make it more likely that the project
objectives will be accomplished.
THE APPLICATION OF PERT
TO LARGE-SCALE EDUCATIONAL RESEARCH AND EVALUATION STUDIES
C. Marston Case, Educational Testing Service
Part of the Symposium:
Management Problems in Conducting Large-Scale Research and Evaluation Studies
1969 American Educational Research Association Meetings
February 8, 1969, Los Angeles
PROJECT BASE OATE t JUL 68 PROJECT DURATION 667.0 PROJECT COMPLETION OATE 10 OCT 69
i r\
<D
W
cA
PM
3
<
UJ
<
o
Ok
*
8
j
.o
o
H
2 4
*
S?
5
8
CC
H-
Z
o
u
►-
o
Ui
-*
o
0C
3 t
Cfi
Z
o
N
UI
u
ec
©
O
00
UJ
cc
CO
*C
o
<
UJ
*
ac
o
o
z
o0
UJ
u.
c
ec
a
~ z
*
ec
a
*
u
z
ac
a
UJ
Z to
UJ
»
to
Ui
OC
UJ
L>
Z
UJ
UJ
00
z
©
ac
UJ
UJ
©
a
UJ
to
to
I H-
15
15
12 ;
© !
Or
t ui
I
l
a
o
o ►-
c z
1-4 3
O' O
UJ T
a. <
o
a x
a k
1-4 DC
ac <
ULJ H
a to
44 tn
55
kk
*4 M
& a
33333.
•4 -* * 1 ! *4 -» <
M • Iff ft# » m!
•4N «M
• «< 0 fl 0 o 0 flO!att<oicott»«o«Ofl 0 acoflefaa«i»c» 0 >iO>* 9 >d»»
~ <o <o< 0 i« <o «««<:« 4 *<o «o -O'# *
S»»a 88 iS|SiSSgijl!!S 3552 S 2 £S 2 |^
Mj N in «4fMM «4 (M -^NW «4MN MHN • •* «*
l i ! :
> e 0 00 9 « 4 <|R>e 0 4 l 4 N 4 i nO> 0 :i 4 ^ 4 l 9 >
• • ♦ •••'•**•• # #«*••**• tit •
ooooNN<oocD^irkO>^^f o^iAO^me^i
ttnt 0 4 ’ONKacO (^OOf 4 « 4 Ni
w © * * 6 * 6
« o
UJ (0 )
i »h i
| MM I
I
I
ftO o
■ t* in
fO
•* O O *
* o
er>
22
30 I
I a I i
I * I
ii ;
I* i
I
I
!
I
I
• o ©
to m
rv
• o o
4 * o
fNJ
• o ©
m m
S I
• o o
CM O
• o o
m in
#;##*>•***# *6
*66
* 6 *
6 m 6
I
*
o ou
u
o u
o u
o
o
o
o
* • # u
u *
o o
♦ * *
* * # *
« * * # * * « * #
* a
a
« #
« « *
# *
* # «
UJ UJ
-J «j
< <
o o
00 00
O UJ
C >
M M
ac o*
uj <
a. .j
©
a
• •••••••••••••§•§•§•••••••••••••••••••
ooco^JCT'ococcpofr. iTN-omirmir. ♦
OOOOOOCOaDOOOOCOCCCOCOOCOOOOQOCOCOXcOXOOCOaOCOOOCO^a'OC'O'a'C'a'CrC'a'C'
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ o ^ ^ ^ >n e e o e e e o o %© ^ *£ ^ ^ ^ ^ ^ ^ ^
jcoooaQ.&a.ah^hk>>>>uuuuazzzzecoeccDtfoc 2
Z Z ZZZUJUJUJUJUJOUOOOOOOUJUJUJUJUJ<< 4 C<UJUJUJUJ<< 4 C
.o<‘C<<^iflWin^cocorE 2 zoQDOQooooiLiu.iii:xr
HcotTN(MrMa*'Oc\)OMOno^^^cc%rH©ircNj^c<oo^^ONnoh' 6 ff k .CK
<\l f\! HpHVJ M f\J fO MfVJfVJ M M <VJ H ^ fO M N fSJ H H ^ MM
ERiC
3
*0
VO
<D
HO
05
PL|
O'
o
M
2 g
<
O
<
Q
ui
00
3
Ui
J c*>
< 3 <
$ n *
o
x
o
x
u.
x
Ui
k
to
>
to
o
00
k
Z
o
o
o
Ui
-3
c
X
a
z
ce
*
*
co
i/d
Ui
O
t/i
3
i/>
>0
x
o
3:
0
1
oo
00
tn
o
3
k
1A
k
V)
o
<
UI
X
co
k
UI
UI
-J
k
0 C
o
3
k
Ul
z
CO
0
CD -H
3
in
Ui
5
o
z
3
a
o
oc
a
Ui
u
z
Ui
5 IS
ui 2
CO UJ
z §
3 UJ
ae *y)
I I
SI
H < I
CO Q I
UJ |
kX I
< CO I
J 2 I
CO
<0
cd
3
<
N t H
UJ
Qk
UJ «£
k- a
u
Ui k
a. OC
K<
Ui k
t/D
1 a i
1 pH
H
pH
p
pH
pH
pH
pH
p*
ft.
«u
<
i i
i i
f *
IT
CD
ID
ID
in
in
in
in
•n
♦ k I
I
! >
V
>
k
k
k
>-
k
>
V
pH
1 ** 1
1 <
<
<
<
<
<
<
4 t
<
<
o
o
3
1 Z 1
1 3 1
1 o
1
o
e
O
O
o
o
o
o
c
k
1 |
z
pp
1 Jk |
1 o
o
o
C
o
o
o
o
o
O
1 CD < 1
1 •
•
0
0
•
0
0
0
0
0
<
z
1 « OC 1
t c
o
o
M
o
in
o
o
in
o
o
1 X 31
i rt>
pH
m
pH
fVJ
rvj
pH
pH
<_!
1 O Q 1
1
k k |
5a |
OCX I
uj a t
a. a i
i
GO
<0
3
-3
O
o
o
CO
00
CO
CO
C0
00
00
3
00
HD
H>
<0
HD
H>
HD
HD
>
52
>
k-
a
k
>
>
U
CD
3
O
U
Ui
a
o
a
UI
Z
<
Z
a
to
5
z
z
Q
HD
O
k
a
PM
3
k
c*
*n
m
pH
pH
pH
pH
CO
<30
00
00
00
CO
00
3
3
HD
HD
0
PD
HD
HD
0
HD
HD
CO
CD
a
Cl
a
ft.
ki
>
O
3
3
UI
cu
LU
UI
a
O
UJ
<
PC
CO
t/D
to
CO
o
z
Q
k
CD
pn
pn
m
*
nj
o
pH
N
pH
eg
IM
pH
PH
« k
< <
X oc
s§
-J
<
I 2
HD
o
o
o
CD
o
c
O
o
o
0
0
0
0
•
0
0
0
0
0
O
o
o
o
o
O
o
o
o
o
o
o
o
o
o
o
o
o
o
o
0
0
0
0
•
0
0
0
0
0
PM
o
o
PM
o
in
o
o
m
o
pH
4-
pH
m
pH
PM
eg
p4
pH
j
H
CO
l—
1
CO
k-
CO
1
UI
3
U
3
UI
i
k-
HD
UJ
HD
k
i
u>
3
UJ
i
QC
U
3
C
X
i
a
3
3
3
a
i
CO
<
i
>
X
Z 1
X
<v
k
H>
UJ
_ 1
a
CO
X
C I
k*
LL>
X
1
<
<
Um
3
p* 1
X
to
1
G
o
k* I
-J
Ui
H?
u.
z
1
a
k
O
k
o
a i
X
<
<
i
UJ
O
3
k
PH |
z
X
1
Z
c
X 1
PH
<
a
1
X
X
UJ
a i
a
X
i
<
u.
CO 1
m
c
o
1
UJ
z
ui t
X
z
<
1
<
c
a i
a
fcH
k
i
UI
k-
UI
i
X
o
<
eu
X
i
a
o
z
o
a
i
p
a
CD
a
X
i
z
PH
UJ
i
<
PH
CO
PH
k
i
•j
3
UI
2
Z
i
a
a
o
CL
pp
CO
>0
o
3
<
CO
>0
CD
3
<
00
*0
CO
3
<r
k
CO
CD
<
00
<0
-J CD
LU 3
oc <
00
hd
Ui
tn
k
a
3
k
c0
00 00
hd %o
gg
CD
Z
oc
a.
oc
o
UL
CO
CD 00
> >
a o
z z
z
Ui
X
3
OC 00
k <0
to
Z >
f* o
z
0
o
o
X
o
PVJ
>
c o
k
k k
2
O'
pH
m
pn
o
m
k
»H
k
pH k
Z
PH
n.
a.
O
PH
UJ
k
3
mJ
X
CO
c0
k
PH
3
UI
mJ
CO
3
X
k
UJ
Uj
Ui
<
UJ
Ui
<
Ui Ui
k
UJ Ui
UJ
k
k
CD
k
PH
k
>
PH
k k
CO
k k
O
k
<
<
Z
<r
X
<
k
HI
< <
z
< <
z
<
a
3
PH
o
UJ
a
p«
Q
tu
cd a
PH
o o
<
G
z
k
pj
X
z
<t
PH
UJ
CD
<r
X
3
UJ
X
Z
mJ
<
JNi
<
PH
eg
o
a
<6
O
a
QC M
U 3
CO ft.
CO
PP
lO
k
-I
<
z
<
<
Q
HD
O
a.
3
ft.
O
z
<
co
a.
c
CL
hd
z o
~ a
00
a
UJ
oc
QC
UJ
to
CO
5
I
<
O
z
<
2 ft. Ui
»-« OC
X >-» c
O 3 U
< CL tO
<
eg *
pH pH
a cl
3 2
a ft.
a
ui
oc
a
Q
Z
<
UJ
to
-D
HD 00
pH O
a CL
> PH PH
Ui 3 2
oc a a
<
oc
CD
z
oc
a
to
cc
2
&
I I
I ~!
9 2 1
l i
o
"D
<
i/%
PM
t
HD
3
PH
m
3
3
O
O
o
O
C
P0
k
k
pH
pH
a
X
X
a
* ft.
a
a
ft.
&
a
o
ERIC
