Skip to main content

Full text of "DTIC ADA530086: AFOTEC Cyberspace Testing"

See other formats



AFOTEC 

AIR FORCE OPERATIONAL TEST AND EVAUJATIOIM CENTER 











Report Documentation Page 


Form Approved 
OMB No. 0704-0188 


Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and 
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, 
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington 
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it 
does not display a currently valid OMB control number. 


1. REPORT DATE 

12 MAY 2010 


2. REPORT TYPE 


3. DATES COVERED 

00-00-2010 to 00-00-2010 


4. TITLE AND SUBTITLE 

AFOTEC Cyberspace Testing 


5a. CONTRACT NUMBER 


5b. GRANT NUMBER 


5c. PROGRAM ELEMENT NUMBER 


6. AUTHOR(S) 


5d. PROJECT NUMBER 


5e. TASK NUMBER 


5f. WORK UNIT NUMBER 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

Air Force Operational Test and Evaluation Center,8500 Gibson Blvd. 
S.E.,Kirtland AFB,NM,87117 


8. PERFORMING ORGANIZATION 
REPORT NUMBER 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS (ES) 


10. SPONSOR/MONITOR’S ACRONYM(S) 


11. SPONSOR/MONITOR’S REPORT 
NUMBER(S) 


12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited 


13. SUPPLEMENTARY NOTES 

7th Annual Acquisition Research Symposium to be held May 12-13, 2010 in Monterey, California. U.S. 
Government or Federal Rights License 


14. ABSTRACT 


15. SUBJECT TERMS 


16. SECURITY CLASSIFICATION OF: 


a. REPORT 

unclassified 


b. ABSTRACT 

unclassified 


c. THIS PAGE 

unclassified 


17. LIMITATION OF 

18. NUMBER 

ABSTRACT 

OF PAGES 

Same as 

16 

Report (SAR) 



19a. NAME OF 
RESPONSIBLE PERSON 


Standard Form 298 (Rev. 8-98) 

Prescribed by ANSI Std Z39-18 






AFOTEC 

Cyberspace Testing 


Mr Jeff Olinger 
AFOTEC Technical Advisor 


NPS Acquisition Research Symposium 

12 May 2010 


2 







Purpose 




* Provide a brief overview of AFOTEC, our responsibilities 
and test initiatives, and a quick look at several business 
system-related test programs 





















Scope & Responsibilities 




* Independent operational test 
agency for AF 

- Reports directly to CSAF 

* Operationally tests all 
Acquisition Category I, II, & 
oversight programs 

- Required by USC Title 10 

- Guidance & direction from 
DoD 5000-series 



4 











AFOTEC Locations 




NelliS;AFB 
(Det 6 & OL-NN) 


Buckley AFB 
(Det 4, OL-BC) 

Inactivating NLT 30 Jun 11 

O 


eterson AFB (Det 4) 


Los Angeles AFB 

(SMC LNO) ' 



dwards AFB 
iet 1 & Det 5) 


< LP Kirtland AFB 
v (HQ AFOTEC) 


Ft Worth, TX O 
(Det 1, OL-FT) 


Detachment (Det) 

O Operating Location (OL) - Test 
O OL - Liaison Officer (LNO) 




Hanscom AFB 

J (ESC LNO) O 


1 / 


right-Patterson AFB 

(ASC LNO) 


O Washington, D.C. 
F (NCR LNO) 
(Det 1, OL-AV) 


Marietta, GA 
(OL-MG) 


(inactivating NLT 15 May 10) 
Hurlburt AFB - r® 

< De,5 ' 0L - HF ®A V 

(Det 2, AAC LNO) 


5 






Approach to Testing 




* Early & continuous influence throughout the acquisition 
process 

* Integrated Developmental Test /Operational Test (IDT/OT) 

* Determine effectiveness, suitability & mission capability 

- Clearly identifies capabilities delivered to warfighter 

* Reports provide decision-quality data to acquisition 
decision makers 

- Occasionally inform major command fielding decisions 








AFOTEC Initiatives 




* Early Influence 

* Integrated Developmental Test/Operational Test (IDT/OT) 


* Space OT&E Model 

* Cyberspace OT&E Initiative 















Early Influence 




* Formalized approach to: 

- Refine capability requirements and acquisition strategies 

- Develop test and evaluation strategies and plans 

* Greatest opportunity to affect emerging capabilities 

- Apply standardized OT&E methodologies 

- Typically begins before Milestone A 

* Raises interaction level & brings more people to our area 

- Summits, High Performance Teams, AF Test and Evaluation 
Symposium 

Pushing Influence to the Left 


AF and Joint Influence - AF/A5R HPT, AFROC, and JCIDS documents 


B 


Materiel 

Solution 

Analysis 

Materiel 

Development 

Decision 


Technology 

Development 


Engineering and 
Manufacturing 
Development 


Production & 
Deployment 


Operations & 
Support 


PDR 


o 


0 


Post-CDR 

Assessment 


LRIP/IOT&E 


0 


FRP 

Decision 

Review 















4 k 



dis¬ 


integrated DT/OT 


% i A f 
k A 


m 


Mi 


* Work early and consistently throughout program 

* Reduce duplication and share data 

* Shorten the timeline from concept to deployment 

- Reduce cost and time by leveraging developmental test 

* Optimal use of limited resources 

- Test Articles, Ranges, Funding and Time 

- > $58.9M & 56 Months saved since 2008 


DT Data 


Plans 


f 


- 


DT/OT 

Data 


OT Data 


DT 


DT 

Events 


Time 

Saved 





Plans 




|j 


ans 




ans 


DT 

Activites 


$$ 

Saved 


OT 

Activities 




9 


















































Space OT&E Model 




* Acquisition & OT&E policy and execution disconnect 

* AFOTEC hosted a Space Test Initiative Summit at 
Kirtland in June 2008 

* Early Funding, Early Decisions 

- Front-loaded acquisition decisions 

* OT&E did not inform acquisition decisions on time 

- Effort 90% post-launch 

* Tenets of transformed space OT&E model 

- Maximize early influence 

- Integrated DT/OT throughout 

- Agile analysis and reporting 

- System-of-system evaluations 

- Effort now 90% pre-launch 


10 









Cyber Acquisition 
and Sustainment 





* AFSPC and AFMC anchored Agile Cyber Acquisition with 
Ops & Innovation and Rapid Tiers 

- AFOTEC involved from the onset; remains cognizant and 
collaborative for synergy and integration 

* AFOTEC is evolving Cyber OT&E Model for AF and Joint 
foundational acquisitions 

- Synchronized with on-going OSD efforts evolving 
acquisition policy for IT Systems 



11 























Cyberspace OT&E 


4 k 



% i A f 
k A 


m 


^r!%- 


S» 


Collaborating with acquisition & warfighter communities 

- Influence requirements process early to ensure “fight 
through” capabilities are testable and measurable 

- Ensure STAR contains relevant cyberspace threats 


* Collaborating with the Air Force Research Labs (AFRL) 


- AFRL is supporting AFOTEC development of Cyberspace 
OT&E methodologies for dirty and contested environments 

- AFRL providing critical opportunities to identify the latest 
technologies, capabilities, vulnerabilities and threats 


12 








Cyberspace OT&E 




• OT&E Tenets of Excellence 

- Early and continuous influence to refine requirements 

- Integrated DT/OT throughout acquisition life cycle 

- Agile to support emerging cyberspace acquisition strategy 

- Conducted in operationally realistic environments 

(e.g. dirty and contested environments) 

* Cyberspace acquisition strategy is emerging and will 
demand a comprehensive OT&E strategy 


OT&E of Cyberspace capabilities should enable timely acquisition and 
fielding of capabilities that ensure mission assurance 


13 











Program Examples 



\ i i / 
i i 


• Defense Enterprise Accounting and Management System 
(DEAMS) 

- Joint OSD Business Management Modernization Program 

- AFOTEC involved Jun 09—2 yrs prior to MS B (EOA) 

- IDT/OT: integration into existing system/processes 

- Impacted schedule for Increments 1&2/Spirals 1-4 

* Expeditionary Combat Support System (ECSS) 

- Provide AF-wide integrated logistics system 

- Involvement since program initiation 2007 

- IDT/OT: completed EOA Apr 10 

■ Results based on real data: functional/systems integration 
testing (FIT/SIT) with contractor/RTO and AFOTEC 


14 








Summary 




* AFOTEC conducts operational test and evaluation to 
inform acquisition decision makers and major using 
commands 


* Engaged early and often across the acquisition 
community to improve acquisition processes 

* Integrated DT/OT is foundation for all testing, including 
IT-based systems 

* AFOTEC is actively engaged as part of entire acquisition 
team from earliest program stages 


15 








Questions 





16