At A Glance:

Capstone Sponsor: NASA
Capstone Team Lead: Annie Wu
Capstone Team Members: Sofia Campos, Lynn Chitwood, Nick Kidd, and Anthony Vo
Key Problem: Assess how NASA can responsibly integrate generative AI into operations.
Solution: Created a practical assessment framework that is scalable as generative AI improves and provided key recommendations for implmentation.

The Challenge

MSBA-BA graduate students Nicholas Kidd, Annie Wu, Sofia Campos, Anthony Vo, and Lynn Chitwood took on the task of integrating generative AI into NASA's operations to improve mission outcomes. NASA's missions rely on precision, and the output of generative AI is still prone to errors. The team focused on answering the question of how to assess and grade AI technologies effectively, while addressing concerns about AI-generated errors and inaccuracies.

The Solution

The team developed a practical framework with five assessment techniques:

  1. Human Judgment: Involving human evaluators to test AI outputs for coherence, emotional impact, and creativity using tailored rubrics and subject matter experts.
  2. Criteria-Based Assessment: Setting predefined benchmarks to evaluate the accuracy and precision of AI outputs, ensuring trustworthiness.
  3. AI vs. AI: Comparing multiple AI models by giving them the same prompts to identify their strengths and weaknesses.
  4. AI Voting: Using multiple AI models to generate predictions and then combining their outputs through a voting mechanism to assess accuracy and reliability.
  5. Standardized Testing: Adapting traditional and creative tests to fit NASA’s needs, helping to measure AI’s decision-making and ethical standards.

This framework included an adaptable assessment tool that is scalable as the technology improves, providing NASA with a clear method of evaluating and integrating generative AI. Finally, the team delivered several key recommendations to ensure that personnel are prepared for the eventual integration of generative AI. Through their thoughtful and practical approach, the NASA AI team laid the groundwork for improving mission outcomes and advancing scientific research at NASA with the help of generative AI.

 

2024 NASA Team Public Presentation